“The idea, put forward by a California smart utility box company called Span, is to put the GPUs where the power has already been allocated—at the home. Span says the average household uses only about 42% of the electricity allotted to it, and rarely reaches peak usage. Span’s smart utility boxes detect that, and steer the extra available power over to the GPUs, which live inside a ”node” that sits beside the house and looks something like an HVAC unit. The boxes contain 16 Nvidia GPUs, 4 AMD CPUs, 4 terabytes of memory, and a cooling system. When a large number of homes have these, the servers could be connected together in a network and work together on distributed computing jobs (workloads), Span says.
In exchange for hosting a node, Span pays a big chunk of the homeowner’s electricity and broadband internet bills.
And there may even be advantages for putting the compute power closer to the end users that are using the chatbots or AI services, Span says.
It’s a cool idea on paper, but it’s almost completely unproven in real-world use. Span has been prototyping the units but has yet to install any of them beside real homes. I asked Span VP Chris Lander if his company has done technical studies showing that its brand of distributed computing will be fast and robust enough to handle real AI workloads. ‘We’ve done a bunch of technical studies internally [and] a bunch of modeling for different kinds of workloads, both from the business point of view [and] the product point of view and from the technical architecture point of view,’ he replies.
The idea of asking homeowners to host boxes full of GPUs is a symptom of the woeful dearth of data center space needed for AI computing.
