As drones survey forests, robots navigate warehouses and sensors monitor city streets, more of the world’s decision-making is occurring autonomously on the edge—on the small devices that gather information at the ends of much larger networks.
But making that shift to edge computing is harder than it seems. Although artificial intelligence (AI) models continue to grow larger and smarter, the hardware inside these devices remains tiny.
Engineers typically have two options, neither are ideal. Storing an entire AI model on the device requires significant memory, data movement and computing power that drains batteries. Offloading the model to the cloud avoids those hardware constraints, but the back-and-forth introduces lag, burns energy and presents security risks.









