Large language models (LLMs) are ushering in a revolutionary era with their remarkable capabilities. From enhancing everyday applications to transforming complex systems, generative AI is becoming an integral part of our lives.

However, the surge in demand for AI-powered solutions exposes a critical challenge: the scarcity of computational resources required to meet the growing appetite for logic and voice-based interfaces. This scarcity leads to a pressing need for cost-efficient platforms that can support the development and deployment of LLMs.

Industrializing AI software development will require transforming the processes for developing, deploying and maintaining AI systems from a research or ad-hoc approach into a structured, systematic and scalable industrial process. By focusing on cloud cost optimization and platform engineering, businesses can foster growth, profitability, and innovation in the field of AI.

Less than a year into the AI boom and startups are already grappling with what may become an industry reckoning.

Take Jasper, a buzzy AI startup that raised $125 million for a valuation of $1.5 billion last year — before laying off staff with a gloomy note from its CEO this summer.

Now, in a provocative new story, the Wall Street Journal fleshes out where the cracks are starting to form. Basically, monetizing AI is hard, user interest is leveling off or declining, and running the hardware behind these products is often very expensive — meaning that while the tech does sometimes offer a substantial “wow” factor, its path to a stable business model is looking rockier than ever.

“Personalize or Perish.” One of the leading newspapers aptly summarizes the critical nature of personalization 2.0, or hyper-personalization for businesses.

We live in an era where customers expect businesses to understand their wants and needs. Today, companies must meet customers’ needs and anticipate and exceed them. And for this, they must pivot to a digital-first mindset to create stronger, more authentic customer interactions.

How do they do this? Through a hyper-personalized, AI-powered business strategy where products, ads, and interactions are tailor-made for each customer or a group of customers.

The statement that may not be obvious is that the sleeping giant, Google has woken up, and they are iterating on a pace that will smash GPT-4 total pre-training FLOPS by 5x before the end of the year. The path is clear to 100x by the end of next year given their current infrastructure buildout. Whether Google has the stomach to put these models out publicly without neutering their creativity or their existing business model is a different discussion.

Today we want to discuss Google’s training systems for Gemini, the iteration velocity for Gemini models, Google’s Viperfish (TPUv5) ramp, Google’s competitiveness going forward versus the other frontier labs, and a crowd we are dubbing the GPU-Poor.

Access to compute is a bimodal distribution. There are a handful of firms with 20k+ A/H100 GPUs, and individual researchers can access 100s or 1,000s of GPUs for pet projects. The chief among these are researchers at OpenAI, Google, Anthropic, Inflection, X, and Meta, who will have the highest ratios of compute resources to researchers. A few of the firms above as well as multiple Chinese firms will 100k+ by the end of next year, although we are unsure of the ratio of researchers in China, only the GPU volumes.

Are you running a small business? Learn how to use AI for small businesses effectively and what tools to use to get the most out of the AI.