Andrej Karpathy’s keynote at AI Startup School in San Francisco. Slides provided by Andrej: https://drive.google.com/file/d/1a0h1mkwfmV2PlekxDN8isMrDA5evc4wW…

Drawing on his work at Stanford, OpenAI, and Tesla, Andrej sees a shift underway. Software is changing, again. We’ve entered the era of “Software 3.0,” where natural language becomes the new programming interface and models do the rest.

He explores what this shift means for developers, users, and the design of software itself— that we’re not just using new tools, but building a new kind of computer.

More content from Andrej: / @andrejkarpathy.

Chapters and Thoughts (From Andrej Karpathy!)
0:00 — Imo fair to say that software is changing quite fundamentally again. LLMs are a new kind of computer, and you program them *in English*. Hence I think they are well deserving of a major version upgrade in terms of software.
6:06 — LLMs have properties of utilities, of fabs, and of operating systems → New LLM OS, fabbed by labs, and distributed like utilities (for now). Many historical analogies apply — imo we are computing circa ~1960s.
14:39 — LLM psychology: LLMs = \.

For decades, scientists believed that glial cells—the brain’s “support staff”—were just passive helpers to the neurons that do the heavy lifting of thinking and remembering. But that view is rapidly changing.

Astrocytes, the most abundant type of glial cell, play a fundamental role in memory. Despite most hippocampal synapses being contacted by an astrocyte, there are no current theories that explain how neurons, synapses, and astrocytes might collectively contribute to memory function. We demonstrate that fundamental aspects of astrocyte morphology and physiology naturally lead to a dynamic, high-capacity associative memory system. The neuron–astrocyte networks generated by our framework are closely related to popular machine learning architectures known as Dense Associative Memories. Adjusting the connectivity pattern, the model developed here leads to a family of associative memory networks that includes a Dense Associative Memory and a Transformer as two limiting cases.

Inspired by insects, these robots follow local cues like turns and distance to build honeycomb patterns that resist failure.

With more code created by AI, there is more surface area to troubleshoot. There is a need for AI to autonomously troubleshoot, mediate and even prevent complex incidents at scale—self-healing codegen.