For these reasons, and more, it seems unlikely to me that LLM technology alone will provide a route to “true AI.” LLMs are rather strange, disembodied entities. They don’t exist in our world in any real sense and aren’t aware of it. If you leave an LLM mid-conversation, and go on holiday for a week, it won’t wonder where you are. It isn’t aware of the passing of time or indeed aware of anything at all. It’s a computer program that is literally not doing anything until you type a prompt, and then simply computing a response to that prompt, at which point it again goes back to not doing anything. Their encyclopedic knowledge of the world, such as it is, is frozen at the point they were trained. They don’t know of anything after that.

And LLMs have never experienced anything. They are just programs that have ingested unimaginable amounts of text. LLMs might do a great job at describing the sensation of being drunk, but this is only because they have read a lot of descriptions of being drunk. They have not, and cannot, experience it themselves. They have no purpose other than to produce the best response to the prompt you give them.

This doesn’t mean they aren’t impressive (they are) or that they can’t be useful (they are). And I truly believe we are at a watershed moment in technology. But let’s not confuse these genuine achievements with “true AI.” LLMs might be one ingredient in the recipe for true AI, but they are surely not the whole recipe — and I suspect we don’t yet know what some of the other ingredients are.

We’re just at the beginning of an AI revolution that will redefine how we live and work. In particular, deep neural networks (DNNs) have revolutionized the field of AI and are increasingly gaining prominence with the advent of foundation models and generative AI. But running these models on traditional digital computing architectures limits their achievable performance and energy efficiency. There has been progress in developing hardware specifically for AI inference, but many of these architectures physically split the memory and processing units. This means the AI models are typically stored in a discrete memory location, and computational tasks require constantly shuffling data between the memory and processing units. This process slows down computation and limits the maximum achievable energy efficiency.

Dunno if anyone has already posted this.

The chip showcases critical building blocks of a scalable mixed-signal architecture.

Venture capitalist a16z rebuilds a research paper with “AI Town” and releases the code. AI Town uses a language model to simulate a Sims-like virtual world in which all characters can flexibly pursue their motives and make decisions based on prompts.

In April, a team of researchers from Google and Stanford published the research paper Smallville. OpenAI’s GPT-3.5 simulates AI agents in a small digital town based solely on prompts.

Each character has an occupation, personality, and relationships with other characters, which are specified in an initial description. With further prompts, the AI agents begin to observe, plan, and make decisions.