Toggle light / dark theme

View a PDF of the paper titled When Models Manipulate Manifolds: The Geometry of a Counting Task, by Wes Gurnee and 6 other authors

When you look at text, you subconsciously track how much space remains on each line. If you’re writing “Happy Birthday” and “Birthday” won’t fit, your brain automatically moves it to the next line. You don’t calculate this—you *see* it. But AI models don’t have eyes. They receive only sequences of numbers (tokens) and must somehow develop a sense of visual space from scratch.

Inside your brain, “place cells” help you navigate physical space by firing when you’re in specific locations. Remarkably, Claude develops something strikingly similar. The researchers found that the model represents character counts using low-dimensional curved manifolds—mathematical shapes that are discretized by sparse feature families, much like how biological place cells divide space into discrete firing zones.

The researchers validated their findings through causal interventions—essentially “knocking out” specific neurons to see if the model’s counting ability broke in predictable ways. They even discovered visual illusions—carefully crafted character sequences that trick the model’s counting mechanism, much like optical illusions fool human vision.

2. Attention mechanisms are geometric engines: The “attention heads” that power modern AI don’t just connect related words—they perform sophisticated geometric transformations on internal representations.

1. What other “sensory” capabilities have models developed implicitly? Can AI develop senses we don’t have names for?


Language models can perceive visual properties of text despite receiving only sequences of tokens-we mechanistically investigate how Claude 3.5 Haiku accomplishes one such task: linebreaking in fixed-width text. We find that character counts are represented on low-dimensional curved manifolds discretized by sparse feature families, analogous to biological place cells. Accurate predictions emerge from a sequence of geometric transformations: token lengths are accumulated into character count manifolds, attention heads twist these manifolds to estimate distance to the line boundary, and the decision to break the line is enabled by arranging estimates orthogonally to create a linear decision boundary. We validate our findings through causal interventions and discover visual illusions—character sequences that hijack the counting mechanism.

How scientists are trying to use AI to unlock the human mind

Compared with conventional psychological models, which use simple math equations, Centaur did a far better job of predicting behavior. Accurate predictions of how humans respond in psychology experiments are valuable in and of themselves: For example, scientists could use Centaur to pilot their experiments on a computer before recruiting, and paying, human participants. In their paper, however, the researchers propose that Centaur could be more than just a prediction machine. By interrogating the mechanisms that allow Centaur to effectively replicate human behavior, they argue, scientists could develop new theories about the inner workings of the mind.

But some psychologists doubt whether Centaur can tell us much about the mind at all. Sure, it’s better than conventional psychological models at predicting how humans behave—but it also has a billion times more parameters. And just because a model behaves like a human on the outside doesn’t mean that it functions like one on the inside. Olivia Guest, an assistant professor of computational cognitive science at Radboud University in the Netherlands, compares Centaur to a calculator, which can effectively predict the response a math whiz will give when asked to add two numbers. “I don’t know what you would learn about human addition by studying a calculator,” she says.

Even if Centaur does capture something important about human psychology, scientists may struggle to extract any insight from the model’s millions of neurons. Though AI researchers are working hard to figure out how large language models work, they’ve barely managed to crack open the black box. Understanding an enormous neural-network model of the human mind may not prove much easier than understanding the thing itself.

Can Physics Use Inconsistent Mathematics?

Discussion with logician Graham Priest on the existence of true contradictions in reality.

Please consider financially supporting my work (ONLY if you have the means).

Patreon: https://patreon.com/trsam?utm_medium=unknown&utm_source=join…t=copyLink.
Substack: https://rsampod.substack.com.
PayPal: https://paypal.me/rsampod?country.x=AU&locale.x=en_AU

{Podcast}
Substack: https://rsampod.substack.com/podcast.
Spotify: https://open.spotify.com/show/4ryEqjut4r6SMtfxLdM1Le.
SFP: https://podcasters.spotify.com/pod/show/rahul-samaranayake.

{Website}

What Ultimately Is There? Metaphysics and the Ruliad

Stephen Wolfram shares surprising new ideas and results from a scientific approach to metaphysics. Discusses time, spacetime, computational irreducibility, significance of the observer, quantum mechanics and multiway systems, ruliad, laws of nature, objective reality, existence, mathematical reality.

Long-Sought Proof Tames Some of Math’s Unruliest Equations

The trajectory of a storm, the evolution of stock prices, the spread of disease — mathematicians can describe any phenomenon that changes in time or space using what are known as partial differential equations. But there’s a problem: These “PDEs” are often so complicated that it’s impossible to solve them directly.

Mathematicians instead rely on a clever workaround. They might not know how to compute the exact solution to a given equation, but they can try to show that this solution must be “regular,” or well-behaved in a certain sense — that its values won’t suddenly jump in a physically impossible way, for instance. If a solution is regular, mathematicians can use a variety of tools to approximate it, gaining a better understanding of the phenomenon they want to study.

But many of the PDEs that describe realistic situations have remained out of reach. Mathematicians haven’t been able to show that their solutions are regular. In particular, some of these out-of-reach equations belong to a special class of PDEs that researchers spent a century developing a theory of — a theory that no one could get to work for this one subclass. They’d hit a wall.

A ‘crazy’ dice proof leads to a new understanding of a fundamental law of physics

Right now, molecules in the air are moving around you in chaotic and unpredictable ways. To make sense of such systems, physicists use a law known as the Boltzmann distribution, which, rather than describe exactly where each particle is, describes the chance of finding the system in any of its possible states. This allows them to make predictions about the whole system even though the individual particle motions are random. It’s like rolling a single die: Any one roll is unpredictable, but if you keep rolling it again and again, a pattern of probabilities will emerge.

Developed in the latter half of the 19th century by Ludwig Boltzmann, an Austrian physicist and mathematician, this Boltzmann distribution is used widely today to model systems in many fields, ranging from AI to economics, where it is called “multinomial logit.”

Now, economists have taken a deeper look at this universal law and come up with a surprising result: The Boltzmann distribution, their mathematical proof shows, is the only law that accurately describes unrelated, or uncoupled, systems.

Joscha Bach presents “Machine Consciousness and Beyond” | dAGI Summit 2025

Bach reframes AI as the endpoint of a long philosophical project to “naturalize the mind,” arguing that modern machine learning operationalizes a lineage from Aristotle to Turing in which minds, worlds, and representations are computational state-transition systems. He claims computer science effectively re-discovers animism—software as self-organizing, energ†y-harvesting “spirits”—and that consciousness is a simple coherence-maximizing operator required for self-organizing agents rather than a metaphysical mystery. Current LLMs only simulate phenomenology using deepfaked human texts, but the universality of learning systems suggests that, when trained on the right structures, artificial models could converge toward the same internal causal patterns that give rise to consciousness. Bach proposes a biological-to-machine consciousness framework and a research program (CIMC) to formalize, test, and potentially reproduce such mechanisms, arguing that understanding consciousness is essential for culture, ethics, and future coexistence with artificial minds.

Key takeaways.

▸ Speaker & lens: Cognitive scientist and AI theorist aiming to unify philosophy of mind, computer science, and modern ML into a single computationalist worldview.
▸ AI as philosophical project: Modern AI fulfills the ancient ambition to map mind into mathematics; computation provides the only consistent language for modeling reality and experience.
▸ Computationalist functionalism: Objects = state-transition functions; representations = executable models; syntax = semantics in constructive systems.
▸ Cyber-animism: Software as “spirits”—self-organizing, adaptive control processes; living systems differ from dead ones by the software they run.
▸ Consciousness as function: A coherence-maximizing operator that integrates mental states; second-order perception that stabilizes working memory; emerges early in development as a prerequisite for learning.
▸ LLMs & phenomenology: Current models aren’t conscious; they simulate discourse about consciousness using data full of “deepfaked” phenomenology. A Turing test cannot detect consciousness because performance ≠ mechanism.
▸ Universality hypothesis: Different architectures optimized for the same task tend to converge on similar internal causal structures; suggests that consciousness-like organization could arise if it’s the simplest solution to coherence and control.
▸ Philosophical zombies: Behaviorally identical but non-conscious agents may be more complex than conscious ones; evolution chooses simplicity → consciousness may be the minimal solution for self-organized intelligence.
▸ Language vs embodiment: Language may contain enough statistical structure to reconstruct much of reality; embodiment may not be strictly necessary for convergent world models.
▸ Testing for machine consciousness: Requires specifying phenomenology, function, search space, and success criteria—not performance metrics.
▸ CIMC agenda: Build frameworks and experiments to recreate consciousness-like operators in machines; explore implications for ethics, interfaces, and coexistence with future minds.

AI systems could identify math anxiety from student inputs and change feedback

Math anxiety is a significant challenge for students worldwide. While personalized support is widely recognized as the most effective way to address it, many teachers struggle to deliver this level of support at scale within busy classrooms. New research from Adelaide University shows how artificial intelligence (AI) could help address challenges such as math anxiety by using a student’s inputs and identifying signs of anxiety or disengagement during learning.

Published in npj Science of Learning, the study suggests that when AI systems are designed to use the right data and goals, they can adapt their responses to help counteract negative emotional experiences associated with math, before these feelings escalate.

Lead researcher Dr. Florence Gabriel says AI has the potential to transform how math anxiety is supported, by offering timely, tailored interventions that step through learning and build student well-being.

Using duality to construct and classify new quantum phases

A team of theoretical researchers has found duality can unveil non-invertible symmetry protected topological phases, which can lead to researchers understanding more about the properties of these phases, and uncover new quantum phases. Their study is published in Physical Review Letters.

Symmetry is one of the most fundamental concepts for understanding phases of matter in modern physics—in particular, symmetry-protected topological (SPT) phases, whose quantum mechanical properties are protected by symmetries, with possible applications in quantum computing and other fields.

Over the past few years, non-invertible symmetries, which extend the framework of conventional symmetries, have attracted significant attention in high energy physics and condensed matter physics. However, their complex mathematical structures have made it difficult to understand their corresponding phases of matter, or SPT phases.

/* */