\u201cHumans are ~1,000,000x more sample efficient than LLMs, and there are no good explanations.\u201d<\/p>\n
\u201cWe\u2019ve found that a big part of the answer is that LLMs are *massively* underparameterized by many orders of magnitude. The brain has 100T synapses learning from ~200M tokens of language. Chinchilla prescribes a 1T-parameter model on 20T tokens. That\u2019s 7 OOMs apart in per token.\u201d<\/p>\n
\u201cAfter a lot of painstaking work, we present new data-constrained scaling laws and learning algorithms to scale to (extrapolated) quadrillion-parameter transformers.\u201d<\/p>\n
The current consensus in frontier pre-training is clear: high-quality human text is finite, but compute scaling is not slowing down. The key question for AI leaders is: How do we scale compute when data is constrained?<\/p>\n
A breakthrough paper from Q Labs (*Baron et al., Oct 2026\u2022<\/span> provides a rigorous roadmap: Quadrillion-parameter architectures, aggressive weight decay, and first-class ensembling.<\/p>\n Here is the breakdown of the operational shift every AI research and infrastructure team should track:<\/p>\n 1. The Parameter-to-Token Shift.<\/p>\n