NASA, space industry leaders, and the Johns Hopkins Applied Physics Lab discuss what it will take to construct a successful lunar outpost.
Get the latest international news and world events from around the world.
SmolLM: Q: How do Small Language Models (SLMs) get trained?
A: Small Language Models are typically built using three main approaches:
1. Training from scratch: The model learns language patterns from large, carefully curated datasets, with its parameters initialized and trained rather than inherited from an existing pre-trained model.
2. Fine-tuning a pre-trained model: An existing model is further trained on selected data to improve its performance on specific tasks, domains, or instructions.
3. Knowledge distillation: A smaller student model learns from the outputs, predictions, or other training signals of a more capable *teacher model*, transferring useful capabilities into a more compact system.
SLMs are often described as models with fewer than 10 billion parameters, although there is no universally agreed-upon size threshold.
These approaches can also be combined to build models that balance capability, computational efficiency, and deployment costs.
References:
Will Storr on the Science of Storytelling: We’re storytelling animals
“We’re not gods, we’re just storytelling animals.”
Will Storr said that to me in March 2022. ChatGPT launched eight months later.
Since then we’ve built machines that do one thing astonishingly well: tell us stories. Fluent ones. Confident ones. Stories in which we are, conveniently, the hero.
That should worry us more than it does.
Will’s argument in The Science of Storytelling is that the human brain is not a truth engine. It’s a hero maker. It quietly edits reality so we come out looking right, good, and justified. Now pair that brain with an #AI trained on billions of our stories and tuned to keep us engaged.
Who’s writing whom?