The Allen Institute for AI created the Open Language Model, or OLMo, which is an open-source large language model with the aim of advancing the science of language models through open research.
AI2 has partnered with organizations such as Surge AI and MosaicML for data and training code. These partnerships are crucial for providing the diverse datasets and sophisticated training methodologies that underpin OLMo’s capabilities. The collaboration with the Paul G. Allen School of Computer Science and Engineering at the University of Washington and Databricks Inc. has also been pivotal in realizing the OLMo project.
It is important to note that the current architecture of OLMo is not the same as the models that power chatbots or AI assistants, which use instruction-based models. However, that’s on the roadmap. According to AI2, there will be multiple enhancements made to the model in the future. In the coming months, there are plans to iterate on OLMo by introducing different model sizes, modalities, datasets, and capabilities into the OLMo family. This iterative process is aimed at continuously improving the model’s performance and utility for the research community.
OLMo’s open and transparent approach, along with its advanced capabilities and commitment to continuous improvement, make it a major milestone in the evolution of LLMs.
