{"id":178162,"date":"2023-12-12T13:27:39","date_gmt":"2023-12-12T19:27:39","guid":{"rendered":"https:\/\/lifeboat.com\/blog\/2023\/12\/phi-2-the-surprising-power-of-small-language-models"},"modified":"2023-12-12T13:27:39","modified_gmt":"2023-12-12T19:27:39","slug":"phi-2-the-surprising-power-of-small-language-models","status":"publish","type":"post","link":"https:\/\/lifeboat.com\/blog\/2023\/12\/phi-2-the-surprising-power-of-small-language-models","title":{"rendered":"Phi-2: The surprising power of small language models"},"content":{"rendered":"

<\/a><\/p>\n

Microsoft research releases Phi-2 and promptbase.<\/p>\n

Phi-2 outperforms other existing small language models, yet it\u2019s small enough to run on a laptop or mobile device.<\/p>\n

\n

Over the past few months, our Machine Learning Foundations team at Microsoft Research has released a suite of small language models (SLMs) called \u201cPhi\u201d that achieve remarkable performance on a variety of benchmarks. Our first model, the 1.3 billion parameter Phi-1<\/strong> (opens in new tab)<\/a>, achieved state-of-the-art performance on Python coding among existing SLMs (specifically on the HumanEval and MBPP benchmarks). We then extended our focus to common sense reasoning and language understanding and created a new 1.3 billion parameter model named Phi-1.5<\/strong> (opens in new tab)<\/a>, with performance comparable to models 5x larger.<\/p>\n

We are now releasing Phi-2<\/strong> (opens in new tab)<\/a>, a 2.7 billion-parameter language model that demonstrates outstanding reasoning and language understanding capabilities, showcasing state-of-the-art performance among base language models with less than 13 billion parameters. On complex benchmarks Phi-2 matches or outperforms models up to 25x larger, thanks to new innovations in model scaling and training data curation.<\/p>\n

With its compact size, Phi-2 is an ideal playground for researchers, including for exploration around mechanistic interpretability, safety improvements, or fine-tuning experimentation on a variety of tasks. We have made Phi-2<\/strong> (opens in new tab)<\/a> <\/strong>available in the Azure AI Studio model catalog to foster research and development on language models.<\/p>\n","protected":false},"excerpt":{"rendered":"

Microsoft research releases Phi-2 and promptbase. Phi-2 outperforms other existing small language models, yet it\u2019s small enough to run on a laptop or mobile device. Over the past few months, our Machine Learning Foundations team at Microsoft Research has released a suite of small language models (SLMs) called \u201cPhi\u201d that achieve remarkable performance on a [\u2026]<\/p>\n","protected":false},"author":709,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1522,6],"tags":[],"class_list":["post-178162","post","type-post","status-publish","format-standard","hentry","category-innovation","category-robotics-ai"],"_links":{"self":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/178162","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/users\/709"}],"replies":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/comments?post=178162"}],"version-history":[{"count":0,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/178162\/revisions"}],"wp:attachment":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/media?parent=178162"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/categories?post=178162"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/tags?post=178162"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}