{"id":128971,"date":"2021-10-14T09:22:59","date_gmt":"2021-10-14T16:22:59","guid":{"rendered":"https:\/\/lifeboat.com\/blog\/2021\/10\/microsofts-massive-new-language-ai-is-triple-the-size-of-openais-gpt-3"},"modified":"2021-10-14T09:22:59","modified_gmt":"2021-10-14T16:22:59","slug":"microsofts-massive-new-language-ai-is-triple-the-size-of-openais-gpt-3","status":"publish","type":"post","link":"https:\/\/lifeboat.com\/blog\/2021\/10\/microsofts-massive-new-language-ai-is-triple-the-size-of-openais-gpt-3","title":{"rendered":"Microsoft\u2019s Massive New Language AI Is Triple the Size of OpenAI\u2019s GPT-3"},"content":{"rendered":"<p><a class=\"aligncenter blog-photo\" href=\"https:\/\/lifeboat.com\/blog.images\/microsofts-massive-new-language-ai-is-triple-the-size-of-openais-gpt-32.jpg\"><\/a><\/p>\n<p>\ud83d\ude03<\/p>\n<hr>\n<p>Microsoft\u2019s <a href=\"https:\/\/www.microsoft.com\/en-us\/research\/blog\/using-deepspeed-and-megatron-to-train-megatron-turing-nlg-530b-the-worlds-largest-and-most-powerful-generative-language-model\/\">blog post<\/a> on Megatron-Turing says the algorithm is skilled at tasks like completion prediction, reading comprehension, commonsense reasoning, natural language inferences, and word sense disambiguation. But stay tuned\u2014there will likely be more skills added to that list once the model starts being widely utilized.<\/p>\n<p>GPT-3 turned out to have capabilities beyond what its creators anticipated, like writing code, doing math, translating between languages, and autocompleting images (oh, and writing a <a href=\"https:\/\/singularityhub.com\/2020\/10\/23\/an-ai-wrote-this-short-film-and-its-sort-of-fascinating\/\">short film<\/a> with a twist ending). This led some to <a href=\"https:\/\/venturebeat.com\/2020\/09\/03\/were-entering-the-ai-twilight-zone-between-narrow-and-general-ai\/\">speculate<\/a> that GPT-3 might be the gateway to <a href=\"https:\/\/singularityhub.com\/2018\/07\/22\/from-here-to-human-level-artificial-general-intelligence-in-four-not-all-that-simple-steps\/\">artificial general intelligence<\/a>. But the algorithm\u2019s variety of talents, while unexpected, still fell within the language domain (including programming languages), so that\u2019s a bit of a stretch.<\/p>\n<p>However, given the tricks GPT-3 had up its sleeve based on its 175 billion parameters, it\u2019s intriguing to wonder what the Megatron-Turing model may surprise us with at 530 billion. The algorithm likely won\u2019t be commercially available for some time, so it\u2019ll be a while before we find out.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\ud83d\ude03 Microsoft\u2019s blog post on Megatron-Turing says the algorithm is skilled at tasks like completion prediction, reading comprehension, commonsense reasoning, natural language inferences, and word sense disambiguation. But stay tuned\u2014there will likely be more skills added to that list once the model starts being widely utilized. GPT-3 turned out to have capabilities beyond what its [\u2026]<\/p>\n","protected":false},"author":513,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[41,2229,6],"tags":[],"class_list":["post-128971","post","type-post","status-publish","format-standard","hentry","category-information-science","category-mathematics","category-robotics-ai"],"_links":{"self":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/128971","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/users\/513"}],"replies":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/comments?post=128971"}],"version-history":[{"count":0,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/128971\/revisions"}],"wp:attachment":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/media?parent=128971"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/categories?post=128971"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/tags?post=128971"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}