{"id":160828,"date":"2023-03-22T22:25:41","date_gmt":"2023-03-23T03:25:41","guid":{"rendered":"https:\/\/lifeboat.com\/blog\/2023\/03\/new-ligo-technique-accelerates-training-of-large-machine-learning-models"},"modified":"2023-03-22T22:25:41","modified_gmt":"2023-03-23T03:25:41","slug":"new-ligo-technique-accelerates-training-of-large-machine-learning-models","status":"publish","type":"post","link":"https:\/\/lifeboat.com\/blog\/2023\/03\/new-ligo-technique-accelerates-training-of-large-machine-learning-models","title":{"rendered":"New LiGO technique accelerates training of large machine-learning models"},"content":{"rendered":"<p><a class=\"aligncenter blog-photo\" href=\"https:\/\/lifeboat.com\/blog.images\/new-ligo-technique-accelerates-training-of-large-machine-learning-models2.jpg\"><\/a><\/p>\n<p>It\u2019s no secret that OpenAI\u2019s ChatGPT has some incredible capabilities\u2014for instance, the chatbot can write poetry that resembles Shakespearean sonnets or debug code for a computer program. These abilities are made possible by the massive machine-learning model that ChatGPT is built upon. Researchers have found that when these types of models become large enough, extraordinary capabilities emerge.<\/p>\n<p>But bigger models also require more time and money to train. The training process involves showing hundreds of billions of examples to a model. Gathering so much data is an involved process in itself. Then come the monetary and <a href=\"https:\/\/techxplore.com\/tags\/environmental+costs\/\" rel=\"tag\" class=\"\">environmental costs<\/a> of running many powerful computers for days or weeks to train a model that may have billions of parameters.<\/p>\n<p>\u201cIt\u2019s been estimated that training models at the scale of what ChatGPT is hypothesized to run on could take millions of dollars, just for a single training run. Can we improve the efficiency of these training methods, so we can still get good models in less time and for less money? We propose to do this by leveraging smaller language models that have previously been trained,\u201d says Yoon Kim, an assistant professor in MIT\u2019s Department of Electrical Engineering and Computer Science and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL).<\/p>\n","protected":false},"excerpt":{"rendered":"<p>It\u2019s no secret that OpenAI\u2019s ChatGPT has some incredible capabilities\u2014for instance, the chatbot can write poetry that resembles Shakespearean sonnets or debug code for a computer program. These abilities are made possible by the massive machine-learning model that ChatGPT is built upon. Researchers have found that when these types of models become large enough, extraordinary [\u2026]<\/p>\n","protected":false},"author":662,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[],"class_list":["post-160828","post","type-post","status-publish","format-standard","hentry","category-robotics-ai"],"_links":{"self":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/160828","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/users\/662"}],"replies":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/comments?post=160828"}],"version-history":[{"count":0,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/160828\/revisions"}],"wp:attachment":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/media?parent=160828"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/categories?post=160828"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/tags?post=160828"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}