{"id":201255,"date":"2024-12-11T10:29:03","date_gmt":"2024-12-11T16:29:03","guid":{"rendered":"https:\/\/lifeboat.com\/blog\/2024\/12\/leaner-large-language-models-could-enable-efficient-local-use-on-phones-and-laptops"},"modified":"2024-12-11T10:29:03","modified_gmt":"2024-12-11T16:29:03","slug":"leaner-large-language-models-could-enable-efficient-local-use-on-phones-and-laptops","status":"publish","type":"post","link":"https:\/\/lifeboat.com\/blog\/2024\/12\/leaner-large-language-models-could-enable-efficient-local-use-on-phones-and-laptops","title":{"rendered":"Leaner Large Language Models could enable Efficient Local Use on Phones and Laptops"},"content":{"rendered":"<p><a class=\"aligncenter blog-photo\" href=\"https:\/\/lifeboat.com\/blog.images\/leaner-large-language-models-could-enable-efficient-local-use-on-phones-and-laptops2.jpg\"><\/a><\/p>\n<p>Large language models (LLMs) are increasingly automating tasks like translation, text classification and customer service. But tapping into an LLM\u2019s power typically requires users to send their requests to a centralized server\u2014a process that\u2019s expensive, energy-intensive and often slow.<\/p>\n<p>Now, researchers have introduced a technique for compressing an LLM\u2019s reams of data, which could increase privacy, save energy and lower costs. Their findings are published on the arXiv preprint server.<\/p>\n<p>The new algorithm, developed by engineers at Princeton and Stanford Engineering, works by trimming redundancies and reducing the precision of an LLM\u2019s layers of information. This type of leaner LLM could be stored and accessed locally on a device like a phone or laptop and could provide performance nearly as accurate and nuanced as an uncompressed version.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Large language models (LLMs) are increasingly automating tasks like translation, text classification and customer service. But tapping into an LLM\u2019s power typically requires users to send their requests to a centralized server\u2014a process that\u2019s expensive, energy-intensive and often slow. Now, researchers have introduced a technique for compressing an LLM\u2019s reams of data, which could increase [\u2026]<\/p>\n","protected":false},"author":707,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1523,38,41,1512],"tags":[],"class_list":["post-201255","post","type-post","status-publish","format-standard","hentry","category-computing","category-engineering","category-information-science","category-mobile-phones"],"_links":{"self":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/201255","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/users\/707"}],"replies":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/comments?post=201255"}],"version-history":[{"count":0,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/201255\/revisions"}],"wp:attachment":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/media?parent=201255"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/categories?post=201255"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/tags?post=201255"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}