{"id":194542,"date":"2024-08-14T11:22:27","date_gmt":"2024-08-14T16:22:27","guid":{"rendered":"https:\/\/lifeboat.com\/blog\/2024\/08\/ai-study-reveals-dramatic-reasoning-breakdown-in-large-language-models"},"modified":"2024-08-14T11:22:27","modified_gmt":"2024-08-14T16:22:27","slug":"ai-study-reveals-dramatic-reasoning-breakdown-in-large-language-models","status":"publish","type":"post","link":"https:\/\/lifeboat.com\/blog\/2024\/08\/ai-study-reveals-dramatic-reasoning-breakdown-in-large-language-models","title":{"rendered":"AI Study reveals Dramatic Reasoning Breakdown in Large Language Models"},"content":{"rendered":"<p><a class=\"aligncenter blog-photo\" href=\"https:\/\/lifeboat.com\/blog.images\/ai-study-reveals-dramatic-reasoning-breakdown-in-large-language-models2.jpg\"><\/a><\/p>\n<p>Even the best AI large language models (LLMs) fail dramatically when it comes to simple logical questions. This is the conclusion of researchers from the J\u00fclich Supercomputing Center (JSC), the School of Electrical and Electronic Engineering at the University of Bristol and the LAION AI laboratory.<\/p>\n<p>In their paper posted to the arXiv preprint server, titled \u201cAlice in Wonderland: Simple Tasks Showing Complete Reasoning Breakdown in State-Of-the-Art Large Language Models,\u201d the scientists attest to a \u201cdramatic breakdown of function and reasoning capabilities\u201d in the tested state-of-the-art LLMs and suggest that although language models have the latent ability to perform basic reasoning, they cannot access it robustly and consistently.<\/p>\n<p>The authors of the study\u2014Marianna Nezhurina, Lucia Cipolina-Kun, Mehdi Cherti and Jenia Jitsev\u2014call on \u201cthe scientific and technological community to stimulate urgent re-assessment of the claimed capabilities of the current generation of LLMs.\u201d They also call for the development of standardized benchmarks to uncover weaknesses in language models related to basic reasoning capabilities, as current tests have apparently failed to reveal this serious failure.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Even the best AI large language models (LLMs) fail dramatically when it comes to simple logical questions. This is the conclusion of researchers from the J\u00fclich Supercomputing Center (JSC), the School of Electrical and Electronic Engineering at the University of Bristol and the LAION AI laboratory. In their paper posted to the arXiv preprint server, [\u2026]<\/p>\n","protected":false},"author":707,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6,44],"tags":[],"class_list":["post-194542","post","type-post","status-publish","format-standard","hentry","category-robotics-ai","category-supercomputing"],"_links":{"self":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/194542","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/users\/707"}],"replies":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/comments?post=194542"}],"version-history":[{"count":0,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/194542\/revisions"}],"wp:attachment":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/media?parent=194542"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/categories?post=194542"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/tags?post=194542"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}