{"id":197223,"date":"2024-10-08T09:31:06","date_gmt":"2024-10-08T14:31:06","guid":{"rendered":"https:\/\/lifeboat.com\/blog\/2024\/10\/ai-challenge-seeks-questions-to-test-human-level-intelligence"},"modified":"2024-10-08T09:31:06","modified_gmt":"2024-10-08T14:31:06","slug":"ai-challenge-seeks-questions-to-test-human-level-intelligence","status":"publish","type":"post","link":"https:\/\/lifeboat.com\/blog\/2024\/10\/ai-challenge-seeks-questions-to-test-human-level-intelligence","title":{"rendered":"AI challenge seeks questions to test human-level intelligence"},"content":{"rendered":"<p><a class=\"aligncenter blog-photo\" href=\"https:\/\/lifeboat.com\/blog.images\/ai-challenge-seeks-questions-to-test-human-level-intelligence3.jpg\"><\/a><\/p>\n<p>Two of San Francisco\u2019s leading players in artificial intelligence <a href=\"https:\/\/scale.com\/blog\/humanitys-last-exam\" target=\"_blank\">have challenged<\/a> the public to come up with questions capable of testing the capabilities of large language models (LLMs) like Google Gemini and OpenAI\u2019s o1. Scale AI, which specializes in preparing the vast tracts of data on which the LLMs are trained, teamed up with the Center for AI Safety (CAIS) to launch the initiative, Humanity\u2019s Last Exam.<\/p>\n<p>Featuring prizes of US$5,000 (\u00a33,800) for those who come up with the top 50 questions selected for the test, Scale and CAIS say the goal is to test how close we are to achieving \u201cexpert-level AI systems\u201d using the \u201clargest, broadest coalition of experts in history.\u201d<\/p>\n<p>Why do this? The leading LLMs are already acing many established tests in intelligence, <a href=\"https:\/\/www.nature.com\/articles\/s41586-023-06747-5\" target=\"_blank\">mathematics<\/a> and <a href=\"https:\/\/law.stanford.edu\/2023\/04\/19\/gpt-4-passes-the-bar-exam-what-that-means-for-artificial-intelligence-tools-in-the-legal-industry\/\" target=\"_blank\">law<\/a>, but it\u2019s hard to be sure how meaningful this is. In many cases, they may have pre-learned the answers due to the gargantuan quantities of data on which they are trained, including a significant percentage of everything on the internet.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Two of San Francisco\u2019s leading players in artificial intelligence have challenged the public to come up with questions capable of testing the capabilities of large language models (LLMs) like Google Gemini and OpenAI\u2019s o1. Scale AI, which specializes in preparing the vast tracts of data on which the LLMs are trained, teamed up with the [\u2026]<\/p>\n","protected":false},"author":718,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1496,2229,6],"tags":[],"class_list":["post-197223","post","type-post","status-publish","format-standard","hentry","category-law","category-mathematics","category-robotics-ai"],"_links":{"self":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/197223","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/users\/718"}],"replies":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/comments?post=197223"}],"version-history":[{"count":0,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/197223\/revisions"}],"wp:attachment":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/media?parent=197223"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/categories?post=197223"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/tags?post=197223"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}