{"id":193220,"date":"2024-07-21T00:23:19","date_gmt":"2024-07-21T05:23:19","guid":{"rendered":"https:\/\/lifeboat.com\/blog\/2024\/07\/cognition-devin-was-evaluated-on-a-random-25-subset-of-the-dataset"},"modified":"2024-07-21T00:23:19","modified_gmt":"2024-07-21T05:23:19","slug":"cognition-devin-was-evaluated-on-a-random-25-subset-of-the-dataset","status":"publish","type":"post","link":"https:\/\/lifeboat.com\/blog\/2024\/07\/cognition-devin-was-evaluated-on-a-random-25-subset-of-the-dataset","title":{"rendered":"Cognition: Devin was evaluated on a random 25% subset of the dataset"},"content":{"rendered":"<p><a class=\"aligncenter blog-photo\" href=\"https:\/\/lifeboat.com\/blog.images\/cognition2.jpg\"><\/a><\/p>\n<p>Devin was unassisted, whereas all other models were assisted (meaning the model was told exactly which files need to be edited).<\/p>\n<p>We plan to publish a more detailed technical report soon\u2014stay tuned for more details.<\/p>\n<p>We are an applied AI lab focused on reasoning. \u200d We\u2019re building AI teammates with capabilities far beyond today\u2019s existing AI tools. By solving reasoning, we can unlock new possibilities in a wide range of disciplines\u2014code is just the beginning. We want to help people around the world turn their ideas into reality.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Devin was unassisted, whereas all other models were assisted (meaning the model was told exactly which files need to be edited). We plan to publish a more detailed technical report soon\u2014stay tuned for more details. We are an applied AI lab focused on reasoning. \u200d We\u2019re building AI teammates with capabilities far beyond today\u2019s existing [\u2026]<\/p>\n","protected":false},"author":513,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[],"class_list":["post-193220","post","type-post","status-publish","format-standard","hentry","category-robotics-ai"],"_links":{"self":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/193220","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/users\/513"}],"replies":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/comments?post=193220"}],"version-history":[{"count":0,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/193220\/revisions"}],"wp:attachment":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/media?parent=193220"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/categories?post=193220"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/tags?post=193220"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}