{"id":186683,"date":"2024-04-03T08:24:33","date_gmt":"2024-04-03T13:24:33","guid":{"rendered":"https:\/\/lifeboat.com\/blog\/2024\/04\/apple-researchers-develop-ai-that-can-see-and-understand-screen-context"},"modified":"2024-04-03T08:24:33","modified_gmt":"2024-04-03T13:24:33","slug":"apple-researchers-develop-ai-that-can-see-and-understand-screen-context","status":"publish","type":"post","link":"https:\/\/lifeboat.com\/blog\/2024\/04\/apple-researchers-develop-ai-that-can-see-and-understand-screen-context","title":{"rendered":"Apple researchers develop AI that can \u2018see\u2019 and understand screen context"},"content":{"rendered":"<p><a class=\"aligncenter blog-photo\" href=\"https:\/\/lifeboat.com\/blog.images\/apple-researchers-develop-ai-that-can-see-and-understand-screen-context.jpg\"><\/a><\/p>\n<p>Apple researchers have developed a new artificial intelligence system that can understand ambiguous references to on-screen entities as well as conversational and background context, enabling more natural interactions with voice assistants, according to a paper published on Friday.<\/p>\n<p>The system, called <a href=\"https:\/\/arxiv.org\/pdf\/2403.20329.pdf\">ReALM (Reference Resolution As Language Modeling)<\/a>, leverages large language models to convert the complex task of reference resolution \u2014 including understanding references to visual elements on a screen \u2014 into a pure language modeling problem. This allows ReALM to achieve substantial performance gains compared to existing methods.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Apple researchers have developed a new artificial intelligence system that can understand ambiguous references to on-screen entities as well as conversational and background context, enabling more natural interactions with voice assistants, according to a paper published on Friday. The system, called ReALM (Reference Resolution As Language Modeling), leverages large language models to convert the complex [\u2026]<\/p>\n","protected":false},"author":556,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[],"class_list":["post-186683","post","type-post","status-publish","format-standard","hentry","category-robotics-ai"],"_links":{"self":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/186683","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/users\/556"}],"replies":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/comments?post=186683"}],"version-history":[{"count":0,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/186683\/revisions"}],"wp:attachment":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/media?parent=186683"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/categories?post=186683"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/tags?post=186683"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}