{"id":113615,"date":"2020-09-30T14:23:18","date_gmt":"2020-09-30T21:23:18","guid":{"rendered":"https:\/\/lifeboat.com\/blog\/2020\/09\/this-ai-generates-photos-using-only-text-captions-as-a-guide"},"modified":"2020-09-30T14:23:18","modified_gmt":"2020-09-30T21:23:18","slug":"this-ai-generates-photos-using-only-text-captions-as-a-guide","status":"publish","type":"post","link":"https:\/\/lifeboat.com\/blog\/2020\/09\/this-ai-generates-photos-using-only-text-captions-as-a-guide","title":{"rendered":"This AI Generates Photos Using Only Text Captions as a Guide"},"content":{"rendered":"

<\/a><\/p>\n

Researchers at the Allen Institute for Artificial Intelligence<\/a> (AI2) have created a machine learning algorithm that can produce images using only text captions as its guide. The results are somewhat terrifying\u2026 but if you can look past the nightmare fuel, this creation represents an important step forward in the study of AI and imaging.<\/p>\n

Unlike some of the genuinely mind-blowing machine learning algorithms we\u2019ve shared in the past\u2014see here<\/a>, here<\/a>, and here <\/a>\u2014this creation is more of a proof-of-concept experiment. The idea was to take a well-established computer vision model that can caption photos based on what it \u201csees\u201d in the image, and reverse it: producing an AI that can generate images from captions, instead of the other way around.<\/p>\n

This is a fascinating area of study and, as MIT Technology Review<\/em><\/a> points out, it shows in real terms how limited these computer vision algorithms really are. While even a small child can do both of these things readily\u2014describe an image in words, or conjure a mental picture of an image based on those words\u2014when the Allen Institute researchers tried to generate a photo from a text caption using a model called LXMERT, it generated nonsense in return.<\/p>\n","protected":false},"excerpt":{"rendered":"

Researchers at the Allen Institute for Artificial Intelligence (AI2) have created a machine learning algorithm that can produce images using only text captions as its guide. The results are somewhat terrifying\u2026 but if you can look past the nightmare fuel, this creation represents an important step forward in the study of AI and imaging. Unlike [\u2026]<\/p>\n","protected":false},"author":396,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[41,6],"tags":[],"class_list":["post-113615","post","type-post","status-publish","format-standard","hentry","category-information-science","category-robotics-ai"],"_links":{"self":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/113615","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/users\/396"}],"replies":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/comments?post=113615"}],"version-history":[{"count":0,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/113615\/revisions"}],"wp:attachment":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/media?parent=113615"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/categories?post=113615"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/tags?post=113615"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}