{"id":149664,"date":"2022-11-07T23:23:09","date_gmt":"2022-11-08T05:23:09","guid":{"rendered":"https:\/\/lifeboat.com\/blog\/2022\/11\/why-neural-networks-can-learn-almost-anything"},"modified":"2022-11-07T23:23:09","modified_gmt":"2022-11-08T05:23:09","slug":"why-neural-networks-can-learn-almost-anything","status":"publish","type":"post","link":"https:\/\/lifeboat.com\/blog\/2022\/11\/why-neural-networks-can-learn-almost-anything","title":{"rendered":"Why Neural Networks can learn (almost) anything"},"content":{"rendered":"<p><\/p>\n<p><iframe style=\"display: block; margin: 0 auto; width: 100%; aspect-ratio: 4\/3; object-fit: contain;\" src=\"https:\/\/www.youtube.com\/embed\/0QczhVg5HaI?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; encrypted-media; gyroscope;\n   picture-in-picture\" allowfullscreen><\/iframe><\/p>\n<p>A video about neural networks, how they work, and why they\u2019re useful.<\/p>\n<p>My twitter: <a href=\"https:\/\/twitter.com\/max_romana\">https:\/\/twitter.com\/max_romana<\/a>.<\/p>\n<p>SOURCES<br \/> Neural network playground: <a href=\"https:\/\/playground.tensorflow.org\/\">https:\/\/playground.tensorflow.org\/<\/a><\/p>\n<p>Universal Function Approximation:<br \/> Proof: <a href=\"https:\/\/cognitivemedium.com\/magic_paper\/assets\/Hornik.pdf\">https:\/\/cognitivemedium.com\/magic_paper\/assets\/Hornik.pdf<\/a>.<br \/> Covering ReLUs: <a href=\"https:\/\/proceedings.neurips.cc\/paper\/2017\/hash\/32cbf687880eb1674a07bf717761dd3a-Abstract.html\">https:\/\/proceedings.neurips.cc\/paper\/2017\/hash\/32cbf687880eb\u2026tract.html<\/a>.<br \/> Covering discontinuous functions: <a href=\"https:\/\/arxiv.org\/pdf\/2012.03016.pdf\">https:\/\/arxiv.org\/pdf\/2012.03016.pdf<\/a>.<\/p>\n<p>Turing Completeness:<br \/> Networks of infinite size are turing complete: Neural Computability I &amp; II (behind a paywall unfourtunately, but is cited in following paper)<br \/> RNNs are turing complete: <a href=\"https:\/\/binds.cs.umass.edu\/papers\/1992_Siegelmann_COLT.pdf\">https:\/\/binds.cs.umass.edu\/papers\/1992_Siegelmann_COLT.pdf<\/a>.<br \/> Transformers are turing complete: <a href=\"https:\/\/arxiv.org\/abs\/2103\">https:\/\/arxiv.org\/abs\/2103<\/a>.<\/p>\n<p>More on backpropagation:<\/p>\n<div class=\"more-link-wrapper\"> <a class=\"more-link\" href=\"https:\/\/lifeboat.com\/blog\/2022\/11\/why-neural-networks-can-learn-almost-anything\">Continue reading \u201cWhy Neural Networks can learn (almost) anything\u201d | &gt;<\/a><\/div>\n","protected":false},"excerpt":{"rendered":"<p>A video about neural networks, how they work, and why they\u2019re useful. My twitter: https:\/\/twitter.com\/max_romana. SOURCES Neural network playground: https:\/\/playground.tensorflow.org\/ Universal Function Approximation: Proof: https:\/\/cognitivemedium.com\/magic_paper\/assets\/Hornik.pdf. Covering ReLUs: https:\/\/proceedings.neurips.cc\/paper\/2017\/hash\/32cbf687880eb\u2026tract.html. Covering discontinuous functions: https:\/\/arxiv.org\/pdf\/2012.03016.pdf. Turing Completeness: Networks of infinite size are turing complete: Neural Computability I &amp; II (behind a paywall unfourtunately, but is cited in following [\u2026]<\/p>\n","protected":false},"author":556,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[],"class_list":["post-149664","post","type-post","status-publish","format-standard","hentry","category-robotics-ai"],"_links":{"self":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/149664","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/users\/556"}],"replies":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/comments?post=149664"}],"version-history":[{"count":0,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/149664\/revisions"}],"wp:attachment":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/media?parent=149664"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/categories?post=149664"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/tags?post=149664"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}