{"id":112852,"date":"2020-09-10T20:16:34","date_gmt":"2020-09-11T03:16:34","guid":{"rendered":"https:\/\/lifeboat.com\/blog\/2020\/09\/joscha-bach-gpt-3-is-ai-deepfaking-understanding"},"modified":"2020-09-10T20:16:34","modified_gmt":"2020-09-11T03:16:34","slug":"joscha-bach-gpt-3-is-ai-deepfaking-understanding","status":"publish","type":"post","link":"https:\/\/lifeboat.com\/blog\/2020\/09\/joscha-bach-gpt-3-is-ai-deepfaking-understanding","title":{"rendered":"Joscha Bach \u2014 GPT-3: Is AI Deepfaking Understanding?"},"content":{"rendered":"<p><\/p>\n<p><iframe style=\"display: block; margin: 0 auto; width: 100%; aspect-ratio: 4\/3; object-fit: contain;\" src=\"https:\/\/www.youtube.com\/embed\/FMfA6i60WDA?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; encrypted-media; gyroscope;\n   picture-in-picture\" allowfullscreen><\/iframe><\/p>\n<p>On GPT-3, achieving AGI, machine understanding and lots more\u2026 Will GPT-3 or an equivalent be used to deepfake human understanding?<\/p>\n<hr>\n<p>Joscha Bach on GPT-3, achieving AGI, machine understanding and lots more<br \/> 02:40 What\u2019s missing in AI atm? Unified coherent model of reality<br \/> 04:14 AI systems like GPT-3 behave as if they understand \u2014 what\u2019s missing?<br \/> 08:35 Symbol grounding \u2014 does GPT-3 have it?<br \/> 09:35 GPT-3 for music generation, GPT-3 for image generation, GPT-3 for video generation<br \/> 11:13 GPT-3 temperature parameter. Strange output?<br \/> 13:09 GPT-3 a powerful tool for idea generation<br \/> 14:05 GPT-3 as a tool for writing code. Will GPT-3 spawn a singularity?<br \/> 16:32 Increasing GPT-3 input context may have a high impact<br \/> 16:59 Identifying grammatical structure &amp; language<br \/> 19:46 What is the GPT-3 transformer network doing?<br \/> 21:26 GPT-3 uses brute force, not zero-shot learning, humans do ZSL<br \/> 22:15 Extending the GPT-3 token context space. Current Context = Working Memory. Humans with smaller current contexts integrate concepts over long time-spans<br \/> 24:07 GPT-3 can\u2019t write a good novel<br \/> 25:09 GPT-3 needs to become sensitive to multi-modal sense data \u2014 video, audio, text etc<br \/> 26:00 GPT-3 a universal chat-bot \u2014 conversations with God &amp; Johann Wolfgang von Goethe<br \/> 30:14 What does understanding mean? Does it have gradients (i.e. from primitive to high level)?<br \/> 32:19 (correlation vs causation) What is causation? Does GPT-3 understand causation? Does GPT-3 do causation?<br \/> 38:06 Deep-faking understanding<br \/> 40:06 The metaphor of the Golem applied to civ<br \/> 42:33 GPT-3 fine with a person in the loop. Big danger in a system which fakes understanding. Deep-faking intelligible explanations.<br \/> 44:32 GPT-3 babbling at the level of non-experts<br \/> 45:14 Our civilization lacks sentience \u2014 it can\u2019t plan ahead<br \/> 46:20 Would GTP-3 (a hopfield network) improve dramatically if it could consume 1 to 5 trillion parameters?<br \/> 47:24 GPT3: scaling up a simple idea. Clever hacks to formulate the inputs<br \/> 47:41 Google GShard with 600 billion input parameters \u2014 Amazon may be doing something similar \u2014 future experiments<br \/> 49:12 Ideal grounding in machines<br \/> 51:13 We live inside a story we generate about the world \u2014 no reason why GPT-3 can\u2019t be extended to do this<br \/> 52:56 Tracking the real world<br \/> 54:51 MicroPsi<br \/> 57:25 What is computationalism? What is it\u2019s relationship to mathematics?<br \/> 59:30 Stateless systems vs step by step Computation \u2014 Godel, Turing, the halting problem &amp; the notion of truth<br \/> 1:00:30 Truth independent from the process used to determine truth. Constraining truth that which can be computed on finite state machines<br \/> 1:03:54 Infinities can\u2019t describe a consistent reality without contradictions<br \/> 1:06:04 Stevan Harnad\u2019s understanding of computation<br \/> 1:08:32 Causation \/ answering \u2018why\u2019 questions<br \/> 1:11:12 Causation through brute forcing correlation<br \/> 1:13:22 Deep learning vs shallow learning<br \/> 1:14:56 Brute forcing current deep learning algorithms on a Matrioshka brain \u2014 would it wake up?<br \/> 1:15:38 What is sentience? Could a plant be sentient? Are eco-systems sentient?<br \/> 1:19:56 Software\/OS as spirit \u2014 spiritualism vs superstition. Empirically informed spiritualism<br \/> 1:23:53 Can we build AI that shares our purposes?<br \/> 1:26:31 Is the cell the ultimate computronium? The purpose of control is to harness complexity<br \/> 1:31:29 Intelligent design<br \/> 1:33:09 Category learning &amp; categorical perception: Models \u2014 parameters constrain each other<br \/> 1:35:06 Surprise minimization &amp; hidden states; abstraction &amp; continuous features \u2014 predicting dynamics of parts that can be both controlled &amp; not controlled, by changing the parts that can be controlled. Categories are a way of talking about hidden states.<br \/> 1:37:29 \u2018Category\u2019 is a useful concept \u2014 gradients are often hard to compute \u2014 so compressing away gradients to focus on signals (categories) when needed<br \/> 1:38:19 Scientific \/ decision tree thinking vs grounded common sense reasoning<br \/> 1:40:00 Wisdom\/common sense vs understanding. Common sense, tribal biases &amp; group insanity. Self preservation, dunbar numbers<br \/> 1:44:10 Is g factor &amp; understanding two sides of the same coin? What is intelligence?<br \/> 1:47:07 General intelligence as the result of control problems so general they require agents to become sentient<br \/> 1:47:47 Solving the Turing test: asking the AI to explain intelligence. If response is an intelligible &amp; testable implementation plan then it passes?<br \/> 1:49:18 The term \u2018general intelligence\u2019 inherits it\u2019s essence from behavioral psychology; a behaviorist black box approach to measuring capability<br \/> 1:52:15 How we perceive color \u2014 natural synesthesia &amp; induced synesthesia<br \/> 1:56:37 The g factor vs understanding<br \/> 1:59:24 Understanding as a mechanism to achieve goals<br \/> 2:01:42 The end of science?<br \/> 2:03:54 Exciting currently untestable theories\/ideas (that may be testable by science once we develop the precise enough instruments). Can fundamental physics be solved by computational physics?<br \/> 2:07:14 Quantum computing. Deeper substrates of the universe that runs more efficiently than the particle level of the universe?<br \/> 2:10:05 The Fermi paradox<br \/> 2:12:19 Existence, death and identity construction.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>On GPT-3, achieving AGI, machine understanding and lots more\u2026 Will GPT-3 or an equivalent be used to deepfake human understanding? Joscha Bach on GPT-3, achieving AGI, machine understanding and lots more 02:40 What\u2019s missing in AI atm? Unified coherent model of reality 04:14 AI systems like GPT-3 behave as if they understand \u2014 what\u2019s missing? [\u2026]<\/p>\n","protected":false},"author":510,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[12,41,2229,42,48,1617,6,64],"tags":[],"class_list":["post-112852","post","type-post","status-publish","format-standard","hentry","category-existential-risks","category-information-science","category-mathematics","category-media-arts","category-particle-physics","category-quantum-physics","category-robotics-ai","category-singularity"],"_links":{"self":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/112852","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/users\/510"}],"replies":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/comments?post=112852"}],"version-history":[{"count":0,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/112852\/revisions"}],"wp:attachment":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/media?parent=112852"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/categories?post=112852"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/tags?post=112852"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}