Toggle light / dark theme

The 80,000 Hours Podcast features unusually in-depth conversations about the world’s most pressing problems and what you can do to solve them. Learn more, read the summary and find the full transcript on the 80,000 Hours website: https://80000hours.org/podcast/episodes/jan-leike-superalignment.

In July, OpenAI announced a new team and project: Superalignment. The goal is to figure out how to make superintelligent AI systems aligned and safe to use within four years, and the lab is putting a massive 20% of its computational resources behind the effort.

Today’s guest, Jan Leike, is Head of Alignment at OpenAI and will be co-leading the project. As OpenAI puts it, “…the vast power of superintelligence could be very dangerous, and lead to the disempowerment of humanity or even human extinction. … Currently, we don’t have a solution for steering or controlling a potentially superintelligent AI, and preventing it from going rogue.”

Given that OpenAI is in the business of developing superintelligent AI, it sees that as a scary problem that urgently has to be fixed. So it’s not just throwing compute at the problem — it’s also hiring dozens of scientists and engineers to build out the Superalignment team.

The media frenzy surrounding ChatGPT and other large, language model, artificial intelligence systems spans a range of themes, from the prosaic – large language models could replace conventional web search – to the concerning – AI will eliminate many jobs – and the overwrought – AI poses an extinction-level threat to humanity. All of these themes have a common denominator: large language models herald artificial intelligence that will supersede humanity.

But large language models, for all their complexity, are actually really dumb. And despite the name “artificial intelligence,” they’re completely dependent on human knowledge and labor. They can’t reliably generate new knowledge, of course, but there’s more to it than that.

ChatGPT can’t learn, improve or even stay up to date without humans giving it new content and telling it how to interpret that content, not to mention programming the model and building, maintaining and powering its hardware. To understand why, you first have to understand how ChatGPT and similar models work, and the role humans play in making them work.

Elon Musk delves into the groundbreaking potential of Neuralink, a revolutionary venture aimed at interfacing with the human brain to tackle an array of brain-related disorders. Musk envisions a future where Neuralink’s advancements lead to the resolution of conditions like autism, schizophrenia, memory loss, and even spinal cord injuries.

Elon Musk discusses the transformative power of Neuralink, highlighting its role in restoring motor control after spinal cord injuries, revitalizing brain function post-stroke, and combating genetically or trauma-induced brain diseases. Musk’s compelling insights reveal how interfacing with neurons at an intricate level can pave the way for repairing and enhancing brain circuits using cutting-edge technology.

Discover the three-layer framework Musk envisions: the primary layer akin to the limbic system, the more intelligent cortex as the secondary layer, and the potential tertiary layer where digital superintelligence might exist. Musk’s thought-provoking perspective raises optimism about the coexistence of a digital superintelligence with the human brain, fostering a harmonious relationship between these layers of consciousness.

Elon Musk emphasises the urgency of Neuralink’s mission, stressing the importance of developing a human brain interface before the advent of digital superintelligence and the elusive singularity. By doing so, he believes we can mitigate existential risks and ensure a stable future for humanity and consciousness as we navigate the uncharted territories of technological evolution.

New research indicates that Australia and New Zealand are the two best places on Earth to survive a nuclear war. The recently published set of calculations don’t focus on blast-related deaths or even deaths caused by radiation fall-out, which most estimates say would number in the hundreds of millions, but instead look at how a nuclear winter caused by nuclear bomb explosions would affect food supplies, potentially leading to the starvation of billions.

Nuclear War Simulations Performed For Decades

Since the first atomic bombs were dropped on the Japanese cities of Hiroshima and Nagasaki in 1945, effectively spelling the end of World War II, war game theorists have looked at a myriad of simulations to determine the potential effects of a full-blown nuclear battle. Many simulations look at the potentially hundreds of millions that would likely die in the initial blasts, while others have tried to model the slower but equally as deadly body count from radiation sickness.

On this episode, Daniel Schmachtenberger returns to discuss a surprisingly overlooked risk to our global systems and planetary stability: artificial intelligence. Through a systems perspective, Daniel and Nate piece together the biophysical history that has led humans to this point, heading towards (and beyond) numerous planetary boundaries and facing geopolitical risks all with existential consequences. How does artificial intelligence, not only add to these risks, but accelerate the entire dynamic of the metacrisis? What is the role of intelligence vs wisdom on our current global pathway, and can we change course? Does artificial intelligence have a role to play in creating a more stable system or will it be the tipping point that drives our current one out of control?

About Daniel Schmachtenberger:
Daniel Schmachtenberger is a founding member of The Consilience Project, aimed at improving public sensemaking and dialogue.

The throughline of his interests has to do with ways of improving the health and development of individuals and society, with a virtuous relationship between the two as a goal.

Towards these ends, he’s had particular interest in the topics of catastrophic and existential risk, civilization and institutional decay and collapse as well as progress, collective action problems, social organization theories, and the relevant domains in philosophy and science.

Hypnotized LLMs can help leak confidential financial information, generate malicious code and even cross red lights.

Tech pundits worldwide have been fluctuating between marking artificial intelligence as the end of all of humanity and calling it the most significant thing humans have ever touched since the internet.

We are in a phase where we are unsure what the AI Pandora’s box will reveal. Are we heading for doomsday or utopia?

There’s no shortage of AI doomsday scenarios to go around, so here’s another AI expert who pretty bluntly forecasts that the technology will spell the death of us all, as reported by Bloomberg.

This time, it’s not a so-called godfather of AI sounding the alarm bell — or that other AI godfather (is there a committee that decides these things?) — but a controversial AI theorist and provocateur known as Eliezer Yudkowsky, who has previously called for bombing machine learning data centers. So, pretty in character.

“I think we’re not ready, I think we don’t know what we’re doing, and I think we’re all going to die,” Yudkowsky said on an episode of the Bloomberg series “AI IRL.”

An asteroid discovery algorithm—designed to uncover near-Earth asteroids for the Vera C. Rubin Observatory’s upcoming 10-year survey of the night sky—has identified its first “potentially hazardous” asteroid, a term for space rocks in Earth’s vicinity that scientists like to keep an eye on.

The roughly 600-foot-long asteroid, designated 2022 SF289, was discovered during a test drive of the algorithm with the ATLAS survey in Hawaii. Finding 2022 SF289, which poses no risk to Earth for the foreseeable future, confirms that the next-generation algorithm, known as HelioLinc3D, can identify near-Earth asteroids with fewer and more dispersed observations than required by today’s methods.

“By demonstrating the real-world effectiveness of the software that Rubin will use to look for thousands of yet-unknown potentially hazardous asteroids, the discovery of 2022 SF289 makes us all safer,” said Rubin scientist Ari Heinze, the principal developer of HelioLinc3D and a researcher at the University of Washington.