First thoughts and preliminary insights into OpenAI’s GPT o1 Strawberry in the medical domain, with some expected and unexpected findings. We have a \.
Our final estimate of the achievable inter data center bandwidth by 2030 is 4 to 20 Pbps, which would allow for training runs of 3e29 to 2e31 FLOP. In light of this, bandwidth is unlikely to be a major constraint for a distributed training run compared to achieving the necessary power supply in the first place.
Expanding bandwidth capacity for distributed training networks presents a relatively straightforward engineering challenge, achievable through the deployment of additional fiber pairs between data centers. In the context of AI training runs potentially costing hundreds of billions of dollars, the financial investment required for such bandwidth expansion appears comparatively modest.44
We conclude that training runs in 2030 supported by a local power supply could likely involve 1 to 5 GW and reach 1e28 to 3e29 FLOP by 2030. Meanwhile, geographically distributed training runs could amass a supply of 2 to 45 GW and achieve 4 to 20 Pbps connections between data center pairs, allowing for training runs of 2e28 to 2e30 FLOP.45 All in all, it seems likely that training runs between 2e28 to 2e30 FLOP will be possible by 2030.46 The assumptions behind these estimates can be found in Figure 3 below.
I found this a bit disturbing.
The Committee on Energy and Commerce is the oldest standing legislative committee in the U.S. House of Representatives and is vested with the broadest jurisdiction of any congressional authorizing committee.
And could entire civilizations seek to leave this reality behind?
Watch my exclusive video Exploring The Multiverse: https://nebula.tv/videos/isaacarthur–…
Get Nebula using my link for 40% off an annual subscription: https://go.nebula.tv/isaacarthur.
Get a Lifetime Membership to Nebula for only $300: https://go.nebula.tv/lifetime?ref=isa…
Use the link gift.nebula.tv/isaacarthur to give a year of Nebula to a friend for just $30.
Check out https://www.johnmsmart.com to learn more about Transcension Hypothesis.
Visit our Website: http://www.isaacarthur.net.
Enjoy the videos and music you love, upload original content, and share it all with friends, family, and the world on YouTube.
From AI-generated video advancements to new AI-powered AR glasses, here were the 5 biggest announcements.
The breakthrough AI needs
Posted in innovation, robotics/AI
Only competition between countries will prevent progress from being stopped.
Beijing opted out of a pact to keep humans in control of decision-making on nuclear weapons, a move observers say underlines its dilemma.
Year 2023 face_with_colon_three
Mice overexpressing Has2 from the naked mole-rat showed an increase in hyaluronan levels in several tissues, and a lower incidence of spontaneous and induced cancer, attenuated inflammation through several pathways, extended lifespan and improved healthspan.
A research team led by Prof. Ye Hong from the University of Science and Technology of China has developed an alumina ceramic bionic wick with finger-like pores inspired by the stomatal array of natural leaves. Their research is published in Langmuir.
As the performance of electronic chips continues to improve, their power consumption also increases, posing new challenges for cooling strategies. Loop heat pipes (LHPs) are a compelling cooling solution due to their high heat transfer capability, antigravity heat transfer, and absence of moving parts.
However, the differing requirements for flow resistance and capillary force make designing the pore structure of the capillary wick within an LHP challenging. Specifically, larger pores are needed for gaseous working fluids to reduce flow resistance, while smaller pores are necessary to provide sufficient capillary force for liquid suction.