{"id":196532,"date":"2024-09-22T03:24:53","date_gmt":"2024-09-22T08:24:53","guid":{"rendered":"https:\/\/lifeboat.com\/blog\/2024\/09\/can-ai-scaling-continue-through-2030"},"modified":"2024-09-22T03:24:53","modified_gmt":"2024-09-22T08:24:53","slug":"can-ai-scaling-continue-through-2030","status":"publish","type":"post","link":"https:\/\/lifeboat.com\/blog\/2024\/09\/can-ai-scaling-continue-through-2030","title":{"rendered":"Can AI Scaling Continue Through 2030?"},"content":{"rendered":"<p><a class=\"aligncenter blog-photo\" href=\"https:\/\/lifeboat.com\/blog.images\/can-ai-scaling-continue-through-20303.jpg\"><\/a><\/p>\n<p>Our final estimate of the achievable inter data center bandwidth by 2030 is <strong>4 to 20 Pbps<\/strong>, which would allow for training runs of <strong>3e29 to 2e31 FLOP<\/strong>. In light of this, bandwidth is unlikely to be a major constraint for a distributed training run compared to achieving the necessary power supply in the first place.<\/p>\n<p>Expanding bandwidth capacity for distributed training networks presents a relatively straightforward engineering challenge, achievable through the deployment of additional fiber pairs between data centers. In the context of AI training runs potentially costing hundreds of billions of dollars, the financial investment required for such bandwidth expansion appears comparatively modest.<sup id= fnref:44><a href=\"https:\/\/epochai.org\/blog\/can-ai-scaling-continue-through-2030#fn:44\" class=\"\" rel=\"footnote\">44<\/a><\/sup><\/p>\n<p>We conclude that training runs in 2030 supported by a local power supply could likely involve <strong>1 to 5 GW<\/strong> and reach <strong>1e28 to 3e29<\/strong> <strong>FLOP<\/strong> by 2030. Meanwhile, geographically distributed training runs could amass a supply of <strong>2 to 45 GW<\/strong> and achieve <strong>4 to 20 Pbps<\/strong> connections between data center pairs, allowing for training runs of <strong>2e28 to 2e30 FLOP<\/strong>.<sup id= fnref:45><a href=\"https:\/\/epochai.org\/blog\/can-ai-scaling-continue-through-2030#fn:45\" class=\"\" rel=\"footnote\">45<\/a><\/sup> All in all, it seems likely that training runs between <strong>2e28 to 2e30 FLOP<\/strong> will be possible by 2030.<sup id= fnref:46><a href=\"https:\/\/epochai.org\/blog\/can-ai-scaling-continue-through-2030#fn:46\" class=\"\" rel=\"footnote\">46<\/a><\/sup> The assumptions behind these estimates can be found in Figure 3 below.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Our final estimate of the achievable inter data center bandwidth by 2030 is 4 to 20 Pbps, which would allow for training runs of 3e29 to 2e31 FLOP. In light of this, bandwidth is unlikely to be a major constraint for a distributed training run compared to achieving the necessary power supply in the first [\u2026]<\/p>\n","protected":false},"author":709,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[45,6],"tags":[],"class_list":["post-196532","post","type-post","status-publish","format-standard","hentry","category-finance","category-robotics-ai"],"_links":{"self":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/196532","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/users\/709"}],"replies":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/comments?post=196532"}],"version-history":[{"count":0,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/196532\/revisions"}],"wp:attachment":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/media?parent=196532"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/categories?post=196532"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/tags?post=196532"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}