What is the limiting factor for AI scaling?

"those same GPUs ended up being very useful, huge boost to AI"

"if you start going like two, three years out, it's all about total how many gigawatt hours of battery are produced, that will be the limiting factor."

"Dojo also has the potential to become a sellable service"

"Dojo, which we believe has the potential for an order of magnitude improvement in the cost of training."

"A lot of AI is data curation. It's like shocking. It's not like vast numbers of lines of code."

"My prediction is that we will go from an extreme silicon shortage today to probably voltage transformer shortage in about a year and then an electricity shortage a year in two years."

"the metric that will be most important in a few years is useful compute per unit of energy."

"The rate of change of the ratio of digital to biological compute is hyper exponential"

"It's pretty hard to put together 100 gigawatts of AI compute."

"Given the pace of technology improvement, it’s not worth sinking 1GW of power into H100s. The @xAI 100k H100 liquid-cooled training cluster will be online in a few months. Next big step would probably be ~300k B200s with CX8 networking next summer."

"Nice work by @xAI team, @X team, @Nvidia & supporting companies getting Memphis Supercluster training started at ~4:20am local time. With 100k liquid-cooled H100s on a single RDMA fabric, it’s the most powerful AI training cluster in the world!"

"I'm quite concerned about actually being able to get state of the art Nvidia GPUs when we want them."

"we are going to double down on Dojo and we do see a path to being competitive with Nvidia"

"This weekend, the @xAI team brought our Colossus 100k H100 training cluster online. From start to finish, it was done in 122 days. Colossus is the most powerful AI training system in the world. Moreover, it will double in size to 200k (50k H200s) in a few months. Excellent work by the team, Nvidia and our many partners/suppliers."

"the training needs for Optimus, for Optimus humanoid robot are probably at least ultimately 10x what's needed for the car"

"Colossus 2 will be the first Gigawatt AI training supercluster"

"230k GPUs, including 30k GB200s, are operational for training Grok @xAI in a single supercluster called Colossus 1 (inference is done by our cloud providers). At Colossus 2, the first batch of 550k GB200s & GB300s, also for training, start going online in a few weeks. As Jensen Huang has stated, @xAI is unmatched in speed. It’s not even close."

"Colossus 2, built by @xAI, will be the world’s first Gigawatt+ AI training supercomputer"

"say for argument sake like 10x more compute will double the intelligence"

"Colossus II, the world’s first Gigawatt AI training cluster"

"1.21 Gigawatts of training compute! (Actually, slightly more)"

"Just as we will be the first to bring a Gigawatt of coherent training compute online, we will also be the first to 10GW, 100GW, 1TW, …"

"five years to get to blind production. I'm like five years to me is infinity"

"xAI will have more AI compute than everyone else combined in <5 years"

"xAI has bought a third building called MACROHARDRR. Will take @xAI training compute to almost 2GW. Try @Grok. Download latest app."

"limiting factor for um AI deployment is fundamentally electrical power"

"it's basically electricity generation and cooling um are limiting factors for AI"

"The Colossus 2 supercomputer for @Grok is now operational. First Gigawatt training cluster in the world. Upgrades to 1.5GW in April."

"In the one year timeframe, it's energy"

"the three things on you need are mass to orbit, lot of solar power, and and radiators"

"power constraints are going to be, they really are a major issue for AI"

"there will be at least a, a 15 gigawatt shortfall of power in 2027 for AI chips"