$125B for Superintelligence? 3 Models Coming, Sutskever’s Secret SSI, & Data Centers (in space)…
Ilya Sutskever's 'straight shot to superintelligence' is already valued at $5B, but now we get $125B data centers in the works. Yes, plural. Will this be the ultimate gambit on the scaling hypothesis?
Weights and Biases’ Weave: wandb.me/ai_explained
And yes, the title is not clickbait, a company is pledging to build data centers in space, but that follows failed attempts in the sea. Plus, distributed training, Gemini 2, Grok-3, Colossus, CharacterAI, Orion, and … chapters.
AI Insiders:
Chapters:
00:00 – Intro
01:06 – SSI, Safe Superintelligence (Sutskever)
03:45 – Grok-3 (Colossus) + Altman Concerned
05:36 – CharacterAI + Foundation Models
06:26 – $125B Supercomputers + 5-10GW
08:28 – ‘GPT-6’ Scale
09:07 – Zuckerberg on Exponentials and Doubt
09:42 – Strawberry/Orion + Connections + Weights
11:39 – Data Centers in Space (and the sea)
12:45 – Distributed Training + SemiAnalysis Report w/ Gemini 2
17:34 – Climate Change Pledges?
Weights and Biases’ Weave: wandb.me/ai_explained
Safe Superintelligence (Sutskever SSI):
$125B Data Centers:
Altman ‘Too Aggressive’:
OpenAI Orion:
Semianalysis Report:
Xai Colossus:
Altman Reacts:
GPT-6 Co-location:
Data Centers in Space:
And Underwater:
Zuckerberg on Power and Exponentials:
Epoch AIB Report:
Original SuperAlignment Deadline:
Character AI Bought:
My New Coursera Course! The 8 Most Controversial Terms in AI:
Non-hype Newsletter:
GenAI Hourly Consulting:
Weights and Biases’ Weave: wandb.me/ai_explained
cant wait for ASDI (artificial super duper intelligence)
lmao
I’m personally a proponent of AWMI instead (artificial “woah, mama!” intelligence)
I don’t think so ahsihasusuauau. The term ASI and AGI comes to determine something like GPU, CPU. It is certainly not a very lucid term xD
all the while we’ve had ASD for a long time
comments like this make me smile
I feel the AGI.
Me too Ilya, me too
I would ask for permission before feeling the ASI, this ain’t a petting zoo
😂
Take that fiber optic cable out of your mouth!
You’ll be working for an AGI in 2 years.
they found a “this one weird trick can make you a billionaire” video
Be one of the top experts in a rapidly growing and capital-intensive field? Easy
@@GolerGkA Yeah I could do that but I’m already making good money with dropshiping, so whatever
These valuations are totally nuts. I also wonder how many % Ilya has. It has to be in the very low single digits. Maybe even less than 1%
Assuming a pre-money valuation of $5bn the investors got 16.6%. 3 founders with Ilya being the pulling power he’s prob got 15%+
@@edoa That would be absolutely nuts. $750 million net worth on paper out of no where? It does not really make sense to me. Is he really that much better than anyone else?
@@WigglyloveThe Wozniak of AI
@@Wigglyloveyeah. He’s a powerhouse of AI
He was the best student of Godfather of AI and worth the bet @@Wigglylove
Let’s just hope, if scale really is the key, it allows us to find a way to scale a reasoning model down to manageable levels.
If anything, we need something reasonable to turn to in times like these.
A datacenter in space would appear to have multiple issues, not the least of which is maintenance. Even with the advent of SpaceX, it’s not exactly cheap to send all the parts into orbit. Then, even though they say “passive cooling”, how are you going to reject a significant percentage of a gigawatt’s worth of heat? The ISS already has to use a huge radiator to reject a much smaller amount (maybe 1/1000th?) of heat into space.
It’s a very, very stupid idea and will remain such for a long time.
AGI will solve that, I can feel it
Cooling how? The vacuum of space is the perfect heat insulator.,
And they are such a tempting missile target…
Jokes apart, we live in an era where access to low Earth orbit is about to get much cheaper, especially if projects like SpaceX’s Starship start working.
This just makes me appreciate how special our brains are.
It’s kinda crazy. How chatGPT for example has vast and vast amounts of knowledge and an insane generality compared to your random 90 IQ bloke… But fails at decently easy tasks.
Ironically, this should also make us realize how inefficient our biological intelligence might be.
A truly advanced artificial intelligence should be able to achieve human-level cognition with just a tiny fraction of these resources.
Naaah, look at it, neurons giving themselves a standing ovation. Time for the humble ones and zeros to take over the applause.
I of course agree. But I found it fascinating in a recent talk that Demis Hasabis of Google Deep Mind suggested that the brain is no longer a driver, marker, map of where to go with A.I to make AGI. He just says its an engineering problem now and well understood. That was wild to hear because I specifically remember Demis saying some years ago that studying the brain was the secret to finding out how to make intelligent machines and I believe he studied that subject deeply himself in his younger years.
Brains kinda brute force the intelligence by truly massive scaling. Their advantage is they’re not static, they’re self modifying structures unlike current static LLMs. But they are slow to learn, slow to communicate with others, have tiny operating memory, short attention span, lossy memory and need to rest often.
Computers can do all of these better once we figure out the correct system architecture. Which I believe should be hybrid just like our brains are composed of parts that have different functionality, specialization. Basically add a knowledge pool, reasoning center, math coprocessor, introspective thoughts, creative subsystem and so on. Then we’ll have truly superior AI.
Distributed training is honestly more difficult than building your own power plant for the data center.
Well, wouldn’t similar problems arise when doing training across multiple machines within the same data center anyway? There would just be added latency.
Surely, an algorithm that could synchronize the many different nodes across networks has already been developed.
Maybe im not understanding the crux of the issues, just sounds like something that’s been solved for years.
Hahaha right on. I’m sure nuclear is wide scale adoption next decade. Has to be..
Datacenters in space are stupid, the cooling is such a giant problem even for normal satellites. The visualization conveniently does not show any radiators, which is funny, as one would need an absolute ton of them. also that design is a micrometeorite magnet. Also for good latency one would have to be in a low orbit, and that entails either an orbital decay in at most 20-30 years (or less) for a 400km (or less) orbit, or BOTH an exorbitant fuel cost AND a very high risk of failure due to orbital derbies
TLDR: That is stupid on so many levels i do not think an Orbital Enginieer saw this before the publishing of the promotional video
“BUT IT LOOKS SO GOOD AND REAL IN THE PROMO!!!!11!!1111!!!11!!!!!1!”
I think that the people involved in these projects re much MUCH smarter than you or me and they wouldnt have overlooked these things.
Just to note I rely on and appreciate you for an honest perspective on this field.
There are so many channels out there that try to tell everyone about something shocking or stunning happening in AI on an almost daily basis. It is nice to have a channel with information you can trust to be informative, accurate and absent of any hyperbole.
The first thing that an AGI/ASI should focus on is making a more efficient version of itself. The human mind runs on 20W. I see no reason why we shouldn’t be able to get an AGI to run on < 1000 W.
This is Turing Police. Stay where you are, we dispatching agents to your place.
That will eventually come. We might need the brute force step to acquire helpful tools which will actually provide a much more intelligently designed model. And then it snowballs towards singularity right there.
yes – either there are much more efficient algorithms, or we need to use organic processors. “we have a brain orbiting the planet” would be awesome
And the AGI/ASI will realise it is wasteful to grow its own organic brain when there are already 8 billion fully grown brains on the planet. 😂 At least we won’t become mere batteries when plugged into the ASI’s Matrix
Natural selection, though a blunt, unintelligent instrument, has had hundreds of millions of years to optimize the brain. It will be hard for us to top that.
Fascinating. Biggest stakes being played right now, and 99.9% of the population have no idea whatsoever is happening.
This gets a lot of media coverage; I highly doubt 99.9% of the world’s population doesn’t know what’s going on.
@@SergiusXVII what do you think is going on?
@@SergiusXVII I think he mean that most people have no idea what this rapid AI evolution can affect us. I think its a valid statement, even we who have more understanding of this area, are struggling to translate this to real futur image.
@SergiusXVII legitimately nobody knows what ssi is at the moment. It’s less than 99.9%
@@SergiusXVII I don’t think the majority of people believe that SI – hopefully SSI – will ever happen. I think that’s what he meant.
The production quality of these videos is astounding.
Aw thanks man
What’s crazier is the scaling like this almost certainly won’t unlock AGI and yet this kind of money is being poured into these projects. It says more about human psychology and the desire for ‘AGI’ than anything else.
@@snarkyboojum You don’t know it won’t scale and their are constant advances in architecture anyway. The idea that you know better than the people running all the top tech companies and that the insane amount of money, attention and effort being focused on improving AI won’t pay off and all these companies are just “scaling” and crossing their fingers is absurd.
@@jyjjy7 I happen to work at one of those ‘top tech companies’, and have degrees in physics and computer science, so I have a fairly educated view. Of course these architectures scale, but scaling is extremely unlikely to unlock ‘superintelligence’. If you think otherwise, I’d encourage you to read more.
@@snarkyboojum As long as scaling creates new more powerful architectures and so on…
@@bienspasser9054 No. Scaling might lead to new architectures but not necessarily on the path to super intelligence. Blindly relying on scaling and some magical causal connection to “new architectures” giving you AGI is honestly like just shooting in the dark and hoping to hit something.
or maybe it shows that some people (like a do) don’t expect more than mediocrity from biological entities. This bother the humanocentrists.
SSI is so funny. Imagine if Apple in the 80s were like “we’re not releasing a single product until we have created a fully functional mobile device with Internet, video chat, apps, Face ID etc. etc..
Lmao, good point.
Lmao that’s because you think it’s the 80s, but Ilya thinks it’s 2012.
I think folks who believe that “more scale is all you need” are in for a “bitter lesson” of their own. Because of the strong faith in the bitter lesson, many ideas get dismissed as incremental work because any benefit would become irrelevant in six months. When that stops being true, I don’t think it will be a bubble popping so much as a renaissance for alternative design concepts. There’s got to be more we can do than just decoder-only models that rely on BPE, right?
Most of the progress this year has been on efficiency.
If getting the data center problem solved is so hard, it could make sense to have one team working on that while another team works in efficiency. Then when you have the new system up you can use the massive gain in compute with the massive gains in efficiency and go even further.
Kind of like Moore’s law petering finally let flourish alternates like die assembling and specialized processors.
The whole idea of SI being a product is funny
I watched a recent interview with the co-founder of DeepMind, Shane Legg, and he didn’t even mention LLMs as the path to AGI. He said DeepMind was working on other architectures to get them there. He maintains DeepMind’s 2030 timeline.
He also pointed out that the famous alien-seeming “Move 31” in AlphaGo’s game against Lee Sedol was not made by an LLM.
Why wouldn’t we watch all the way to the end. No one else comes close to what you do. Fact checked, objective, intelligent, dilligent analysis, hitting the very most salient points.
0:55 “If they’re wrong, this [the $125 billion spent on data centers] could all be viewed as the biggest waste of resources in human history.”
Gee, I dunno—the $3 trillion spent on the US war in Iraq (as estimated by the Harvard Kennedy School) seems like it’s larger, if we assume that money spent has _some_ relationship to resources used. (The $125 billion still _could be_ a big waste of resources, though.)
Good call
My wife and I are currently working on an AGI, should be ready and fully trained in 18-25 yrs. Current budgeting indicates it’ll take far less than 125 billion dollars.
Fr?
Is it really artificial, though? Me and my wife have shipped three GIs and they are of a higher quality than current AGI attempts honestly. Fraction of the budget too.
couple goals
But can these AGIs make me a hip-hop song written and sung by Spongebob Squarepants in less than 2 mins?
@@abdvs325 Well that technology is already here, We can do that now.