New Google Model Ranked ‘No. 1 LLM’, But There’s a Problem

A new and mysterious Gemini model appears at the top of the leaderboard, but is that the full story? I dig behind the headline to show you some anti-climactic results, give some context with leaks in the last 48 hours of diminishing returns to scaling, and add the response of Altman, OpenAI and co. The future is about to look a lot stranger.

80,000 hours Podcast + Channel:

You can now gift memberships to AI Insiders (my Patreon w/ exclusive vids, network):

Chapters:
00:00 – Introduction
01:25 – LM Leaderboard
02:35 – Benchmarks and Leaks
05:31 – Low EQ
07:37 – Other labs have issues too though
10:31 – OpenAI claim and counter-claim
14:13 – Other news

‘There is no wall’:

Gemini Ranking:
API not yet up:
‘Just Die Chat’:
Google CEO tweet:
Sutskever Quote:
Another OpenAI Staffer Leaves:
Bloomberg Report:
Noam Brown on what OpenAI Researchers Believe:
Clive Chan:
Chollet Responds to Altman:

Altman Emails:
Change of Heart:
Amodei on ‘Empirical Regularities’:
Verge Report:
OpenAI Agents in January:

The 8 Most Controversial Terms in AI:

Non-hype Newsletter:

Podcast:

I use Descript to edit my videos: