How Playing Games Is Making AI Smarter

TL;DR: Today we’re releasing a new episode of our podcast AI & I. Dan Shipper sits down with Alex Duffy, our head of AI training, to discuss the spin-out of Alex’s new company, Good Start Labs, from Every. Watch on X or YouTube, or listen on Spotify or Apple Podcasts. Here’s a link to the episode transcript.

Was this newsletter forwarded to you? Sign up to get it in your inbox.

This episode of AI & I is a little different from our usual fare: It’s a conversation with our head of AI training Alex Duffy about Good Start Labs, a company he incubated inside Every.

Today, Good Start Labs is spinning out of Every as a separate company with $3.6 million in funding from General Catalyst, Inovia, Every, and a group of angel investors from top-tier AI labs like DeepMind.

Dan Shipper and Alex get into how Alex learned some of his biggest lessons about the real world from games, starting with RuneScape, which taught him how markets work and how not to get scammed. He explains why the static benchmarks we use to evaluate LLMs today are breaking down, and how games like Diplomacy offer a richer, more dynamic way to test and train large language models. Finally, Alex shares where he sees the most promise in AI—software, life sciences, and education—and why he believes games can make the models we use smarter, while helping people understand and use AI more effectively. Here’s a link to the episode transcript.

You can check out their full conversation here:

Here’s a link to the episode transcript.

Timestamps

1. Introduction: 00:01:48
2. Why evals and benchmarks are broken: 00:04:14
3. The sneakiest LLMs in the market: 00:07:13
4. A competition that turns prompting into a sport: 00:13:00
5. Building a business around using games to make AI better: 00:15:49
6. Can language models learn how to be funny: 00:22:39
7. Why games are a great way to evaluate and train new models: 00:25:31
8. What child psychology tells us about games and AI: 00:26:58
9. Using games to unlock continual learning in AI: 00:30:10
10. Why Alex cares deeply about games: 00:36:42
11. Where Alex sees the most promise in AI: 00:44:37
12. Rethinking how young people start their careers in the age of AI: 00:50:54

You can check out the episode on X, Spotify, Apple Podcasts, or YouTube. Links are below:

1. Watch on X
2. Watch on YouTube
3. Listen on Spotify (make sure to follow to help us rank!)
4. Listen on Apple Podcasts

Miss an episode? Catch up on Dan’s recent conversations with founding executive editor of Wired Kevin Kelly, star podcaster Dwarkesh Patel, LinkedIn cofounder Reid Hoffman, ChatPRD founder Claire Vo, economist Tyler Cowen, writer and entrepreneur David Perell, founder and newsletter operator Ben Tossell, and others, and learn how they use AI to think, create, and relate.

If you’re enjoying the podcast, here are a few things I recommend:

1. Subscribe to Every
2. Follow Dan on X
3. Subscribe to Every’s YouTube channel

Rhea Purohit is a contributing writer for Every focused on research-driven storytelling in tech. You can follow her on X at @RheaPurohit1 and on LinkedIn, and Every on X at @every and on LinkedIn.

We build AI tools for readers like you. Write brilliantly with Spiral. Organize files automatically with Sparkle. Deliver yourself from email with Cora. Dictate effortlessly with Monologue.

We also do AI training, adoption, and innovation for companies. Work with us to bring AI into your organization.

Get paid for sharing Every with your friends. Join our referral program.