Home
Categories
EXPLORE
True Crime
Comedy
Society & Culture
Business
Sports
TV & Film
Technology
About Us
Contact Us
Copyright
© 2024 PodJoint
00:00 / 00:00
Sign in

or

Don't have an account?
Sign up
Forgot password
https://is1-ssl.mzstatic.com/image/thumb/Podcasts211/v4/6a/24/22/6a242243-a886-3562-51aa-5b0137909c8b/mza_6305134645633578970.jpg/600x600bb.jpg
The AI Research Deep Dive
The AI Research Deep Dive
37 episodes
1 week ago
From arXiv to insight: a daily tour of cutting-edge AI papers. The AI Research Deep Dive podcast dives into a new groundbreaking research paper every day. It combs through the most important details and results to give you a great idea of what the paper accomplishes and how it gets there.
Show more...
Science
RSS
All content for The AI Research Deep Dive is the property of The AI Research Deep Dive and is served directly from their servers with no modification, redirects, or rehosting. The podcast is not affiliated with or endorsed by Podjoint in any way.
From arXiv to insight: a daily tour of cutting-edge AI papers. The AI Research Deep Dive podcast dives into a new groundbreaking research paper every day. It combs through the most important details and results to give you a great idea of what the paper accomplishes and how it gets there.
Show more...
Science
https://d3t3ozftmdmh3i.cloudfront.net/staging/podcast_uploaded_nologo/43949260/43949260-1750798569136-3391783a0fb9a.jpg
BeyondWeb: Lessons from Scaling Synthetic Data for Trillion-scale Pretraining
The AI Research Deep Dive
15 minutes 38 seconds
2 months ago
BeyondWeb: Lessons from Scaling Synthetic Data for Trillion-scale Pretraining

This episode of "The AI Research Deep Dive" explores "BeyondWeb," a paper from DatologyAI that offers a rigorous, scientific solution to the AI "data wall"—the problem of running out of high-quality web data for training. The host explains how BeyondWeb moves beyond messy, ad-hoc methods for creating synthetic data by introducing a principled framework based on "source rephrasing." Listeners will learn the paper's key lessons: start with high-quality web text, transform it using a diverse portfolio of styles and formats, and use surprisingly small models to do it efficiently. The episode highlights the stunning results, where a model trained on BeyondWeb data not only learns up to 7.7 times faster but also allows a 3-billion-parameter model to outperform an 8-billion-parameter model, providing a practical roadmap for building more capable and efficient AI in a data-constrained world.


The AI Research Deep Dive
From arXiv to insight: a daily tour of cutting-edge AI papers. The AI Research Deep Dive podcast dives into a new groundbreaking research paper every day. It combs through the most important details and results to give you a great idea of what the paper accomplishes and how it gets there.