Beyond Accuracy: Evaluating the learned representations of Generative AI models

https://is1-ssl.mzstatic.com/image/thumb/Podcasts112/v4/8c/60/0c/8c600c41-9d2f-37c8-9fd4-6533713a40ee/mza_15383876713238264269.jpg/600x600bb.jpg

Jay Shah Podcast

Jay Shah

95 episodes

2 weeks ago

Interviews with scientists and engineers working in Machine Learning and AI, about their journey, insights, and discussion on latest research topics.

Science

RSS

All content for Jay Shah Podcast is the property of Jay Shah and is served directly from their servers with no modification, redirects, or rehosting. The podcast is not affiliated with or endorsed by Podjoint in any way.

Interviews with scientists and engineers working in Machine Learning and AI, about their journey, insights, and discussion on latest research topics.

Science

https://d3t3ozftmdmh3i.cloudfront.net/staging/podcast_uploaded_nologo/38361858/cc657ce8428915a7.jpeg

Beyond Accuracy: Evaluating the learned representations of Generative AI models | Aida Nematzadeh

Jay Shah Podcast

53 minutes 17 seconds

1 month ago

Beyond Accuracy: Evaluating the learned representations of Generative AI models | Aida Nematzadeh

Dr. Aida Nematzadeh is a Senior Staff Research Scientist at Google DeepMind where her research focused on multimodal AI models. She works on developing evaluation methods and analyze model’s learning abilities to detect failure modes and guide improvements. Before joining DeepMind, she was a postdoctoral researcher at UC Berkeley and completed her PhD and Masters in Computer Science from the University of Toronto. During her graduate studies she studied how children learn semantic information through computational (cognitive) modeling. Time stamps of the conversation00:00 Highlights01:20 Introduction02:08 Entry point in AI03:04 Background in Cognitive Science & Computer Science 04:55 Research at Google DeepMind05:47 Importance of language-vision in AI10:36 Impact of architecture vs. data on performance 13:06 Transformer architecture 14:30 Evaluating AI models19:02 Can LLMs understand numerical concepts 24:40 Theory-of-mind in AI27:58 Do LLMs learn theory of mind?29:25 LLMs as judge35:56 Publish vs. perish culture in AI research40:00 Working at Google DeepMind42:50 Doing a Ph.D. vs not in AI (at least in 2025)48:20 Looking back on research careerMore about Aida: http://www.aidanematzadeh.me/About the Host:Jay is a Machine Learning Engineer at PathAI working on improving AI for medical diagnosis and prognosis. Linkedin: shahjay22 Twitter: jaygshah22 Homepage: https://jaygshah.github.io/ for any queries.Stay tuned for upcoming webinars!**Disclaimer: The information in this video represents the views and opinions of the speaker and does not necessarily represent the views or opinions of any institution. It does not constitute an endorsement by any Institution or its affiliates of such video content.**