We talk IoT – The Internet of Things Business Podcast
Avnet Silica
75 episodes
5 days ago
The We talk IoT podcast keeps listeners updated on the most important developments in the world of the internet of things, iIoT, artificial intelligence, and cognitive computing. Our guests are leading industry experts, business professionals, and experienced journalists as they discuss some of today’s hottest tech topics and how they can help boost your bottom line.
You can follow We talk IoT – the Internet of Things Business podcast anywhere you get your podcasts.
About the host
Stefanie Ruth Heyduck is a science and tech journalist with over 20 years of experience. She has worked for renowned magazines, research institutions, and tech companies. Also, she is an experienced communications consultant and supports her clients with digital and content strategies and new work transformations.
All content for We talk IoT – The Internet of Things Business Podcast is the property of Avnet Silica and is served directly from their servers
with no modification, redirects, or rehosting. The podcast is not affiliated with or endorsed by Podjoint in any way.
The We talk IoT podcast keeps listeners updated on the most important developments in the world of the internet of things, iIoT, artificial intelligence, and cognitive computing. Our guests are leading industry experts, business professionals, and experienced journalists as they discuss some of today’s hottest tech topics and how they can help boost your bottom line.
You can follow We talk IoT – the Internet of Things Business podcast anywhere you get your podcasts.
About the host
Stefanie Ruth Heyduck is a science and tech journalist with over 20 years of experience. She has worked for renowned magazines, research institutions, and tech companies. Also, she is an experienced communications consultant and supports her clients with digital and content strategies and new work transformations.
Billion-parameter brains in pocket-sized chips: The local AI revolution - We talk IoT #75
We talk IoT – The Internet of Things Business Podcast
36 minutes 7 seconds
5 days ago
Billion-parameter brains in pocket-sized chips: The local AI revolution - We talk IoT #75
In this episode, we explore how engineers are embedding powerful AI directly into hardware – no cloud connection required.
Michaël Uyttersprot from Avnet Silica and Cedric Vincent from Tria Technologies reveal how they run ChatGPT-quality language models on resource-constrained embedded devices. What once demanded data centre infrastructure now fits onto chips with just 2GB of RAM.
The conversation covers the technical challenges of cramming billion-parameter models into embedded systems, real-world applications from conference rooms to industrial robotics, and the three compelling reasons driving this shift: data privacy, power efficiency, and cost control.
Michaël and Cedric discuss hardware platforms from AMD, NXP, and Qualcomm, explain techniques like quantisation and mixture of experts, and demonstrate applications including a vintage telephone box that lets you call avatars from different time periods.
Tune in to learn why the future of AI might not be in the cloud at all – and what that means for industries from manufacturing to healthcare.
#AI #LLM #embeddedsystems #IoT #privacy #wetalkiot
Summary of this week's episode:
02:48 What makes large language models special
05:27 Why run LLMs locally on embedded devices
07:42 Real-world applications: Vision LLMs and OCR
11:12 Technical deep dive: How to fit billions of parameters into tiny devices
18:52 Understanding temperature: Making AI creative or accurate
22:41 Industries moving fastest: OCR, security, and robotics
24:52 Future applications: Robotic arms and time series analysis
28:00 The biggest technical hurdle: Power consumption
30:55 Advice for engineers: Start with llama.cpp
Show notes:
Michaël Uyttersprot: https://www.linkedin.com/in/micha%C3%ABl-uyttersprot-aaa971211/
Cedric Vincent: https://www.linkedin.com/in/cedric-vincent-19222910/
Tria Technologies: https://www.tria-technologies.com/
Generative AI at the Edge: https://my.avnet.com/silica/solutions/technologies/artificial-intelligence/generative-ai/
The podcast episode where the generative AI examples where discussed: https://www.podbean.eu/ep/pb-9juiy-d4dec4
How to enhance embedded systems with Generative AI and Local LLMs | Michael Uyttersprot at HWPMAX25: https://www.youtube.com/watch?v=wL9g2wJ1a7c
Listen to the "We Talk IoT" Soundtrack on:
Spotify: https://open.spotify.com/playlist/05MOV4OV2MH2in2txsAGtG?si=ad08112cb8d443f4
YouTube: https://www.youtube.com/watch?v=D-NvQ6VJYtE&list=PLLqgVFfZhDRVYmpEqbgajzDvGL4kACRDp
The Llama song: https://players.brightcove.net/4890384868001/H1lw2qObHx_default/index.html?videoId=6385342213112
About Avnet Silica:
This podcast is brought to you by Avnet Silica — the Engineers of Evolution.
Subscribe to our newsletters here: https://my.avnet.com/silica/resources/newsletter/
You can connect with us on LinkedIn: https://www.linkedin.com/company/silica-an-avnet-company/. Or find us at www.avnet-silica.com.
We talk IoT – The Internet of Things Business Podcast
The We talk IoT podcast keeps listeners updated on the most important developments in the world of the internet of things, iIoT, artificial intelligence, and cognitive computing. Our guests are leading industry experts, business professionals, and experienced journalists as they discuss some of today’s hottest tech topics and how they can help boost your bottom line.
You can follow We talk IoT – the Internet of Things Business podcast anywhere you get your podcasts.
About the host
Stefanie Ruth Heyduck is a science and tech journalist with over 20 years of experience. She has worked for renowned magazines, research institutions, and tech companies. Also, she is an experienced communications consultant and supports her clients with digital and content strategies and new work transformations.
