OpenTelemetry for GenAI and the OpenLLMetry project

https://is1-ssl.mzstatic.com/image/thumb/Podcasts123/v4/86/d2/73/86d27364-ec10-a48e-fdbd-88122c7102ea/mza_12047764457078744174.jpg/600x600bb.jpg

OpenObservability Talks

Dotan Horovits

66 episodes

4 days ago

On OpenObservability Talks we discuss harnessing the power of open source to advance observability initiatives for developers, DevOps and SRE practitioners around the world. We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat. https://www.youtube.com/@openobservabilitytalks You can find us on X (Twitter) @openobserv and BlueSky @openobservability.bsky.social

Technology

RSS

All content for OpenObservability Talks is the property of Dotan Horovits and is served directly from their servers with no modification, redirects, or rehosting. The podcast is not affiliated with or endorsed by Podjoint in any way.

Technology

https://d3t3ozftmdmh3i.cloudfront.net/staging/podcast_uploaded_episode/6432187/6432187-1763924578583-a6da5c3247165.jpg

OpenTelemetry for GenAI and the OpenLLMetry project - OpenObservability Talks S6E06

OpenObservability Talks

1 hour 1 minute 8 seconds

1 month ago

OpenTelemetry for GenAI and the OpenLLMetry project - OpenObservability Talks S6E06

Generative AI is everywhere, but how do we monitor and observe it? OpenTelemetry has been a prominent tool and standard for observability, and recently the OTel community has been aiming to expand its scope and cover GenAI workloads with semantic conventions and tools.

In this episode, Horovits is joined by Nir Gazit, creator of the OpenLLMetry project, and member of the OpenTelemetry Generative AI SIG. We discuss new semantic conventions, tracing prompts and model behavior, the OpenLLMetry project’s journey, and what observability even means for modern AI systems.

Nir Gazit is the CEO and co-founder of Traceloop, and brings a wealth of data and AI experience, with previous experience leading AI teams at Google and serving as the Chief Architect at Fiverr.

You can read the recap post: https://medium.com/p/81b9cea6a771/

Show Notes:

00:00 - intro

04:09 - what is observability for AI

18:07 - AI observability differences from traditional observability

25:22 - OpenLLMetry intro

41:21 - OpenLLMetry latest updates and roadmap

47:00 - OpenTelemetry GenAI Semantic Conventions SIG

56:03 - KubeCon updates: CrossPlane, Knative, Dragonfly, in-toto reached CNCF graduation

1:00:08 - outro

Resources:

OpenTelemetry Generative AI Observability SIG: https://github.com/open-telemetry/community/blob/1c71595874e5d125ca92ec3b0e948c4325161c8a/projects/llm-semconv.md

https://github.com/traceloop/openllmetry

https://github.com/traceloop/hub

https://github.com/traceloop/opentelemetry-mcp-server

Socials:

BlueSky: https://bsky.app/profile/openobservability.bsky.social

Twitter: ⁠https://twitter.com/OpenObserv⁠

LinkedIn: https://www.linkedin.com/company/openobservability/

YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠

Dotan Horovits

============

Twitter: @horovits

LinkedIn: www.linkedin.com/in/horovits

Mastodon: @horovits@fosstodon

BlueSky: @horovits.bsky.social

Nir Gazit

========

Twitter: https://x.com/nir_ga

LinkedIn: https://www.linkedin.com/in/nirga/

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.