Monday, November 16, 2026 · 9:00–10:00 AM PST · Online
Zoom link: us02web.zoom.us/j/82308186562
Join us for two deep dives into hidden AI performance costs: surprise PyTorch recompilations during inference and compute, memory, and communication bottlenecks in Jev training.
Talk #0: Introductions and Meetup Updates
by Chris Fregly and Antje Barth
Talk #1: Overcoming the PyTorch Recompilation Dilemma in Real-Time Inference Workloads
by Chaim Rand
`torch.compile` can accelerate steady-state inference, yet a guard failure on a new input shape or layout can trigger an expensive recompilation in the request path. Drawing on his latest post, Chaim will compare six approaches using a dynamic graph-neural-network example: cache preloading, bounded recompilation with eager fallback, Dynamo and input-layout tuning, warmup seeding, background compilation, and ahead-of-time compilation with `torch.export` and AOTInductor. He’ll explain their tradeoffs and how the right choice depends on the model, input distribution, runtime environment, and latency requirements.
Bio: Chaim Rand is a machine-learning algorithm developer who writes about PyTorch and practical AI/ML performance optimization.
Based on: Overcoming the PyTorch Recompilation Dilemma in Real-Time Inference Workloads
Talk #2: From Profiles to Throughput: How TypeSafe Optimized Jev with Zymtrace
by Israel Ogbole, Co-founder and CEO of Zymtrace
Israel will explore the latest Zymtrace/TypeSafe case study on training and serving Jev. He’ll discuss how continuous CPU/GPU profiling and MCP-accessible profile data supported an iterative optimization loop across compute, memory, communication, and inference synchronization. The published case study reports approximately 46% higher training throughput on the same hardware after three successive changes, plus approximately 10% better inference performance after removing a hidden device-to-host synchronization. The talk will focus on how the team found each bottleneck, mea