World Models with DreamerV3 - AI Build & Learn

Welcome to AI Build & Learn, a weekly AI engineering stream where we pick a new topic and learn by building together.

This event kicks off a run on world models: models that learn an internal representation of how an environment works, then use it to predict what happens next and to plan. It's a natural next step after the generation and RL events, tying both threads together.

We're starting with DreamerV3, a model-based RL agent and a great on-ramp to the idea. Dreamer learns a compact world model of its environment from experience, then trains its policy almost entirely inside imagined rollouts of that model rather than the real environment. It's lightweight (trains on a single GPU), works across many tasks with the same settings, and connects directly to the RL and MuJoCo work.

Some things to look up to get started:
Model:

DreamerV3 (Danijar Hafner): github.com/danijar/dreamerv3
Paper: "Mastering Diverse Domains through World Models" arxiv.org/abs/2301.04104

Background

The original "World Models" paper (Ha and Schmidhuber): worldmodels.github.io/

Resources

GitHub: github.com/sagecodes/ai-build-and-learn
Events Calendar: luma.com/ai-builders-and-learners
Slack (Discuss during the week): slack.flyte.org/
Hosted by Sage Elliott: linkedin.com/in/sageelliott/

In this stream

Intro to topic
Community Discussion
Practical examples

Community challenge (optional)
Try spending 30–90 minutes during the week learning or building something related to the topic, then share what you’re working on in Slack.

Note on Flyte / Union
You may see Flyte used in some demos. Flyte is an open-source AI orchestration platform maintained by Union (where I work) for building scalable, durable, and observable AI workflows. You do not need to use Flyte to participate.

Union: union.ai/
Flyte: flyte.org/

Drop a comment with ideas for future t