AivexaNewsSearch
AI news for builders and product teamsChecked every hour

Why FLUX 3 Trains on Images, Video and Sound — Andreas Blattmann & Dustin Podell, Black Forest Labs

Collected Nov 16, 2026

Andreas Blattmann and Dustin Podell of Black Forest Labs appeared on the Latent Space podcast to explain why FLUX 3 was trained jointly on images, video and audio rather than on still images alone. Blattmann co-created Stable Diffusion and co-invented the latent diffusion method behind it; Podell led FLUX Kontext and was lead author of SDXL. Both came out of the Stability AI image-generation group, which spun out to form BFL with help from Anj Midha, and the lab's Flux series followed.

The core argument is about what each signal contributes. A still image shows where something is. A video shows how it got there. Audio can mark when an event happened, even if the camera missed the contact. Blattmann's illustration is a person catching a tossed ball: the hand starts moving before the ball arrives, and a person can do that without being able to write down the trajectory. That kind of implicit, predictive knowledge is what the team means by visual intelligence, and the three modalities are meant to supply complementary evidence toward it.

Camera motion gets particular attention. If a camera pans around a person, at some point the model has to render the back of their head, so ordinary two-dimensional video carries a lot of information about what objects look like from angles the training frames never showed directly. The episode treats this as a stronger test than asking whether any single generated frame looks convincing, while leaving open the question of whether such a model genuinely understands physics.

A concrete engineering example concerns upscaling. Sharpening each frame of a clip independently can make every frame look excellent on its own, yet playback shimmers because the model made slightly different decisions from frame to frame. BFL derives its video upscaler from the same FLUX 3 backbone, and temporal consistency turns out to matter even when the user only asked for a higher-resolution file.

The conversation also contrasts diffusion with autoregression. An autoregressive model can learn from every token in a training example at once, while the diffusion side can distill a long sampling process down to a handful of steps. Which advantage wins depends on whether you are paying to train the model or waiting for an output, which is a useful way to think about the trade-off: training-time efficiency and inference-time latency pull in different directions. In practice, this suggests the two approaches are less rivals than tools whose costs land at different points in the pipeline.

Podell's early Kontext experiment shows how the team decides where to invest. He reasoned that an image model able to draw a dog wearing a hat already knows something about where hats belong. He and a colleague assembled a small set of photos labeled with relationships, and a training run of a few hours taught the model to put a logo on a shirt. The result was rough but good enough to take to the next offsite, after which the team spent months on data and modeling. That pattern — a cheap probe to validate an idea before a long investment — is a familiar one for teams deciding whether an editing capability is worth a full training run.

Podell also described an unexpected use: people feeding FLUX old, degraded family photos, restoring them, and showing the results to their families. He said he had not predicted it. It sits alongside the harder technical problem of preserving a person's identity across edits and across frames, and it is a reminder that consumers notice fidelity failures sharply even when the surrounding pipeline is sound.

The published chapter list runs from the Black Forest roots and Podell's path through visual intelligence, SDXL, learning motion, flow matching, starting BFL, early FLUX use cases, editing with Kontext, training data, testing an edit, autoregression, world action models, robotics in 2027, and who uses FLUX. The episode is sponsored by Deepgram.

Why it matters: teams building image and video pipelines should expect temporal consistency, not per-frame quality, to be the binding constraint on anything that outputs clips, and BFL's bet is that joint training on video and audio is the route to it. For product teams, editing features that preserve identity and untouched regions — the Kontext direction — look like the near-term battleground, with video and sound generation arriving on the same backbone. A reasonable inference is that model choice will hinge increasingly on sampling cost and latency rather than raw sample quality alone.

Read at Latent Space

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

What can a model learn about the world by watching video and listening to it? Black Forest Labs trained FLUX 3 on images, video and audio together. Andreas Blattmann and Dustin Podell join swyx to explain why. A moving camera reveals the far side of an...