How NVIDIA Speeds Up FLUX 3 Image Klein on Blackwell GPUs

NVIDIA’s November 6 technical post reports faster inference for Black Forest Labs’ FLUX 3 Image Klein on Blackwell GPUs using TensorRT LLM VisualGen. In its 2048 × 2048 tests on one B200, successive optimizations reduced Base text-to-image latency from 22.36 to 6.46 seconds and image-to-image latency from 54.64 to 12.28 seconds. Those are reported results for specified configurations, not independent measurements or a promise that every prompt and deployment will see the same gains.

Klein uses Qwen3-VL-4B-Instruct to condition generation on text, a diffusion transformer to refine image latents, and an autoencoder to encode reference images and decode the result. The Base checkpoint uses 50 denoising steps with classifier-free guidance in NVIDIA’s tests; the distilled checkpoint uses 16 steps without it. This distinction matters when comparing latency: the faster checkpoint performs less iterative work, while its output must be assessed separately for the intended editing or generation task.

The post identifies denoising as the main target. In one compiled, NVFP4, single-B200 pipeline measurement at 2048 × 2048, it accounted for 96.8% of Base text-to-image latency and 97.8% of Base image-to-image latency. VisualGen first applies lower-precision NVFP4 transformer weights and fused matrix multiplication kernels, including fused activation and attention projections. NVIDIA reports up to 1.62 times faster text-to-image and 1.58 times faster image-to-image generation for NVFP4 than BF16 on Blackwell. Precision changes can alter pixels, so speed alone does not establish equivalent output.

Attention becomes expensive as image resolution increases. NVIDIA says BF16 attention consumed about 60% of transformer GPU kernel time in its 2048 × 2048 NVFP4 text-to-image tests. With an FP8 attention path, VC-Attention uses a repair budget to correct a selected fraction of tokens with the largest quantization error. At a budget of 0.005, the post reports 1.18 times faster generation for Base and 1.16 times for distilled Klein versus its BF16 attention baseline. Their LPIPS differences against corresponding baseline outputs were 0.086 and 0.151. These paired results show a speed and output-change tradeoff, not a universal quality ranking.

A second attention option reduces computation through sparsity. For Base at 2048 × 2048, Sol-Attention with τ=1.0 sped up denoising plus decoding by 1.24 times when the first 16 steps remained dense, or 1.30 times when only the first 12 did. NVIDIA’s example images show larger deviations in some scenes with fewer dense steps. The practical inference is that the dense-step cutoff should be chosen against the kinds of fine detail and edit preservation a workload requires; the published examples cannot establish a safe cutoff for every prompt.

Cache-DiT attacks repetition across denoising steps by reusing intermediate transformer outputs when a residual-difference rule permits it. At the reported threshold of 0.24, Base text-to-image generation at 2048 × 2048 was 1.94 times faster than the uncached NVFP4 configuration, with LPIPS 0.033 against its uncached output. The distilled checkpoint gained 1.20 times at that resolution. A plausible explanation for the smaller gain is its shorter denoising sequence, which offers fewer opportunities to reuse work; that explanation is an inference, not a separate experiment in the post.

For eight B200 GPUs, VisualGen assigns Base’s positive and negative guidance branches to separate GPU groups and partitions each branch’s sequence with Ulysses parallelism. The distilled model uses sequence parallelism alone. NVIDIA reports median 2048 × 2048 text-to-image latency falling from 14.90 to 3.00 seconds for Base and from 2.73 to 1.08 seconds for distilled Klein when moving from one to eight GPUs. Those figures describe latency, not throughput or economic efficiency. The post also notes that work outside denoising takes roughly half of the eight-GPU distilled runtime, limiting further end-to-end gains.

A useful reproduction should pin the checkpoint, VisualGen configuration, GPU count, resolution, prompt and reference image, seed, step count, and guidance setting. NVIDIA averaged its single-GPU pipeline measurements over ten requests after two warmups, used seed 0 for the optimization benchmarks unless noted, and excluded optional prompt upsampling. Measure each optimization against its stated baseline before combining them, then inspect multiple prompts and edits for subject identity, text, small details, and preservation of requested regions. LPIPS measures similarity to a baseline image; it cannot alone decide whether either image better satisfies the prompt. Until broader workloads are tested, the reported speedups remain evidence for NVIDIA’s measured configurations.