Skip to content

Conversation

@Pfannkuchensack
Copy link
Collaborator

Summary

Add support for alternative diffusers Flow Matching schedulers for Z-Image models:

  • Euler (default) - 1st order, optimized for Z-Image-Turbo (~8 steps)
  • Heun (2nd order) - Better quality, 2x slower (double model evaluations per step)
  • LCM - Optimized for few-step generation (1-4 steps)

The scheduler can be selected in both the Linear UI (Generation Settings → Advanced) and the Workflow Editor (Z-Image Denoise node).

Backend changes:

  • Extended invokeai/backend/flux/schedulers.py with Z-Image scheduler types and mapping
  • Added scheduler InputField to z_image_denoise invocation (version 1.2.0 → 1.3.0)
  • Refactored denoising loop to support both built-in Euler and diffusers schedulers

Frontend changes:

  • Added zImageScheduler to Redux state in paramsSlice
  • Created ParamZImageScheduler component for Linear UI dropdown
  • Integrated scheduler selection into buildZImageGraph

Note: Only Flow Matching schedulers are compatible with Z-Image. Traditional schedulers like DDIM cannot be used because Z-Image uses velocity prediction (v-prediction) with a linear interpolation schedule, while DDIM expects noise prediction (ε-prediction) with a variance-preserving schedule.

Related Issues / Discussions

QA Instructions

  1. Select a Z-Image model (e.g., Z-Image-Turbo)
  2. Open Generation Settings → Advanced Options
  3. Verify the Scheduler dropdown appears with options: Euler, Heun (2nd order), LCM
  4. Generate images with each scheduler:
    • Euler: Should produce identical results to previous behavior (default, 8 steps recommended)
    • Heun: Takes ~2x longer, may produce slightly different/improved results
    • LCM: Works best with 1-4 steps
  5. Test in Workflow Editor: The Z-Image Denoise node should have a scheduler dropdown

Merge Plan

Standard merge, no special considerations. This PR depends on the Flux scheduler PR being merged first (shared scheduler infrastructure in schedulers.py).

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • ❗Changes to a redux slice have a corresponding migration
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

Add support for alternative diffusers Flow Matching schedulers:
- Euler (default, 1st order)
- Heun (2nd order, better quality, 2x slower)
- LCM (optimized for few steps)

Backend:
- Add schedulers.py with scheduler type definitions and class mapping
- Modify denoise.py to accept optional scheduler parameter
- Add scheduler InputField to flux_denoise invocation (v4.2.0)

Frontend:
- Add fluxScheduler to Redux state and paramsSlice
- Create ParamFluxScheduler component for Linear UI
- Add scheduler to buildFLUXGraph for generation
Add support for alternative diffusers Flow Matching schedulers for Z-Image:
- Euler (default) - 1st order, optimized for Z-Image-Turbo (8 steps)
- Heun (2nd order) - Better quality, 2x slower
- LCM - Optimized for few-step generation

Backend:
- Extend schedulers.py with Z-Image scheduler types and mapping
- Add scheduler InputField to z_image_denoise invocation (v1.3.0)
- Refactor denoising loop to support diffusers schedulers

Frontend:
- Add zImageScheduler to Redux state in paramsSlice
- Create ParamZImageScheduler component for Linear UI
- Add scheduler to buildZImageGraph for generation
@github-actions github-actions bot added python PRs that change python files invocations PRs that change invocations backend PRs that change backend files frontend PRs that change frontend files labels Dec 26, 2025
@lstein
Copy link
Collaborator

lstein commented Dec 28, 2025

Euler and Heun are working as expected. However, when I generate with the LCM scheduler, I get the same error as in PR #8704 :

pydantic_core._pydantic_core.ValidationError: 1 validation error for InvocationProgressEvent
percentage
  Input should be less than or equal to 1 [type=less_than_equal, input_value=1.1666666666666667, input_type=float]
    For further information visit https://errors.pydantic.dev/2.12/v/less_than_equal

I am just using the linear UI at the moment.

LCM scheduler may have more internal timesteps than user-facing steps,
causing user_step to exceed total_steps. This resulted in progress
percentage > 1.0, which caused a pydantic validation error.

Fix: Only call step_callback when user_step <= total_steps.
@lstein
Copy link
Collaborator

lstein commented Dec 28, 2025

I can confirm that all three schedulers now run to completion on my system in both the linear and workflow editor views. There is a step count problem. Both Euler and LCM are doing one more step than I ask for (e.g. 9 steps when I ask for 8). Heun is giving me N2-1 steps, i.e. 15 steps when I ask for 8. But maybe this is the expected behavior, because I also get N2-1 when using Heun with SDXL.

@Pfannkuchensack
Copy link
Collaborator Author

Heun N×2-1: Heun is a 2nd-order solver that performs two model evaluations per "user step":
First pass: Estimates the slope at the starting point
Second pass: Estimates the slope at the predicted endpoint, then averages
For 8 steps: 8×2 = 16 internal evaluations, but the state_in_first_order mechanism only counts "completed" steps, resulting in 15 (N×2-1 when counting starts at 1). This behaves consistently with SDXL Heun, so this is also expected behavior.

@Pfannkuchensack
Copy link
Collaborator Author

Euler/LCM N+1 Issue: This likely comes from the initial step_callback at step 0:

step_callback(
PipelineIntermediateState(
step=0, # ← This is the "extra" step
...
),
)
Plus N additional steps in the loop. This is actually expected behavior - the first callback shows the initial state (pure noise), followed by N denoising steps.

@lstein
Copy link
Collaborator

lstein commented Dec 29, 2025

Euler/LCM N+1 Issue: This likely comes from the initial step_callback at step 0:

step_callback( PipelineIntermediateState( step=0, # ← This is the "extra" step ... ), ) Plus N additional steps in the loop. This is actually expected behavior - the first callback shows the initial state (pure noise), followed by N denoising steps.

I'm happy with the explanation for the N+1 steps with Euler and Heun, but the user experience is inconsistent with the SD and SDXL models, which display N steps in the server log. Some users will ask why SDXL shows N steps and Flux/ZImage show N+1 steps. It's a small thing, and if it is too hard or hacky to fix let me know and I'll merge it in.

…ount

Remove the initial step_callback at step=0 to match SD/SDXL behavior.
Previously Flux/Z-Image showed N+1 steps (step 0 + N denoising steps),
while SD/SDXL showed only N steps. Now all models display N steps
consistently in the server log.
@Pfannkuchensack
Copy link
Collaborator Author

Now it should show the "right" step count.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend PRs that change backend files frontend PRs that change frontend files invocations PRs that change invocations python PRs that change python files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants