-
Notifications
You must be signed in to change notification settings - Fork 2.8k
Feature/zimage scheduler support #8705
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Feature/zimage scheduler support #8705
Conversation
Add support for alternative diffusers Flow Matching schedulers: - Euler (default, 1st order) - Heun (2nd order, better quality, 2x slower) - LCM (optimized for few steps) Backend: - Add schedulers.py with scheduler type definitions and class mapping - Modify denoise.py to accept optional scheduler parameter - Add scheduler InputField to flux_denoise invocation (v4.2.0) Frontend: - Add fluxScheduler to Redux state and paramsSlice - Create ParamFluxScheduler component for Linear UI - Add scheduler to buildFLUXGraph for generation
Add support for alternative diffusers Flow Matching schedulers for Z-Image: - Euler (default) - 1st order, optimized for Z-Image-Turbo (8 steps) - Heun (2nd order) - Better quality, 2x slower - LCM - Optimized for few-step generation Backend: - Extend schedulers.py with Z-Image scheduler types and mapping - Add scheduler InputField to z_image_denoise invocation (v1.3.0) - Refactor denoising loop to support diffusers schedulers Frontend: - Add zImageScheduler to Redux state in paramsSlice - Create ParamZImageScheduler component for Linear UI - Add scheduler to buildZImageGraph for generation
|
Euler and Heun are working as expected. However, when I generate with the LCM scheduler, I get the same error as in PR #8704 : I am just using the linear UI at the moment. |
LCM scheduler may have more internal timesteps than user-facing steps, causing user_step to exceed total_steps. This resulted in progress percentage > 1.0, which caused a pydantic validation error. Fix: Only call step_callback when user_step <= total_steps.
…/Pfannkuchensack/InvokeAI into feature/zimage-scheduler-support
|
I can confirm that all three schedulers now run to completion on my system in both the linear and workflow editor views. There is a step count problem. Both Euler and LCM are doing one more step than I ask for (e.g. 9 steps when I ask for 8). Heun is giving me N2-1 steps, i.e. 15 steps when I ask for 8. But maybe this is the expected behavior, because I also get N2-1 when using Heun with SDXL. |
|
Heun N×2-1: Heun is a 2nd-order solver that performs two model evaluations per "user step": |
|
Euler/LCM N+1 Issue: This likely comes from the initial step_callback at step 0: step_callback( |
I'm happy with the explanation for the N+1 steps with Euler and Heun, but the user experience is inconsistent with the SD and SDXL models, which display N steps in the server log. Some users will ask why SDXL shows N steps and Flux/ZImage show N+1 steps. It's a small thing, and if it is too hard or hacky to fix let me know and I'll merge it in. |
…ount Remove the initial step_callback at step=0 to match SD/SDXL behavior. Previously Flux/Z-Image showed N+1 steps (step 0 + N denoising steps), while SD/SDXL showed only N steps. Now all models display N steps consistently in the server log.
|
Now it should show the "right" step count. |
Summary
Add support for alternative diffusers Flow Matching schedulers for Z-Image models:
The scheduler can be selected in both the Linear UI (Generation Settings → Advanced) and the Workflow Editor (Z-Image Denoise node).
Backend changes:
invokeai/backend/flux/schedulers.pywith Z-Image scheduler types and mappingschedulerInputField toz_image_denoiseinvocation (version 1.2.0 → 1.3.0)Frontend changes:
zImageSchedulerto Redux state inparamsSliceParamZImageSchedulercomponent for Linear UI dropdownbuildZImageGraphNote: Only Flow Matching schedulers are compatible with Z-Image. Traditional schedulers like DDIM cannot be used because Z-Image uses velocity prediction (v-prediction) with a linear interpolation schedule, while DDIM expects noise prediction (ε-prediction) with a variance-preserving schedule.
Related Issues / Discussions
QA Instructions
Merge Plan
Standard merge, no special considerations. This PR depends on the Flux scheduler PR being merged first (shared scheduler infrastructure in
schedulers.py).Checklist
What's Newcopy (if doing a release after this PR)