Blurry images since today

[lhovav]

Hi,

I'm getting blurry sample images every 250 images,

started today:
What I did that I think it's a bug:

1. tried different seed - still blurry
2. tried two different datasets from yesterday (which were perfect) - still blurry,
3. deployed another pod (I'm running RunPod) - still blurry

also, the linkess is meh, compared to yesterday (the .safetensor doesn't produce blurry images on local ComfyUI)

Everything is the same, I'm using the same config.yaml file, the same file structure, just changing the lora's name.

does anyone experience this issue too?

Thanks

[steveant]

I've had the same issue. Besides what you mentioned I also:

• reinstalled locally on windows and on linux with latest (followed README.md process)
• I noticed the samples generated have different levels of fogginess and some are consistently clear. (odd).
• Next I'm going to try to revert to an an older changeset hoping it's an issue in the main repo and not the submodules...

FWIW - the first round of latents that get saved to output//samples before any training are foggy.

[duchamps0305]

yes here too.

[julien-blanchon]

I also have blurry image but it's mainly on white background which is a know Flux problem