DeepSpeed zero3

[Abolfazl-kr]

what should we do to use deep speed stage3?
what changes should apply on deep speed config and the codes?

---

Admin Note: Transfer to Discussion, as this is not an issue.

[ymcui]

Such general questions should refer to the documentation of DeepSpeed.
We did not use ZeRO-3 when training our models so we cannot provide specific comments.

[Abolfazl-kr]

Ok.
I'm struggling with overflow and minimizing loss-scale, and i couldn't use bp16 because of my hardware. i want to try zero-3 maybe mixed precision work.

Thanks a lot.

[feifei05]

ValueError: ZeRO inference only makes sense with ZeRO Stage 3 - please adjust your config.
Should I replace it with zero-3?

[feifei05]

[yuchengml]

@Abolfazl-kr @feifei05
Please refer to sft_with_ds_zero3