How to pretrain Qwen 2.5 7B with 128K context length?

[tjoymeed]

Hi Team,

Thank you for your excellent work.

Could you please tell me how to pretrain Qwen 2.5 7B with 128K context length?

Where to find this recipe?

Thanks again!

[tjoymeed]

If I use the premade training script in NeMo, will that give me the 128K context length, or 4096 context length?