Ablation studies for DeepSeekMoE

[JinjieNi]

Hi Authors,

Thanks for presenting the great work of DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models.

Regarding the ablation study "Shared Expert Isolation." in section 4.4 and figure 3, may I check whether the models in Figure 3 are all trained from scratch or directly use the GShard results, especially the blue setting?

Thank you!