New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

我想知道一下技术报告里面激活37B是怎么算出来的？ #231

Open

ciaoyizhen opened this issue Jan 6, 2025 · 0 comments

ciaoyizhen commented Jan 6, 2025

We present DeepSeek-V3, a strong Mixture-of-Experts (MoE) language model with 671B total parameters with 37B activated for each token.

就是上面的话，37B是怎么计算出来的呢？
是直接根据专家数人工手算出来的吗？

比如总参数是可以调p.numel()来获取
这个37B有什么方法是获取的呢？

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment