Skip to content

Issues: NVIDIA/Megatron-LM

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

149 Open 648 Closed
149 Open 648 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[QUESTION] I encountered the following issue when executing your command. What could be the cause? args.exit_on_missing_checkpoint is: True >> '--exit-on-missing-checkpoint' set ... exiting. <<
#1317 opened Dec 10, 2024 by Alinanini
[QUESTION]Does Megatron support tracing computation graphs with torch.fx?
#1315 opened Dec 7, 2024 by fy-j
[BUG] When using LLaVA with freeze-LM, training text only sample occurs error.
#1314 opened Dec 6, 2024 by liveseongho
[QUESTION] How to specify the implementation of Attention？
#1313 opened Dec 6, 2024 by renyinCheng001
[QUESTION] Gradient Propagation in backward pass
#1312 opened Dec 5, 2024 by arul-lm
[QUESTION]UnboundLocalError：local variable ‘output tensor’ referenced before assignmnet
#1311 opened Dec 5, 2024 by zmtttt
[ENHANCEMENT]When load_ckpt is called and the obtained iteration count equals args.train_iters, the train_step process will be directly skipped. If, at this point, the save_checkpoint function may encounter an error.
#1310 opened Dec 5, 2024 by bphwk
[QUESTION]
#1308 opened Dec 2, 2024 by eliird
[BUG] Source for releases 0.8.0 and 0.9.0 are not available
#1307 opened Nov 29, 2024 by dmgcsilva
[BUG] The problem of splitting transformer layers when pipeline parallelism cannot be evenly divided.
#1304 opened Nov 27, 2024 by Baibaifan
[QUESTION] How to split the Transform layer when the pipeline is uneven?
#1303 opened Nov 27, 2024 by renyinCheng001
1
[QUESTION] Why is the initialization of the router and experts different in the MoE part?
#1302 opened Nov 27, 2024 by mxymxy77
[BUG] an illegal memory access was encountered in MOE-MLP(GroupGemm)
#1301 opened Nov 26, 2024 by hgdhrt
[BUG] validate_yaml() isn't in sync with arguments check
#1297 opened Nov 21, 2024 by pierric
1
[QUESTION] deepseek v2 compatility?
#1295 opened Nov 21, 2024 by wavy-jung
[BUG] 0.9.0 release version got param_gather_handle error with 3d parallel
#1292 opened Nov 19, 2024 by SeunghyunSEO
6
[QUESTION] How to convert torch_dist format checkpoint to torch format?
#1291 opened Nov 19, 2024 by zhangyilalala
1
[QUESTION] SGD support in distrib_optimizer.py
#1287 opened Nov 13, 2024 by zstreeter
[QUESTION] There is already a 32-bit model parameter in the optimizer state. Why do we need to store a separate copy of the model parameters in the checkpoint?
#1283 opened Nov 12, 2024 by leondada
Where can I download the tokenizer for the model mcore-llava-mistral-7b-instruct-clip336-pretraining?
#1281 opened Nov 11, 2024 by herolxl
[BUG]Megatron-LM doesn't support transformer-engine 1.13
#1280 opened Nov 11, 2024 by klhhhhh
1
[BUG] Encountering NaN gradients when using CUDA Graph
#1279 opened Nov 11, 2024 by DXZDXZ
1
[QUESTION] is there any restriction to use allgather with moe_expert_capacity_factor?
#1277 opened Nov 7, 2024 by Louis-J
[QUESTION] scaleing MFU calculate
#1276 opened Nov 6, 2024 by ltm920716
[BUG] TP-comm-overlap bug when replacing TELayerNormColumnParallelLinear into TEColumnParallelLinear .
#1275 opened Nov 6, 2024 by wplf
ProTip! Adding no:label will show everything without a label.