Skip to content

Pull requests: deepspeedai/Megatron-DeepSpeed

New pull request New
20 Open 258 Closed
20 Open 258 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

[BUG]Fix the error issue for q/k/v stride is not match for non FPDT scenarios.
#469 opened Mar 17, 2025 by ys950902 Loading…
[Bug]Add sequence_parallel in layernorm init to enable 3D parallelism with DeepSpeed for non CUDA device.
#468 opened Feb 28, 2025 by ys950902 Loading…
1
FastPersist rebase
#467 opened Feb 25, 2025 by tjruwase Loading…
support split qkv linear and sp overlap comm
#415 opened Jul 5, 2024 by inkcherry Loading…
9
fix --use-cpu-initialization error when expert is not tensor-parallel
#413 opened Jul 3, 2024 by taozhiwei Loading…
3
fix NAN loss of rope long context training
#399 opened Jun 5, 2024 by inkcherry Loading…
1
convert mds checkpoint to Hf Llama model
#394 opened May 31, 2024 by vksastry Loading…
1
ds-sequence-parallel(ulysses) for rope.
#392 opened May 30, 2024 by inkcherry Loading…
1
add HFTokenizer option for preprocess_data
#388 opened May 17, 2024 by Jianhong-Zhang Loading…
1
Add layer norm weight plus 1
#378 opened Apr 18, 2024 by Yejing-Lai Loading…
4
Fix ConstantGradScaler and loss-scale argument not match
#376 opened Apr 12, 2024 by BeingGod Loading…
1
Support Llama2Tokenizer
#375 opened Apr 11, 2024 by jinyouzhi Loading…
collect grad_norm for non pipeline path
#370 opened Mar 21, 2024 by inkcherry Loading…
optimize the generation of attention mask
#331 opened Jan 13, 2024 by imh966 Loading…
Enable torch.compile
#322 opened Dec 28, 2023 by tohtana • Draft
Simplify SP - Opportunity to improve SP scalability
#301 opened Nov 28, 2023 by RezaYazdaniAminabadi Loading…
support transfer llama hf weight to megatron weight
#246 opened Sep 12, 2023 by uygnef Loading…
27
add vit training with TP/PP
#146 opened Jun 9, 2023 by etoilestar Loading…
3
attempt at pipelining
#78 opened Aug 18, 2022 by siddharth9820 Loading…
Add support for DS comms
#50 opened Jun 13, 2022 by Quentin-Anthony Loading…
1
ProTip! Filter pull requests by the default branch with base:main.