Skip to content

Pull requests: bigscience-workshop/Megatron-DeepSpeed

New pull request New
45 Open 215 Closed
45 Open 215 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Bump black from 21.4b0 to 24.3.0 dependencies Pull requests that update a dependency file
#402 opened Mar 20, 2024 by dependabot bot Loading…
Add xPos embeddings
#370 opened Mar 7, 2023 by janEbert Loading…
Fix various small problems
#367 opened Feb 28, 2023 by janEbert Loading…
Bloom model training with AML
#365 opened Feb 21, 2023 by savitamittal1 Loading…
Add UL2 data sampling and pretraining
#358 opened Dec 13, 2022 by janEbert Loading…
5
Add FlashAttention
#357 opened Dec 12, 2022 by NouamaneTazi Loading…
3
Distill BLOOM - tentative 2
#354 opened Oct 11, 2022 by younesbelkada • Draft
4
Enable rocm-support
#353 opened Oct 7, 2022 by luukkonenr Loading…
Encoding checkpoint reshaping guide
#349 opened Sep 20, 2022 by tjruwase • Draft
4
Add multiple evaluation compat
#336 opened Aug 30, 2022 by Muennighoff Loading…
[checkpoints] replace bf16 with fp32 checkpoint weights
#327 opened Aug 10, 2022 by stas00 Loading…
5
Prefix LM Eval
#313 opened Jul 16, 2022 by Muennighoff Loading…
4
Add Bitfit
#311 opened Jul 10, 2022 by Muennighoff Loading…
Tool for CKPT averaging
#310 opened Jul 10, 2022 by Muennighoff Loading…
Enable loading ckpt for t0 finetuning
#309 opened Jul 10, 2022 by Muennighoff Loading…
32
[WIP] Hack my way to get OPT running
#301 opened Jul 4, 2022 by thomasw21 • Draft
[MLM] Train script for non causal decoder
#300 opened Jul 4, 2022 by thomasw21 • Draft
a branch combining layer-norm-auto-sync and ds_ckpt_reshape
#292 opened Jun 29, 2022 by stas00 Loading…
BigScience Eval Harness
#291 opened Jun 29, 2022 by Muennighoff Loading…
No-ZeRO reshaping
#289 opened Jun 23, 2022 by Muennighoff Loading…
WIP: Shared t5 code
#286 opened Jun 21, 2022 by thomasw21 Loading…
2 of 4 tasks
[WIP] add debug utils
#275 opened Mar 28, 2022 by stas00 Loading…
Sync 4 layer norms - bf16, fp32, optimizer states on restart
#274 opened Mar 28, 2022 by tjruwase Loading…
Sync layer norm
#271 opened Mar 24, 2022 by thomasw21 • Draft
6
Test different layer norm
#270 opened Mar 24, 2022 by thomasw21 • Draft
ProTip! Exclude everything labeled bug with -label:bug.