cerndb / dist-keras Star 623 Code Issues Pull requests Distributed Deep Learning, with a focus on distributed training, using Keras and Apache Spark. data-science machine-learning apache-spark deep-learning hadoop tensorflow keras optimization-algorithms data-parallelism distributed-optimizers Updated Jul 25, 2018 Python
xrsrke / pipegoose Star 73 Code Issues Pull requests Discussions Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)* transformers moe data-parallelism distributed-optimizers model-parallelism megatron mixture-of-experts pipeline-parallelism huggingface-transformers megatron-lm tensor-parallelism large-scale-language-modeling 3d-parallelism zero-1 sequence-parallelism Updated Dec 14, 2023 Python