Repositories list

• Megatron-Sailor2

  Public
  0•0•0•0•Updated Feb 19, 2025Feb 19, 2025

• regmix

  Public
  [ICLR 2025] 🧬 RegMix: Data Mixture as Regression for Language Model Pre-training (Spotlight)

  Jupyter Notebook

  •

  MIT License

  •5•106•0•0•Updated Feb 17, 2025Feb 17, 2025

• LongSpec

  Public
  Official Implementation of LongSpec

  JavaScript

  •

  MIT License

  •0•0•0•0•Updated Feb 15, 2025Feb 15, 2025

• sailor2

  Public
  🔱 Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMs

  vietnameseindonesiathai+ 14tamiltagalogcebuanolanguage-modelburmesekhmerlao+ 7

  3•34•0•0•Updated Feb 13, 2025Feb 13, 2025

• Rigging-ChatbotArena

  Public
  Improving Your Model Ranking on Chatbot Arena by Vote Rigging

  Python

  •0•16•0•0•Updated Feb 12, 2025Feb 12, 2025

• sailcompass

  Public
  🧭 SailCompass: Towards Reproducible and Robust Evaluation for Southeast Asian Languages

  Python

  •0•13•0•0•Updated Feb 12, 2025Feb 12, 2025

• zero-bubble-pipeline-parallelism

  Public
  Zero Bubble Pipeline Parallelism

  Python

  •

  Other

  •2.6k•336•20•0•Updated Feb 11, 2025Feb 11, 2025

• oat

  Public
  🌾 OAT: A research-friendly framework for LLM online alignment, including preference learning, reinforcement learning, etc.

  thompson-samplingalignmentreasoning+ 13distributed-trainingppodueling-banditsdpodistributed-rlllmonline-rl+ 6

  Python

  •

  Apache License 2.0

  •12•191•3•0•Updated Feb 8, 2025Feb 8, 2025

• oat-zero

  Public
  A lightweight reproduction of DeepSeek-R1-Zero with indepth analysis of self-reflection behavior.

  Python

  •

  MIT License

  •9•156•1•0•Updated Feb 6, 2025Feb 6, 2025

• autofd

  Public
  Automatic Functional Differentiation in JAX

  automatic-differentiationjaxneural-operator+ 1variational-calculus

  Python

  •

  Apache License 2.0

  •1•63•6•0•Updated Jan 17, 2025Jan 17, 2025

• I-FSJ

  Public
  Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defenses (NeurIPS 2024)

  Jupyter Notebook

  •

  MIT License

  •9•56•1•0•Updated Jan 11, 2025Jan 11, 2025

• InfNeRF

  Public
  InfNeRF: Towards Infinite Scale NeRF Rendering with O(log n) Space Complexity

  Python

  •

  Apache License 2.0

  •1•5•1•0•Updated Jan 7, 2025Jan 7, 2025

• sailcraft

  Public
  🚢 Data Toolkit for Sailor Language Models

  data-deduplicationdata-cleaning

  Python

  •8•85•0•0•Updated Dec 21, 2024Dec 21, 2024

• sailor-llm

  Public
  [EMNLP-2024] ⚓️ Sailor: Open Language Models for South-East Asia

  indonesiathailanguage-model+ 4seavietnamlaomalay

  Python

  •

  MIT License

  •10•126•0•0•Updated Dec 21, 2024Dec 21, 2024

• Meta-Unlearning

  Public
  Python

  •1•24•1•0•Updated Dec 8, 2024Dec 8, 2024

• closer-look-LLM-unlearning

  Public
  The official code of the paper "A Closer Look at Machine Unlearning for Large Language Models".

  Python

  •5•21•0•0•Updated Dec 4, 2024Dec 4, 2024

• inceptionnext

  Public
  InceptionNeXt: When Inception Meets ConvNeXt (CVPR 2024)

  convolutional-neural-networks

  Python

  •

  Apache License 2.0

  •22•276•14•1•Updated Dec 2, 2024Dec 2, 2024

• stde

  Public
  Official implementation of Stochastic Taylor Derivative Estimator (STDE) NeurIPS2024

  Python

  •5•99•2•0•Updated Nov 27, 2024Nov 27, 2024

• optim4rl

  Public
  Optim4RL is a Jax framework of learning to optimize for reinforcement learning.

  reinforcement-learningoptimizationoptimizer+ 9reinforcement-learning-algorithmsoptimization-algorithmsmeta-learningjaxlearning-to-learnoptimizersmeta-learning-algorithms+ 2

  Python

  •

  Apache License 2.0

  •2•24•0•0•Updated Nov 27, 2024Nov 27, 2024

• VocabularyParallelism

  Public
  Vocabulary Parallelism

  Python

  •

  Other

  •2.6k•17•0•0•Updated Nov 11, 2024Nov 11, 2024

• sdft

  Public
  [ACL 2024] The official codebase for the paper "Self-Distillation Bridges Distribution Gap in Language Model Fine-tuning".

  language-modelself-distillationsupervised-finetuning

  Shell

  •4•110•4•0•Updated Nov 2, 2024Nov 2, 2024

• Cheating-LLM-Benchmarks

  Public
  [ICLR 2025] Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates (Oral)

  Jupyter Notebook

  •

  MIT License

  •0•69•0•0•Updated Oct 23, 2024Oct 23, 2024

• P-DoS

  Public
  [ArXiv 2024] Denial-of-Service Poisoning Attacks on Large Language Models

  Python

  •2•16•0•0•Updated Oct 22, 2024Oct 22, 2024

• CPO

  Public
  [NeurIPS 2024] The official implementation of paper: Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs.

  Python

  •4•95•3•1•Updated Oct 18, 2024Oct 18, 2024

• SimLayerKV

  Public
  The official implementation of paper: SimLayerKV: A Simple Framework for Layer-Level KV Cache Reduction.

  Python

  •0•42•3•0•Updated Oct 18, 2024Oct 18, 2024

• Attention-Sink

  Public
  [ICLR 2025] When Attention Sink Emerges in Language Models: An Empirical View (Spotlight)

  language-modelattention-mechanismlarge-language-models+ 1attention-sink

  Python

  •

  MIT License

  •1•49•0•0•Updated Oct 17, 2024Oct 17, 2024

• scaling-with-vocab

  Public
  [NeurIPS-2024] 📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623

  Python

  •5•76•1•0•Updated Sep 26, 2024Sep 26, 2024

• envpool

  Public
  C++-based high-performance parallel environment execution engine (vectorized env) for general RL environments.

  roboticsgymhigh-performance-computing+ 13cpp17box2dvizdoomparallel-processingthreadpoolpybind11atari-games+ 6

  C++

  •

  Apache License 2.0

  •106•1.1k•63•11•Updated Aug 12, 2024Aug 12, 2024

• dice

  Public
  Official implementation of Bootstrapping Language Models via DPO Implicit Rewards

  alignmentpreference-learninglarge-language-models+ 1rlhf

  Python

  •

  MIT License

  •3•42•0•0•Updated Jul 29, 2024Jul 29, 2024

• lorahub

  Public
  [COLM 2024] LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition

  Python

  •

  MIT License

  •36•614•3•1•Updated Jul 22, 2024Jul 22, 2024