MLX Community Projects #654
Replies: 24 comments 11 replies
-
|
Text generation: mlx-tuning-fork |
Beta Was this translation helpful? Give feedback.
-
|
text generation: https://github.com/mzbac/mlx-moe-models |
Beta Was this translation helpful? Give feedback.
-
|
An implementation of Reinforcement Learning algorithms in MLX based in the Implementations from CleanRL. Still WIP because it’s missing a benchmark and some other minor things, but the implementations work correctly. |
Beta Was this translation helpful? Give feedback.
-
|
mlx-models. Currently supporting vision models by loading/converting from PyTorch checkpoints. Will later add support for text and audio models as well. |
Beta Was this translation helpful? Give feedback.
-
|
Could you share a bit more about what it does? |
Beta Was this translation helpful? Give feedback.
-
|
Hi I would love to add chat-with-mlx. It is a Chat UI + RAG Implementation on MLX. I wIll add more features later on (more advanced RAG pipeline + multimodal) |
Beta Was this translation helpful? Give feedback.
-
|
I have an example of training a simple language model using BitLinear instead of nn.Linear. It's a port of Karpathy's minGPT to MLX along with a custom implementation of a BitLinear module. https://github.com/adhulipa/mlx-mingpt I noticed this collection already has the far more meatier |
Beta Was this translation helpful? Give feedback.
-
|
Transformer Lab https://github.com/transformerlab/transformerlab-app is an LLM research platform that allows you to run, train, perform RAG, and evaluate LLMs through a GUI. |
Beta Was this translation helpful? Give feedback.
-
|
MLX RAG with GGUF Models: https://github.com/Jaykef/mlx-rag-gguf The code here builds on https://github.com/vegaluisjose/mlx-rag, it has been optimized to support RAG-based inferencing for .gguf models. I am using BAAI/bge-small-en for the embedding model, TinyLlama-1.1B-Chat-v1.0-GGUF as base model and the custom vector database script for indexing texts in a pdf file. Inference speeds can go up to ~413 tokens/sec for prompts and ~36 tokens/sec for generation on my 8G M2 Air. |
Beta Was this translation helpful? Give feedback.
-
|
@Jaykef Very cool, thanks for sharing |
Beta Was this translation helpful? Give feedback.
-
|
Vision: MLX3D A library for deep learning with 3D data using mlx. |
Beta Was this translation helpful? Give feedback.
-
|
@awni |
Beta Was this translation helpful? Give feedback.
-
|
very cool! thanks for working on the 3D support |
Beta Was this translation helpful? Give feedback.
-
|
Done! Cool library btw! |
Beta Was this translation helpful? Give feedback.
-
|
JSON schema decoding (allowing function calling, including an OpenAI-compatible server with tools) using MLX: https://github.com/otriscon/llm-structured-output |
Beta Was this translation helpful? Give feedback.
-
|
Hello for text generation part, I'm happy to share with you that I've proposed and contributed to the integration of MLX with LibreChat.ai. So now you can use your local LLM powered by MLX through a fancy interface privately, enjoy! :D See danny-avila/LibreChat#2580 If in the future the community proposes an API servers supporting also multimodality, transcription, image generation for example, I will add them into LibreChat ;) It could be great also to have and LLM API supporting /models endpoint and multiple models simultaneously :D
|
Beta Was this translation helpful? Give feedback.
-
|
Awesome!! |
Beta Was this translation helpful? Give feedback.
-
|
Hello, mlx community, we are happy to share with you that we have contributed the first strong sub-4 bit LLM model zoo for MLX community.
The modern LLM families include Llama3/2, Phi-3, Mistral, 01-Yi, and Qwen. A mlx-style inference toolkit is also shared for the local web chatting.
We are an active team here, supporting the better low-bit community on the local platform. Enjoy! |
Beta Was this translation helpful? Give feedback.
-
|
This looks really interesting! I'm curious if there is somewhere one can learn more about how you reduce the precision of the models? |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for your interest! The MLX models are a subset of our main work green-bit-llm. We construct the lower-bit models by combining neural architecture search and post-training quantization technique. All models are built from a mix precision of 4/2 bit group-wise min-max quantization only (for better large-scale deployment, e.g., MLX only supports 4/2 bit min-max quantization currently). The MLX models in HF collection is the layer-mix version in the model zoo. Except better but not harder inference, we are also interested in the low-cost fine-tuning. We released Bitorch Engine for low-bit quantized neural network operations. Our release supports full parameter fine-tuning directly in quantized space, even under extremely constrained GPU resource conditions. We are preparing a blog in huggingface for a better understanding of everything. Please stay tuned and star us in git if you like our project QvQ. |
Beta Was this translation helpful? Give feedback.
-
Heck yeah, this is truly awesome! I've been banging my head trying to get a 3-bit Omniquant model to work but this'll be so much better. I'd love to see a low-bit precision Wizard LM2-8x22B (2.2, 2.5, 3.0 range)! It'd also be great to see some metrics versus exllama/QUIP/AQLM as far as speed/accuracy. I'm (trying) to run an AQLM model right now and I believe it is still considered SotA but its about as much degradation as I'm willing to deal with. IIRC -10 to -15 (so say 78-80% Llama3 -> 65-68% accuracy). In any case, I'll be giving your project an updoot and following it |
Beta Was this translation helpful? Give feedback.
-
|
mlx_micrograd - mlx port of Karpathy's micrograd - a tiny scalar-valued autograd engine with a small PyTorch-like neural network library on top. Installationpip install mlx_microgradExample usageExample showing a number of possible supported operations: from mlx_micrograd.engine import Value
a = Value(-4.0)
b = Value(2.0)
c = a + b
d = a * b + b**3
c += c + 1
c += 1 + c + (-a)
d += d * 2 + (b + a).relu()
d += 3 * d + (b - a).relu()
e = c - d
f = e**2
g = f / 2.0
g += 10.0 / f
print(f'{g.data}') # prints array(24.7041, dtype=float32), the outcome of this forward pass
g.backward()
print(f'{a.grad}') # prints array(138.834, dtype=float32), i.e. the numerical value of dg/da
print(f'{b.grad}') # prints array(645.577, dtype=float32), i.e. the numerical value of dg/db
|
Beta Was this translation helpful? Give feedback.
-
|
@awni this should be under Educational. |
Beta Was this translation helpful? Give feedback.
-
|
This one is a little stale, but I've taken the approach used for adding LoRA to LLMs and applied it to LlaVA in mlx-examples Can use this as a starting point for fine tuning VLMs as datasets get more popular, like https://huggingface.co/datasets/HuggingFaceM4/the_cauldron |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
Hi, I wanted to share my project here - MLX Hub |
Beta Was this translation helpful? Give feedback.
-
|
I noticed the project description I provided in my previous comment looks a bit too verbose. Do you mind updating it to this one - mlx-hub - "A command-line tool to search, download & manage MLX AI models on macOS." Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
🤖✨ChatMLX is a modern, open-source, high-performance chat application for MacOS based on large language models. Features 🚀
output.mp4 |
Beta Was this translation helpful? Give feedback.
-
|
aggressor: A simplest possible implementation of Autoregressive Image Generation without Vector Quantization in Apple MLX. |
Beta Was this translation helpful? Give feedback.
-
|
DINO_DETR_MLX : Port of the DINO DETR model for object detection in MLX. API to load pre-trained PyTorch model weights, training/fine-tuning and evaluation using COCO API. This implementation uses Data Loader from torchvision.datasets and also provides a simple custom data loader. Also added a synthetic dataset to run profiler for time/memory cost analysis without the need to download COCO dataset. Please feel free to open an issue / pull request or start a discussion. |
Beta Was this translation helpful? Give feedback.
-
|
e2tts-mlx: A single-file implementation of Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTS model in MLX. |
Beta Was this translation helpful? Give feedback.
-
|
whisper-turbo-mlx: A blazing fast single-file implementation of the openai's Whisper Turbo (all in less than 250 lines of code) |
Beta Was this translation helpful? Give feedback.
-
|
This is so rad! |
Beta Was this translation helpful? Give feedback.
-
|
Thanks so much! I really appreciate it! |
Beta Was this translation helpful? Give feedback.
and others
-
Let's collect some cool MLX integrations and community lead projects here for visibility!
If you have a project you would like to feature, leave a comment, and we will add it.
Text Generation
Vision
Speech and Audio
Multi-modal
Misc
Educational
picoGPTMLX Swift
Beta Was this translation helpful? Give feedback.
All reactions