What it is
MLX is an array framework from Apple's machine learning research group, built for the unified memory of Apple silicon, where the CPU and GPU share one pool of RAM. Its Python API follows NumPy, its nn and optimizers packages follow PyTorch, and C++, C and Swift APIs mirror it. Computation is lazy and graphs are built dynamically, so changing input shapes does not trigger recompilation.
It is no longer Mac-only. `pip install mlx[cuda12]` (or `mlx[cuda13]`) runs it on Linux with Nvidia GPUs of SM 7.5 or newer, and `mlx[cpu]` gives a CPU-only Linux build. Most people meet it through mlx-lm, the companion package for running, quantising and LoRA fine-tuning LLMs from the Hugging Face Hub, where the mlx-community organisation hosts about 5,800 converted models.
Version 0.32.4 was released on 14 October 2026 under the MIT licence.
Our review
MLX matters because of one hardware fact: on Apple silicon the GPU can address the whole of system memory. A Mac with a lot of RAM can hold a quantised model that would need a data-centre GPU elsewhere, and MLX is written around that, with arrays that live in shared memory and run on CPU or GPU without copies. The repo has about 28,900 GitHub stars and released 0.32.4 on 14 October 2026.
Most of the practical value sits in mlx-lm (about 7,400 stars) rather than the core library. It loads models straight from the Hugging Face Hub, quantises and uploads them, serves them over HTTP with mlx_lm.server, and runs LoRA and QLoRA fine-tuning. Its own docs give a concrete figure: LoRA on Mistral 7B with batch size 1 and four tuned layers runs at about 250 tokens a second on an M1 Max with 32 GB.
The limits are the Mac's. Models close to the size of RAM run slowly unless you raise the GPU wired-memory limit with a sysctl, which needs macOS 15; the base package needs macOS 14. Open issues include Metal's GPU watchdog killing LoRA training while a display is active and a kernel panic on an M4 Max during a 173K-token prefill. Two low-severity CVEs in the weight loaders, a heap overflow in load() and a wild pointer in load_gguf(), were published in November 2025.
The CUDA backend widens the audience less than it sounds. It needs Linux with glibc 2.35 or newer and an Nvidia card of SM 7.5 or later, and its real use is letting a team write MLX code on Macs and run the same code on a GPU server. It does not bring the vLLM and Hugging Face training stack that PyTorch has on that hardware, and ROCm support for AMD is still an open experiment. For inference alone, llama.cpp runs on everything and Ollama hides the setup; MLX earns its place when you want Python-level control on the Mac itself.
Where it earns its place
An ML engineer at a support-software start-up with a 32 GB M-series MacBook Pro
Fine-tune a 7B model on support transcripts without renting a GPU
Run mlx_lm.lora with batch size 1, a few tuned layers and gradient checkpointing; the docs report about 250 tokens a second for Mistral 7B on a 32 GB M1 Max, so a modest dataset trains on the laptop. The cost is time on a machine already owned; the risk is the open bug where the Metal watchdog kills training while a display is active.
A university researcher prototyping new model architectures
Iterate on custom layers without waiting on recompilation
Write models with mlx.nn's PyTorch-like modules; graphs are dynamic, so shape changes do not recompile, and composable transforms give autodiff and vectorisation. The same code can later run on a Linux GPU box with mlx[cuda12], though large-scale training still points to PyTorch.
An IT lead at a 20-person law firm with strict confidentiality rules
Run a private chat model on office Macs instead of sending documents to a cloud API
Pull a 4-bit model from mlx-community and serve it with mlx_lm.server on a high-memory Mac, or through a desktop app that uses MLX as its engine. Nothing leaves the machine and the MIT licence costs nothing; the spend is the hardware and someone to maintain it.
Before you commit
- Setup
- `pip install mlx mlx-lm` on a native arm64 Python 3.10+ with macOS 14+; a first mlx_lm.generate from a Hub model is mostly download time. Linux uses mlx[cuda12], mlx[cuda13] or mlx[cpu]; a non-native (Rosetta) Python is the documented install pitfall.
- Integrations
- Hugging Face Hub for loading, converting, quantising and uploading; safetensors and GGUF loading; mx.distributed for multi-machine inference and fine-tuning; Swift bindings for apps; used as a backend by several Mac chat apps.
- Data and privacy
- Runs locally; the README describes no telemetry, and network access is for Hub downloads. MIT licence, so commercial use is unrestricted, though model licences still apply. CVE-2025-62608 and CVE-2025-62609, both low severity, affected the weight loaders.
- Cost at scale
- The software is free; the bill is Apple hardware, and RAM size rather than GPU class decides which models fit. Going past one Mac means mx.distributed across several machines, and an open issue reports out-of-memory failures when those machines have different amounts of RAM.
- Limits
- The CUDA build needs SM 7.5+ and glibc 2.35+; AMD is experimental. Models near RAM size are slow without raising iogpu.wired_limit_mb. Open issues include the Metal watchdog stopping LoRA runs and a kernel panic on very long prefills.
What's good
- GPU can use all of a Mac's unified memory
- NumPy-style Python API plus C++, C and Swift
- LoRA and QLoRA via mlx-lm, about 250 tok/s on an M1 Max per the docs
- About 5,800 converted models on mlx-community
What's not
- CUDA build lacks PyTorch's surrounding ecosystem
- A library, not an app: you write Python or use a front end
- Large models need macOS 15 and a sysctl tweak
- Open Metal watchdog and kernel-panic bugs on long jobs
Best for
Mac-based developers and researchers who want native speed and Python control on Apple silicon.
Alternatives
MLX: common questions
- Is MLX worth it?
- MLX turns a high-memory Mac into a credible box for local inference and LoRA tuning; off Apple silicon it is a port, not a reason to switch. We rate it 4.4 out of 5.
- How much does MLX cost?
- Free and open source under the MIT license.
- What are the downsides of MLX?
- CUDA build lacks PyTorch's surrounding ecosystem; a library, not an app: you write Python or use a front end; large models need macOS 15 and a sysctl tweak; open Metal watchdog and kernel-panic bugs on long jobs.
- Who is MLX best for?
- Mac-based developers and researchers who want native speed and Python control on Apple silicon.
- How much does MLX cost for a team or in production?
- The software is free; the bill is Apple hardware, and RAM size rather than GPU class decides which models fit. Going past one Mac means mx.distributed across several machines, and an open issue reports out-of-memory failures when those machines have different amounts of RAM.
- Is MLX safe for company data?
- Runs locally; the README describes no telemetry, and network access is for Hub downloads. MIT licence, so commercial use is unrestricted, though model licences still apply. CVE-2025-62608 and CVE-2025-62609, both low severity, affected the weight loaders.
- What are the best MLX alternatives?
- The closest alternatives we have reviewed are llama.cpp and Ollama.
Answers checked Oct 23, 2026.
Make MLX? Show this rating on your site
