💡MLX Runs Trillion-Param Muse Spark Open on Mac Studio
AI on Mac? MLX makes it happen
TL;DR
MLX, an open-source array framework, now runs the trillion-parameter Muse Spark Open model on Apple silicon. This opens up new possibilities for AI research and development on Mac hardware.
MLX, an open-source array framework, just made it possible to run the trillion-parameter Muse Spark Open model on a Mac Studio. This is huge for AI researchers and developers looking to leverage Apple's hardware for cutting-edge work. Muse Spark Open runs at over 36 tokens per second on a cluster of four Mac Studios, showing the potential for local AI development. Key details: Muse Spark Open has a context size of 8,192 tokens and is evaluated in NVFP4 precision.
Key Points
MLX enables running Muse Spark Open, a trillion-parameter model, on Mac Studio hardware.
Muse Spark Open runs at over 36 tokens per second on a cluster of four Mac Studios.
The model's context size is 8,192 tokens, crucial for evaluating performance.
MLX supports pipeline parallelization over two Mac Studio computers for large models.
M5 Ultra chip delivers 1.2 TB/s of unified memory bandwidth, crucial for high-performance AI.
Why It Matters
If you're running AI models locally on Mac Studio, MLX now lets you handle massive models like Muse Spark Open. This is a game-changer for researchers needing high-performance AI on Apple hardware, but the context size and parallelization requirements mean it's not for everyone.
Comments
Be the first to comment
Enjoyed this article?
Get it daily. 7am. Free. Reads in 5 minutes.
Join 3,568 builders reading daily.