Mistral.rs
Mistral.rs is a fast, flexible LLM inference engine written in Rust. It supports any Hugging Face model with zero configuration, true multimodality (text, vision, video, audio, speech, image generation, embeddings), full quantization control (ISQ, GGUF, GPTQ, AWQ, HQQ, FP8, BNB), built-in web UI, hardware-aware tuning, and flexible SDKs (Python and Rust). It features agentic capabilities like server-side tool loops, web search, MCP client, and HTTP tool dispatch. Performance optimizations include continuous batching, FlashAttention, PagedAttention, and multi-GPU tensor parallelism.








