LocalAI

Hugging Face's production-ready toolkit for deploying LLMs. Optimized for high throughput with tensor parallelism, quantization, and Flash Attention.

LLM inference in pure C/C++. Run LLaMA and other models on consumer hardware with CPU and GPU support. The engine behind many local AI apps.

Desktop app to discover, download, and run local LLMs. User-friendly GUI for running open-source models on your computer.