Text Generation Inference

Free, open source alternative to OpenAI API. Run LLMs, generate images, audio, and more locally or on-prem with no GPU required.

LLM inference in pure C/C++. Run LLaMA and other models on consumer hardware with CPU and GPU support. The engine behind many local AI apps.

Desktop app to discover, download, and run local LLMs. User-friendly GUI for running open-source models on your computer.