📄️ Quick Start
Quick start CLI, Config, Docker
📄️ Proxy Config.yaml
Set model list, apibase, apikey, temperature & proxy server settings (master-key) on the config.yaml.
🔗 📖 All Endpoints
📄️ Use with Langchain, OpenAI SDK, Curl
Input, Output, Exceptions are mapped to the OpenAI format for all supported models
📄️ Virtual Keys
Track Spend, Set budgets and create virtual keys for the proxy
📄️ 💰 Budgets, Rate Limits per user
Requirements:
📄️ [BETA] Self-serve UI
Allow your users to create their own keys through a UI
📄️ Model Management
Add new models + Get model info without restarting proxy.
📄️ Health Checks
Use this to health check all LLMs defined in your config.yaml
🗃️ 🔥 Load Balancing
2 items
🗃️ Logging, Alerting, Caching
4 items
🗃️ Admin Controls
2 items
📄️ 🐳 Docker, Deploying LiteLLM Proxy
You can find the Dockerfile to build litellm proxy here
📄️ CLI Arguments
Cli arguments, --host, --port, --num_workers