What is LiteLLM?
LiteLLM is a lightweight, open-source abstraction layer for large language models—designed to unify how developers interact with dozens of LLM APIs. Rather than writing custom integration code for each provider (OpenAI, Anthropic, Gemini, Mistral, Ollama, and more), LiteLLM delivers a single, consistent Python interface for completions, embeddings, streaming, retries, fallbacks, and logging—accelerating prototyping, testing, and production deployment.
How to use LiteLLM?
Getting started takes seconds: install via pip install litellm, set your API keys as environment variables (e.g., OPENAI_API_KEY, ANTHROPIC_API_KEY, or OLLAMA_BASE_URL), then call litellm.completion() or litellm.embedding() with any supported model name—like "gpt-4o", "claude-3-haiku", or "llama3". No model-specific SDKs required. Its built-in playground lets you rapidly experiment across providers side-by-side, visualize latency/cost tradeoffs, and validate prompts before integrating into your application stack.