According to the developer’s claims, by eliminating the proxy server, caching layer, cost tracking, and dozens of features that many users do not use from Here is the comparison regarding the presence or absence of features between On the other hand, operation is confirmed (Verified: Yes) for OpenAI, Anthropic, Groq, Mistral, xAI, OpenRouter, and Azure.<\/p>\n This project is currently in Alpha status. According to reports from the developer, the following verifications have been conducted:<\/p>\n It can be installed using The API mirrors Provider errors are mapped to By specifying litelm<\/code>, a lightweight LLM routing library, has been released. This library aims to extract only the core features\u2014model routing and message format translation\u2014from the existing litellm<\/code>, minimizing code size and dependencies.<\/p>\nClaims and Rationale<\/h2>\n
litellm<\/code>, litelm<\/code> is composed of only about 2,900 lines of code and two dependencies (openai<\/code>, httpx<\/code>).<\/p>\nFeature Comparison<\/h3>\n
litellm<\/code> and litelm<\/code>.<\/p>\n\n\n
\n \nFeature<\/th>\n litellm<\/th>\n litelm<\/th>\n<\/tr>\n<\/thead>\n \n Model routing (provider\/model \u2192 right endpoint)<\/td>\n \u2713<\/td>\n \u2713<\/td>\n<\/tr>\n \n Message translation (Anthropic, Bedrock, Cloudflare, Mistral)<\/td>\n \u2713<\/td>\n \u2713<\/td>\n<\/tr>\n \n Streaming + stream_chunk_builder<\/td>\n \u2713<\/td>\n \u2713<\/td>\n<\/tr>\n \n Tool use (function calling)<\/td>\n \u2713<\/td>\n \u2713<\/td>\n<\/tr>\n \n Embeddings<\/td>\n \u2713<\/td>\n \u2713<\/td>\n<\/tr>\n \n Text completions<\/td>\n \u2713<\/td>\n \u2713<\/td>\n<\/tr>\n \n OpenAI Responses API<\/td>\n \u2713<\/td>\n \u2713<\/td>\n<\/tr>\n \n Mock responses<\/td>\n \u2713<\/td>\n \u2713<\/td>\n<\/tr>\n \n Router (load balancing, fallbacks)<\/td>\n \u2713<\/td>\n \u2717<\/td>\n<\/tr>\n \n Proxy server<\/td>\n \u2713<\/td>\n \u2717<\/td>\n<\/tr>\n \n Caching \/ budgeting \/ cost tracking<\/td>\n \u2713<\/td>\n \u2717<\/td>\n<\/tr>\n \n Token counting<\/td>\n \u2713<\/td>\n \u2717<\/td>\n<\/tr>\n \n Image gen, audio, OCR, fine-tuning<\/td>\n \u2713<\/td>\n \u2717<\/td>\n<\/tr>\n \n Agents, guardrails, scheduler<\/td>\n \u2713<\/td>\n \u2717<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n Provider Support Status<\/h3>\n
litelm<\/code> supports routing to 19 providers using the syntax “provider\/model-name”. However, the documentation reports that operation is unverified (Verified: No) for the following providers:<\/p>\n\n
Development Status and Verification<\/h3>\n
\n
Hardware Requirements<\/h2>\n
\n
openai<\/code>, httpx<\/code> (standard installation)<\/li>\nanthropic<\/code>, boto3<\/code>, etc.)<\/li>\n<\/ul>\nHow to Get It<\/h2>\n
pip<\/code>.<\/p>\npip install litelm # openai + httpx\npip install litelm[anthropic] # + anthropic SDK\npip install litelm[bedrock] # + boto3\npip install litelm[all] # everything\n<\/code><\/pre>\nBasic Usage<\/h3>\n
litellm<\/code>, sharing function names, arguments, and response types. Asynchronous versions (acompletion<\/code>, aembedding<\/code>, aresponses<\/code>, atext_completion<\/code>) are provided for all functions.<\/p>\nimport litelm\n\n# Basic Completion\nresponse = litelm.completion("openai\/gpt-4o", messages=[{"role": "user", "content": "Hello!"}])\nprint(response.choices[0].message.content)\n\n# Streaming\nfor chunk in litelm.completion("groq\/llama-3.1-70b-versatile", messages=[...], stream=True):\n print(chunk.choices[0].delta.content or "", end="")\n\n# Embeddings\nresponse = litelm.embedding("openai\/text-embedding-3-small", input=["hello world"])\n<\/code><\/pre>\nError Handling<\/h3>\n
litelm<\/code>‘s exception hierarchy.<\/p>\nfrom litelm import ContextWindowExceededError, RateLimitError, AuthenticationError\n\ntry:\n response = litelm.completion("openai\/gpt-4o", messages=messages)\nexcept ContextWindowExceededError:\n # Handling when the prompt is too long\n pass\nexcept RateLimitError:\n # Handling rate limits\n pass\nexcept AuthenticationError:\n # Handling invalid API keys\n pass\n<\/code><\/pre>\nLocal and Custom Providers<\/h3>\n
api_base<\/code>, any server with an OpenAI-compatible endpoint can be used.<\/p>\n# vLLM\nlitelm.completion("openai\/my-model", messages=[...], api_base="http:\/\/localhost:8000\/v1")\n\n# Ollama\nlitelm.completion("ollama\/llama3", messages=[...], api_base="http:\/\/localhost:11434\/v1")\n\n# LM Studio\nlitelm.completion("openai\/local-model", messages=[...], api_base="http:\/\/localhost:1234\/v1")\n<\/code><\/pre>\nUsing Tool Calling<\/h3>\n
tools = [{\n "type": "function",\n "function": {\n "name": "get_weather",\n "parameters": {"type": "object", "properties": {"city": {"type": "string"}}},\n }\n}]\n\nresponse = litelm.completion(\n "openai\/gpt-4o",\n messages=[{"role": "user", "content": "Weather in Paris?"}],\n tools=tools,\n tool_choice="required",\n)\n\ntool_call = response.choices[0].message.tool_calls[0]\nprint(tool_call.function.name, tool_call.function.arguments)\n<\/code><\/pre>\nWhat the Sources Do Not Cover<\/h2>\n