litelm: Lightweight LLM Routing Library Released

September 12, 2026September 20, 2026

Overview

litelm, a lightweight LLM routing library, has been released. This library aims to extract only the core features—model routing and message format translation—from the existing litellm, minimizing code size and dependencies.

Claims and Rationale

According to the developer’s claims, by eliminating the proxy server, caching layer, cost tracking, and dozens of features that many users do not use from litellm, litelm is composed of only about 2,900 lines of code and two dependencies (openai, httpx).

Feature Comparison

Here is the comparison regarding the presence or absence of features between litellm and litelm.

Provider Support Status

litelm supports routing to 19 providers using the syntax “provider/model-name". However, the documentation reports that operation is unverified (Verified: No) for the following providers:

• Bedrock, Cloudflare, Together, Fireworks, DeepSeek, Perplexity, DeepInfra, Gemini, Cohere, Ollama, vLLM, LM Studio

On the other hand, operation is confirmed (Verified: Yes) for OpenAI, Anthropic, Groq, Mistral, xAI, OpenRouter, and Azure.

Development Status and Verification

This project is currently in Alpha status. According to reports from the developer, the following verifications have been conducted:

• Reviewed 360 core path commits and fixed compatibility gaps.
• Local scope tests: 262 passed, 55 skipped.
• All 45 available provider live tests passed.
• All 10 DSPy smoke tests passed.
• DSPy drop-in compatibility (Predict, CoT, typed signatures, streaming, embeddings, tool use, multi-output) has also been verified.

Hardware Requirements

• Language: Python
• Dependencies: openai, httpx (standard installation)
• Additional SDKs as needed (anthropic, boto3, etc.)

How to Get It

It can be installed using pip.

pip install litelm                # openai + httpx
pip install litelm[anthropic]     # + anthropic SDK
pip install litelm[bedrock]       # + boto3
pip install litelm[all]           # everything

Basic Usage

The API mirrors litellm, sharing function names, arguments, and response types. Asynchronous versions (acompletion, aembedding, aresponses, atext_completion) are provided for all functions.

import litelm

# Basic Completion
response = litelm.completion("openai/gpt-4o", messages=[{"role": "user", "content": "Hello!"}])
print(response.choices[0].message.content)

# Streaming
for chunk in litelm.completion("groq/llama-3.1-70b-versatile", messages=[...], stream=True):
    print(chunk.choices[0].delta.content or "", end="")

# Embeddings
response = litelm.embedding("openai/text-embedding-3-small", input=["hello world"])

Error Handling

Provider errors are mapped to litelm's exception hierarchy.

from litelm import ContextWindowExceededError, RateLimitError, AuthenticationError

try:
    response = litelm.completion("openai/gpt-4o", messages=messages)
except ContextWindowExceededError:
    # Handling when the prompt is too long
    pass
except RateLimitError:
    # Handling rate limits
    pass
except AuthenticationError:
    # Handling invalid API keys
    pass

Local and Custom Providers

By specifying api_base, any server with an OpenAI-compatible endpoint can be used.

# vLLM
litelm.completion("openai/my-model", messages=[...], api_base="http://localhost:8000/v1")

# Ollama
litelm.completion("ollama/llama3", messages=[...], api_base="http://localhost:11434/v1")

# LM Studio
litelm.completion("openai/local-model", messages=[...], api_base="http://localhost:1234/v1")

Using Tool Calling

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "parameters": {"type": "object", "properties": {"city": {"type": "string"}}},
    }
}]

response = litelm.completion(
    "openai/gpt-4o",
    messages=[{"role": "user", "content": "Weather in Paris?"}],
    tools=tools,
    tool_choice="required",
)

tool_call = response.choices[0].message.tool_calls[0]
print(tool_call.function.name, tool_call.function.arguments)

What the Sources Do Not Cover

• Specific figures regarding improvements in latency or memory usage compared to litellm.
• Specific operational stability for providers reported as unverified (Verified: No).

Sources

• https://github.com/kennethwolters/litelm

Update History

• 2026-09-19: Rewrote the article from re-collected sources and restored it from draft to published.