Skip to content

InferenceBerriAI

LiteLLM

AI gateway: call many LLMs through one OpenAI-format interface.

Category
Inference
Pricing
FREEMIUM
Source
Open core
Hosting
Hybrid
Platforms
APIWebCLI
Models
Multi-model
Verified
Jun 7, 2026

Open-source Python SDK and proxy server (AI gateway) that exposes 100+ LLM providers through a single OpenAI-compatible API, with cost tracking, load balancing, fallbacks, caching, and guardrails. Self-host the proxy or use the managed cloud; a paid Enterprise tier adds SSO, audit logs, and support.

Capabilities 4

What it actually does — grouped by capability family.

  • LLM gateway / routing (primary capability)
  • Multi-model access (primary capability)
  • Guardrails (secondary capability)
  • LLM observability (secondary capability)

Pros & cons

  • Load balancing and guardrails built in
  • Open source SDK + proxy
  • Cost tracking, fallbacks, caching
  • Self-host or managed cloud
  • Proxy adds an extra hop
  • Enterprise features are paid
  • Operational upkeep self-hosted

Tags

Alternatives to LiteLLM (4)

View all Inference →
  • View OpenRouter details
    InferenceFREEMIUM

    OpenRouter

    OpenRouter

    One OpenAI-compatible API in front of models from every provider.

    A unified gateway that routes a single endpoint and API key to models from Anthropic, OpenAI, Google, Meta, DeepSeek, xAI, and more — swap models by changing one parameter, with automatic fallbacks and one consolidated bill. Pass-through token pricing plus dozens of free models.

    Swap models by changing one parameter
    Adds a routing hop vs direct provider
    • gateway
    • routing
    • multi-model
    • fallbacks
    Open
  • View Portkey details
    InferenceFREEMIUMOpen core

    Portkey

    Portkey

    AI gateway with observability, guardrails, and governance.

    A production AI gateway that gives apps and agents unified access to 1,600+ LLMs across providers behind a single API, with built-in observability, prompt management, guardrails, and governance. Portkey adds routing, caching, fallbacks, cost limits, PII redaction, RBAC, and an MCP gateway. Its core gateway is open-source; run it self-hosted/hybrid or use the managed cloud, which offers a free tier.

    One API across many providers
    Acquired by Palo Alto Networks (closed 2025)
    • ai-gateway
    • llm-routing
    • observability
    • guardrails
    Open
  • View Vercel AI SDK details
    OrchestrationFREEOSS

    Vercel AI SDK

    Vercel

    TypeScript SDK for streaming, tool-calling, and structured outputs.

    Vercel's batteries-included TypeScript framework for LLM-powered apps. Streaming primitives, structured outputs, tool calling, and React hooks for chat UIs — works with every major provider out of the box.

    Provider-agnostic, swap models in one line
    TypeScript/JS ecosystem only
    • typescript
    • streaming
    • tool-use
    • structured-output
    Open
  • View Ollama details
    InferenceFREEMIUMOpen core

    Ollama

    Ollama

    Run open-weight LLMs locally with one command. OpenAI-compatible API.

    The de-facto way to pull and run open-weight models (Llama, Qwen, Gemma, DeepSeek, gpt-oss) on your own machine — no API key, no data leaving the device. Ships native macOS/Windows/Linux apps, an OpenAI-compatible server, and official Python/JS libraries. MIT-licensed and free locally; an optional paid Ollama Cloud runs larger models.

    One-command pull-and-run
    Local performance bound by your hardware
    • local
    • open-source
    • llm-runner
    • self-hosted
    Open
Open LiteLLM