OpenRouter vs LiteLLM vs Ollama: Which One Should You Use?

Hello builders,

You can shop the entire cheap tier in one afternoon instead of one vendor a quarter, and walk out with a ranked shortlist that came off your own tasks rather than somebody’s upvotes. The openrouter vs litellm vs ollama argument gets fought as though picking one settles anything, and it doesn’t, because all three answer the same question and it isn’t the question you have. They answer reachability: how do I call a lot of models without writing a lot of clients. What you actually have is a selection problem, and a gateway hands you three hundred models and no opinion.

You have met the short version. Under a post titled “How to Control LLM API Costs?”, the top comment reads, in full:

Pretty sure it exists and it’s called OpenRouter.

That’s a real answer to a question nobody asked. OpenRouter is a good product and we are going to wire it up. It just cannot tell you which of your requests to send there.

Three shapes, one interface

There are exactly three shapes, and our choice between them is less about which model is best than about what we will operate.

Run it yourself. Ollama takes the short path: its OpenAI-compatible base URL is http://localhost:11434/v1/, and the api_key the client demands is ignored by the server, so put any string there. vLLM takes the throughput path, one server per model with vllm serve <model>, listening on http://localhost:8000. You operate hardware, a model file and a runtime, and your data never leaves the building.

Go through a gateway. LiteLLM self-hosts, needs Python 3.10 or newer, and listens on port 4000, where you call it exactly like the OpenAI API. OpenRouter is the hosted version of the same idea. You operate an API key.

Rent capacity from somebody hosting open weights. Also an API key, and the biggest price gap on the menu.

Notice the column that table is missing. It has price, latency, and what you operate. It has no column for passes my tasks, and nobody but you can fill that in. Every other comparison table you will find online is this table with a made-up quality column bolted on.

One function, every provider

All three speak the OpenAI chat-completions shape, which is the quiet reason our adapter is twenty lines and not a project.

# providers.py - every model on the menu behind one call.
import os
from openai import OpenAI

# Local endpoints are documented and fixed, so they are hardcoded. Hosted base
# URLs come from env vars on purpose: read them off the provider's quickstart.
# A URL typed from memory is a URL that goes stale.
PROVIDERS = {
    "ollama":  dict(base_url="http://localhost:11434/v1/", api_key="ollama"),
    "vllm":    dict(base_url="http://localhost:8000/v1",
                    api_key=os.environ.get("VLLM_API_KEY", "EMPTY")),
    "litellm": dict(base_url=os.environ.get("LITELLM_URL", "http://0.0.0.0:4000"),
                    api_key=os.environ.get("LITELLM_KEY", "sk-anything")),
    "hosted":  dict(base_url=os.environ.get("HOSTED_URL"),
                    api_key=os.environ.get("HOSTED_API_KEY")),
}

# your alias -> (provider, the id THAT provider expects)
CATALOG = {"local-small": ("ollama", os.environ.get("OLLAMA_MODEL", "qwen3:8b")),
           "gateway":     ("litellm", os.environ.get("GATEWAY_MODEL", "")),
           "ds-flash":    ("hosted", "deepseek-v4-flash")}

_clients = {}

def complete(alias, messages, max_tokens=2000, **kw):
    """The single call site. Swapping providers is now one string."""
    provider, model_id = CATALOG[alias]
    cfg = PROVIDERS[provider]
    if not cfg.get("base_url"):
        raise RuntimeError(f"{provider}: set the base_url env var, do not guess it")
    if provider not in _clients:
        _clients[provider] = OpenAI(base_url=cfg["base_url"],
                                    api_key=cfg["api_key"] or "EMPTY")
    return _clients[provider].chat.completions.create(
        model=model_id, messages=messages, max_tokens=max_tokens, **kw)

Prove each endpoint is up before you debug a single client against it, then send one real call through:

curl http://localhost:11434/v1/models      # ollama
curl http://localhost:8000/v1/models       # vllm
curl http://0.0.0.0:4000/v1/models         # litellm

>>> complete("local-small", [{"role": "user", "content": "ping"}]).choices[0].message.content
'Pong.'

Three filters before price

Before we spend an afternoon wiring providers, we spend twenty minutes eliminating the ones that cannot work for us at all. None of these filters is about money, and testing a model that fails one is measuring something you cannot buy.

Does it fit? We already have a size distribution for our own prompts, which most people do not. Read the p95, not the median, because your biggest inputs correlate with your hardest problems. Then watch for the failure that costs money instead of erroring: some stacks silently truncate an over-long prompt rather than refusing it, the model answers confidently about the half it received, and your pass test may wave it through. If a candidate’s limit sits near your p95, send one deliberately oversized input and confirm you get an error rather than a plausible answer.

Where does your data go? Is it used for training, can you turn that off, what is the retention period, and does that jurisdiction work for contracts you already signed. Read the terms, not a forum. This is the one piece of homework where a wrong answer gets genuinely expensive, and it is the strongest argument for the local option, which has nothing to do with tokens.

Can it take your volume? Price per token is irrelevant if the provider will not accept your requests. OpenRouter is plain about both halves of this:

“We pass through the pricing of the underlying providers; there is no markup on inference pricing (however we do charge a fee when purchasing credits).”

Free models are capped at 50 requests per day under 10 credits, rising to 1,000 per day at 10 credits or more (OpenRouter FAQ). Fifty a day is a hard stop that arrives before lunch. Paid tiers run far higher, but the shape generalises: at volume, rate limits are usually the binding constraint, not price. It is why somebody posts that small models “are not that performant” when what happened is they hit a throughput ceiling and never reached a quality question.

Now let’s do the arithmetic, because the gap is real and badly understated by the “one eighth to one tenth” figure people repeat. Against a frontier model at $5 per million in and $25 out:

input:   $5  / $0.14   =  35.7x cheaper
         $5  / $0.435  =  11.5x cheaper
output:  $25 / $0.28   =  89.3x cheaper
         $25 / $0.87   =  28.7x cheaper

Eleven times to eighty-nine times, depending on the model and the direction, and output usually dominates a bill. But an 89x per-token discount is not an 89x bill reduction. Retries, verbosity and extra turns eat some of that gap before it reaches the invoice, and the only way to learn how much is to score each candidate on our own work.

So wire all three, run the same test against each by changing one word, and rank them per task type, because the model that wins your changelog will not necessarily win your code review.

Now go build something this weekend!

John Cook