# LiteLLM

> Puts one uniform interface in front of many model providers, so keys, cost, limits and logs sit in one place instead of scattered across every service that calls a model.

- Vendor: Berrie AI Incorporated (BerriAI)
- Canonical URL: https://www.convios.com/en/toolbox/litellm
- Language version: https://www.convios.com/de/werkzeugkasten/litellm
- Area: Governance & security · Cluster: model gateway
- Role: Building block · Origin: AI-native
- As of: 2026-07-30 · Reviewed: 2026-07-30 · Author: Dr. Oliver Gausmann, Convios GmbH
- Toolbox: https://www.convios.com/en/toolbox — Markdown: https://www.convios.com/en/toolbox.md

## Verdict

Prevents falling behind: A model gateway makes nobody distinguishable. Without one you lose sight of spend and access, you cannot swap a model when price or quality shifts, and you have no evidence when someone audits you. With one you have merely removed the risk of falling behind. The edge comes from the applications on top, not from the routing layer.

## Suitability by company size

- Solo: suitable with caveats — The library pays off immediately. Running the proxy with its own database, updates and security upkeep is usually more work than a single application justifies for one person.
- Mid-market: suitable — From the second team that calls models onward, the proxy is cheaper than scattered keys. Self-hosting costs no licence, only operating time, and the credentials stay in house.
- Enterprise: suitable with caveats — Directory based sign-in, audit logs and access controls sit behind the paid edition, whose price is not published. On top of that the vendor is small, and the compliance reports are only handed out under contract.

## Vendor staying power

Young: Three years in market, a small team and a funding round that is tiny relative to how widely the software is used. Adoption of the code is the strongest signal here, not the vendor's balance sheet.

- Founded and cohort: 2023, Y Combinator Winter 2023, founders Krrish Dholakia and Ishaan Jaffer (source: https://www.ycombinator.com/companies/litellm, as of 2026-07-30)
- Funding: USD 1.6 million seed funding (Y Combinator, Gravity Fund, Pioneer Fund); higher figures circulate in databases without vendor confirmation (source: https://www.ycombinator.com/companies/litellm, as of 2026-07-30)
- Team size: 10 people according to the company's own entry in the Y Combinator directory (source: https://www.ycombinator.com/companies/litellm, as of 2026-07-30)
- Code adoption: around 55,100 stars and 10,200 forks on GitHub, with releases shipping continuously (source: https://github.com/BerriAI/litellm, as of 2026-07-30)
- Attestations: SOC 2 Type 2 and ISO 27001 per the company page; elsewhere the documentation still names SOC 2 Type I with a Type II process under way (source: https://www.litellm.ai/enterprise, as of 2026-07-30)
- Security incident: 24.03.2026: compromised packages v1.82.7 and v1.82.8 on PyPI carrying a credential stealer, available for around 40 minutes; Docker images and source code not affected (source: https://docs.litellm.ai/blog/security-update-march-2026, as of 2026-07-30)

## Cost of leaving

Low: The core is MIT licensed and runs on your own infrastructure. The interface is OpenAI compatible, the configuration lives in a file, and keys and spend logs live in a Postgres database you own. Leaving means pointing the applications back at the model providers directly and taking your data with you. The only real cost is giving up the paid features once you depend on them.

## Regulation and data

| Point | Finding | Evidence | As of |
|---|---|---|---|
| Data processing agreement | not required when self-hosted, not publicly substantiated for the managed cloud — In self-hosted mode the vendor processes no content, so no processing agreement is needed. A publicly available data processing agreement for the managed edition could not be found on litellm.ai or docs.litellm.ai. | partially evidenced | 2026-07-30 |
| Storage location | your own infrastructure when self-hosted; managed edition either in the EU region Frankfurt (eu-central-1) or in the USA (us-west-1, us-east-1) — For self-hosting the vendor states explicitly that no data and no telemetry reach its servers. The location is therefore determined solely by your own environment. | evidenced | 2026-07-30 |
| Subprocessors | none when self-hosted; the managed edition runs on GCP and AWS, with the database partly on NeonDB — No complete, maintained subprocessor list with change notification is published. Only these three names appear in the documentation. | partially evidenced | 2026-07-30 |
| Third-country transfer | controllable when self-hosted, depends on the model providers connected — The gateway itself transfers nothing to a third country. The transfer arises from the model behind it, and that selection can be restricted per key and per team in the configuration. | evidenced | 2026-07-30 |
| Training on customer data | no vendor access to content when self-hosted — Because the calls never leave your own environment, the training question does not arise against this vendor but against whichever model provider sits behind it. That stays a separate check per model. | evidenced | 2026-07-30 |
| Retention and deletion | set by the operator itself, spend logs deletable automatically via a retention period — The documentation shows a configuration value that deletes logs after a set period. The operator sets that period, not the vendor. | evidenced | 2026-07-30 |
| Certifications | SOC 2 Type 2 and ISO 27001 per the company page, reports only on request on the paid plan — The statements contradict each other. The company page names SOC 2 Type 2, while the documentation page still says SOC 2 Type I with a Type II process whose target date has long passed. Ask for the actual report before signing. | partially evidenced | 2026-07-30 |
| EU AI Act, Article 50 | unclear — Neither the company page nor the documentation addresses the transparency duties. The gateway is not a model provider, so the duty usually falls on whoever operates the application. What helps is that every call can be logged in one place. | partially evidenced | 2026-07-30 |
| Audit logging | spend and usage logs in the free edition, audit logs with a retention rule only in the paid edition — Anyone who needs an auditable trail over key issuance and permission changes needs the licence. Plain cost and usage transparency comes without one. | partially evidenced | 2026-07-30 |

## Cost

- Entry: The core is free under the MIT licence, runs as a Docker image and needs a Postgres database of your own. Directory based sign-in is free for up to five people. At first the only cost is running it. (as of 2026-07-30)
- Where it gets expensive: In two places. First at the step into the paid edition: directory based sign-in beyond five people, audit logs, IP based access limits and the response times only come with a licence, and its price is published nowhere. Second in operations: the March 2026 incident shows this layer has to be patched fast. Anyone without that as a fixed routine pays for it in another currency later.

## Three routes compared

### Straight to the model provider or through its cloud gateway

Every application talks to the model directly, or everything runs through one large cloud provider's gateway. Fastest to start, and billing already sits inside an existing contract. The price is lock-in: keys spread out, swapping a model turns into a rebuild, and the spend view stops at that provider's boundary.

### Managed gateway as a service

Services such as Portkey, OpenRouter or Helicone deliver the same routing as a ready made endpoint with nothing to operate. That saves time and costs the decisive point: every call and every log passes through a third party. For companies with audit duties that is the harder conversation, not the easier one.

### Your own thin proxy in front of the provider APIs

A two hundred line service that holds keys and forwards calls is built in a day. The expensive part is what follows: fallbacks, retries, per model cost accounting, streaming, tool calls, onboarding each new provider. That aftermath is precisely what the tool contains, and it is available as open source.

Recommendation by size:

- Solo: Use the library, do not run the proxy yet.
- Mid-market: Self-host. The free scope covers key management, budgets and fallbacks.
- Enterprise: Self-host, but get price, audit report and response times in writing before licensing.

## Context

- Implements method: ["Context: Moat or Wall?" Test](https://www.convios.com/en/methods/context-moat-or-wall-test) — The test demands a separation of what is interchangeable foundation, and a gateway makes that separation practical, because the model underneath stays swappable while your own context sits above it.
- Implements method: [DORA AI Capabilities Model (Seven AI Capabilities)](https://www.convios.com/en/methods/dora-ai-capabilities) — An internal platform for governing and evidencing AI use is one of the seven capabilities, and a gateway is its narrowest form, because keys, cost and limits all sit in one place.
- Displaces: Model keys scattered across individual services and people, Vendor specific libraries buried in application code, Cost visibility per model provider instead of per team

## Evidence

- Core under MIT licence, the enterprise directory under a separate licence — https://github.com/BerriAI/litellm/blob/main/LICENSE (as of 2026-07-30)
- Paid scope, response times, attestations, free sign-in for up to five people, price on request only — https://www.litellm.ai/enterprise (as of 2026-07-30)
- No data and no telemetry at the vendor when self-hosted, regions of the managed edition, attestations — https://docs.litellm.ai/docs/data_security (as of 2026-07-30)
- Audit logs, access restrictions, tenant separation and guardrails in the paid scope — https://docs.litellm.ai/docs/enterprise (as of 2026-07-30)
- Compromised packages v1.82.7 and v1.82.8 on 24 March 2026, Docker images unaffected — https://docs.litellm.ai/blog/security-update-march-2026 (as of 2026-07-30)
- Founding year, cohort, founders, team size and seed funding — https://www.ycombinator.com/companies/litellm (as of 2026-07-30)
- Retention period for spend logs is configurable — https://docs.litellm.ai/docs/proxy/ui_logs (as of 2026-07-30)