In my experience most teams find out what their LLM workload costs from the invoice. Usage grows, a retry loop misbehaves, a feature ships with a larger context window, and the month closes on a number nobody forecast. Dashboards and budget alerts do not prevent that. They report money that has already left.
Tollgate turns the budget into an admission decision, taken before the request reaches the model. It is an open-source AI gateway and spend-control proxy, out now as v0.1.1 under MIT (Stevens, 2026).
What it does
Tollgate sits between your applications and their LLM providers, currently Anthropic and Google Vertex, both config-gated, plus a built-in mock for testing. There is no OpenAI adapter yet. Each request is authenticated, priced by token, and checked against every budget that applies to it before anything is forwarded. If a hard cap would be exceeded the request is refused and never reaches the provider. Spend is written to a durable, append-only ledger in Postgres, and a read-only web console shows live budget meters, the tokens-to-cost breakdown, and the gateway’s own added latency.
It is written in Rust and deploys to Google Cloud Run. The Terraform module in the repository provisions the full stack and keeps the database URL and API-key pepper in Secret Manager, injected by reference rather than as plaintext environment values.
A refused request returns 402 Payment Required with an x-tollgate-reason header. An unroutable or unpriced model returns 400, also with a reason header.
That is worth unpacking, because the obvious alternative is a trap and the choice made carries its own caveat. 429 is the rate-limiting code, and provider SDKs commonly auto-retry it, so a monthly cap answering 429 would invite well-behaved clients to keep knocking at a gate that cannot open for weeks. 402 avoids that. But 402 is reserved rather than standardised, and the x402 payment standard, now a Linux Foundation project, is busy teaching agent clients that a 402 means pay and retry (x402 Foundation, 2026). Tollgate advertises no payment challenge and its callers authenticate with an issued key, so x402 middleware has nothing to act on here. If you run agents with payment middleware in the stack, check that interaction before relying on the status code alone. The reason header is there to be read.
What it does not do yet
The README carries a Limitations section. Three entries decide whether Tollgate fits a given workload today, so they belong here rather than three clicks away.
There is no streaming. Requests must be non-streaming, because usage cannot be metered from a partial stream in this release.
In the default admission mode, requests referencing external media by fileUri, file_id or URL are refused with a 400. The token cost of fetched media is not bounded by the size of the request body, so the reservation would be far too low to be safe. Those requests need exact admission, described below. If your traffic is multimodal, that decides your configuration on day one.
Prompt-caching cost is approximate. Cache-read and cache-write token classes are not yet priced separately, so on cache-heavy workloads the recorded cost can diverge from the provider’s bill. Standard non-caching usage is exact. Metrics are minimal too: /metrics exposes only a liveness gauge, so Postgres and the console are where spend actually lives.
How a request moves through it
Pricing happens before the forward, which is what separates an enforceable cap from an advisory one. The worst-case cost of a request is reserved up front as a single atomic check-and-increment against Valkey, and once the provider responds with reported token usage the reservation is settled to the actual figure.
Four properties hold that together. Money is handled in integer micros with no floating point anywhere on the cost path. Postgres is the system of record and the Valkey counters are only a hot-path cache, reconciled from the ledger at startup in a direction that can raise a counter but never lower it. A request matching no budget at all is denied, so a missing configuration row cannot leave a deployment unenforced without anyone noticing. And a cache failure fails closed, which is the right call for a spend gate but carries a real cost: Tollgate is an in-path dependency, so if Valkey is unreachable your LLM traffic stops with a 503 rather than flowing unmetered.
Budgets resolve by scope: a specific key, a provider, a model, or the mandatory global backstop. Periods are daily, weekly, or monthly, and a request has to clear every scope that applies to it, so a per-key cap and a deployment-wide monthly ceiling operate together.
Two admission modes decide how strict the cap really is. Output tokens are bounded by the request’s own max_tokens or maxOutputTokens. Input tokens are counted either by fast, the default, which estimates from the request body with no extra call and on token-dense input can settle a little over the cap, or by exact, which makes a pre-flight token-count call to the provider and enforces strictly. exact costs a round trip, and it adds a second provider dependency: if the count call fails the request fails with a 503 rather than falling back to the weaker estimate. If you need the cap strict to the token, that is the trade.
What v0.1.1 gives you
Per-key and global token budgets with a hard stop before the provider. Token-level cost accounting, currency-agnostic and integer-precise. Anthropic and Vertex adapters plus the mock. Budget and price changes that apply within the reload interval without a restart. Ledger retention by monthly partition, with partitions created ahead of need and old ones dropped. Per-request gateway latency on an x-tollgate-overhead-us header, with median and p95 in the console, so the overhead figure for your own traffic is something you read rather than something I assert.
To see it without provisioning anything, cargo run -- demo boots an in-memory gateway with the mock provider, prints a key and two budgets, and serves the console on loopback.
Tollgate builds on proxy and cost-control patterns that LiteLLM established (BerriAI, 2026), implemented in Rust.
Open core
The open-source core is single-operator and single-tenant by design. Keys and budgets are managed through the CLI, the console is observe-only with no write endpoints, and authorization on the observe endpoints is flat: any valid key can view the whole deployment’s budgets and usage. Enforcement stays per key. A Tollgate Enterprise edition will follow for teams running across many tenants and operators, adding web and API management of keys and budgets, role-based access control and SSO, budget-change approval workflows, multi-org scoping, richer analytics, and audit export.
Try it
The repository is at github.com/sakura-sky/tollgate and the release is v0.1.1. Run the demo, and open an issue if it breaks in your environment. This is beta and pre-1.0, so interfaces and the schema may still change, and a report from someone who put a real budget in front of real traffic is worth more to me than any amount of design discussion. If the Enterprise edition is relevant to you, open a GitHub Discussion or contact Sakura Sky. Please keep the SECURITY.md inbox for vulnerability reports.
Disclosure: I created Tollgate. It is published as a Sakura Sky open-source project under MIT, with copyright held by me and its contributors, and there is no contributor licence agreement. Sakura Sky intends to offer a commercial Enterprise edition alongside the free core, which is a commercial interest worth stating. Sakura Sky also builds and runs managed LLM gateway environments for customers, including LiteLLM deployments, and continues to do so; Tollgate is a narrower tool for a specific job and is not a migration recommendation for anyone.
References
BerriAI (2026) LiteLLM. Available at: https://github.com/BerriAI/litellm (Accessed: 29 August 2026).
Stevens, A. (2026) Tollgate v0.1.1. Available at: https://github.com/sakura-sky/tollgate/releases/tag/v0.1.1 (Accessed: 29 August 2026).
x402 Foundation (2026) x402: an open standard for internet-native payments. The Linux Foundation. Available at: https://x402.org (Accessed: 29 August 2026).