Skip to content

Virtual keys

You want to give a teammate, an application or an agent access to your models without handing over the credential that can spend without limit. A virtual key is that: a credential your gateway issues, scoped to what you decide, and revocable on its own.

If you have used scoped API tokens - a deploy key rather than your password, an IAM role rather than root credentials - this is the same idea applied to model spend.

Prerequisites

The step-by-step guide and the console walkthrough cover the operations. The rest of this page explains how it fits together.

One key per caller, not one key per team

flowchart TB
    %% the invisible link stacks the two cases vertically instead of side by side,
    %% which keeps both readable at page width
    subgraph SHARED["one shared key"]
        direction TB
        S1["Ana"] --> K1["sk-team"]
        S2["batch job"] --> K1
        S3["CI"] --> K1
        K1 --> B1["one bill, no attribution<br/>rotating it breaks all three"]
    end
    subgraph PER["a key per caller"]
        direction TB
        P1["Ana"] --> K2["key<br/>Ana"]
        P2["batch job"] --> K3["key<br/>batch, capped"]
        P3["CI"] --> K4["key<br/>CI, cheap models only"]
        K2 --> B2["spend attributed per key<br/>revoke one, the rest keep working"]
        K3 --> B2
        K4 --> B2
    end
    SHARED ~~~ PER

A shared key gives you neither attribution nor blast radius control. When the month's spend doubles you cannot tell which caller did it, and when you rotate the key everything stops at once. Both problems disappear if a key never has more than one holder.

The rule of thumb: a virtual key should belong to exactly one thing that can be switched off independently. A person, a service, a CI pipeline, one agent.

A key, or the identity that maps to one

The gateway accepts two kinds of credential on the same endpoint, and they end up in the same place.

The caller presents The gateway does Spend is attributed to
a virtual key uses it as-is that key
an identity token from the platform's identity provider validates it, reads the email claim, looks up that person's virtual key, and substitutes it the same key 
flowchart TB
    ID["identity token<br/>from single sign-on"] -->|"email claim"| MAP["identity map<br/>email to virtual key"]
    KEY["virtual key<br/>copied to the caller"] --> VK
    MAP --> VK["the request now carries<br/>one virtual key"]
    VK --> LIM["budget, rate limit,<br/>model scope, logging"]
    LIM --> M["model backend"]

So single sign-on does not bypass a virtual key or create a second kind of budget. It is a way of reaching your key without copying it onto a laptop - the request arrives holding the key either way, and every limit on it applies identically.

Two consequences worth knowing before you rely on it:

  • An identity only works if it has been mapped to a key. The lookup is by email address against a map the gateway administrator maintains. A perfectly valid token belonging to someone not in that map is refused, and the failure is a rejected request rather than an unmetered one - the lookup fails closed.
  • The mapping is one key per person. Two people cannot share an identity, which is the same rule as one key per caller arriving by a different route.

The exchange is drawn step by step, for a coding agent, in Connecting over OAuth.

What you can attach to a key

Four controls, and they answer different questions. A key can carry any combination.

Control Question it answers Enforced
Budget how much may this key spend in total? per window, then requests are refused
Rate limit how fast may it spend it? per window, on requests and on tokens
Model scope which models may it reach at all? per request, before routing
Active flag may it be used right now? immediately, without deleting the key

Budgets and rate limits are not the same control and neither substitutes for the other. A budget stops a slow leak - a job that quietly costs a little too much for a month. A rate limit stops a fast one - a retry loop that spends the whole budget in four minutes. A key with only a budget can still burn all of it before anyone reads an alert.

Model scope is the cheapest of the four and the most often skipped. A key that only ever needs a small model should not be able to reach an expensive one, whatever its budget says.

Scope to models, not just to spend

Most runaway cost is a cheap workload accidentally pointed at an expensive model. A model scope makes that a refused request instead of an invoice.

What is fixed after creation

Changeable
The budget, rate limit and model scope yes
Whether the key is active yes - deactivating is reversible, deleting is not
The key's own secret value no - issue a new key and retire the old one
Spend already recorded against it no

A virtual key is a credential

Send one the way you would send a password, and never commit one to Git. If a key is pasted somewhere it should not be, deactivate it first and investigate afterwards - it is one object and nothing else depends on it.

Deactivate before you delete

Deactivating stops the key working while leaving its spend history and its identity in place. Deleting removes the object. Reach for deactivation first: it is instant, it is reversible, and it keeps the record of what the key spent while it was live.

stateDiagram-v2
    [*] --> Active: create
    Active --> Deactivated: deactivate
    Deactivated --> Active: reactivate
    Active --> Gone: delete
    Deactivated --> Gone: delete
    Gone --> [*]

    note right of Deactivated
        requests refused
        spend history kept
        same key value on reactivation
    end note

    note right of Gone
        the key value cannot be reissued
        callers need a new key
    end note

The route through Deactivated is the one to take when you are not certain. The route straight to deletion is the one that cannot be walked back: a leaked key you deleted is as unrecoverable as a leaked key you kept, but you also lose the record of what it spent.