Skip to content

Two ways to run models

There are two ways to get a model answering on this platform, and the AI guides split along that line. Reading the wrong half is the main way to get lost, so this page decides which half applies to you.

The short version: use ours until you need to own it.

The two paths

flowchart TB
    APP["your application<br/>base URL, model name, key"]
    subgraph MANAGED["managed - we operate it"]
        direction TB
        GW1["the platform gateway"] --> M1["models served for you"]
    end
    subgraph OWN["your own - you operate it"]
        direction TB
        GW2["a gateway in your project"] --> M2["your models,<br/>or any OpenAI-compatible endpoint"]
    end
    APP --> MANAGED
    APP --> OWN
Managed Your own
The gateway ours, shared, already running yours, deployed into your project
Serving a model one object and it is served you assemble it, or use the managed path underneath
Backends we register and operate them you add providers
Keys and budgets issued to you you issue them
Upgrades, capacity, failover ours yours
Start here Virtual keys Deploy an LLM gateway

What does not change when you move

Your application talks to a model through three things: a base URL, a model name, and a key. Both paths present the same OpenAI-compatible interface, so moving between them changes configuration rather than code.

That is the point of running your own gateway rather than replacing ours: a gateway is a router, so you can put yours in front of anything that speaks the same API - your own models, an external provider, or the platform gateway itself. Adding control does not mean starting again.

Model names are the part that does not move by itself

The base URL and the key are configuration. A model name is a contract, and the same model can be registered under different names on different gateways. Decide the names you want before anything depends on them - see Model names are the contract.

Which to choose

Take the managed path if you want a model answering and would rather not operate a serving stack. It is the right default, and most projects never need anything else.

Take your own if any of these is true:

  • You need routing, failover or budget policy the shared gateway does not offer you
  • You want your traffic to reach an external provider under your own commercial agreement
  • Your compliance position requires the gateway itself inside your project boundary
  • You are already running a gateway elsewhere and want one consistent configuration

Running your own costs you the upgrades, the capacity planning and the on-call. That is the whole trade.