Console walkthrough: add a model provider¶

Providers are configured in the gateway, not in the platform console. This page covers how to reach the gateway's own interface without exposing it, and what the console can tell you afterwards.

The platform console has no provider form

A provider is gateway configuration, so it is created through the gateway's own API or dashboard. What the console gives you is the workload the gateway runs in and the log that shows whether a provider is working - which is where most of the diagnosis happens.

Prerequisites¶

A running gateway in your project - see Deploy an LLM gateway
The backend you are registering, reachable from that project

1. Reach the gateway without publishing it¶

Forward the Service to your own machine rather than creating a Route:

oc port-forward -n <project> svc/<gateway> 8080:8080

The dashboard and the configuration API are then on http://localhost:8080, reachable only by you.

Check whether the dashboard requires a password before you publish it

Setting an administrator username and password as environment variables does not by itself turn authentication on. Ask the gateway what it thinks its own configuration is:

curl -sS http://localhost:8080/api/config | jq .auth_config

An is_enabled of false means the configuration API answers to anyone who can reach it, and publishing a Route in that state exposes provider credentials and the ability to add backends. Turn authentication on first - see step 2 of the gateway guide.

2. Register the provider¶

The manifest-driven route is in the step-by-step guide. Through the API it is two calls, and the second is the one people miss:

# the provider
curl -sS -X POST http://localhost:8080/api/providers \
  -H 'Content-Type: application/json' \
  -d '{"provider":"<provider>"}'

# a key on that provider - without this it routes nothing
curl -sS -X POST http://localhost:8080/api/providers/<provider>/keys \
  -H 'Content-Type: application/json' \
  -d '{"name":"<key-name>","value":"<credential>","models":["<model-name>"],"weight":1}'

A provider with no keys reports itself as active

GET /api/providers shows provider_status: active for a provider that has no keys at all, because the provider record was created successfully. The keys are a separate collection, and only GET /api/providers/<provider>/keys shows them. A request against such a provider fails with no keys found for provider, which is the first sign that the second call never happened.

3. Confirm it from the console¶

Workloads > Deployments > <gateway>, then the pod's Logs tab. The gateway logs each configuration change and each request it routes:

Three lines are worth finding:

successfully updated provider configuration for provider <name> - the change landed
failed to list models with key <id> - the gateway could not reach the backend, or the credential was refused
"http.target":"/v1/chat/completions","http.status_code":200 - a request actually routed

The model list is the other half of the check, and it is the one a caller sees:

curl -sS http://localhost:8080/v1/models

A backend that registered but cannot be reached is absent from this list while still appearing in /api/providers.

Troubleshoot¶

The provider is registered but its models never appear

The key's status says why. Read it directly rather than inferring from the provider:

curl -sS http://localhost:8080/api/providers/<provider>/keys | jq '.keys[] | {name, status, description}'

failed to execute HTTP request to provider API is a transport failure - the gateway could not open a connection at all. Before changing the configuration, establish whether anything else in the project can reach that address:

oc run netcheck -n <project> --rm -it --restart=Never \
  --image=registry.access.redhat.com/ubi9/ubi-minimal -- \
  curl -sS -o /dev/null -w '%{http_code}\n' http://<backend-service>:<port>/v1/models

A 200 from that pod and a transport failure from the gateway means the address is fine and the fault is in the gateway - most often the version. A gateway that worked against a self-hosted backend before an upgrade, and does not after, is worth pinning back to the version that worked before spending time on the configuration.

The first completion after registering fails, then later ones succeed

A single the server closed connection before returning the first response byte immediately after registration, followed by successful requests with no change in between, is connection reuse rather than a configuration fault. Send a second request before investigating.

Where the console differs from the CLI¶

	Console	`oc` and the gateway API
Creating a provider	not possible - it is gateway configuration	two API calls, or the values file
Reaching the gateway	port-forward, then a browser	`oc port-forward`, then `curl`
Seeing what routed	the pod's Logs tab	`oc logs`
Checking a key's health	not surfaced	`GET /api/providers/<provider>/keys`

Next¶

Issue a virtual key → - so a caller can use the model this provider serves.