afterburnDevelopers

Change two lines.
Keep the rest.

Afterburn’s gateway speaks the OpenAI API. Point your client at it, use a route name as the model, and the console decides which endpoint answers. The control API manages endpoints, benchmarks and routes the same way the console does.

Get a gateway key Control API

/v1/chat/completions
/v1/completions
/v1/models
Server-sent events

Gateway

Any OpenAI client
works as it is.

The model field names a route. Everything else, from streaming to tools, passes through to the endpoint the route picks.

curlShell

curl https://afterburn.your-company.com/v1/chat/completions \
  -H "Authorization: Bearer $AFTERBURN_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "chat",
    "stream": true,
    "messages": [{"role": "user", "content": "Hello"}]
  }'

index.tsTypeScript · openai

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://afterburn.your-company.com/v1",
  apiKey: process.env.AFTERBURN_KEY,
});

const reply = await client.chat.completions.create({
  model: "chat", // a route
  messages: [{ role: "user", content: "Hello" }],
});

Responses

Every answer says
where it came from.

Each gateway response carries headers naming the endpoint that answered, the route revision that chose it, and how many attempts it took.

Response headersSample data

x-afterburn-route: chat
x-afterburn-revision: 4
x-afterburn-endpoint: sglang-l40s
x-afterburn-attempts: 1
x-request-id: req_8f2c…

Fallbacks
A connection error, a 5xx, a 429 or a timeout moves the request to the next endpoint on the route, as long as nothing has been sent back yet.
Errors you can read
When every endpoint on a route fails, the gateway answers with each attempt’s status in an OpenAI-shaped error.
Usage counted
Tokens come from the endpoint’s usage report, or are counted from the stream when it sends none; cost comes from the prices you set.

Control API

The console,
as an API.

Everything the console does goes through the same JSON API under /api/v1, with the same roles and the same audit trail. Sign in with a session, or call it with a key that has the admin scope.

endpointsREST

# add an endpoint
POST   /api/v1/endpoints
POST   /api/v1/endpoints/{id}/check
GET    /api/v1/endpoints/{id}/metrics

# run and read a benchmark
POST   /api/v1/benchmarks
GET    /api/v1/benchmarks/{id}

# routes and their revisions
POST   /api/v1/routes
POST   /api/v1/routes/{id}/decide
POST   /api/v1/routes/{id}/rollback

benchmark.jsonPOST /api/v1/benchmarks

{
  "endpointIds": ["ep_3f2a…", "ep_91c0…"],
  "suite": "chat-short",
  "requests": 200,
  "concurrency": 16,
  "maxTokens": 256
}

Questions

Which OpenAI features pass through?

Everything in the request body is forwarded to the chosen endpoint unchanged, apart from the model name, which becomes the endpoint’s own model. What works depends on the endpoint: tools, JSON mode and the rest behave as that engine supports them.

Does the gateway add latency?

It adds one network hop inside your own network. Afterburn measures first token at the gateway, so any delay it adds shows up in its own numbers.

Can I call an endpoint directly, without a route?

Yes. Use the endpoint’s name as the model and the gateway sends the request there, still logged and timed.

Two lines
to change.

Create a route, make a gateway key, point your client at it.

Open the console Read the API

Change two lines.Keep the rest.

Two linesto change.

Change two lines.
Keep the rest.

Two lines
to change.