
OpenAI Ultrafast: Pay 6x Only When Latency Matters
Summary
A Python router that buys Ultrafast speed only where users wait, with fallback and cost tracking.
OpenAI's DevDay on September 29 shipped a lot: Dots, the Agents API, GPT-6.1 Sol, a Decisions API in limited preview. The line item most developers will feel in their bill is smaller and easier to miss. Ultrafast is a new service tier, requested with service_tier="ultrafast", that OpenAI says generates up to 300 tokens per second, up to 6x faster than standard in the API. It costs 6x the standard rate.
That trade is only good in some places. A chat reply a user is staring at is worth it. A nightly summarization job is not. This guide builds a small Python router that picks the cheapest tier that still meets a latency budget, falls back gracefully when your account is not entitled to Ultrafast, learns your real tokens-per-second as it runs, and bills each call at the tier that actually served it.
You will end up with about 60 lines of router code, a test harness that runs without an API key, and a cost model you can drop your own numbers into.
What Ultrafast is, and what is still unclear
Facts from OpenAI's DevDay recap and the model docs: Ultrafast is a premium speed tier for time-sensitive work, up to 6x faster in the API and up to 8x faster in Codex, priced at 6x standard. It launched with GPT-6 Astra, where OpenAI lists $60 per million input tokens and $300 per million output tokens. A GPT-6.1 Sol Ultrafast variant was described as coming soon, without a published price. The Python SDK's own docs describe the tier as access-controlled: setting service_tier to ultrafast requests it, and the response body reports the tier that actually served the request.
Two things to keep straight. First, access-controlled means some accounts will get an error, so your code needs a fallback path. Second, the served tier can differ from the tier you asked for, and you are billed for the served one. Both points shape the design below.
Prerequisites
- Python 3.10+ and a recent
openaipackage (tested with 3.22.1). TheServiceTiertype in that release already includesultrafast. - An OpenAI API key. For Ultrafast itself you need entitlement; the router works without it.
- No key is needed to run the test harness in step 4, because it uses a mock HTTP transport.
Step 1: The pricing model
Standard GPT-6.1 Sol pricing from the model page is $2.00 per million input tokens, $0.10 cached input and $10.00 output. The docs also list Fast mode at 2x and Flex at 50%. We map those to a multiplier per tier and add 6x for Ultrafast.
| Tier | Multiplier | Approx tokens/sec (prior) | Use for |
|---|---|---|---|
| flex | 0.5x | 25 | Batch, backfills, evals |
| default | 1x | 50 (derived) | Background agents, async work |
| priority (fast) | 2x | 100 (assumed) | Interactive with some slack |
| ultrafast | 6x | 300 (published, up to) | Voice, live coding, tight UI loops |
Only the 300 tokens/sec figure and the 6x speedup claim are published, so the standard figure is derived by dividing. The others are placeholders. That is why the router learns real speed at runtime instead of trusting this table.
Step 2: The router
The logic is simple. Estimate finish time as time-to-first-token plus output tokens divided by the tier's speed. Walk the tiers from cheapest to most expensive and return the first one that fits the budget. Skip Flex for interactive calls, since Flex can queue.
"""tier_router.py - pick the cheapest OpenAI service tier that meets a latency budget."""
import time
from dataclasses import dataclass, field
import openai
from openai import OpenAI
# USD per 1M tokens: (input, cached_input, output). Standard rates from the
# model page; ultrafast = 6x standard (announced multiplier). Cached-input
# prices for fast/ultrafast are ASSUMED to scale the same way. Verify on your invoice.
BASE = {
"gpt-6.1-sol": (2.00, 0.10, 10.00),
"gpt-6-astra": (10.00, 0.50, 50.00), # 60/300 ultrafast / 6; cached assumed
}
MULT = {"flex": 0.5, "default": 1.0, "priority": 2.0, "ultrafast": 6.0}
# Tokens/sec priors. Only ultrafast (~300) and "6x faster than standard" are
# published, so standard ~50 is derived. Real numbers get learned below.
PRIOR_TPS = {"flex": 25.0, "default": 50.0, "priority": 100.0, "ultrafast": 300.0}
ORDER = ["flex", "default", "priority", "ultrafast"] # cheapest first
def cost_usd(model, tier, usage):
inp, cached_rate, out = BASE[model]
m = MULT.get(tier, 1.0)
cached = getattr(getattr(usage, "input_tokens_details", None), "cached_tokens", 0) or 0
fresh = usage.input_tokens - cached
return m * (fresh * inp + cached * cached_rate + usage.output_tokens * out) / 1e6
@dataclass
class TierRouter:
client: OpenAI
model: str = "gpt-6.1-sol"
ultrafast_enabled: bool = True
ttft: float = 0.6 # seconds before first token (prior)
tps: dict = field(default_factory=lambda: dict(PRIOR_TPS))
spent: float = 0.0
def eta(self, tier, out_tokens):
return self.ttft + out_tokens / self.tps[tier]
def pick(self, budget_s, expect_out, interactive=True):
for tier in ORDER:
if tier == "flex" and interactive:
continue # flex can queue for a long time
if tier == "ultrafast" and not self.ultrafast_enabled:
continue
if self.eta(tier, expect_out) <= budget_s:
return tier
return "ultrafast" if self.ultrafast_enabled else "priority"
def _create(self, tier, **kw):
t0 = time.perf_counter()
r = self.client.responses.create(model=self.model, service_tier=tier, **kw)
return r, time.perf_counter() - t0
def run(self, prompt, budget_s=3.0, expect_out=300, interactive=True, **kw):
tier = self.pick(budget_s, expect_out, interactive)
try:
r, dt = self._create(tier, input=prompt, **kw)
except openai.APIStatusError as e:
if tier == "ultrafast" and e.status_code in (400, 403, 404):
self.ultrafast_enabled = False # not entitled: stop trying
tier = "priority"
r, dt = self._create(tier, input=prompt, **kw)
else:
raise
served = getattr(r, "service_tier", None) or tier # what you are BILLED for
out = r.usage.output_tokens
if out >= 50 and dt > 0: # learn real speed (EWMA)
self.tps[served] = 0.7 * self.tps.get(served, PRIOR_TPS["default"]) + 0.3 * (out / dt)
c = cost_usd(self.model, served, r.usage)
self.spent += c
return r, {"asked": tier, "served": served, "seconds": round(dt, 2), "usd": round(c, 6)}
Three details matter. pick returns Ultrafast only when nothing cheaper fits, so most traffic stays cheap. run catches 400, 403 and 404 on an Ultrafast request, flips a flag, and retries on priority, so an unentitled account degrades instead of failing. And cost is computed from r.service_tier, the echoed tier, falling back to what you asked for only when the field is missing.
Step 3: Use it
from openai import OpenAI
from tier_router import TierRouter
router = TierRouter(OpenAI())
# A user is waiting: 1.5 second budget, roughly 300 output tokens
resp, meta = router.run("Explain this stack trace in 3 bullets: ...", budget_s=1.5, expect_out=300)
print(resp.output_text)
print(meta)
# A nightly job: no one is waiting, so let Flex take it
resp, meta = router.run("Summarize yesterday's tickets: ...", budget_s=600, expect_out=800, interactive=False)
print(meta)
Example metadata for the first call, on an entitled account:
{'asked': 'ultrafast', 'served': 'ultrafast', 'seconds': 1.9, 'usd': 0.0309}
The second call returns asked: 'flex' at half the standard price. Same code path, different budget.
Step 4: Test it without spending a cent
The OpenAI client accepts a custom httpx client, so you can swap in httpx.MockTransport and exercise the real SDK request and response parsing with a fake server. This test checks the tier picks, the request body, the 403 fallback and the cost math.
import json, httpx
from openai import OpenAI
from tier_router import TierRouter, cost_usd
seen = []
def handler(req):
body = json.loads(req.content); seen.append(body.get("service_tier"))
if body.get("service_tier") == "ultrafast" and NOT_ENTITLED[0]:
return httpx.Response(403, json={"error": {"message": "not enabled", "type": "invalid_request_error"}})
served = "priority" if body["service_tier"] in ("fast","priority") else body["service_tier"]
return httpx.Response(200, json={
"id":"resp_1","object":"response","created_at":0,"model":body["model"],"status":"completed",
"service_tier": served, "parallel_tool_calls":True,"tool_choice":"auto","tools":[],
"output":[{"id":"m1","type":"message","role":"assistant","status":"completed",
"content":[{"type":"output_text","text":"ok","annotations":[]}]}],
"usage":{"input_tokens":2000,"output_tokens":400,"total_tokens":2400,
"input_tokens_details":{"cached_tokens":1500},"output_tokens_details":{"reasoning_tokens":0}}})
NOT_ENTITLED=[False]
def mk(): return OpenAI(api_key="x", max_retries=0, http_client=httpx.Client(transport=httpx.MockTransport(handler)))
r = TierRouter(mk())
print("pick 3s/300tok :", r.pick(3.0, 300))
print("pick 10s/300tok:", r.pick(10.0, 300))
print("pick 1.5s/300 :", r.pick(1.5, 300))
print("pick batch :", r.pick(600, 300, interactive=False))
resp, meta = r.run("hi", budget_s=1.5); print(meta, seen)
NOT_ENTITLED[0]=True; r2 = TierRouter(mk())
resp, meta = r2.run("hi", budget_s=1.5); print(meta, seen, r2.ultrafast_enabled)
class U: input_tokens=2000; output_tokens=400; input_tokens_details=type("d",(),{"cached_tokens":1500})
for t in ("default","priority","ultrafast"): print(t, round(cost_usd("gpt-6.1-sol", t, U),6))
Output from running it:
pick 3s/300tok : ultrafast
pick 10s/300tok: default
pick 1.5s/300 : ultrafast
pick batch : flex
{'asked': 'ultrafast', 'served': 'ultrafast', 'seconds': 0.19, 'usd': 0.0309} ['ultrafast']
{'asked': 'priority', 'served': 'priority', 'seconds': 0.0, 'usd': 0.0103} ['ultrafast', 'ultrafast', 'priority'] False
default 0.00515
priority 0.0103
ultrafast 0.0309
Read the last list on the second result line: it holds every tier the mock server received. The first entry is the earlier successful Ultrafast call, the second is the unentitled router's Ultrafast attempt that got a 403, and the third is its fallback to priority. The trailing False shows Ultrafast is now switched off for that router, so it never pays for a failed round trip again.
A worked example: what 6x does to a real bill
Take a coding assistant that makes 10,000 calls a day. Each call sends 2,000 input tokens, 1,500 of them cached, and gets back 400 output tokens, on GPT-6.1 Sol.
| Setup | Cost per call | Cost per 10k calls |
|---|---|---|
| All default | $0.00515 | $51.50 |
| All priority | $0.0103 | $103.00 |
| All ultrafast (assumes 6x on cached too) | $0.0309 | $309.00 |
| Router: 80% default, 20% ultrafast | - | $103.00 |
The router row is 0.8 x $51.50 + 0.2 x $309 = $103. You keep the speed on the fifth of traffic where a human is waiting and pay twice the baseline instead of six times. Latency for those calls drops from roughly 0.6 + 400/50 = 8.6 seconds to 0.6 + 400/300 = 1.9 seconds, if the published throughput holds for your workload. Measure it, because 'up to' is doing real work in that claim.
A rule of thumb: Ultrafast pays off when the saved seconds are worth more than five times the standard cost of the call. For a voice agent where 6 seconds of silence loses the user, that is easy. For a background agent that already runs for minutes, it is not.
Common pitfalls
- Billing follows the served tier. The response echoes
service_tier, and OpenAI notes the value can differ from what you requested. Log both, and compute cost from the echo. The router does. - Ultrafast is access-controlled. Do not hard-code it on the happy path. Catch the error, degrade to
priority, and turn it off for the process so you are not paying a failed round trip on every call. The exact error status for an unentitled account is not documented in what I found, so the router handles 400, 403 and 404; confirm on your own account. - Model coverage moves. The SDK docstring I tested with says the tier is available for
gpt-5.6-sol, DevDay coverage says it launched with GPT-6 Astra, and GPT-6.1 Sol's Ultrafast price was still pending. Treat any model and tier pair as unverified until a request returnsservice_tier=ultrafast. - Cached-input price at 6x is an assumption. The 6x multiplier is published for input and output. I applied it to cached tokens too. If your invoice differs, change the multiplier table, not the code.
- Output length dominates. Speed gains come from token generation, so the win is large for long answers and tiny for a 20-token classification. For short routing choices, a cheaper model beats a faster tier. OpenAI's new Decisions API targets exactly that case, but it is in limited preview with no public schema yet, so I did not build on it.
- Reasoning tokens count as output. With
reasoning.effortset high, most of the generation may be hidden reasoning. Ultrafast speeds it up, and you pay 6x for it. Drop effort first, then consider the tier. - Streaming changes the math. If you stream, the user perceives time-to-first-token, not total time. A standard tier with streaming may already feel fast enough at short outputs.
- Do not learn speed from tiny samples. The router ignores calls under 50 output tokens when updating throughput, since fixed overhead swamps the estimate.
Quick reference
| Item | Value | Source status |
|---|---|---|
| Request parameter | service_tier="ultrafast" | SDK docs, verified in 3.22.1 |
| Response field | service_tier (served tier) | SDK docs |
| Price multiplier | 6x standard | DevDay recap and reporting |
| Astra Ultrafast price | $60 in / $300 out per 1M | Reported at launch |
| Sol standard price | $2 in / $0.10 cached / $10 out | Model page |
| Sol Ultrafast price | Not published (about $12 / $60 implied) | Pending |
| Throughput | Up to 300 tok/s | OpenAI claim, measure yourself |
| Access | Access-controlled | SDK docs |
Next steps
- Log
metafrom every call to a table and chart spend by served tier per day. - Add a per-request deadline: if a call has already burned half its budget in retries, jump a tier.
- Put the latency budget in config per endpoint, so the product team, not the code, decides what deserves 6x.
- Once the Decisions API opens up, route trivial classification there and reserve Ultrafast for long generations.
- Compare Ultrafast against Fast mode on your own prompts for a week before committing traffic.
Sources: OpenAI DevDay 2026 recap (openai.com/index/devday-2026-recap), the GPT-6.1 Sol model page on developers.openai.com, the OpenAI Python SDK 3.22.1 type docs for service_tier, VentureBeat and The Decoder coverage of the launch.
Comments
Be the first to comment
Found this useful?
Get new AI guides for builders by email. Free.
Join 3,594 builders reading daily.