Skip to content
OpenAI Ultrafast: Pay 6x Only When Latency Matters — ContentBuffer guide

OpenAI Ultrafast: Pay 6x Only When Latency Matters

K
Kodetra Technologies··9 min read Intermediate

Summary

A Python router that buys Ultrafast speed only where users wait, with fallback and cost tracking.

OpenAI's DevDay on September 29 shipped a lot: Dots, the Agents API, GPT-6.1 Sol, a Decisions API in limited preview. The line item most developers will feel in their bill is smaller and easier to miss. Ultrafast is a new service tier, requested with service_tier="ultrafast", that OpenAI says generates up to 300 tokens per second, up to 6x faster than standard in the API. It costs 6x the standard rate.

That trade is only good in some places. A chat reply a user is staring at is worth it. A nightly summarization job is not. This guide builds a small Python router that picks the cheapest tier that still meets a latency budget, falls back gracefully when your account is not entitled to Ultrafast, learns your real tokens-per-second as it runs, and bills each call at the tier that actually served it.

You will end up with about 60 lines of router code, a test harness that runs without an API key, and a cost model you can drop your own numbers into.

What Ultrafast is, and what is still unclear

Facts from OpenAI's DevDay recap and the model docs: Ultrafast is a premium speed tier for time-sensitive work, up to 6x faster in the API and up to 8x faster in Codex, priced at 6x standard. It launched with GPT-6 Astra, where OpenAI lists $60 per million input tokens and $300 per million output tokens. A GPT-6.1 Sol Ultrafast variant was described as coming soon, without a published price. The Python SDK's own docs describe the tier as access-controlled: setting service_tier to ultrafast requests it, and the response body reports the tier that actually served the request.

Two things to keep straight. First, access-controlled means some accounts will get an error, so your code needs a fallback path. Second, the served tier can differ from the tier you asked for, and you are billed for the served one. Both points shape the design below.

Prerequisites

  • Python 3.10+ and a recent openai package (tested with 3.22.1). The ServiceTier type in that release already includes ultrafast.
  • An OpenAI API key. For Ultrafast itself you need entitlement; the router works without it.
  • No key is needed to run the test harness in step 4, because it uses a mock HTTP transport.

Step 1: The pricing model

Standard GPT-6.1 Sol pricing from the model page is $2.00 per million input tokens, $0.10 cached input and $10.00 output. The docs also list Fast mode at 2x and Flex at 50%. We map those to a multiplier per tier and add 6x for Ultrafast.

TierMultiplierApprox tokens/sec (prior)Use for
flex0.5x25Batch, backfills, evals
default1x50 (derived)Background agents, async work
priority (fast)2x100 (assumed)Interactive with some slack
ultrafast6x300 (published, up to)Voice, live coding, tight UI loops

Only the 300 tokens/sec figure and the 6x speedup claim are published, so the standard figure is derived by dividing. The others are placeholders. That is why the router learns real speed at runtime instead of trusting this table.

Step 2: The router

The logic is simple. Estimate finish time as time-to-first-token plus output tokens divided by the tier's speed. Walk the tiers from cheapest to most expensive and return the first one that fits the budget. Skip Flex for interactive calls, since Flex can queue.

"""tier_router.py - pick the cheapest OpenAI service tier that meets a latency budget."""
import time
from dataclasses import dataclass, field
import openai
from openai import OpenAI

# USD per 1M tokens: (input, cached_input, output). Standard rates from the
# model page; ultrafast = 6x standard (announced multiplier). Cached-input
# prices for fast/ultrafast are ASSUMED to scale the same way. Verify on your invoice.
BASE = {
    "gpt-6.1-sol": (2.00, 0.10, 10.00),
    "gpt-6-astra": (10.00, 0.50, 50.00),   # 60/300 ultrafast / 6; cached assumed
}
MULT = {"flex": 0.5, "default": 1.0, "priority": 2.0, "ultrafast": 6.0}

# Tokens/sec priors. Only ultrafast (~300) and "6x faster than standard" are
# published, so standard ~50 is derived. Real numbers get learned below.
PRIOR_TPS = {"flex": 25.0, "default": 50.0, "priority": 100.0, "ultrafast": 300.0}
ORDER = ["flex", "default", "priority", "ultrafast"]  # cheapest first


def cost_usd(model, tier, usage):
    inp, cached_rate, out = BASE[model]
    m = MULT.get(tier, 1.0)
    cached = getattr(getattr(usage, "input_tokens_details", None), "cached_tokens", 0) or 0
    fresh = usage.input_tokens - cached
    return m * (fresh * inp + cached * cached_rate + usage.output_tokens * out) / 1e6


@dataclass
class TierRouter:
    client: OpenAI
    model: str = "gpt-6.1-sol"
    ultrafast_enabled: bool = True
    ttft: float = 0.6                       # seconds before first token (prior)
    tps: dict = field(default_factory=lambda: dict(PRIOR_TPS))
    spent: float = 0.0

    def eta(self, tier, out_tokens):
        return self.ttft + out_tokens / self.tps[tier]

    def pick(self, budget_s, expect_out, interactive=True):
        for tier in ORDER:
            if tier == "flex" and interactive:
                continue                    # flex can queue for a long time
            if tier == "ultrafast" and not self.ultrafast_enabled:
                continue
            if self.eta(tier, expect_out) <= budget_s:
                return tier
        return "ultrafast" if self.ultrafast_enabled else "priority"

    def _create(self, tier, **kw):
        t0 = time.perf_counter()
        r = self.client.responses.create(model=self.model, service_tier=tier, **kw)
        return r, time.perf_counter() - t0

    def run(self, prompt, budget_s=3.0, expect_out=300, interactive=True, **kw):
        tier = self.pick(budget_s, expect_out, interactive)
        try:
            r, dt = self._create(tier, input=prompt, **kw)
        except openai.APIStatusError as e:
            if tier == "ultrafast" and e.status_code in (400, 403, 404):
                self.ultrafast_enabled = False      # not entitled: stop trying
                tier = "priority"
                r, dt = self._create(tier, input=prompt, **kw)
            else:
                raise
        served = getattr(r, "service_tier", None) or tier   # what you are BILLED for
        out = r.usage.output_tokens
        if out >= 50 and dt > 0:                            # learn real speed (EWMA)
            self.tps[served] = 0.7 * self.tps.get(served, PRIOR_TPS["default"]) + 0.3 * (out / dt)
        c = cost_usd(self.model, served, r.usage)
        self.spent += c
        return r, {"asked": tier, "served": served, "seconds": round(dt, 2), "usd": round(c, 6)}

Three details matter. pick returns Ultrafast only when nothing cheaper fits, so most traffic stays cheap. run catches 400, 403 and 404 on an Ultrafast request, flips a flag, and retries on priority, so an unentitled account degrades instead of failing. And cost is computed from r.service_tier, the echoed tier, falling back to what you asked for only when the field is missing.

Step 3: Use it

from openai import OpenAI
from tier_router import TierRouter

router = TierRouter(OpenAI())

# A user is waiting: 1.5 second budget, roughly 300 output tokens
resp, meta = router.run("Explain this stack trace in 3 bullets: ...", budget_s=1.5, expect_out=300)
print(resp.output_text)
print(meta)

# A nightly job: no one is waiting, so let Flex take it
resp, meta = router.run("Summarize yesterday's tickets: ...", budget_s=600, expect_out=800, interactive=False)
print(meta)

Example metadata for the first call, on an entitled account:

{'asked': 'ultrafast', 'served': 'ultrafast', 'seconds': 1.9, 'usd': 0.0309}

The second call returns asked: 'flex' at half the standard price. Same code path, different budget.

Step 4: Test it without spending a cent

The OpenAI client accepts a custom httpx client, so you can swap in httpx.MockTransport and exercise the real SDK request and response parsing with a fake server. This test checks the tier picks, the request body, the 403 fallback and the cost math.

import json, httpx
from openai import OpenAI
from tier_router import TierRouter, cost_usd

seen = []
def handler(req):
    body = json.loads(req.content); seen.append(body.get("service_tier"))
    if body.get("service_tier") == "ultrafast" and NOT_ENTITLED[0]:
        return httpx.Response(403, json={"error": {"message": "not enabled", "type": "invalid_request_error"}})
    served = "priority" if body["service_tier"] in ("fast","priority") else body["service_tier"]
    return httpx.Response(200, json={
      "id":"resp_1","object":"response","created_at":0,"model":body["model"],"status":"completed",
      "service_tier": served, "parallel_tool_calls":True,"tool_choice":"auto","tools":[],
      "output":[{"id":"m1","type":"message","role":"assistant","status":"completed",
                 "content":[{"type":"output_text","text":"ok","annotations":[]}]}],
      "usage":{"input_tokens":2000,"output_tokens":400,"total_tokens":2400,
               "input_tokens_details":{"cached_tokens":1500},"output_tokens_details":{"reasoning_tokens":0}}})
NOT_ENTITLED=[False]
def mk(): return OpenAI(api_key="x", max_retries=0, http_client=httpx.Client(transport=httpx.MockTransport(handler)))

r = TierRouter(mk())
print("pick 3s/300tok :", r.pick(3.0, 300))
print("pick 10s/300tok:", r.pick(10.0, 300))
print("pick 1.5s/300  :", r.pick(1.5, 300))
print("pick batch     :", r.pick(600, 300, interactive=False))
resp, meta = r.run("hi", budget_s=1.5); print(meta, seen)
NOT_ENTITLED[0]=True; r2 = TierRouter(mk())
resp, meta = r2.run("hi", budget_s=1.5); print(meta, seen, r2.ultrafast_enabled)
class U: input_tokens=2000; output_tokens=400; input_tokens_details=type("d",(),{"cached_tokens":1500})
for t in ("default","priority","ultrafast"): print(t, round(cost_usd("gpt-6.1-sol", t, U),6))

Output from running it:

pick 3s/300tok : ultrafast
pick 10s/300tok: default
pick 1.5s/300  : ultrafast
pick batch     : flex
{'asked': 'ultrafast', 'served': 'ultrafast', 'seconds': 0.19, 'usd': 0.0309} ['ultrafast']
{'asked': 'priority', 'served': 'priority', 'seconds': 0.0, 'usd': 0.0103} ['ultrafast', 'ultrafast', 'priority'] False
default 0.00515
priority 0.0103
ultrafast 0.0309

Read the last list on the second result line: it holds every tier the mock server received. The first entry is the earlier successful Ultrafast call, the second is the unentitled router's Ultrafast attempt that got a 403, and the third is its fallback to priority. The trailing False shows Ultrafast is now switched off for that router, so it never pays for a failed round trip again.

A worked example: what 6x does to a real bill

Take a coding assistant that makes 10,000 calls a day. Each call sends 2,000 input tokens, 1,500 of them cached, and gets back 400 output tokens, on GPT-6.1 Sol.

SetupCost per callCost per 10k calls
All default$0.00515$51.50
All priority$0.0103$103.00
All ultrafast (assumes 6x on cached too)$0.0309$309.00
Router: 80% default, 20% ultrafast-$103.00

The router row is 0.8 x $51.50 + 0.2 x $309 = $103. You keep the speed on the fifth of traffic where a human is waiting and pay twice the baseline instead of six times. Latency for those calls drops from roughly 0.6 + 400/50 = 8.6 seconds to 0.6 + 400/300 = 1.9 seconds, if the published throughput holds for your workload. Measure it, because 'up to' is doing real work in that claim.

A rule of thumb: Ultrafast pays off when the saved seconds are worth more than five times the standard cost of the call. For a voice agent where 6 seconds of silence loses the user, that is easy. For a background agent that already runs for minutes, it is not.

Common pitfalls

  • Billing follows the served tier. The response echoes service_tier, and OpenAI notes the value can differ from what you requested. Log both, and compute cost from the echo. The router does.
  • Ultrafast is access-controlled. Do not hard-code it on the happy path. Catch the error, degrade to priority, and turn it off for the process so you are not paying a failed round trip on every call. The exact error status for an unentitled account is not documented in what I found, so the router handles 400, 403 and 404; confirm on your own account.
  • Model coverage moves. The SDK docstring I tested with says the tier is available for gpt-5.6-sol, DevDay coverage says it launched with GPT-6 Astra, and GPT-6.1 Sol's Ultrafast price was still pending. Treat any model and tier pair as unverified until a request returns service_tier=ultrafast.
  • Cached-input price at 6x is an assumption. The 6x multiplier is published for input and output. I applied it to cached tokens too. If your invoice differs, change the multiplier table, not the code.
  • Output length dominates. Speed gains come from token generation, so the win is large for long answers and tiny for a 20-token classification. For short routing choices, a cheaper model beats a faster tier. OpenAI's new Decisions API targets exactly that case, but it is in limited preview with no public schema yet, so I did not build on it.
  • Reasoning tokens count as output. With reasoning.effort set high, most of the generation may be hidden reasoning. Ultrafast speeds it up, and you pay 6x for it. Drop effort first, then consider the tier.
  • Streaming changes the math. If you stream, the user perceives time-to-first-token, not total time. A standard tier with streaming may already feel fast enough at short outputs.
  • Do not learn speed from tiny samples. The router ignores calls under 50 output tokens when updating throughput, since fixed overhead swamps the estimate.

Quick reference

ItemValueSource status
Request parameterservice_tier="ultrafast"SDK docs, verified in 3.22.1
Response fieldservice_tier (served tier)SDK docs
Price multiplier6x standardDevDay recap and reporting
Astra Ultrafast price$60 in / $300 out per 1MReported at launch
Sol standard price$2 in / $0.10 cached / $10 outModel page
Sol Ultrafast priceNot published (about $12 / $60 implied)Pending
ThroughputUp to 300 tok/sOpenAI claim, measure yourself
AccessAccess-controlledSDK docs

Next steps

  • Log meta from every call to a table and chart spend by served tier per day.
  • Add a per-request deadline: if a call has already burned half its budget in retries, jump a tier.
  • Put the latency budget in config per endpoint, so the product team, not the code, decides what deserves 6x.
  • Once the Decisions API opens up, route trivial classification there and reserve Ultrafast for long generations.
  • Compare Ultrafast against Fast mode on your own prompts for a week before committing traffic.

Sources: OpenAI DevDay 2026 recap (openai.com/index/devday-2026-recap), the GPT-6.1 Sol model page on developers.openai.com, the OpenAI Python SDK 3.22.1 type docs for service_tier, VentureBeat and The Decoder coverage of the launch.

Comments

Subscribe to join the conversation...

Be the first to comment

Found this useful?

Get new AI guides for builders by email. Free.

Join 3,594 builders reading daily.

Also get