
Inside OpenAI's Real-Time Access Engine
Highlights from our AMA with Jonah Cohen, Tech Lead for Financial Engineering at OpenAI
The model runs in your VPC. The meter runs in someone else's. Why that matters, and the four other things a usage layer has to get right.

Microsoft just stood up a $2.5 billion “Frontier Company” and pointed 6,000 engineers at deploying AI inside customer environments. Amazon committed a billion dollars to the same bet two days earlier. In May, OpenAI and Anthropic each launched their own forward-deployed ventures, backed by TPG, Goldman Sachs, and Blackstone. Palantir built an entire company on the idea two decades ago.
The whole industry is moving AI into the customer's environment. Sovereign models, private VPCs, air-gapped inference, ZDR contracts with every provider. Enterprises are spending millions to make sure the intelligence never leaves their walls.
And nearly every one of them still runs one wire out the back: to a third-party metering cloud.
We made the inference sovereign and left the usage data behind. The model runs in your VPC. The meter doesn't. That is not a smaller problem than the one we just solved. It is the same problem, one layer down.
Two things happened in the last year that turned this from the exception into the default.
First, Zero Data Retention became table stakes. Every major provider, OpenAI, Anthropic, Google, AWS Bedrock, now offers ZDR: they process your data to generate a response, then delete it. No storage, no logging, no training. It used to be a premium ask. Now it is a line on the procurement form. If your data is regulated or your customers' data is involved, legal has already had this conversation.
Second, the open-weight models caught up. Z.ai shipped GLM-5.2, a 744-billion-parameter frontier model, under a fully open MIT license: download it, self-host it, fine-tune it, no restrictions. Moonshot's Kimi K2.6 and K2.7 are trillion-parameter open-weight models that enterprises are already running on their own hardware for data sovereignty and cost control. DeepSeek, Llama, Qwen, Mistral, all of them now ship models you can run entirely inside your own environment. For most production workloads, the capability gap with the closed frontier has effectively closed.
Put those together and the conclusion is hard to avoid. There is no longer a good reason for sensitive inference to leave your environment, so it doesn't.
Which makes the exception glaring. You pulled the model in-house. You self-host the weights. You signed ZDR so the provider forgets your prompts the moment it answers them. And then you hand every token of consumption data, every credit balance, every usage event, to a metering SaaS running in someone else's cloud. The one system you didn't bring home is the one that sees everything you spend.
BYOC means Bring Your Own Cloud. The Stigg usage runtime deploys inside your own environment: your AWS, GCP, or Azure account, your VPC, your network, your security policies. Not our servers. Yours. You get the full runtime, metering, entitlements, credits, and governance, and none of the consumption data leaves your infrastructure. SaaS metering tools route every usage event to their own cloud. That is the design, not a setting you can flip. BYOC is a different architecture, not a different pricing plan.
Six things change the moment enforcement runs inside your VPC instead of over the public internet to a vendor's API.
Scale without a vendor ceiling. Third-party metering hits rate limits and throughput caps at enterprise volume. In your own cloud you scale the meter the way you scale everything else: horizontally, on your infra, with no external bottleneck. In BYOC, Stigg handles 1M+ events per second.
Enforcement in milliseconds. When the entitlement check runs in your VPC, it travels your internal network, not a round trip over the public internet to someone else's API. At AI scale that difference compounds into latency your users can feel. Keep the check where the workload is and it stays in milliseconds.
Costs that don't scale with your success. SaaS metering charges per event or per transaction. The better your product does, the bigger their invoice. BYOC flips it: a fixed infrastructure cost, no per-event markup. Your metering bill stops being a tax on growth.
Data that never leaves. Every API call, every token, every credit balance. With SaaS metering, all of it lives on someone else's servers. BYOC means it stays in your environment. You own it, you audit it, you decide who sees it.
Security with no new front door. No extra attack surface. No new vendor in your SOC 2 scope. No data egress to defend in the next security review. The meter inherits the posture you already built: your IAM, your encryption, your network policy.
Rate limiting that happens in real time. When enforcement sits next to your application, limits are synchronous. You can cut off a runaway agent before it burns through a customer's entire credit balance, not thirty seconds later when a batch job finally notices. For agentic workloads, that gap is the difference between an alert and a bill.
Here is the objection you will hear: we already have metering, it works, why change it. Fair. But that is not the real choice. The real choice is between metering you brought home and metering you didn't. You already decided that sensitive inference belongs inside your walls. The usage data that records exactly how your customers consume your product, what they spend, where the money moves, is at least as sensitive as the prompts. Probably more. It is the financial record of your business. So the question isn't whether your metering engine is good. Plenty of them are. Sovereignty is the first thing to ask about it. It is not the last.
Sovereignty is the most visible failure, which is why it is the one people notice first. It is not the root of the others. A meter can run entirely inside your VPC and still enforce on a batch cycle. It can be fully sovereign and still be a counter rather than a ledger. Where a usage layer runs is one property of it. It does not confer the rest.
After 18 months of building this layer, the more useful question turned out not to be where your metering runs. It is what any metering layer has to get right before you put the financial record of your business inside it. We landed on five. They spell GREAT.
Governed. Governance is the thing teams add after the incident. An agent loops, a customer burns through a month of credits in an afternoon, and the postmortem asks who could have stopped it. The answer is usually that nobody could, because the limits lived in three places: some application code, a vendor dashboard, and a spreadsheet that finance maintains by hand. That happens because most metering systems treat governance as reporting. They tell you what was consumed. Deciding what is allowed to be consumed is left to you. The alternative is to make the limit part of the entitlement itself, so that a hard limit means the customer consumes exactly what they have and cannot spill past it, and a soft limit means you are allowing the overdraft on purpose and have decided in advance how it settles. A limit that lives outside the entitlement is a suggestion.
Real-time enforcement. You have just read the case for this one. An agentic workload does not consume in a straight line, it spikes, and one retry loop can burn a customer's entire balance in the time it takes a batch job to run. Enforcement that happens on a cycle is not enforcement. It is a report that arrives after the money is gone. The check has to sit in the request path, before the action runs, and it has to return fast enough that nobody is tempted to route around it.
Every billing system. Teams collapse metering and billing into one system because at the beginning they look like one problem. Usage logic ends up inside the billing vendor, and it works, right up until pricing changes. Then the asymmetry shows. Your pricing changes far more often than your billing stack does: a new plan, a credit pack, a different unit, an enterprise deal with its own terms. Replacing your payment rails is not a quarterly event. When usage logic lives inside billing, every pricing experiment becomes a migration, and every billing migration becomes a pricing freeze. Stripe, Zuora, homegrown, more than one at once during a transition: your usage runtime should sit above billing, not inside it.
Autonomously deployed. This is the letter the rest of this post has already argued for. Your cloud, your data, fixed costs. If your models do not leave your VPC, your consumption data should not either.
Transaction-grade ledger. Credits almost always start the same way: a number in a row, decremented on use. It works for a year. Then finance asks what a customer's balance was on the last day of the quarter and why, and nobody can answer, because the only record of the balance is the balance. The fix is not a bigger number. It is to stop storing the balance at all. Every grant is an entry, every deduction is an entry, and the balance is what the entries add up to. Once that is true, the questions that were unanswerable become ordinary: which grant did this deduction draw from, what happens when a retried request arrives twice, which credits expire first when a customer holds three grants with different terms. The finance side follows from the same structure. When each grant carries what it cost, the ledger already knows what has been recognized, what is still deferred, and what expired unused. The question finance asks is never what the balance is. It is what the balance was, and why.
Two of the five are architectural, which is the part worth paying attention to. Enforcement that runs on a cycle does not become synchronous with a config change, and a counter does not become a ledger retroactively. Those belong on the evaluation checklist, not the roadmap.
Bring the meter home.