Usage Metering Explained: Architecture and Pipelines

Usage metering works fine until an AI agent starts generating 3,000 events per minute across a multi-step workflow and the pipeline falls behind. Enforcement is reading stale state. A customer blows past their limit. The metering layer catches up an hour later, by which point the overage has already been charged.

This guide covers how usage metering works, where pipelines break under AI workloads, and what the infrastructure needs to handle it at scale.

What is usage metering?

Usage metering is the process of tracking and converting product usage into measurable data that can be billed or enforced.

It captures, transforms, and aggregates raw consumption into structured metrics. Billing then uses those metrics to apply pricing and generate invoices.

For engineering teams, usage metering is the pipeline that records what was consumed, by whom, and when. Without it, there is no reliable way to enforce limits, calculate charges, or provide visibility into usage.

How usage metering works

Usage metering works by capturing raw usage events, transforming them into measurable metrics, and applying pricing rules to determine charges.

The pipeline has three stages, each adding structure before data reaches billing or enforcement. The table below maps each stage to its infrastructure requirement:

1. Ingestion

Ingestion collects raw telemetry such as API calls, tokens consumed, storage written, and agent actions completed.

This data arrives in different formats and volumes, often at very high scale. At this stage, the system standardizes and cleans the data so it can be processed consistently.

2. Metering

Metering converts that normalized event stream into billable, aggregated metrics.

For example, a storage system tracks usage over time and converts it into billable units like GB per month by applying aggregation logic.

3. Rating

Rating applies pricing rules to those metrics to calculate charges. This includes handling tiered pricing, discounts, credits, and overage fees.

The aggregation method depends on what you are measuring:

• Count: Number of events such as API calls or actions
• Sum: Total volume such as tokens or data transferred
• Average: Mean value over a period, such as average request size
• Max: Highest observed value, such as peak concurrent users

Choosing the wrong aggregation method leads to incorrect billing and makes reconciliation difficult.

Real-time vs. batch usage metering

Real-time and batch usage metering differ in how they process events and how they support enforcement in a system.

Batch metering processes events at set intervals such as hourly, daily, or at the end of a billing period. It works for billing reconciliation where accuracy matters more than speed, but it can’t enforce limits before usage exceeds them.

Real-time usage metering processes events as they arrive and updates usage state immediately. This allows the system to check balances, validate entitlements, and allow or block actions before they run.

For AI products where a single agent session can generate thousands of events in minutes, batch metering creates an enforcement gap. By the time the pipeline processes the events, the usage has already happened, and the overage has already posted.

Real-time usage metering requires a pipeline that can handle bursts of events, a low-latency layer for usage reads, and fallback behavior when parts of the system are unavailable.

Why usage metering is harder in AI products

Usage metering is harder in AI products because consumption is more granular, costs are tied to infrastructure, and workflows span multiple steps.

AI workloads introduce three challenges that standard SaaS systems do not face:

• Granular and variable units: Token usage changes based on input length, output length, model type, and reasoning mode. A single request can vary widely in cost, even with similar inputs.
• Direct cost impact: Each unit of usage carries real infrastructure cost. Metering errors affect margin as well as billing accuracy. AI pricing models are typically based on token usage, where each request maps directly to cost, as reflected in OpenAI’s pricing model.
• Multi-step workflows: A single user action can trigger multiple model calls, tool executions, and retrieval steps. Each step consumes usage that must be tracked and attributed correctly.

These factors make attribution more complex, since usage is spread across multiple steps within a workflow. The system needs to track consumption at each step so the total reflects what actually happened behind a single user action.

A single event at the request level misses that detail when usage is distributed across multiple internal operations.

Usage metering vs. enforcement: What’s the difference?

Usage metering tracks what was consumed, while enforcement uses that data to decide what can happen next in real time. These two layers serve different roles in the system:

Usage-based billing tools focus on metering. They capture events, count usage, and pass that data to billing systems for invoicing. Their role centers on accurate usage reporting over time, while enforcement happens separately at request time.

Enforcement lives in the entitlements layer. It defines what a customer can consume based on their plan and current usage, including limits, current balance, and what happens when those limits are reached.

When usage approaches a limit, enforcement acts before the request completes. Metering records the outcome after the fact. Both layers work together, but they solve different problems.

Where in-house usage metering breaks in production

In-house usage metering starts to break when systems move beyond simple counters and need to handle shared usage, real-time updates, and changing plans.

Most teams build a basic metering setup early on. A credits table and a decrement function work at low scale, and the decision to build in-house makes sense at that stage.

The issues show up as usage grows and the system needs to stay consistent under load:

• Concurrent usage: Multiple users drawing from a shared balance can create race conditions. Atomic debit operations against an append-only ledger keep usage accurate.
• State consistency: Cached entitlement data becomes outdated when plans change mid-session. The system needs a clear strategy for cache refresh and fallback behavior.
• Attribution: AI workflows often span multiple model calls and tool steps. Each step needs to be mapped back to the correct customer, feature, and session.
• Auditability: Finance requires an append-only record of usage and credits. Mutable balances make reconciliation and dispute handling difficult.
• Plan changes: Mid-cycle upgrades introduce edge cases around prorated credits, active limits, and legacy plan handling.

At this stage, the problem shifts from tracking usage to maintaining consistent state across the system.

Enterprise customers make this harder. They need usage tracked and enforced independently across teams, departments, products, and agents within the same organization.

Most in-house systems were never designed for that cardinality and get retrofitted under deadline when the first enterprise contract demands it.

Stigg handles this natively per-agent, per-team, and per-department enforcement without retrofitting.

Why usage metering needs enforcement to work

Usage metering works only when the system can act on that data before a request completes.

Accurate usage tracking alone does not control behavior. The system needs to read current usage, resolve entitlement state, and decide whether to allow or block the action in the request path.

That decision has to happen locally to keep latency predictable. A remote call on every request adds delay that compounds under load.

At production scale, systems handle this by:

• Keeping entitlement state in a local cache
• Resolving checks on cache hits without network calls
• Falling back to an upstream service on cache misses
• Defining clear timeout and fallback behavior

For teams with data residency requirements, the enforcement layer needs to run inside their own infrastructure. A BYOC deployment keeps entitlement data within the VPC and removes any dependency on external uptime for read-path decisions.

If the upstream service loses availability, the local cache keeps enforcement running without falling back to open access or blocking all requests.

This structure keeps enforcement fast and consistent while usage continues to change in real time.

How Stigg handles usage metering and enforcement

Stigg is the usage runtime for AI products. Entitlements, credits, usage limits, and spend governance run synchronously in the request path before compute is consumed.

For engineering teams building or rethinking this layer:

• Sidecar deployment runs as a Docker container alongside your application, keeping enforcement in the request path without a remote dependency on every check
• Local Redis cache stores entitlement state so access decisions resolve immediately on cache hits, with the Sidecar fetching from Stigg's Edge API at around 100ms on misses
• BYOC Sidecar runs inside your own VPC, keeping enforcement operational under high concurrency and upstream interruptions with no external data exposure
• Failure isolation means cached state continues serving decisions during upstream outages, so enforcement stays stable when external systems are unavailable
• Real-time metering processes usage events continuously at scale, supporting high event volumes without delaying enforcement decisions
• Usage events sync to billing systems like Stripe while enforcement stays independent of the billing layer

You don't need to adopt the full stack since each component works independently, so teams can start with credits or enforcement and expand from there.

When metering and enforcement are separated correctly, pricing changes stop being engineering work. Usage decisions stay fast under load, and the audit trail is already there. If that's not where your system is, see how Stigg’s infrastructure works.

FAQs

1. What is the difference between usage metering and billing?

The main difference between usage metering and billing is that metering records what was consumed while billing applies pricing rules to generate invoices. Metering runs continuously as usage occurs. Billing runs on that data after the fact to calculate charges and process payments.

2. What is the difference between real-time and batch usage metering?

The main difference between real-time and batch usage metering is when events are processed. Batch metering processes usage at fixed intervals and works well for billing reconciliation, but cannot enforce limits before they are exceeded. 

Real-time metering processes events as they arrive, which allows the system to check balances and block requests before usage completes.

3. How does usage metering work in AI products?

Usage metering in AI products tracks token consumption, agent actions, and multi-step workflow costs at the event level. 

Because a single request can vary widely in cost depending on model type, input length, and reasoning mode, the metering layer needs to attribute usage at each step rather than recording a single event at the request level.

4. Why does usage metering break under concurrent load?

Usage metering breaks under concurrent load when multiple sessions draw from a shared balance simultaneously and the system lacks atomic operations to prevent race conditions. 

Without atomic debit operations against an append-only ledger, two concurrent requests can read the same balance before either commits, both go through, and the balance goes negative without triggering a hard limit.

5. What infrastructure does real-time usage metering require?

Real-time usage metering requires a high-throughput ingestion pipeline, a low-latency read layer for enforcement decisions, and fallback behavior when parts of the system are unavailable. 

At scale, this means keeping entitlement state in a local cache so checks resolve without a network call on every request, with a configurable timeout to prevent upstream latency from cascading into the application.