The credit ledger: AI cost as integers, not floats

Why AI cost is harder than SaaS cost

Traditional SaaS billing is a flat number per seat per month. The unit economics are a spreadsheet. AI cost is none of those things.

A single chat completion request carries:

• Per-token pricing — input tokens, output tokens, reasoning tokens, cached tokens, each at a different rate.
• Per-provider rates — OpenAI, Anthropic, Google, Groq, DeepSeek all price differently, with different billing units and different discount mechanics (prompt caching, batch, volume tiers).
• FX exposure — provider list prices are in USD. Customers in Japan, the EU, and the UK pay in JPY, EUR, GBP. The rate moves daily.
• Workload-driven variance — the same prompt can cost $0.0003 or $3.00 depending on model choice, output length, and tool use.

The result: cost is not predictable from the request alone. It’s only known after the upstream call returns and the provider reports usage.

This breaks the naive billing model where you charge what the provider charges you. By the time you know the cost, the request is over. If the customer had no budget for it, you’ve already eaten it.

The representation: micro-USD, integers

SchneeAI stores cost as BIGINT micro-USD. One US dollar is 1,000,000 micros. A typical GPT-4o completion costs 1,000-3,000 micros ($0.001-$0.003 for a medium chat turn). A DeepSeek-V3 completion costs 100-300 micros.

Why integers, not NUMERIC(10,4) or FLOAT?

Floats lose money. Repeated addition of small floating-point values accumulates rounding error. Across millions of requests per day, the error is real. Worse, it’s not deterministic — the same workload charged twice produces slightly different totals. That’s unacceptable for a billing system.

Decimals are correct but slow. NUMERIC is exact, but arithmetic on it is meaningfully slower than on BIGINT, and it can’t be atomically incremented in Postgres without taking a lock. For a hot path that updates the ledger on every request, that matters.

Micros as integers hit the sweet spot. Six decimal places is enough resolution for realistic per-request charges (a minimum-viable 10-input/10-output-token DeepSeek request costs ~14 micros and rounds cleanly). Integer arithmetic is exact and fast. Atomic UPDATE ... SET balance = balance + $1 works without locking surprises. And BIGINT’s 2^63 ceiling holds ~$9.2 trillion — roughly a third of US GDP, and more than any individual account will ever accrue.

The same convention is applied to customer-facing JPY amounts (amount_jpy BIGINT) and credits (credits BIGINT, where 1 credit = 1 micro-USD by default). Exchange rates are stored as NUMERIC(20,10) because they’re ratios, not money.

The timing problem

The fundamental design problem is this:

You must decide whether the customer can afford the call before you make it. You only learn what the call cost after it returns.

If you skip the upfront check, a $5 request from a customer with $0 credit is a $5 loss. If you naively reserve the maximum possible cost, you lock up balances for hours and customers complain.

The pattern that solves this is two-phase reserve-and-settle — borrowed from payment processing, where it shows up as “authorize” and “capture”.

Phase 1: Reserve (Tx1)

Before the upstream call, SchneeAI opens a Postgres transaction and:

1. Estimates cost based on the model, prompt size, and max_tokens. The estimate doesn’t have to be exact — it has to be an upper bound with high confidence.
2. Reserves credit by inserting a row into the credit ledger with type=reservation and amount=estimate. The running balance is the sum of settled entries minus open reservations.
3. Checks budget enforcement — service-level, tenant-level, user-level. Budgets see the reserved amount as already spent.
4. Commits.

If the customer has insufficient credit or has hit a budget cap, the transaction rolls back and the request never reaches the provider. The caller sees a structured insufficient_credit or budget_exceeded error.

The reservation row carries the interaction ID and a short TTL. It’s the placeholder that gets reconciled against reality.

Phase 2: Settle (Tx2)

The upstream call completes. The provider’s response includes token usage. SchneeAI now knows the actual cost.

A second Postgres transaction:

1. Inserts a settled entry with type=charge and the actual cost (computed from the provider’s token counts × the model’s per-token rates).
2. Deletes the reservation row — or, equivalently, inserts a compensating release entry. We chose delete because reservations are ephemeral by definition; their existence in the table means the request is in flight.
3. Updates the usage record with the actual token counts and cost.
4. Commits.

The ledger is the source of truth for the balance. The usage record is the source of truth for why the balance changed.

Why this split matters

Two transactions, with the external LLM call between them, is more complex than one transaction. Why bother?

Because the LLM call cannot be inside a Postgres transaction. A 30-second streaming response keeps a transaction open for 30 seconds, holding locks and exhausting the connection pool. Multiply by concurrent requests and the database falls over.

Because failures are asymmetric. The upstream call can fail, time out, return partial output, or return output the customer rejects. Each case has a different financial consequence:

The two-phase pattern makes these cases handleable. A single-phase “charge after the call” model would either block the response until the database is available (terrible latency) or accept that some charges are lost (terrible revenue).

Idempotency

The credit ledger has an idempotency_key column with a unique index. The key is derived from the interaction ID and the operation type, so a retry of the same logical operation hits the unique constraint and is safely rejected.

This matters because retries are everywhere:

• The gateway retries a transient provider failure. The retry is a new interaction with a new ID — two reservations, two potential charges, but only one provider success. The reconciliation worker spots the duplication and reverses the orphan charge.
• The client retries after a network drop. Same logic: idempotency on the client request key means the second attempt is a no-op.
• The reconciliation worker itself retries. Idempotent insert means it can re-process safely.

Idempotency at the ledger level is what makes the system robust to exactly-once billing semantics without requiring exactly-once delivery from any of the upstream components.

Budgets as a control layer

A budget is a soft cap on spend over a window: per day, per month, per feature. It’s a layer above the credit ledger.

When a reservation is created, the budget counters are incremented. When the reservation settles, the budget counter is adjusted by the difference between estimate and actual. When the reservation is released, the budget counter is decremented.

Budgets have threshold actions:

• Notify at 50% / 80% / 100% — email or webhook.
• Throttle at 90% — start rejecting low-priority requests.
• Hard stop at 100% — all requests return budget_exceeded.

The hard stop uses the same code path as insufficient credit. From the caller’s perspective, the error is the same — the difference is whether the cap is financial (no money) or operational (no budget for this feature this month).

The interesting design choice: budgets check against the estimated cost at reservation time, not the actual cost at settle time. This means a workload that consistently underestimates will gradually drift over the cap. We accept that — budgets are a soft control, not a hard wall. Customers who need a hard wall set their budget threshold lower than their actual ceiling.

Refunds, corrections, and the append-only rule

The credit ledger is append-only. There is no UPDATE to a ledger row. Corrections are new rows:

• type=refund with negative amount — reverses a charge.
• type=adjustment with positive or negative amount — manual correction by ops.
• type=grant with positive amount — monthly credit grant, promotional credit, etc.

Each entry references the original entry it corrects, with a reason and an operator ID. The current balance is always SUM(amount) WHERE account_id = $1. There’s no cached balance column to drift.

This pattern — borrowed from financial accounting — has three properties that matter:

1. Auditability. Every change to a customer’s balance is a row with a timestamp, a reason, and an actor. There’s no “the balance used to be X, now it’s Y” — there’s a chain of entries that proves it.
2. Reproducibility. The current balance is a function of the entry history. If the schema changes, you can replay.
3. No drift. A cached balance column will eventually diverge from the sum of entries, usually at 3am during a billing cycle close. Append-only means there’s nothing to drift.

The cost is a SUM() on every balance read. With a partial index on account_id and materialized views for hot reads, this is fast enough at SchneeAI’s scale. At much larger scale, a periodic snapshot table plus delta-since-snapshot would be the next step.

The reconciliation worker

No real-time system gets every charge right on the first try. SchneeAI runs a reconciliation worker that:

1. Compares the gateway’s record of provider calls against the provider’s usage API.
2. Spots mismatches: calls where the gateway recorded a charge but the provider reports different usage, calls where the gateway crashed before charging, calls where the provider reports usage the gateway didn’t see.
3. Inserts correction entries with type=adjustment and a reason=reconciliation note.

The worker is idempotent and runs hourly. It’s the safety net for the cases the two-phase pattern doesn’t catch — mostly network partitions between gateway and database, or between gateway and provider.

Reconciliation also catches the rare case where a provider’s reported usage decreases after the fact (some providers revise down for aborted generations). The negative adjustment flows through naturally.

What we’d tell someone building this from scratch

1. Use integers. Micro-USD as BIGINT. Don’t reach for NUMERIC until you’ve benchmarked integer arithmetic and found it insufficient.
2. Separate reservation from charge. Even if your external call is fast and reliable today, the pattern is the same and the cost of building it up front is small. Retrofitting it later means a data migration.
3. Append-only ledger, always. Cached balances drift. Don’t keep a balance column.
4. Idempotency keys on every write. Retries are everywhere; dedupe at the database.
5. Build the reconciliation worker before you need it. By the time charges are going missing, you’re already losing money and trust.
6. Budgets are soft, credit is hard. Don’t make budgets block payment-critical paths. The credit ledger is the source of truth; budgets are a usability layer.

The two-phase reserve-and-settle pattern is not novel — it’s how payment processors have worked for decades. What’s specific to AI is the variance: the gap between estimate and actual can be 100x, the calls can take 30 seconds, and the failure modes are stranger than HTTP error codes. The ledger design has to absorb all of that without losing pennies.

The credit ledger is one of the operational primitives SchneeAI ships with. Read the product overview to see how it fits with the gateway, budgets, and reconciliation, or start a conversation about your billing requirements.