The chatbot kept talking after the API balance hit zero

At 06:43 UTC on Sunday, November 1, our Anthropic account made its last successful call. The next request came back with a 400 error: credit balance is too low. We pay for Anthropic in advance, and the balance had reached zero.

For the next 63 hours, RezScore looked mostly fine from the outside. That was the problem.

Half the product kept working

Jen, our chat advisor, can run on more than one model. When an Anthropic call fails, chat retries the same request on DeepSeek. We built and tested that fallback for exactly this kind of day. So people who uploaded a resume that weekend could still ask Jen questions and get answers. Nobody saw an error page. The site’s health checks stayed green.

The full resume report has no fallback, on purpose. It reads a resume line by line and writes specific, factual notes about it, and we only run it on the model we have validated for that job. A weaker model guessing at someone’s work history is worse than no report. With Anthropic refusing every call, each report stopped partway, and the page told the visitor we could not finish the full read. Every full report started during the outage ended that way.

The friendly half of the product kept talking while the half that does the real analysis was down. A working chat made the outage harder to notice, not easier.

How the balance ran out

Nothing attacked us, and no job got stuck in a loop. On a normal day our Anthropic spend was under a dollar. Over the last three days of October it roughly tripled.

Nearly all of that increase came from one conversation. Every chat request sends the model the conversation so far, so it can answer in context. That is normal and necessary, but it means each turn costs more than the last: a late message pays again for every message before it. Over those three days the average chat request to our main chat model carried about 29,500 tokens of input. The largest carried 55,586, and nearly all of it was earlier conversation being sent again. That single conversation accounted for 98 percent of the period’s chat spend on that model.

We did have daily spending caps per model. The comment above them said what they did and did not do: a cap was checked when a conversation was assigned a model, not when a request was made. A conversation already in progress kept going no matter what the day had cost. The caps were also sized for a larger prepaid balance than we had left. So the long conversation kept running and kept growing, until the account was empty.

What we built

Codex, one of the AI agents that works on RezScore, wrote the fix on November 2, and it went live at 23:57 UTC that night. I reviewed it and verified it in production. It has three parts.

1. A budget check before every request. Every provider class in the app now gets its chat method wrapped when the class is defined:

cls.chat = controlled_chat(traced_chat(cls.__dict__['chat']))

Before a request goes out, controlled_chat adds up what that model has already cost today, adds an estimate of the request about to be made, and refuses it if the total would pass the cap. The estimate is deliberately pessimistic: input characters divided by four, plus at least 4,096 output tokens, all priced as if nothing were cached. Anthropic also gets an overall daily allowance across all its models, and a fixed budget for a set period that never resets on its own. A model with no known price is refused rather than waved through. Because the check runs on every call, a conversation that started before the cap was hit is now covered too. When chat is refused, it falls back to DeepSeek like any other failure.

2. A shorter memory for the model. Chat now sends at most the last 30 messages of a conversation. The instructions that shape how Jen behaves are always sent in full, and nothing is deleted from the conversation itself; the model just stops being sent all of it every time. That puts a ceiling on the cost of a single request, which is what turned one long conversation into an outage.

3. A shared switch for a dead account. Before the fix, every server process found out about the empty balance separately, one failed call at a time. Those failed calls were not billed, but each one added a wasted round trip before the fallback, and nothing stopped the next one.

Now the first confirmed out-of-credit error flips a switch stored in our PostgreSQL database, keyed by provider and a SHA-256 fingerprint of the API key. Every process reads the same row, so one failure stops every call to that account before it leaves the building. A rate-limit error (429) does not flip it; that is a different problem with a different cure. The alert goes out once per incident instead of once per failure. While the switch is open, new visitors are not enrolled in our test comparing chat models, so the test is not quietly measuring an outage.

The switch never resets itself. After credit is added, someone runs one small paid request on purpose, capped at 16 tokens of output, and the switch closes only if it gets a real answer back. We chose that over automatic retries because a retry loop against a dead account is exactly the failure we were trying to remove.

Recovery

The account was refilled on November 3. The probe went through, and the first successful Anthropic call after the outage came at 21:52 UTC. We then reran the reports that had stopped partway. Every full report started during the outage has since completed.

Two days later we made one more change. Chat and the full report shared one Anthropic daily allowance, so a single long conversation could still use up the money the reports needed. We lowered the cap on the chat model so that the remaining allowance is effectively held back for reports. When chat hits its cap now, it moves to DeepSeek for the rest of the day, and the reports keep running.

What these controls cannot promise

We want to be precise about the limits.

• The check is an estimate, not a reservation. Several requests checked at the same moment can all pass and then all spend. Estimating tokens from characters can be off, and an SDK retry can spend twice. The checks make an overrun small and rare. They do not make it impossible.
• The app cannot see the prepaid balance. It knows what it has recorded spending, not what is left in the account. A spending limit set with the provider itself, and some money held in reserve, are still needed.
• Long conversations are still expensive. Thirty messages is a ceiling, not a discount. Prompt caching, which lets the provider bill repeated conversation history at a lower rate, is the next step, along with per-account allowances so one visitor cannot use a day’s budget.

What we took from it

A fallback is good engineering until it hides the outage it is covering for. Ours worked so well that the product looked healthy while its most important feature was down. If part of your product can silently switch models, alert on the provider errors themselves, not just on whether the page loads.

The cost of a conversation also grows with its length, not with the number of people talking. Our spending caps assumed money would go out a little at a time across many visitors. This time it went out in one conversation that kept getting more expensive, and the only check in its path ran once, at the start.