Telling clients when to come back after a limit

Predictable behaviour follows, and none of it is helpful to either side.

A client refused by an HTTP response that tells it nothing has two reasonable options: give up, or try again. Most of them try again, usually at once, because nothing in the response suggested that waiting would help. That retry arrives while the condition which caused the refusal is still true, is refused in its turn, and the whole cycle repeats itself.

The result is that a limit intended to reduce load ends up producing additional load, made up entirely of retries that had no chance of succeeding. That is the same self-inflicted amplification described under bursts, arriving from a different direction.

Worse, those retries are indistinguishable from the abuse the limit existed to catch in the first place. A legitimate client hammering because nobody told it anything looks like a client hammering on purpose, and no amount of analysis separates them afterwards.

One thing changes, and it only works on the clients that were already trying to behave properly.

Told when to return, a well-built client waits and then comes back exactly once. Load during the refusal period drops to nothing from that client, and the request that eventually arrives succeeds, which is the outcome both sides were after all along.

This is why the practice is worth adopting even though nothing about it can be enforced on anybody. It costs nothing at all to include, it changes the behaviour of every client whose author cared about behaving well, and it makes the traffic that remains considerably more informative: a client that keeps hammering after being told when to return has told you something about itself.

That last point is the one people miss most often. Guidance is not only a courtesy to well-built clients; it is a filter that separates the merely careless from the genuinely indifferent.

One more property: this is the rare defensive measure whose benefit grows with how well-behaved your traffic already is. On an interface used mostly by careful integrations, almost every refused client will honour the instruction, and the load reduction is close to total. On an interface under deliberate attack, it changes nothing. Knowing which of those describes you tells you how much to expect from it.

Three rules, and all of them are about honesty rather than about technique.

Say something you will honour. If you name a moment and the limit is still in force when that moment arrives, the client is refused a second time and learns to distrust the guidance entirely, which leaves you worse off than saying nothing would have.

Do not name a moment that brings everybody back at once. Telling a thousand refused clients to return at the same instant produces a synchronised burst precisely then, which is a queue you built yourself. Spreading return times, even slightly, avoids building a queue that discharges in one moment.

Prefer a short honest wait to a long safe one. An overly cautious answer makes clients wait when they could have proceeded, and for interactive traffic that difference is felt directly by whoever happens to be sitting in front of the screen at the time.

There is a second, quieter benefit. A refusal that carries an instruction is one somebody can act on without contacting you, so the volume of "my integration stopped working" conversations falls along with the retry traffic. That saving lands on people rather than on servers, and it is the one most likely to be noticed by whoever has to answer those messages.

Three cases where it changes nothing, and the third is by far the most important of them.

Clients that do not read it. Anything that ignores the HTTP response and retries on its own schedule is entirely unaffected. Guidance is advice, and advice reaches only the willing.

Deliberate abuse. Traffic sent specifically to overwhelm you is not waiting for permission to continue, and it will not start waiting now.

A limit that is simply wrong. If ordinary use is what keeps being refused, telling people when to come back fixes nothing whatsoever; it only makes the refusal politer. The correct action is to look at what is being refused and why, which is what watching before enforcing is for, described under operations.

That third case needs a closer look, because guidance can conceal a mistake rather than reveal it. Refusals become tidy, retries become orderly, dashboards look calm, and everything appears managed while legitimate traffic is still quietly being turned away at the same rate as before.

Stated separately from all of the design advice above, and narrowly.

Rate limiting is applied per domain, which is the unit the limits are counted against. Everything above is guidance for designing limits, whatever enforces them, and the choice of what a refusal tells a client belongs wherever those responses are produced.