ModelRefs / Tool Rate Limiter — Tool Pattern
Tool Rate Limiter — Tool Pattern
Apply per-tool, per-user, and per-conversation rate limits with token-bucket policies to protect downstream systems.
Overview
Token-bucket rate limiter applied at the dispatcher. Limits scoped to tool, user, and conversation prevent runaway loops from overwhelming upstream APIs or cost budgets.
When to use it: You need to prevent runaway loops and protect upstream quotas and budgets.
Pattern details
| Pattern class | safety |
|---|---|
| Difficulty | beginner |
| Invocation mode | synchronous |
| Also known as | tool throttle, tool budget guard |
| Last reviewed | 2026-06-07 |
Known failure modes
- Starvation — Heavy user blocks light users on a shared bucket. Mitigation: Use per-user buckets in addition to global.
When not to use it
- Single global bucket without per-user fairness.
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Tool Rate Limiter — Tool Pattern.
Frequently asked questions
When should I use the Tool Rate Limiter tool pattern?
You need to prevent runaway loops and protect upstream quotas and budgets.
What are common failure modes of Tool Rate Limiter?
Starvation
Is Tool Rate Limiter production-ready?
Yes when paired with the safety controls and observability hooks documented on the pattern page.