Changes to V1 API Rate Limiting
As of Oct 27, 2020, Whiplash is switching our V1 API to a new rate limiting strategy. This new rate limiting strategy uses a “Leaky Bucket” algorithm. This new algorithm will allow Whiplash customers to access the necessary data without compromising other users' experience. For the majority of V1 API users, the ability to "burst" will be a major enhancement.
Leaky Bucket Basics
The algorithm works similar to an actual leaky bucket holding water. The leaky bucket method allows requests to come into Whiplash up to a set maximum capacity. As this bucket is being filled up, some of the requests are “leaking” out of the bucket allowing for more capacity. If a request that comes in while the bucket is still full, we will throttle that request. Eventually, even a full bucket will empty.
Dealing with throttles
When the Whiplash V1 API throttles a request, the server will respond with a 429 response code and a message “Rate limit exceeded”. This request did not make it to whiplash and it is the clients responsibility to re-send. On every request sent to Whiplash, it is necessary to check that our server returned a successful response code.
We have our leaky bucket algorithm set up with the following parameters:
- Burst Rate: 200
- Threshold: 2
- Interval: 10 seconds
This means that we allow the bucket to hold 200 requests. Every 10 seconds we add 2 requests.