Product Update
Launching Smart Rate Limits
Hitting 3rd party API rate limit errors can lead to data loss and integration outages for your users. But with Smart Rate Limits, this will no longer be an issue
Brian Yam
,
Head of Marketing
4
mins to read
As your customer base grows and their integration usage scales, you’ll inevitably run into the complex challenges of each 3rd party API’s rate limits. That’s why we’re excited to launch the first phase of Smart Rate Limits - designed to help developers avoid data loss from hitting rate limits with their integrations.
How does Smart Rate Limits work?
Smart Rate Limits tracks how many requests are sent that count towards every rate-limiting policy, over a sliding window of time (be it minutes or days), across all request types (pre-built actions and integration request steps).
When reaching a rate limit, we automatically throttle all subsequent requests and calculate the time until the schedule resets. This mechanism enables Paragon to guarantee that the request will eventually go through, instead of failing on the spot or after multiple retries.
In scenarios where a rate limit is hit due to shared quotas across other apps, Paragon will recover from that error similarly by throttling all subsequent requests until the scheduler resets.
Why are integration rate limits painful to deal with?
A lot went into designing and building this feature due to the inherent challenges with 3rd party APIs, including:
Opaque rate-limiting policy documentation
Multidimensional rate limits
Shared quotas across apps
Varying rate limits per Connected User
Opaque documentation
Rate limits are often poorly documented, if at all, making it nearly impossible for to anticipate prior to hitting errors in load testing or production. Not to mention that these rate limits can also be changed without notice.
To address this challenge, our team has productized our internal knowledge base of all the 3rd party APIs and their rate limits that we’ve built up over the past few years, and our partnerships with many of these companies enables us to be on top of any changes to their rate limits.
Multidimensional rate limits
Rate-limiting policies can take on many forms, depending on the 3rd party API.
Limits across different levels (per API key, per user, per IP address, or per org, per endpoint, per developer app)
Limits across different frequencies (requests per minute, hour, or day)
They impose concurrency limits (requests per minute), daily totals per user, as well as daily totals per developer account (which is especially relevant for multi-tenant SaaS applications).
The challenge is also exponentially harder to build around within a distributed systems architecture because you will need to keep track of request volumes/frequency in aggregate across all the different potential sources of requests.
Smart Rate Limits automatically tracks all requests made to a 3rd party API at every level. Whether you have 100 Connected Users sharing the same developer app client id and secret, multiple workflows for a single integration, or even if you’re using our Connect Proxy API, all requests are tracked in aggregate against every rate-limiting policy.
Shared quotas across apps
Your app may only be one of many that your user has integrated with a particular application. For example, if you’ve built a Slack integration, you will definitely be sharing quotas with the dozens of other Slack integrations your users have enabled.
So even if you keep track of all the requests made to a particular 3rd party API from your own app perfectly, you will never have visibility into all the other requests being made to your user’s Slack account from other applications.
In this case, Smart Rate Limits would recover, throttle, queue, and retry requests automatically if we hit a quota due to requests from your users’ other integrations with that same app.
Varying rate limits per Connected User
Rate limits can also vary depending on a Connected User’s plan - for example, Salesforce allows for higher rate limits on higher plans (ie. Enterprise vs. Professional). Smart Rate Limits will account for this variability at a Connected Users level by pulling in their plan/tier IDs to determine the relevant rate limit.
What’s next?
We’ve started with the most common, high-volume integrations, including Salesforce, HubSpot, and Netsuite, and will continue rolling Smart Rate Limits out to more of our integrations.
Beyond that, we plan to make Smart Rate Limits a generalized feature with configuration options so it can be used for Custom Integrations as well!
