Rate limiting and throttling

[scm_pa]

Hi. For a customer I'm putting together a desktop application that will take all of the conversations in holding queue, parse some information from the conversations attributes/participant data and use that information to move those conversations to other queues. The holding queue can have thousands of conversations in them at one time.

What I did was use PostAnalyticsConversationsDetailsJobs() to get the conversations in the holding queue in one call and then parse the information and move the conversations to where they need to be. This was working for the most part unless the application gets run again that day. Calls that have already been moved that day show as still in the holding queue due to the nature of PostAnalyticsConversationsDetailsJobs() and when the data lake is updated.

I switched to using PostAnalyticsConversationsDetailsQuery() but since that does not return the conversation attributes/participant data that I need to process the conversations I now had to add in calls to GetConversation() to query the individual conversations so that I can get to the attributes/participant data. However I'm now running into rate limiting issues. I have code in place for retry logic but I'm also looking for some suggestions on how best to throttle the api calls.

I've made some changes to cache data when possible and moved some non-api processing to different parts of the code to try and spread out the api calls and I put a sleep in that kicks in at some point. That has given me some incremental progress and I'm still seeing rate limits being hit but I was looking for any other suggestions on throttling or how others handle it. Thanks in advance.

[tim.smith]

The documentation for rate limiting and recommended best practices can be found here: https://developer.genesys.cloud/api/rest/rate_limits. It's generally not helpful to attempt to throttle your app because your app can't know where it stands with regard to remaining rate limit capacity. In most cases, you'll run full throttle until you get a 429, then wait for the prescribed number of seconds and then open the flood gates again. The main rate limiter for the API is 300 RPM, but you may hit some other undisclosed rate limits when doing a lot of bulk querying. Analytics definitely has several of them.

For your specific case, you should look into event bridge. It will allow you to get conversation notifications pushed to your preferred destination in AWS. This will avoid the rate limiting problem entirely, avoid the "if it runs again in the same day" problem, and will also allow you to operate in a timeliness of your choosing (real time, on demand, or interval processing). This is preferred over WebSocket notifications for service type integrations because Event Bridge offers guaranteed delivery and WebSockets don't.

[Eos_Rios]

If buying another product ( AWS ) to make this product actually work for you isn't something your management will sign off on either, you may want to see if this API in preview can help you get your attributes without having to poll every call one by one.

[scm_pa]

Thank you. One of the reasons that I'm throttling is because the link you posted has that as a suggestion on how to avoid 429 errors.

Regulate the rate of API requests using one of the following:

Reduce the frequency of API requests
**Apply a throttle to API requests**
Implement an exponential backoff mechanism in response to 429 errors

If the general recommendation from Genesys is to not throttle then perhaps the page needs to be updated.

[tim.smith]

It depends on the application.

An interactive web app should be more careful and should throttle its requests to avoid being rate limited. When that kind of app gets rate limited, it will make the app unresponsive to the user's commands until the rate limit period has expired. That's a bad experience and should be avoided by reserving a little rate limit capacity for the user's actions to make requests.

For a bulk service process like what you're describing, nobody is sitting there waiting for the rate limit period to expire, so it's probably not worth the effort to spread out the requests to avoid getting rate limited. As long as your app is well-behaved and stops making requests when it should, there's not an issue with that pattern. That's totally your call though; if you have reasons to spread out your requests, go for it.

edit: you did say a desktop app, but the use case sounded more like this app does this one thing as opposed to a general Genesys Cloud client that serves many purposes. Just use your judgment on what approach is best for your users; neither is inherently wrong from Genesys' perspective.