Concurrent API Requests - Too Many Requests

[Adrian_Santamaria]

Hello

(This topic is an extension of this one. )

I've been trying to perform asynchronous requests using the new version of the Python SDK, and although I do not get Warnings anymore, now I am getting Exceptions caused by 429 Too Many Requests errors.

When using the SDK synchronously, it handles the request limit, and if it reaches it, then it stops and continues after the timeout. However, doing the asynchronous requests that does not work properly.

Here you have my code:

import PureCloudPlatformClientV2

client_id = "my_client_id"
client_secret = "my_client_secret"

apiclient = PureCloudPlatformClientV2.api_client.ApiClient()
apiclient.rest_client = PureCloudPlatformClientV2.RESTClientObject(pools_size=10, max_size=10)
apiclient = apiclient.get_client_credentials_token(client_id, client_secret)
PureCloudPlatformClientV2.configuration.api_client = apiclient

user_ids = # List of 400 user ids
usernames = queue.Queue()
put_username_in_queue = lambda user : usernames.put(user.username)

threads = [usersApi.get_user(id, callback=put_username_in_queue) for id in user_ids]

for thread in threads:
    thread.join()

while not usernames.empty():
    print(usernames.get())

As result, I am getting only about 370 usernames out of 400

[tim.smith]

I've tested using the code below and the behavior appears to be the same between version 78.0.0 and 79.2.0 with the exception that the warnings about the thread pool aren't present in the latter. The line creating RESTClientObject was commented out for the testing with the older version.

The behavior I've observed is that the app will immediately make all 200 requests, but only 180 of the responses will be immediately returned (rate limiting for that API operation kicks in at 180 requests). The remaining 20 requests wait to be retried once the retry-after period has elapsed. In both SDK versions, a few of the retries will fail and throw a 429 error. It appears that python isn't waiting quite long enough, which is what is causing a few of the initial retries to fail with a 429 message. Note that Python doesn't have a way of catching errors that are thrown from a thread, so the application isn't able to handle the errors thrown from asynchronous operations.

I'll see what we can do to work around this. For now, I'd suggest implementing logic in your app to validate the results you got and retry requests for any missing data after all threads have completed, or avoid using asynchronous requests.

import os

import PureCloudPlatformClientV2
from PureCloudPlatformClientV2.rest import ApiException, RESTClientObject

# Client config
client_id = os.environ['PURECLOUD_CLIENT_ID']
client_secret = os.environ['PURECLOUD_CLIENT_SECRET']
PureCloudPlatformClientV2.configuration.host = "https://api.mypurecloud.com"

apiclient = PureCloudPlatformClientV2.api_client.ApiClient()
apiclient.rest_client = RESTClientObject(4, 4)
print("Authenticating...")
apiclient = apiclient.get_client_credentials_token(client_id, client_secret)
PureCloudPlatformClientV2.configuration.api_client = apiclient
usersApi = PureCloudPlatformClientV2.UsersApi()
threads = []

c = 0


def got_user(user):
    global c
    c += 1
    print("Got user {0}: {1}".format(c, user.username))


i = 0
while i < 200:
    try:
        i += 1
        print("Fetching #%s..." % i)
        threads.append(usersApi.get_user(
            user_id="0afc8662-90e1-4a94-a77e-ccd7965d9711", callback=got_user))
    except ApiException as e:
        print("Exception when calling UsersApi->get_user: %s\n" % e)
        print("Header: %s" % e.headers['ININ-Correlation-Id'])


# Wait until complete
[t.join() for t in threads]
print("Done")

[Adrian_Santamaria]

Ok, thank you for your response, Tim. For now I will avoid using async requests with the Python SDK until the problem is fixed.

[tim.smith]

FWIW, this appears to be a rounding/precision issue in the API, not an issue with the SDK; the SDK is waiting the correct amount of time specified in the retry-after header, but sometimes the API hasn't quite exited the rate limiting condition by the time the client wait has elapsed. API-5233 has been logged for that issue.

[Adrian_Santamaria]

Hello

Today, using the Python SDK without Concurrent Calls, I got this error:

HTTP response body: {"message":"Rate limit exceeded the maximum [300] requests within [60] seconds","code":"too.many.requests","status":429,"entityName":"token","details":[{"errorCode":"inin-ratelimit-count","fieldName":"304"},{"errorCode":"inin-ratelimit-allowed","fieldName":"300"},{"errorCode":"inin-ratelimit-reset","fieldName":"0"}]}

So this problem is not limited to async requests.

[tim.smith]

Correct. Rate limiting applies to all API requests and your application must adhere to rate limiting best practices no matter how you use the SDK, including if you don't use it at all.

https://developer.mypurecloud.com/api/rest/rate_limits.html