How to handle paging in .NET SDK

[JDunn]

Hi,

The API documentation in general talks about paging in the platform:

## API Paging
(https://developer.genesys.cloud/api/rest/tips/#api-paging)

Seriously, who has a pager these days?

No, no, not that kind of paging. There are many resources in the Platform API that can return very large data sets. To avoid always sending large amounts of data in a response, these resources implement the concept of paged results. When a request is made to a resource with paged results, the resource will return the default number of results for the first page. To retrieve more results, the request can either specify a larger value for *pageSize* or retrieve subsequent pages by specifying *pageNumber* .

For example, calling `GET /api/v2/users` without any parameters will return 25 results. To get more results, the request can be modified as `GET /api/v2/users?pageSize=50` to return 50 results per page (getting users 1-50). Typically, the max *pageSize* for paged resources is 500. Additionally, to retrieve the second page of the default number of results the request can be modified as `GET /api/v2/users?pageNumber=2` to return the next page (getting users 26-50). These can be combined as `GET /api/v2/users?pageSize=50&pageNumber=2` to get users 51-100.

Determining if more results are available can be evaluated in two ways:

1. Compare the response properties of *pageNumber* with *pageCount* . If *pageNumber* == *pageCount* , there are no more pages. If *pageNumber* < *pageCount* , add 1 to *pageNumber* and specify that value in the next request to get the next page.
2. Check for a value in the response property *nextUri* . If a value exists, it will be the path to the next page. If it does not exist, there are no more pages.

But how is this best done when using the SDK? I am not making API requests directly to those endpoints and getting JSON returns, I am using the classes / methods (etc) in the SDK.

For example, I'm using UsersApi.GetUserRoutingSkills() as I want to get a list of all our users and all their skills & proficiencies. We have over 1,000 users so I hit the rate limit early on.

(As a novice) How is best to handle this? Currently I'm getting a list of all users from usersApi.GetUsers(), looping through each of those using their ID for the GetUserRoutingSkills() request. After a short period I get:

: 'Error calling GetUserRoutingskills: {"message":"Rate limit exceeded the maximum. Retry the request in [34] seconds","code":"too.many.requests.retry.after","status":429,"contextId":"77390665-ea94-49ef-9eb8-3cac03291577","details":[],"errors":[]}'

Thank you for any guidance you can give.

Edit: Do I check for that exception and if it occurs, get the last user and then sleep the application for however long is in the 429 response? That doesn't sound right but...

[jacobshaw]

Hi JDunn,
Best practice for 429 responses is to implement an exponential backoff using the Retry-After header.
See more about rate limits here: https://developer.genesys.cloud/api/rest/rate_limits
Additionally, the GetUserRoutingskills function takes pageSize and pageNumber as optional parameters:
https://developer.genesys.cloud/api/rest/client-libraries/dotnet/UsersApi#getuserroutingskills

[Eos_Rios]

The simplest and laziest way to handle is a 1 second sleep between requests to ensure you never exceed 60 a minute.

[JDunn]

Hi Jacob,

I apologise, I'm not sure what exponential backoff means - but is this the principle? To get the retry-after value and just sleep the request like this?

[This is completely rookie code, but, as an example...]

foreach (var user in usersApi.GetUsers(500).Entities.OrderBy(n => n.Name))
{
    try
    {
        foreach (var skill in usersApi.GetUserRoutingskills(user.Id).Entities)
        {
            userInfo.Add($"{user.Username},{skill.Name},{skill.Proficiency}");
        }
    }
    catch (PureCloudPlatform.Client.V2.Client.ApiException ex)
    {
        Console.WriteLine(ex.Headers["Retry-After"].ToString());
        var sleepTimer = int.Parse((ex.Headers["Retry-After"])) * 1000;
        Thread.Sleep(sleepTimer);
    }
}

Additionally, the GetUserRoutingskills function takes pageSize and pageNumber as optional parameters:

I saw that, but this is essentially where I'm struggling - I suppose the question is how can I tell how many pages there are without getting the JSON response?

I'm getting around this in a different API request by manually calling the method five times with a limit of 100 results each time, but that's only because I know how many to expect - maybe I can do the same with users somehow and query how many users in total there are, then split my requests into chunks? I can probably tidy that up a bit but I'm just trying to understand the principle in basic terms first. I've never really dealt with rate limits like this, it wasn't a problem I had to manage when using IceLib on PureConnect as I never exceeded the limit.

[JDunn]

That sounds workable but would mean it takes like 15 minutes to query all our users each time

[Eos_Rios]

We tried implementing a count-the-requests-then-throttle but it was a lot more code for what end up the same amount of waiting.
A 60 request per minute cap is still a 60 requests per minute cap whether you wait one second after every request or wait 60 seconds after the 59th, and 1000/60 is always going to result in 15 no matter how you try to slice it.

Your exponential back off is approximate but you have to do it in a way that saves state, so you resume trying from where you failed after the sleep.

If you're trying to get skills per user you may be better off calling GetUsers with the "skills" expands rather than asking for the skills of every user. GET /api/v2/users

You're doing a subrequest per user when you could just have that info in your users loop already.

[JDunn]

Damn - I only pursued GetUserRoutingSkills() because originally my GetUsers was returning null in the Skills value, I assumed it was just not available like that for some reason. Never thought that I could add it as an Expand value.

That's sorted that then - ran the query this morning and didn't hit the rate limit at all. Perfect.

I've also been looking for that documentation, thank you - that'll be a big help.