How to troubleshoot Limit-0001 Alerts?

I am currently in the build phase of a project and therefore there is minimal traffic on the system. (maybe 10 calls per day), however, I am seeing the Limit-0001 event occuring once every 2 days or so on the token.rate.per.minute.

Looking at the Develoepr center https://developer.genesys.cloud/platform/operational-event-catalog/limit-0001 does not really give me too much as to how I track the offending user that is causing the alerts. The above webpage suggests the following:

" Steps to resolve:

Using the entityId from the eventbridge alert, track down the relevant token."

however, there I cant seem to find anything on how to track down the relevant token.

Has anyone got any ideas on how to track the offending user which is triggering these events?

1 Like

Hi @Sean_Mahon,

I had the same question and ended up with asking Customer Care. They have given a thorough answer on this one, let me just post it here:

We have discussed this with our development team and they have clarified that in order to use the Entity ID that is being shown in the UI for LIMIT events, you'd have to use

/api/v2/audits/query endpoint which is also available in the developer center https://developer.genesys.cloud/platform/audit/#post-api-v2-audits-query. You would have to use it as the 'Entity ID', here is a sample payload:

{
"interval": "2024-04-19T00:00:00.000/2024-04-20T00:00:00.000",
"serviceName": "PeoplePermissions",
"filters": [
{
"property": "EntityType",
"value": "AccessToken"
},
{
"property": "Action",
"value": "Create"
},
{
"property": "EntityId",
"value": "g-1w1HmLq7gEmXaQ6ApLmiiut1WJFjUMC_8SAcL0hbM="
}
],
"sort": [
{
"name": "Timestamp",
"sortOrder": "desc"
}
]
}

Here I used EntityID 'g-1w1HmLq7gEmXaQ6ApLmiiut1WJFjUMC_8SAcL0hbM=' which is an example last April 19th in your Org as shown below:

If you use the interval shown above (April 19 whole day) and query for EntityType 'AccessToken' and Action 'Create', the API result will show us who/which OAuth client created this access token which is exceeding the limit for token.rate.per.minute key. After running the API, you will get a transaction id like something below:

{
"id": "YOUR-ID",
"state": "Running",
"startDate": "2024-04-23T04:21:46Z",
"interval": "2024-04-19T00:00:00.000Z/2024-04-20T00:00:00.000Z",
"serviceName": "PeoplePermissions",

you need to use this id on endpoint GET /api/v2/audits/query/{transactionId}/results https://developer.genesys.cloud/platform/audit/#get-api-v2-audits-query--transactionId--results to fetch the result of your query (previous API call above). Once fetched, the result will look like this:

{
"id": "YOUR-ID",
"pageSize": 25,
"entities": [
{
"id": "YOUR-ID",
"userHomeOrgId": "YOUR-ID",
"user": {
"id": "YOUR-ID",
"selfUri": "/api/v2/users/YOUR-ID"
},
"client": {
"id": "YOUR-ID",
"selfUri": "/api/v2/oauth/clients/YOUR-ID"
},
"remoteIp": [
"IP",
"IP"
],
"serviceName": "PeoplePermissions",
"level": "USER",
"eventDate": "2024-04-19T11:33:23Z",
"message": {
"localizableMessageCode": "ACCESS_TOKEN_CREATE_SUCCESS",
"message": "access token created",
"messageWithParams": "",
"messageParams": {}
},
"action": "Create",
"entity": {
"id": "g-1w1HmLq7gEmXaQ6ApLmiiut1WJFjUMC_8SAcL0hbM="
},
"entityType": "AccessToken",
"status": "SUCCESS",
"application": "Directory Web Client (Unified App)",
"initiatingAction": {
"transactionId": "YOUR-ID"
},
"transactionInitiator": true,
"propertyChanges": [],
"context": {},
"entityChanges": []
}
]
}

The important bit here is the 'application' part which tells you which OAuth client created the token which is almost exceeding the limit. It says here 'Directory Web Client (Unified App)' which basically means that the client belongs to Genesys itself (meaning the error is related to a user context, an actual user in Genesys cloud). This is because users when logging in and using Genesys also performs API calls in the backend when accessing views, initiating calls etc.. via a token.

I would recommend ignoring LIMIT errors if it is related to 'Directory Web Client (Unified App)' application/OAuth client as this just means that users might be accessing a lot of things (Dashboards for example which generates a lot of API calls in the backend) inside Genesys Cloud but this error should not block them from anything.

If you see a different application in the result (an external application for example that is using an OAuth client in your Org) then further investigation might be needed. Audit APIs mentioned above should help you pinpoint the exact application throwing these limit errors.

2 Likes

Thanks so much for this. I am still waiting for a response from Genesys Customer Care for this exact detail. Thankfully my events seem to be related to the "Directory Web Client" applicaiton and I know that users is designing all the Dashboards at the moment.

Sean.