AWS S3 recording bulk actions integration

Hello!,

I have been testing the extraction of recordings from PureCloud to an S3 (https://help.mypurecloud.com/articles/about-the-aws-s3-recording-bulk-actions-integration/), but it only downloads since the in which you create the extraction jobs.

But I need to download the recordings from previous months, I have tried the following codes:

# Build the create job query, for export action, set query.action = "EXPORT"
query = PureCloudPlatformClientV2.RecordingJobsQuery()
query.action = "EXPORT"
query.action_date = "2020-01-01T00:00:00.000Z" # or 2029-01-01T00:00:00.000Z
query.integration_id = "XXXXXXXXXXXXXX_XXXXX_X_X_X_X_X_XXX"
query.includeScreenRecordings = True
query.conversation_query = {
    "interval": "2020-06-01T03:00:00.000Z/2020-06-10T03:00:00.000Z",
    "order": "asc",
    "orderBy": "conversationStart",
    "start_of_day_interval_matching": True
}

But I only get it to download from the day I create the jobs, I don't know if I'm using the code well, or if I'm missing any parameter.

I hope you can help me.

Cheers!

Hello,

I think your issue is coming from one or two things.

The first one is that the action_date should be a future date.
Note that the date is based on UTC time.
It specifies when the action will be executed/processed.
So you could create the recording bulk job with a date/time in one hour in the future (i mean your current UTC date/time + one hour).

The second thing is that you will need to remove a piece of code from the tutorial - the one that cancels the recording job - at the end of the code sample
-> cancel_job_response = recording_api.delete_recording_job(job_id)
I think this was left in the tutorial so people could create and submit a recording job, and then cancel it - just for the sake of trying to create a recording job (without triggering the delete or the export).

As a note - recordings can only be exported once to an AWS S3.

The process is the following:

  • you create a recording bulk job (export) set to start at a later date (the action date)
  • Genesys Cloud starts preparing the job (checking which conversations matching the interval/filter have a recording)
  • you can check the status of the job and when it says it is READY, it means the job is ready to be executed
  • you can then request to execute the job (setting the job state to PROCESSING)
  • Genesys Cloud will start processing the job (job state = PROCESSING). When it has finished completing the job, the job state will show as FULFILLED.

When the recording job is fulfilled, it doesn't mean that the recordings are all exported/copied to the AWS S3.
It just means that Genesys Cloud has tagged the necessary conversations/recording metadata - so they are ready to be exported/deleted at the date specified in action date - setting an exportDate or deleteDate.

When action date occurs (exportDate, deleteDate), Genesys Cloud will then submit the recordings to the AWS S3 integration.
Depending on the number of recordings which are currently exported to S3 (via a policy and/or via the recording bulk job), it can take few minutes/hours for the recordings to appear on AWS S3.
When a recording has been exported to AWS S3, the recording metadata will be updated in Genesys Cloud with a proper exportedDate (GET /api/v2/conversations/{conversationId}/recordingmetadata)
If you go on the Interaction in Genesys Desktop, under Details tab, it should also display the exportedDate.

Regards,

1 Like

Hello, if I am clear that the action_date parameter has to be a current or future date, I attach all the code I have, but it only extracts the recordings from the start of the jobs, (2021-04-01).

But what I need is to extract all the recordings of 2020. Or as much as possible, can you tell me or mention how to do it?

import base64, sys, requests, time
import PureCloudPlatformClientV2
from pprint import pprint
from PureCloudPlatformClientV2.rest import ApiException
print('-------------------------------------------------------------')
print('- Execute Bulk Action on recordings-')
print('-------------------------------------------------------------')
# OAuth when using Client Credentials
client_id = 'XXXXXX'
client_secret = '*************'

# Authenticate client
api_client = PureCloudPlatformClientV2.api_client.ApiClient().get_client_credentials_token(client_id, client_secret)

# Get the api
recording_api = PureCloudPlatformClientV2.RecordingApi(api_client)
access_token = recording_api.api_client.access_token

# Assign the token
PureCloudPlatformClientV2.configuration.access_token = access_token

# Build the create job query, for export action, set query.action = "EXPORT"
query = PureCloudPlatformClientV2.RecordingJobsQuery()
query.action = "EXPORT"
query.action_date = "2021-01-10T00:00:00.000Z"
query.integration_id = "XXXXXXXXXXXXXXXXXXX"
query.includeScreenRecordings = True
query.conversation_query = {
    "interval": "2020-06-01T03:00:00.000Z/2020-06-10T03:00:00.000Z",
    "order": "asc",
    "orderBy": "conversationStart",
    "start_of_day_interval_matching": True
}
print(query)
try:
    # Call create_recording_job api
    create_job_response = recording_api.post_recording_jobs(query)
    job_id = create_job_response.id
    print(f"Succesfully created recording bulk job { create_job_response}")
    print(job_id)
except ApiException as e:
    print(f"Exception when calling RecordingApi->post_recording_jobs: { e }")
    sys.exit()
# Call get_recording_job api
while True:
    try:
        get_recording_job_response = recording_api.get_recording_job(job_id)
        job_state = get_recording_job_response.state
        if job_state != 'PENDING':
            print("IS NOT PENDING")
            break
        else:
            time.sleep(2)
    except ApiException as e:
        print(f"Exception when calling RecordingApi->get_recording_job: { e }")
        sys.exit()
if job_state == 'READY':
    try:
        execute_job_response = recording_api.put_recording_job(job_id, { "state": "PROCESSING"})
        print(f"Succesfully execute recording bulk job { execute_job_response}")
    except ApiException as e:
        print(f"Exception when calling RecordingApi->put_recording_job: { e }")
        sys.exit()
else:
    print(f"Expected Job State is: READY, however actual Job State is: { job_state }")

# create an instance of the API class
api_instance = PureCloudPlatformClientV2.RecordingApi()
page_size = 25 # int | Page size (optional) (default to 25)
page_number = 1 # int | Page number (optional) (default to 1)
sort_by = 'userId' # str | Sort by (optional) (default to userId)
state = 'READY' # str | Filter by state (optional)
show_only_my_jobs = False # bool | Show only my jobs (optional)
job_type = 'EXPORT' # str | Job Type (Can be left empty for both) (optional)

try:
    get_recording_jobs_response = recording_api.get_recording_jobs(page_size=page_size, page_number=page_number, sort_by=sort_by, state=state, show_only_my_jobs=show_only_my_jobs, job_type=job_type)
    print(f"Succesfully get recording bulk jobs { execute_job_response}")
except ApiException as e:
    print(f"Exception when calling RecordingApi->get_recording_jobs: { e }")
    sys.exit()

Thank you.

I don't think it is the job which extracts your recordings from the start of the jobs.
The job will try to extract the conversations which occurred during the specified interval ("2020-06-01T03:00:00.000Z/2020-06-10T03:00:00.000Z").

If you have set the AWS S3 Integration in one of your Policies (Recording), this is what would cause new recordings to be automatically exported.
If you set/configure the AWS S3 Integration in a Policy, it means that the new recordings matching this policy (recordings which will occur after you have configured the Policy) will automatically be extracted and exported to S3. No need to run a recording bulk job for these ones.

Based on the script you have copied above, you have set the action date as January 10th. Which means that Genesys Cloud will start exporting the recordings to S3 (matched by the recording bulk job) at that date. Not before.

As a side comment, when you create the recording job and check the status (waiting for READY job status), you can also see how many recordings are matching this recording job request. In the sample script, this info would come in get_recording_job_response = recording_api.get_recording_job(job_id)
You can maybe print this (logs/console) when the job state is READY. It should show how many recordings the bulk job will take care of (totalRecordings).
When the job is executed and gets to FULFILLED state, you should then see how many recordings were processed (totalProcessedRecordings). At this stage (FULFILLED), as I wrote in my previous post, the recordings are not yet on the AWS S3. Export will start at the date you specified in action date.

If you think this is already what you are doing, please open a case with Genesys Care to investigate as customer-specific data cannot be investigated via the forum.

Regards,

1 Like

From what I understand is that the action_date parameter is the one that starts the export action to an S3 ?, example

action_date = 2020-01-05 // This will start

"interval": "2020-06-01T03: 00: 00.000Z / 2020-06-10T03: 00: 00.000Z" // And this would be the date interval from where the recordings would be obtained.

I copy the response from the api

{
  "entities": [
    {
      "id": "5eeea820-d632-456b-8a02-9e9bb215af35",
      "state": "PROCESSING",
      "recordingJobsQuery": {
        "action": "EXPORT",
        "actionDate": "2020-01-01T00:00:00Z",
        "integrationId": "XXXXXXX",
        "includeScreenRecordings": true
      },
      "dateCreated": "2021-01-04T19:06:04.654Z",
      "totalConversations": 163546,
      "totalRecordings": 94562,
      "totalProcessedRecordings": 94555,
      "percentProgress": 99,
      "selfUri": "/api/v2/recording/jobs/5eeea820-d632-456b-8a02-9e9bb215af35",
      "user": {
        "id": "326f01ba-1613-4095-a67b-9a1193d01377",
        "selfUri": "/api/v2/users/326f01ba-1613-4095-a67b-9a1193d01377"
      }
    }
  ],
  "pageSize": 25,
  "pageNumber": 1,
  "total": 1,
  "firstUri": "/api/v2/recording/jobs?pageSize=25&pageNumber=1&state=PROCESSING&jobType=EXPORT",
  "selfUri": "/api/v2/recording/jobs?pageSize=25&pageNumber=1&state=PROCESSING&jobType=EXPORT",
  "lastUri": "/api/v2/recording/jobs?pageSize=25&pageNumber=1&state=PROCESSING&jobType=EXPORT",
  "pageCount": 1
}

As we are in 2021 - action_date = 2021-01-05T20:00:00Z
Currently (at the time of my post), it is 18:59 UTC.
So let's say I create a job and it gets in READY/FULFILLED before 20:00 UTC. Then at 20:00 UTC, Genesys Cloud will start the export.

Based on the screenshot, it seems you used an acton date in the past (2020-01-01T00:00:00Z).

As you have lots of recordings that would need to be exported (94562), the preparation of the job to get to READY state and the processing of the job (to get to FULFILLED) state probably take some time.

So I would first try to create a recording job on a subset/smaller interval - to see if it works or not - as processing and exporting almost 100000 recordings will take time.
So maybe, try with an export of the recordings for one day only like 2020-06-01T03:00:00.000Z/2020-06-02T03:00:00.000Z with an action date tomorrow (just to make sure your job has time to get to READY and then to FULFILLED).
Then tomorrow, at time of the action date (exportDate), Genesys Cloud will start pushing these recordings to AWS S3. Note that it takes time to get them on S3 - they won't appear immediately at the time of action date. Depending on the number of recordings to export, it can take some time - see comment on the tutorial here: https://developer.mypurecloud.com/api/tutorials/recordings-bulk-actions/index.html?language=python&step=3

Regards,

1 Like

I did what you told me, I am FULFILLED before 20:00. But still it is not exporting the recordings to the S3.

{
  "id": "ac25a627-736d-4b03-90df-2f3a2719e8cf",
  "state": "FULFILLED",
  "recordingJobsQuery": {
    "action": "EXPORT",
    "actionDate": "2021-01-05T20:00:00Z",
    "integrationId": "XXXXXXX",
    "includeScreenRecordings": true
  },
  "dateCreated": "2021-01-05T19:51:03.155Z",
  "totalConversations": 262,
  "totalRecordings": 220,
  "totalProcessedRecordings": 220,
  "percentProgress": 100,
  "selfUri": "/api/v2/recording/jobs/ac25a627-736d-4b03-90df-2f3a2719e8cf",
  "user": {
    "id": "326f01ba-1613-4095-a67b-9a1193d01377",
    "selfUri": "/api/v2/users/326f01ba-1613-4095-a67b-9a1193d01377"
  }
}

It's only 1 day of recordings.

Regards!

Please open a case with Genesys Care to investigate as customer-specific data cannot be investigated via the forum.

Thank you very much for your help, it is already exporting without problem to an S3.

Cheers!

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.