The first release of the Open Data Exporter has been made available on github!
What is the Open Data Exporter?
It's a configurable node.js service that executes PureCloud API requests, performs data calculations and transformations, and exports the data into templates.
What would I want to use it for?
The intended use case is for historical WFM integrations, such as Verint. The application will export any data to a file, so it's useful as a general data export tool for any data that can be retrieved from the Platform APIs.
What are the limitations for using it?
The project is licensed under the MIT License. Basically, you can use, reuse, modify, and sell this project for your (personal or commercial) purposes.
Will it be able to do...
Yes! Although there is no guarantee of support or ongoing maintenance, there is currently active work surrounding this project and additional features, documentation, and examples are being added. However, the intent of making this open source is that the project will benefit from community contributions to add additional configurations and API support. PRs with well-written code written in a style similar to the rest of the code base will generally be accepted. If you'd like to discuss features before building them, just make a post about it here!
I don't get it. Can I see an example?
Sure! Here's a sample of the abandon report:
Hi, sounds pretty good but couln't make it work.
Note that I've got my own node.js server/app that is already talking with the PureCloud API -it's a small frontend to have better/customised dashboards to monitor the queues and change user presence status eg on-queue to busy or offline by a surpervisor.
So it's definitely something in the code, don't have the time right now to investigate so posting it here in case it's something straightforward that I'd missed in the pre-requisites documentation:
> #node index.js /config=./config/examples/abandon_report.json /jobs=abandon_report
> loading config from ./config/examples/abandon_report.json
> [TemplateDefinitions][DEBUG] Loading standard module "dataOperations" from dataOperations.js
> [TemplateDefinitions][DEBUG] Loading standard module "datetimeOperations" from datetimeOperations.js
> ╔════════════════════════════╗
> ║ Open Data Exporter v0.0.1 ║
> ╚════════════════════════════╝
> [executor][ERROR] Error: Bad Request
> at Request.callback (/Users/mandrake74/Code/open-data-exporter/src/node_modules/superagent/lib/node/index.js:698:17)
> at IncomingMessage.<anonymous> (/Users/mandrake74/Code/open-data-exporter/src/node_modules/superagent/lib/node/index.js:922:12)
> at emitNone (events.js:91:20)
> at IncomingMessage.emit (events.js:185:7)
> at endReadableNT (_stream_readable.js:974:12)
> at _combinedTickCallback (internal/process/next_tick.js:74:11)
> at process._tickCallback (internal/process/next_tick.js:98:9)
> [main][ERROR] Error: Bad Request
> at Request.callback (/Users/mandrake74/Code/open-data-exporter/src/node_modules/superagent/lib/node/index.js:698:17)
> at IncomingMessage.<anonymous> (/Users/mandrake74/Code/open-data-exporter/src/node_modules/superagent/lib/node/index.js:922:12)
> at emitNone (events.js:91:20)
> at IncomingMessage.emit (events.js:185:7)
> at endReadableNT (_stream_readable.js:974:12)
> at _combinedTickCallback (internal/process/next_tick.js:74:11)
> at process._tickCallback (internal/process/next_tick.js:98:9)
I think that's due to authentication failing. Make sure you're using a valid client ID and secret from an OAuth client that's configured for client credentials. I'll make a task to addI've added additional tracing/handling around the login process to produce a more useful error.
Thanks Tim, spot on, it definitely looks like a auth issue.
With your additional tracing, it was pretty clear:
[executor][ERROR] Error: Authentication error: Client ID not set
at Api.login (/Users/mandrake74/Code/open-data-exporter/src/api.js:39:19)
at Executor.initialize (/Users/mandrake74/Code/open-data-exporter/src/executor.js:45:6)
at Object.<anonymous> (/Users/mandrake74/Code/open-data-exporter/src/index.js:25:10)
at Module._compile (module.js:571:32)
at Object.Module._extensions..js (module.js:580:10)
at Module.load (module.js:488:32)
at tryModuleLoad (module.js:447:12)
at Function.Module._load (module.js:439:3)
at Module.runMain (module.js:605:10)
at run (bootstrap_node.js:420:7)
[main][ERROR] Error: Authentication error: Client ID not set
at Api.login (/Users/mandrake74/Code/open-data-exporter/src/api.js:39:19)
at Executor.initialize (/Users/mandrake74/Code/open-data-exporter/src/executor.js:45:6)
at Object.<anonymous> (/Users/mandrake74/Code/open-data-exporter/src/index.js:25:10)
at Module._compile (module.js:571:32)
at Object.Module._extensions..js (module.js:580:10)
at Module.load (module.js:488:32)
at tryModuleLoad (module.js:447:12)
at Function.Module._load (module.js:439:3)
at Module.runMain (module.js:605:10)
at run (bootstrap_node.js:420:7)
I realised I forgot to add my credentials in the config/examples/abandon_report.json - I though you just needed to add that once in the config.json file in the root directory.
But it still doesn't work, and yes it's client credentials which are working in another node.js app:
R5002090:src mandrake74$ node index.js /config=./config/examples/abandon_report.json /jobs=abandon_report
loading config from ./config/examples/abandon_report.json
[TemplateDefinitions][DEBUG] Loading standard module "dataOperations" from dataOperations.js
[TemplateDefinitions][DEBUG] Loading standard module "datetimeOperations" from datetimeOperations.js
╔════════════════════════════╗
║ Open Data Exporter v0.0.1 ║
╚════════════════════════════╝
[api][ERROR] Error: Bad Request
[executor][ERROR] Error: Authentication failed! Check your Client ID and Secret
at /Users/mandrake74/Code/open-data-exporter/src/api.js:55:12
at tryCatch (/Users/mandrake74/Code/open-data-exporter/src/node_modules/es6-promise/dist/es6-promise.js:410:12)
at invokeCallback (/Users/mandrake74/Code/open-data-exporter/src/node_modules/es6-promise/dist/es6-promise.js:425:13)
at publish (/Users/mandrake74/Code/open-data-exporter/src/node_modules/es6-promise/dist/es6-promise.js:393:7)
at publishRejection (/Users/mandrake74/Code/open-data-exporter/src/node_modules/es6-promise/dist/es6-promise.js:335:3)
at flush (/Users/mandrake74/Code/open-data-exporter/src/node_modules/es6-promise/dist/es6-promise.js:121:5)
at _combinedTickCallback (internal/process/next_tick.js:67:7)
at process._tickCallback (internal/process/next_tick.js:98:9)
[main][ERROR] Error: Authentication failed! Check your Client ID and Secret
at /Users/mandrake74/Code/open-data-exporter/src/api.js:55:12
at tryCatch (/Users/mandrake74/Code/open-data-exporter/src/node_modules/es6-promise/dist/es6-promise.js:410:12)
at invokeCallback (/Users/mandrake74/Code/open-data-exporter/src/node_modules/es6-promise/dist/es6-promise.js:425:13)
at publish (/Users/mandrake74/Code/open-data-exporter/src/node_modules/es6-promise/dist/es6-promise.js:393:7)
at publishRejection (/Users/mandrake74/Code/open-data-exporter/src/node_modules/es6-promise/dist/es6-promise.js:335:3)
at flush (/Users/mandrake74/Code/open-data-exporter/src/node_modules/es6-promise/dist/es6-promise.js:121:5)
at _combinedTickCallback (internal/process/next_tick.js:67:7)
at process._tickCallback (internal/process/next_tick.js:98:9)
R5002090:src mandrake74$
I would recommend to use env variables instead and process.env.VARIABLENAME, it's more secure and much simpler to use in different script files.
Hi Tim, thanks to your trace, I had a quick look at api.js in order to hardcode my credentials to see if this would work, and I found the issue: I'm using the AU environment, so I need to specify this environment in the purecloud session:
PS: I also noticed a cheeky mistake in the doc/example, at least in the last pull I did. The name of the abandon job example is slightly different than the one mentioned on the wiki: it's abandons_job and not abandon_report, as set in the abandon_report.json example config file
The app loads only the config file you specify with the /config command line parameter. If you don't specify anything, the default path of ./config.json is used.
See the Running the application page on the wiki. There are command line parameters for client ID and secret. I use these to inject values from environment examples, as is shown in the example: /clientId=$CLIENT_ID /clientSecret=$CLIENT_SECRET.
The API will use whatever environment is defined in the config file or will default to mypurecloud.com if you don't specify anything.
Thanks for pointing this out. I'll update the page.
Tim, I'm trying to understand whether the Open Data Exporter platform would be more "complete" than the REST API, as I've been using so far. For example, I need to extract the full contact list data. Using the REST API, I need to place a call for the report and then get the URL to download the results. I was wondering if, using the Open Data Exported, I would get the data right away. What do you say?
That is certainly a use case that this project could handle. However, if all you're looking for is exporting and downloading a contact list, this project will probably be more overhead than it's worth. The key value-add here is that through templates and a little javascript, you can aggregate and format data from multiple API calls into a single data export, which isn't relevant to your use case.
whether the Open Data Exporter platform would be more "complete" than the REST API
To address that directly, that's not really an appropriate comparison. The ODE is an application that was written to consume the Platform API. It has no more or less access to anything in PureCloud than you do by consuming the API directly. There is nothing in this project that any customer couldn't have built on their own.
