Audio Hook - Websocket Integration

Arun_Kumar1 · July 27, 2023, 9:06am

We are following the pattern & practices. - Connection probe steps.

While activate the webhook in Genesys, Client is able to connect with web socket server. After successful connection client is sending response both with Type "Open".
Server is responding back to client with below request body with type "Opened"

{ "version": "2", "type": "opened", "seq": 1, "clientseq": 1, "id": "9aba9a07-8dd5-4654-974a-44075c3d2ba5", "parameters": { "startPaused": false, "media": [ { "type": "audio", "format": "PCMU", "channels": [ "internal" ], "rate": 8000 } ] } }

Immediatley client is sending response with Type "Close". While server replying with type closed. Getting below error.

{"version":"2","id":"9aba9a07-8dd5-4654-974a-44075c3d2ba5","type":"error","seq":3,"position":"PT0S","parameters":{"code":400,"message":"Connection probe error: Message is malformed"},"serverseq":1}

Kindly help what is wrong in the request body with type "Opened"

Jerome.Saint-Marc · July 27, 2023, 10:10am

Hello,

Without seeing the messages sent by the client and the ones sent by your server, it is difficult to say.
Given that you get an error when sending the Closed message, it is possibly this one which is having an issue (could be the seq/serverSeq) and you haven't shared what is the close message you receive and what is the closed message you are sending.

Regards,

Arun_Kumar1 · July 27, 2023, 10:26am

Hi Jermoe,

Kindly find the detailed message below.

Connect : client to server
{
connectionId: 'ITbAHeRmIAMCF3g=',
routeKey: '$connect',
eventType: 'CONNECT',
genesysbody: {}
}

Open: client to server
{
version: '2',
id: '62780b08-3a9d-43b0-9d74-4aa5745fa633',
type: 'open',
seq: 1,
position: 'PT0S',
parameters: {
organizationId: 'aec04f7c-8108-4e07-a9d6-67a8d5df7b3e',
conversationId: '00000000-0000-0000-0000-000000000000',
participant: {
id: '00000000-0000-0000-0000-000000000000',
ani: '',
aniName: '',
dnis: ''
},
media: [{"type":"audio","format":"PCMU","channels":["external","internal"],"rate":8000},
{"type":"audio","format":"PCMU","channels":["external"],"rate":8000},
{"type":"audio","format":"PCMU","channels":["internal"],"rate":8000}],
language: null,
customConfig: {}
},
serverseq: 0
}

Opened : server to client
{
version: '2',
type: 'opened',
seq: 1,
clientseq: 1,
id: '62780b08-3a9d-43b0-9d74-4aa5745fa633',
parameters: { startPaused: false,
media: [
{
"type": "audio",
"format": "PCMU",
"channels": ["external", "internal"],
"rate": 8000
}
] }
}

Close: client to server
{
connectionId: 'ITbAHeRmIAMCF3g=',
routeKey: '$default',
eventType: 'MESSAGE',
genesysbody: {
version: '2',
id: '62780b08-3a9d-43b0-9d74-4aa5745fa633',
type: 'close',
seq: 2,
position: 'PT0S',
parameters: { reason: 'end' },
serverseq: 1
}
}

Closed : server to client
{
version: '2',
type: 'closed',
seq: 2,
clientseq: 2,
id: '62780b08-3a9d-43b0-9d74-4aa5745fa633',
parameters: {}
}
Error message:
{
connectionId: 'ITbAHeRmIAMCF3g=',
routeKey: '$default',
eventType: 'MESSAGE',
genesysbody: {
version: '2',
id: '62780b08-3a9d-43b0-9d74-4aa5745fa633',
type: 'error',
seq: 3,
position: 'PT0S',
parameters: {
code: 400,
message: 'Connection probe error: Message is malformed'
},
serverseq: 1
}
}

Jerome.Saint-Marc · July 27, 2023, 2:12pm

Hello,

I have tried to run the connection probe (audiohook) from my environment and the messages you have posted above look ok in terms of attribute values (I mean the log).

The error raised is related to the closed message that your server sends back.
The client does not seem to parse it properly. I mean the error "says" that last valid message was serverseq:1.

Not knowing how you are sending your message and if it is different from the piece of code where you send the opened message (as this one appears to be caught by the client/GC), I don't really know what is wrong.

I would suggest to open a case with Genesys Cloud Customer Care, as they'd have access to logs/infos from the client (GC) and maybe see something more.

Regards,

Arun_Kumar1 · July 27, 2023, 2:22pm

Hi Jerome,

Kindly send your sample websocket application code. Also in which cloud server you have deployed your web socket?

Jerome.Saint-Marc · July 27, 2023, 2:30pm

No full audiohook code to share - just a basic nodejs script leveraging ws npm to manage the connection probe.

Something like:

const wss = new WebSocket.Server({ port: wssPort }); // the webSocket server

wss.on('connection', function connection(ws, req) {
  console.log('Request HEADERS: ', req.headers);

  ws.on('message', function message(data, isBinary) {
    if (!isBinary) {
      let dataObj = JSON.parse(data);
      console.log('RECEIVED: %s', data);
      if (dataObj.type == 'open') {
        console.log('TRYING TO OPEN .........');
        ...
        ws.send(JSON.stringify(openedMsg));
      } else if (dataObj.type == 'close') {
        console.log('TRYING TO CLOSE .........');
        ...
        ws.send(JSON.stringify(closedMsg));
      }
    }
  });

});

I am running this code locally and using localtunnel npm to get a public uri.
Have this as well in my code:

const localtunnel = require('localtunnel');
const wssDomain = 'my-audiohook-123';

(async () => {
  const tunnel = await localtunnel({ port: wssPort, 
    subdomain: wssDomain });

  console.log("Tunnel: ", tunnel.url);

  tunnel.on('close', () => {
    // tunnels are closed
  });
})();

Regards,

Arun_Kumar1 · July 27, 2023, 2:42pm

Thanks Jerome, Is it possible to have any Zoom session?

Jerome.Saint-Marc · July 27, 2023, 2:51pm

No. I am not available for a zoom session. What's your question?

Arun_Kumar1 · July 27, 2023, 2:52pm

Using ngrok if we generate public url for websocket, will it support in Audio Hook integration?

Jerome.Saint-Marc · July 27, 2023, 3:04pm

I don't know. I have not tried.
Localtunnel worked for me and didn't require account registration or else (that's why I have used it for this test).

Regards,

Arun_Kumar1 · July 27, 2023, 3:12pm

thanks for the information, While test Connection probe, Once we send opened message to the client, we got sample audio packets from client right?

Jerome.Saint-Marc · July 27, 2023, 3:50pm

No, not automatically.
The doc says it is optional.
For a simple probe, the client may omit sending synthesized audio and immediately follow the open transaction with a close transaction.
I didn't receive sample audio packets during connection probe in my test. But I am not a frequent user of audiohook.

Side comment about your ngrok question. I just tried with ngrok (free account) and it seems to be ok (at least for connection probestage).

Arun_Kumar1 · July 27, 2023, 4:15pm

Yes now my code is working from local, i am able to get open type. How to receive sample audio packets from client?

Jerome.Saint-Marc · July 27, 2023, 4:31pm

You have to make a call that will trigger voice transcription/audiohook.
See About AudioHook Monitor - Genesys Cloud Resource Center
This call must go through a trunk where recording is enabled - so it can't be an internal call. It will have to be an inbound call (from external).

Regards,

Arun_Kumar1 · July 27, 2023, 4:37pm

RECEIVED: {"version":"2","id":"994970f0-ad31-4185-8cc4-0ed4641f0efc","type":"open","seq":1,"position":"PT0S","parameters":{"organizationId":"aec04f7c-8108-4e07-a9d6-67a8d5df7b3e","conversationId":"00000000-0000-0000-0000-000000000000","participant":{"id":"00000000-0000-0000-0000-000000000000","ani":"","aniName":"","dnis":""},"media":[{"type":"audio","format":"PCMU","channels":["external","internal"],"rate":8000},{"type":"audio","format":"PCMU","channels":["external"],"rate":8000},{"type":"audio","format":"PCMU","channels":["internal"],"rate":8000}],"language":null,"customConfig":{}},"serverseq":0}
OPENED SEND {"version":"2","type":"opened","seq":1,"clientseq":1,"id":"994970f0-ad31-4185-8cc4-0ed4641f0efc","parameters":{"startPaused":false,"media":[{"type":"audio","format":"PCMU","channels":["external","internal"],"rate":8000}]}}
RECEIVED: {"version":"2","id":"994970f0-ad31-4185-8cc4-0ed4641f0efc","type":"error","seq":2,"position":"PT0S","parameters":{"code":400,"message":"Connection probe error: Message is malformed"},"serverseq":0}
RECEIVED: {"version":"2","id":"994970f0-ad31-4185-8cc4-0ed4641f0efc","type":"close","seq":3,"position":"PT0S","parameters":{"reason":"error"},"serverseq":0}
CLOSE SEND {"version":"2","type":"closed","seq":3,"clientseq":1,"id":"994970f0-ad31-4185-8cc4-0ed4641f0efc","parameters":{}}
RECEIVED: {"version":"2","id":"994970f0-ad31-4185-8cc4-0ed4641f0efc","type":"error","seq":4,"position":"PT0S","parameters":{"code":409,"message":"Connection probe error: Unexpected "opened" message."},"serverseq":1}
RECEIVED: {"version":"2","id":"994970f0-ad31-4185-8cc4-0ed4641f0efc","type":"error","seq":5,"position":"PT0S","parameters":{"code":400,"message":"Connection probe error: Message is malformed"},"serverseq":1}
RECEIVED: {"version":"2","id":"994970f0-ad31-4185-8cc4-0ed4641f0efc","type":"error","seq":6,"position":"PT0S","parameters":{"code":408,"message":"Connection probe error: Close transaction timed out."},"serverseq":1}

What is this error message?

Arun_Kumar1 · July 27, 2023, 4:46pm

Can u help with sample opened message JSON body?

Jerome.Saint-Marc · July 27, 2023, 4:59pm

RECEIVED: {"version":"2","id":"5f020a60-a9cf-4a0a-add7-717401f04817","type":"open","seq":1,"position":"PT0S","parameters":{"organizationId":"xxxxxxx-xxxx-xxxx-xxxxx-xxxxxxxxx","conversationId":"00000000-0000-0000-0000-000000000000","participant":{"id":"00000000-0000-0000-0000-000000000000","ani":"","aniName":"","dnis":""},"media":[{"type":"audio","format":"PCMU","channels":["external","internal"],"rate":8000},{"type":"audio","format":"PCMU","channels":["external"],"rate":8000},{"type":"audio","format":"PCMU","channels":["internal"],"rate":8000}],"language":null,"customConfig":{}},"serverseq":0}
TRYING TO OPEN .........
SENDING: {"version":"2","type":"opened","seq":1,"clientseq":1,"id":"5f020a60-a9cf-4a0a-add7-717401f04817","parameters":{"startPaused":false,"media":[{"type":"audio","format":"PCMU","channels":["external","internal"],"rate":8000}]}}
RECEIVED: {"version":"2","id":"5f020a60-a9cf-4a0a-add7-717401f04817","type":"close","seq":2,"position":"PT0S","parameters":{"reason":"end"},"serverseq":1}
TRYING TO CLOSE .........
SENDING: {"version":"2","type":"closed","seq":2,"clientseq":2,"id":"5f020a60-a9cf-4a0a-add7-717401f04817","parameters":{}}

You might have to investigate your code, and check what you are using for the websocket library.

I don't know if you have tried with a nodejs example like what I posted.
If yes, and if that doesn't work on your side, open a ticket with Genesys Cloud Customer Care.
If no, try it at least to see if the problem could come from your code or the package/library you are using for websocket.

Arun_Kumar1 · July 28, 2023, 5:52am

Kindly help with Client Application code reference link to check web socket URL instead of configuring url in Genesys audio hook. I got some GitHub code link, is it correct refernce?

Jerome.Saint-Marc · July 28, 2023, 7:54am

That is indeed an audiohook sample project which has been published/made available recently.

Regards,

Arun_Kumar1 · July 28, 2023, 8:10am

Can we follow this sample code?