Historical adherence issue - mixed data

Hello everyone,

Need someone`s help.

I really tired of historical adherence mechanism - websocket pool.

In my auto ingesting system there were a lot different problems, when I got mixed data (One creds for different ENVs + local runs to load historical data in multiple processes cause issues that all created channels are subscribed to same notification and there is infinity incoming messages in one websocket poll.

So, if I have 1 process to load data for 2021 January, one process to load 2018 July, I will get duplicated mixed data.

It looks absolutely illogical and inconvenient.
I checked documentation and Google for notifications subscribe a lot of times and did not find any solution or work around. (Maybe I am blind?)

I tried to re-create channels (in the hope that the old ones will be cleansed), but nothing.
It did not work.

Moreover, websocket pool/queue managed to break itself, because I was getting old data (websocket responses from requests which I even did not remember) during 4 hours. (There were no launches during that).

I think behavior of loading of historical adherences needs updates or any attributes to separate requests or attributes to increase safety during parallel launches.

Thanks. Hope to get soon answer.

I'm sorry to hear about your frustrating experience, hopefully the following will help.

The historical adherence API returns an "id" property. When you subscribe to the notification topic, the notifications will also come through with an id that you can use to filter out results that don't correspond to your request. Your application should subscribe to the notification topic prior to making the API request, start saving notifications, then after you make the API request you should compare the id properties of all received notifications to the id returned on the response from the API.

id - (string) The query ID to listen for.

There is a tutorial for subscribing to historical adherence notifications here, and you can find more documentation on the historical adherence API route here

Also I'm not sure what you mean by this, can you give more details?

Hello,

To add to Brian's answers, and to your comment on "I tried to re-create channels (in the hope that the old ones will be cleansed), but nothing"
If you don't close the websocket corresponding to a specific subscription channel, you will continue to receive updates on topics you have subscribed to on this subscription channel.
What you can do is to request to stop receiving topics on a specific channel (if the corresponding websocket is not closed yet).
You can either replace the list of subscriptions with PUT /api/v2/notifications/channels/{channelId}/subscriptions (with an empty array - [] - in the request body).
You can also unsubscribe for all topics with DELETE /api/v2/notifications/channels/{channelId}/subscriptions.

Regards,

1 Like

Hello,

Thank you so much for your reply.
I have already done something similar, but don't compare ids.

Will try.

I created a lot of different channels and historical requests.

In official documentation I see:

There is a limit of 20 channels per user/app combination. Creating a 21st channel will remove the channel with oldest last used date. Channels without an active connection will be removed first.

I have an assumption that it does not work, because all my channels were displayed as opened for several hours. And they continued to get different messages, although I did not make any new requests.

Hello,

Thank your for additions.

I am using Python context manager "with", which, in my opinion, close websocket connection itself.
Perhaps, I am wrong, but I was sure, that all ws connections are closed after execution.

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.