A conversation with conversationStart '2023-09-12 00:32:04.770' 12/09/2023 10:32.04 local time, appears in two intervals:
2023-09-11T14:00:00.0000000Z/2023-09-12T13:59:59.9990000Z (correct). The 12th of Sep, full day
2023-09-12T14:00:00.0000000Z/2023-09-13T13:59:59.9990000Z (incorrect). The 13th of Sep, full day
The page size was set 100. The first occurrence appeared on page 35 of 82, the second on page 3 of 59.
I receive thousands of duplicates per day now from different.
With Analytics Detail Queries, "The interval will filter based on the conversation start timestamp. Conversations will only be included in the results if their start occurs on a day that the interval touches". See here.
Your conversation starts at '2023-09-12 00:32:04.770'.
This is on a day (2023-09-12) that your interval touches -> interval:"2023-09-12T14:00:00.0000000Z/2023-09-13T13:59:59.9990000Z"
So if your conversation is still active at the beginning of the interval (has some activity during the interval -> interval:"2023-09-12T14:00:00.0000000Z/2023-09-13T13:59:59.9990000Z"), it will be included in the results.
You can add additional filters on conversationStart or conversationEnd. They accept time ranges value (i.e. 2023-09-12T14:00:00.0000000Z/2023-09-13T13:59:59.9990000Z)
It is unfortunate that this is the case. We would like to store the page (of total pages) and interval with each conversation. Now we have to discard this, since this is unreliable.
Our ETL loads total pages over a longer period.
If the query allows to run not by full days but also by hours, which is it inaccurate and requires discarding conversations, which are outside the given time range?
I am sorry but I don't know what you mean by the page of total pages and interval.
If you are trying to retrieve conversation details (analytics queries), you can either use the Analytics Conversation Details Query, with interval parameter value being processed as described here.
Your interval can cover more than one day (up to 30 days), and you can use conversationFilters or segmentFilters in your query to refine the selection. As an example defining a date/time range on conversationStart or conversationEnd (value matches '2023-09-12T14:00:00.0000000Z/2023-09-13T13:59:59.9990000Z').
If it is not about retrieving analytics data from the current day (data is pushed to this repository once a day), you can also use the Analytics Conversation Details Job. It is in fact the recommended endpoint when retrieving large quantities of data.
The interval parameter works differently and there is an additional startOfDayIntervalMatching flag. Interval: Conversation detail queries will only include conversations that started on a day (UTC) touched by the interval. Conversation detail jobs will by default include all conversations whose lifetime overlaps with the query interval. If you want the same behavior as for queries, you should set the startOfDayIntervalMatching flag in the query to true.
You can also have a look at the Analytics Integration Guide, to give you dieas about alternative (e.g. AWS EventBridge integration).