Purpose
BioSense Platform users need to know what to do if they suspect duplicate messages. This technical guidance identifies common triggers and outlines a solution for resolving these issues.
High collapse rates point to duplicate messages
When facilities are brought onboard the BioSense Platform, the NSSP team checks incoming data for standard completeness and validity. They also calculate collapse rate, which is the average message count per visit. The team looks for a collapse rate of fewer than 35 messages per visit—although fewer than 10 messages per visit is preferred. When high collapse rates are found, the cause is typically duplicate messages.
Duplicate messages occur when the BioSense Platform receives the same exact message more than once. Sometimes the entire message—including Message_Control_ID—is received multiple times, indicating that the message has been sent more than once. Other times, the Message_Control_ID can change when the rest of the message is the same as a previous message. When this occurs, the cause is often related to message triggers at the data source.
Message duplication vs. duplicate visits
Do not confuse "message duplication" with "duplicate visits." Message duplication—when the BioSense Platform receives the exact same message more than once—is distinct and should be monitored and resolved using different methods.
Solving the problem
To identify duplicate messages, you will need to run SQL code to de-duplicate all messages associated with a visit (or range of visits) by all fields received from the facility and contained in the XX_ST_Processed or XX_PR_Processed table. Exclude fields created by NSSP processing. Then run an additional check while excluding the Message_Control_ID. Although some duplicate messages are almost always identified, the occurrence is usually low.
If you are concerned about message duplication for your site, please contact the onboarding team by submitting a Service Desk ticket. A team member will respond by explaining the process, answering your questions, and providing the SQL code needed to identify duplication.