Engineering field notes
Event time, late arrivals and replay in IoT telemetry
Design a test matrix for delayed and duplicate IoT telemetry. Define event time and receive time, replay a small fixture and preserve evidence for ingestion changes.

Write the timestamp contract
Define event time as the timestamp representing when the device measured the value. Define ingest time as when the receiver accepted the message. Record the time zone convention, expected precision, device clock quality and the decision for absent or impossible device timestamps.
Choose the identity used for idempotency early. Examples include a device ID with sequence or a stable source event ID. Include retransmission, a sequence rollover and an out-of-order arrival in your seed fixture. Set an expected outcome for each record before running the pipeline.
Design a replay fixture with a real boundary
For a one-minute rollup, use sample timestamps at 00:00:00, 00:00:59 and 00:01:00. Give one event an ingest time two minutes later, and repeat its event ID. Assign expected bucket and duplicate handling based on the product’s time policy. Decide whether late data updates published aggregates or goes to a separate reconciliation path.
Include one invalid timestamp to exercise quarantine and one device clock discontinuity to test the defined clock policy. Preserve the raw sequence before normalization so a reviewer can compare what arrived with what was stored.
Verify before and after ingestion
Use the same fixture for the baseline and proposed implementation. Keep replay ordering explicit and read results from the normal consumer path. Compare counts, per-window totals, quarantine outcomes and the raw event identity. Run the same event set twice and inspect whether the state is stable.
Record the schema version, parser commit, boundary conditions and artifact locations. If downstream corrections are allowed, test when and how consumers learn that an aggregate changed. Use the output diff and expecteds as the evidence for a release decision.
Plan the production checks
Choose a concise set of indicators: duplicate rate, transport delay by percentile and the rate of events that miss the accepted lateness threshold. When evaluating a rollout, preserve the traffic window and device population. Sensitive payloads can stay in the source system while a sanitized result summarizes behavior.
Example file
The sample contains UTC event and receive times, a repeated ID, a minute boundary and an invalid timestamp. Compare those cases with your written policy.
Download the telemetry replay fixtureSkills in the catalog
Repository sources
References are pinned to the same upstream commit as the public catalog.