REI Skills

Engineering field notes

Event time, late arrivals and replay in IoT telemetry

Design a test matrix for delayed and duplicate IoT telemetry. Define event time and receive time, replay a small fixture and preserve evidence for ingestion changes.

Three checkpoints in the guide: Device timestamps, Watermarks and dedupe, Replay report
Rootcastle field notes · Engineering field notes

Write the timestamp contract

Define event time as the timestamp representing when the device measured the value. Define ingest time as when the receiver accepted the message. Record the time zone convention, expected precision, device clock quality and the decision for absent or impossible device timestamps.

Choose the identity used for idempotency early. Examples include a device ID with sequence or a stable source event ID. Include retransmission, a sequence rollover and an out-of-order arrival in your seed fixture. Set an expected outcome for each record before running the pipeline.

Design a replay fixture with a real boundary

For a one-minute rollup, use sample timestamps at 00:00:00, 00:00:59 and 00:01:00. Give one event an ingest time two minutes later, and repeat its event ID. Assign expected bucket and duplicate handling based on the product’s time policy. Decide whether late data updates published aggregates or goes to a separate reconciliation path.

Include one invalid timestamp to exercise quarantine and one device clock discontinuity to test the defined clock policy. Preserve the raw sequence before normalization so a reviewer can compare what arrived with what was stored.

Verify before and after ingestion

Use the same fixture for the baseline and proposed implementation. Keep replay ordering explicit and read results from the normal consumer path. Compare counts, per-window totals, quarantine outcomes and the raw event identity. Run the same event set twice and inspect whether the state is stable.

Record the schema version, parser commit, boundary conditions and artifact locations. If downstream corrections are allowed, test when and how consumers learn that an aggregate changed. Use the output diff and expecteds as the evidence for a release decision.

Plan the production checks

Choose a concise set of indicators: duplicate rate, transport delay by percentile and the rate of events that miss the accepted lateness threshold. When evaluating a rollout, preserve the traffic window and device population. Sensitive payloads can stay in the source system while a sanitized result summarizes behavior.

Example file

The sample contains UTC event and receive times, a repeated ID, a minute boundary and an invalid timestamp. Compare those cases with your written policy.

Download the telemetry replay fixture

Skills in the catalog

Repository sources

References are pinned to the same upstream commit as the public catalog.