Do not lose an accepted observation.
Messages are persistent, the publishing channel uses confirms, and the consumer acknowledges only after the SQL transaction commits.
Independent systems-integration lab / 2026
Operational telemetry replay.
A C#/.NET worker consumes equipment events over RabbitMQ, commits them to SQL, and acknowledges only after storage owns the delivery. The publisher deliberately creates duplicates, delay and reordering so the failure semantics have something real to answer.
Deliveries0
SQL inserts0
Duplicates0
Out of order0
initialising replay…
#EventInjectionSQLOrdering
01 / Requirements
Turn delivery ambiguity into a small contract.
The lab starts from failure behaviour, not from the framework template.
Make redelivery harmless.
Every envelope carries a stable event identity. That identity is the SQL primary key, so the second delivery becomes evidence instead of a second row.
Keep late data visible.
Sequence is assessed against a replay-scoped equipment cursor. A late unique observation is stored and marked; it is not silently discarded.
Distinguish poison data from transient failure.
Invalid JSON is rejected without requeue. A failed SQL operation is negatively acknowledged with requeue so it can recover.
02 / Implementation
One owner for each boundary.
A long-lived worker owns consumption. A repository owns SQL semantics. A deterministic schedule owns fault injection.
.NET BackgroundService
Explicit cancellation + long-lived connection/channel
RabbitMQ deliveries remain sequential on one consumer channel. The body is copied inside the callback, persisted, then acknowledged by delivery tag.
RabbitMQ / AMQP 0-9-1
Durable topic exchange + queue + manual ack
Publisher confirmations expose broker rejection. Mandatory routing and stable topology names make an unroutable event an error rather than a silent success.
SQLite in WAL mode
Primary-key idempotency + transactional counters
The local database keeps the lab reproducible. The storage contract is deliberately narrow enough to replace the engine without changing delivery semantics.
Deterministic release schedule
Duplicate identity + delayed sequence + reversed windows
The injector transforms a clean sequence before publishing. Tests assert the exact schedule so the experiment itself is repeatable.
03 / Failure semantics
The evidence says exactly what happened.
| Condition | Decision | Recorded evidence | What it does not prove |
|---|---|---|---|
| Duplicate event ID | Ignore second insert; acknowledge after transaction. | Delivery +1, duplicate +1, stored row unchanged. | Exactly-once delivery. |
| Lower sequence | Store unique event and flag it late. | Insert +1, out-of-order +1. | Whether late data should alter an operational state estimate. |
| Invalid envelope | Reject without requeue. | Structured warning and broker rejection path. | A complete dead-letter policy. |
| SQL failure | Nack with requeue; stable identity protects retry. | Error log and eventual redelivery. | Distributed transaction atomicity. |
04 / Delivery
Tests and tooling are part of the project.
xUnit pins the fault schedule and SQL decisions. A broker-backed integration test crosses the real AMQP boundary. GitHub Actions runs formatting, build and both suites; PowerShell and Docker Compose give another engineer the same entry points locally.