Reliable automation knows what to do when the outcome is uncertain.

Distinguish rejection, failure and an unknown outcome

If an API rejects data because a field is missing, repeating the same request will not repair it. If the service responds slowly and the connection times out, the operation may have completed without confirmation reaching the caller. Treating both outcomes as “retry” can produce incorrect results.

For each integration, define observable states: received, validated, processing, completed, rejected and requiring verification. Preserve the external operation reference when available. The person responding should understand which step succeeded rather than see only a generic “workflow failed” notification. Record enough context to distinguish a new business request from another delivery of an existing one.

Repeat attempts without multiplying their effects

Idempotency allows attempts at the same operation to represent one intention. Amazon describes this principle for safer retries. Actual support depends on the API: some accept a key, while others need checks or an operation register. Do not assume every connector automatically prevents duplicates.

Where an API offers an idempotency key, reuse it only for the same logical operation under that API’s contract. A different request should not inherit an earlier request’s key. Stripe documentation illustrates provider-specific rules rather than a guarantee applicable to other services.

Scroll the table to compare all columns.

Illustrative example: order created, response lost

1. 01

   Initial attempt

   The workflow submits an order. The business system records it, but confirmation does not reach the workflow within the expected time.

2. 02

   Verification

   The workflow preserves the same operation identity. If supported, it checks status using an external reference or retries with the required idempotency key.

3. 03

   Intervention

   If the outcome remains unknown, the case enters a queue showing data and attempts. A person checks the business system before authorising another creation.

Safe retries are not a promise of exactly-once execution across the whole system. Check the boundaries and guarantees of every component.

Periodically check that systems agree

Reconciliation compares expected outcomes with recorded ones: a completed workflow case should have the correct reference in the business system, and vice versa. Define meaningful discrepancies, who examines them and how corrections preserve a trace of the change.

Not every step can be undone. A draft can be deleted; a sent email cannot be recovered as easily. Order actions and design corrective steps around this difference. Rolling back code does not undo effects already produced in external systems. Include those outstanding effects in any recovery plan.

Tests to request before release

Document these tests alongside the operating runbook. The benefit is practical: the team should know how to recover a case without asking the supplier to reconstruct every incident from scratch. Include the person authorised to restart processing once reconciliation is complete.

• Simulate timeouts before and after the external record is created.
• Deliver the same event twice and verify the result.
• Stop processing between two systems and resume without duplicates.
• Exhaust retries and check the queue, alert and case assignment.
• Compare external and internal states after a manual correction.

Practical questions