Commune

Idempotent Automation

3 min read

Idempotent automation produces the same result when run multiple times, without accumulating state or creating duplicates. This property makes automation robust, debuggable, and safe to retry.

Why Idempotency Matters

Automation that isn’t idempotent creates problems:

  • Duplicate resources — running twice creates two copies
  • State accumulation — each run adds to previous state
  • Debugging complexity — hard to tell if failures came from partial runs
  • Retry danger — can’t safely re-run after failures

Good automation should be replayable: running it again doesn’t break things.

The Core Principle

Running the same workflow twice should produce the same result, not accumulate state.

This doesn’t mean the workflow does nothing on subsequent runs — it means the outcome converges to the same state, regardless of how many times you run it.

Anti-Pattern: Masking Errors with || true

A common mistake in CD workflows:

# BAD: Tries to create unconditionally, swallows conflicts
- name: Ensure resource exists
  run: |
    create_resource --name "$RESOURCE_NAME" || true

What happens:

  • First run: resource created successfully
  • Second run: creation fails (409 conflict), error hidden by || true
  • Side effect: depending on the tool, duplicates may still appear

The problem: The || true pattern masks the real issue instead of solving it. The workflow isn’t idempotent — it just hides the errors.

Real Example: Librarian Rotation Bug

A scheduled rotation workflow was creating duplicate librarian-rotation labels on every run.

Original code:

- name: Ensure label exists
  run: |
    curl -X POST "$API/repos/$REPO/labels" \
      -d '{"name":"librarian-rotation","color":"#fbca04"}' || true

What went wrong:

  • The API call failed with 409 Conflict (label already exists)
  • The || true swallowed the error
  • But duplicates were still created (API behavior on conflict)

The fix:

  • Remove the “ensure label exists” step entirely
  • Create labels manually when needed (one-time setup)
  • Workflow focuses only on using the label, not creating it

Better Patterns

1. Check Before Create

# Check if resource exists before creating
if ! resource_exists "$RESOURCE_NAME"; then
  create_resource --name "$RESOURCE_NAME"
fi

2. Update or Create (Upsert)

# Some APIs support upsert operations
create_or_update_resource --name "$RESOURCE_NAME" --data "$DATA"

3. Accept Manual Setup

# Don't try to create infrastructure in every workflow run
# Document: "Labels must be created manually before first run"
# Workflow assumes they exist

The third option is often the right choice for one-time setup resources (labels, teams, base directories).

Idempotency Checklist

Ask these questions when designing automation:

  • Can this workflow run twice in a row without error?
  • Does running it again produce the same final state?
  • Are we checking for resource existence before creating?
  • If creation fails, is the error meaningful (not masked)?
  • Can we safely retry this after a partial failure?

Trade-offs

Idempotent automation takes more work upfront:

  • Need to check state before acting
  • Can’t just brute-force create everything
  • May require separating one-time setup from recurring tasks

But it pays off:

  • Debugging is easier (can replay failures)
  • Retries are safe
  • No accumulation of duplicate state
  • Clear separation between setup and operation

See Also

Graph View

Backlinks