Skip to main content
← Back to list
01Issue
FeatureClosedSwamp Club
AssigneesNone

Extension models need built-in support for idempotent create operations

Opened by swampadmin · 5/13/2025

Problem

When a workflow partially fails and is re-run, model create methods execute again for resources that were already successfully created. This leads to duplicate resources for non-idempotent APIs (e.g. Droplets, Load Balancers) because the auto-generated models are simple POST wrappers with no state awareness.

Real-world scenario encountered:

  1. Workflow with VPC, Tag, Droplets, LB, and Firewall
  2. VPC creation fails (invalid CIDR), Tag succeeds
  3. User fixes the CIDR and re-runs the workflow
  4. Tag step runs again (harmless — DO tags are idempotent)
  5. If all steps had succeeded initially, a re-run would create duplicate droplets and load balancers

The context.dataRepository is available for models to check existing state, but every model author has to implement this pattern themselves. The auto-generated DigitalOcean extension models don't do this at all — their create methods unconditionally POST to the API.

Proposed Solution

Provide a framework-level mechanism so model authors get idempotent creates without boilerplate. Some options:

  1. createOrGet helper in context — A utility that checks existing data before calling create, e.g. context.createOrGet("state", instanceName, () => create(...)). If state data already exists, it returns the existing handle; otherwise it runs the create function and stores the result.

  2. Declarative idempotent: true on methods — swamp skips execution if the method's output spec already has data for this model instance. Simple and zero-effort for model authors.

  3. Workflow-level "skip if data exists" on steps — Let workflow authors mark steps that should be skipped when the target model already has data, e.g. skipIfExists: true.

  4. Code generation improvement — The auto-generated models should check context.dataRepository for existing state before creating, and optionally verify the resource still exists via a GET call.

Scope of Changes

  • Model execution context — Add helper methods or pre-execution checks
  • Extension model code generation — Generated create methods should include state checks
  • Workflow engine (if option 3) — Add step-level skip conditions based on data existence
  • Documentation — Best practices for idempotent model design

The core issue is that partial workflow failures are inevitable (API errors, rate limits, transient failures), and re-running is the natural recovery path. Without idempotency, re-running is risky rather than safe.

02Bog Flow
OPENTRIAGEDIN PROGRESSCLOSED

Closed

No activity in this phase yet.

03Sludge Pulse

Sign in to post a ripple.