Skip to main content
← Back to list
01Issue
FeatureClosedSwamp Club
AssigneesNone

Workflow run output mixes data, errors, and logs without clear separation — hard to find what actually executed

Opened by swampadmin · 7/4/2025

Problem

When running a workflow with swamp workflow run, the output combines resource data, method errors, and execution logs in a single stream. This makes it difficult to determine:

  1. Which methods actually executed vs which were skipped
  2. What data each method returned (if any)
  3. Whether a failure was from the method itself or from Swamp's execution layer

For example, running a workflow that queries 50 EC2 instances across 3 regions produces hundreds of lines where instance data, API errors, and step logs are interleaved. A user has to manually parse through to find which instances were tagged, which API calls failed, and which steps never ran.

Why it matters

Workflows are supposed to be "reviewable and accurate" (from README). Right now they're neither — the output is too noisy to review before running against production. Related: #623 and #596 both flag output readability as a blocker.

What good looks like

Clear separation between three streams:

[WORKFLOW] workflow-name started
[STEP 1/4] describe-instances → running
  └─ Queried 50 instances from us-east-1
  └─ Queried 30 instances from us-west-2
[STEP 1/4] describe-instances → completed (80 resources returned)

[STEP 2/4] filter-untagged → running
  └─ Checking 80 instances for cost-center tag
[STEP 2/4] filter-untagged → ERROR
  └─ Method failed: invalid CEL expression in filter rule
  └─ File: .swamp/workflows/audit.yaml:42
  └─ Expression: `resource.tags.missing.cost_center` (undefined field)

[STEP 3/4] tag-instances → skipped (dependency failed)
[STEP 4/4] report → skipped (dependency failed)

[SUMMARY]
Executed: 2/4 steps
Failed: 1 (filter-untagged)
Skipped: 2 (unmet dependencies)
Data produced: 80 instance records

A user should be able to scan this in 10 seconds and know:

  • Which steps ran and which didn't
  • Where the failure actually occurred
  • How much data moved through the workflow

Current behavior

Looking at how Swamp logs in src/cli/workflow/run.ts and src/workflow/execution.ts, method outputs and internal logs are written to the same sink without markers. The WorkflowRun type tracks execution state but doesn't expose structured output streams.

Proposed solution

  1. Add a --format flag: swamp workflow run --format=human (default, human-readable) or --format=json (machine-parseable)
  2. Restructure output to separate three channels:
    • Step execution trace (what ran, what was skipped, why)
    • Method data output (what each method returned)
    • Errors with file:line context (where things broke)
  3. Add a --quiet flag to suppress logs and show only the summary

This aligns with how CI/CD platforms (GitHub Actions, GitLab CI) present workflow output — structured but readable.


Contributed by Klement Gunndu

02Bog Flow
OPENTRIAGEDIN PROGRESSCLOSED

Closed

No activity in this phase yet.

03Sludge Pulse

Sign in to post a ripple.