Lab: model evaluate does not persist evaluated definitions for all models

Description

swamp model evaluate resolves expressions in model globalArguments and returns correct results, but only persists evaluated definitions to .swamp/definitions-evaluated/ for some models — not all. This causes swamp workflow run --last-evaluated to fail with "No previously evaluated definition found" for the missing models.

Steps to Reproduce

Create 10 model instances of various @john/tailscale-* types, all with expressions in globalArguments:

globalArguments:
  tailnet: ${{ model.my-settings.input.globalArguments.tailnet }}
  apiKey: ${{ model.my-settings.input.globalArguments.apiKey }}
  baseUrl: ${{ model.my-settings.input.globalArguments.baseUrl }}

Run: swamp model evaluate --all --json
- Output reports all 10 models evaluated successfully with hadExpressions: true

Check disk:

find .swamp/definitions-evaluated/ -name "*.yaml" -type f | wc -l
# Returns: 5  (should be 10)

Only 5 of 10 models are persisted. The result is deterministic across repeated runs.

Models that ARE persisted:

@john/tailscale-device
@john/tailscale-user
@john/tailscale-settings
@john/tailscale-acl
@john/tailscale-dns

Models that are NOT persisted:

@john/tailscale-auth-key
@john/tailscale-contact
@john/tailscale-log-config
@john/tailscale-posture
@john/tailscale-webhook

Evaluating missing models individually also fails to persist:

swamp model evaluate my-keys --json  # reports success
find .swamp/definitions-evaluated/ -name "d12cb1c2*.yaml" | wc -l  # returns 0

Running a workflow that uses a non-persisted model:

swamp workflow run <name> --last-evaluated --json
# Fails: "No previously evaluated definition found for 'my-log-config'"

Expected Behavior

swamp model evaluate (both --all and individual) should persist ALL evaluated model definitions to .swamp/definitions-evaluated/, mirroring how swamp workflow evaluate persists to .swamp/workflows-evaluated/.

Actual Behavior

model evaluate returns correct results for all models
Only some models are persisted to disk (5 of 10 in our case)
The result is deterministic — the same 5 succeed/fail every time
No error is reported for the models that fail to persist

Impact

This blocks workflow run --last-evaluated for any workflow that references a model whose evaluated definition wasn't persisted. Combined with the separate bug where workflow run doesn't evaluate expressions at runtime (#537), this means workflows using non-persisted models cannot be executed at all.

Environment

swamp version: 20260227.225234.0-sha.2b2b28be
OS: macOS Darwin 22.6.0

Summary

This affects the model evaluation persistence layer. The evaluate logic works correctly (expressions are resolved and returned), but the file write step silently fails for some model types. The fix would involve investigating why the persistence step in model evaluate only writes for a subset of models — potentially a race condition in parallel writes, a path construction issue for certain type names, or a silent error being swallowed.

yep yep yep

Support named argument syntax for model method run (e.g. --arg key=value)

env var overrides for globalArguments silently change which environment a model targets

deno lint no-import-prefix conflicts with swamp's required npm:/jsr: inline specifiers

AI agents should search community extensions before offering to create custom models

feat: add --content-type filter to extension search

Add a 'reports' feature

Port remaining CLI commands to libswamp + renderer pattern

Persistent runner / server mode to eliminate per-invocation CLI startup overhead

AI agent should prefer fan-out model methods over shell loops for fleet operations

Support unbundled helper scripts in extension packages

Add progress indicator for long-running model methods

Built-in structured output parser for CLI commands

Automatic trace context propagation across workflow steps

Native OpenTelemetry tracing for swamp CLI internals

Claude Code should prefer extending swamp extensions over using external CLI tools

Framework log lines written to stdout pollute model method output

Swamp skill should guide AI agents to create extension models instead of inline shell scripts

Shared team collectives for collaborative extension publishing

Delete methods should return last known state data

Add smoke-test skill for extension models

Workflow run output is unreadable — resource data and errors are buried in log noise

Recommend or manage lockfile strategy for extension model npm dependencies

Output of running a method is not the actual output

Add macOS Keychain vault type

bug: Vault secret shell escaping incomplete — $(), semicolons, and pipes are interpreted

Swamp should support "Scheduled"-type status for execution

feat: add support for Nix via a flake

ModelType.normalize replaces all dots with slashes, corrupting dotted type names

fix: collective validator error message omits drivers and datastores

Skill docs: document configSchema inline requirement and .describe() constraints for vault extensions

It takes 42 seconds to run swamp --help

After today upgrade

Auto-resolver fails when extensions/models/ directory does not exist for vault-only extensions

Azure Key Vault extension vault bundle fails to load in compiled binary

RangeError: Maximum call stack size exceeded when loading extension models with npm dependencies after upgrade to 20260317

Extension Zod schema incompatibility after upgrade to 20260312

Data garbage collection never fully removes expired data from disk

extension push --dry-run requires authentication unnecessarily

Typed context APIs to eliminate no-explicit-any in extension models

AWS extensions fail to load: missing _lib/aws.ts shared dependency

getContent() should resolve vault expressions for sensitive fields

getContent() should return parsed objects, not raw Uint8Array

Add context.readResource() to MethodContext for user extension models

Datastore lock: allow concurrent reads across different models

extension install should be an alias of extension pull

feat: Execution Driver Abstraction — Pluggable Isolation for Model Execution

Support --global-arg on model create

consider a libswamp

Running workflow should not block read-only operations

Update from 20260227 to 20260311 breaks existing model definitions (symlink + path-based type)

loadUserModels ignores --repo-dir, uses cwd instead

datastore lock release --force deadlocks on the stale lock it's trying to release

Datastore lock not released when interactive command fails in non-TTY context

vault put to 1Password stores empty password field

Leaderboard shows 0-event users whose profiles 404

Move AWS, Azure, and 1Password vault providers to extensions

Data instance name uniqueness is global across specs instead of per-spec

Audit timeline misses today's entries in UTC+ timezones

AI agent discoverability: reduce trial-and-error for CLI syntax

GlobalArguments CEL expressions fail when factory inputs are omitted — selective evaluation not working

Document factory pattern for model reuse in skills

Delete method uses identifier as data name instead of instance name

CLI method run ignores required arguments without .default()

Workflow run output mixes data, errors, and logs without clear separation — hard to find what actually executed

data.latest() in workflow step inputs resolved at workflow start, not at step execution time

Update method fails when globalArguments contain cross-model CEL expressions

Track resource deletion in local data model

fix: extension npm packages fail to resolve in compiled binary

swamp repo upgrade replaces entire CLAUDE.md with generic template instead of merging

Add drift detection to reconcile stored state against live cloud resources

Model inputs schema validated on all methods — delete fails requiring create-time inputs

Support wait-until-ready semantics for async resource provisioning

support pushing to organizations

Add data rename/move command for non-destructive refactoring

Schema mismatch warnings lack model context and misrepresent API response shapes

Support pre-flight validate method on models before method execution

Extension models need built-in support for idempotent create operations

Workflow reports succeeded when step outputs indicate failure

Swamp skills don't prevent AI agents from manually generating UUIDs for workflows