@portfolio/eve-world-aws — implementation spec

A custom Workflow World that backs eve's durable runtime with DynamoDB + SQS + S3 instead of local disk, so an eve agent runs on AWS Lambda (scale-to-zero, same account as the OpenNext portfolio — no Vercel, no cross-cloud hop).

Version pins (load-bearing)

npm latest for @workflow/* is 4.2.0. Install the beta explicitly. A 4.x world against a 5.x core fails with ZodError: invalid_union at replay.

The contract

World extends Queue, Storage, Streamer + specVersion?, start?(), close?(). Three concern modules (storage.ts, queue.ts, streamer.ts), composed flat in index.ts. Reference implementation to port: @workflow/world-postgres (dist/storage.js, queue.js, streamer.js, drizzle/schema.js).

Storage → DynamoDB (+ S3) — storage.ts

Event-sourced: events.create() is the only mutation; runs/steps/hooks are materialized views folded from the log. Write event + view in one TransactWriteItems.

• Idempotency: the PG events_entity_creation_unique_index → ConditionExpression: attribute_not_exists(...) so queue redelivery is a no-op.
• Large payloads: run input / step output over the item budget → S3 (runs/<runId>/...), { s3: key } pointer in the item. resolveData:'none' returns the *WithoutData view and skips the S3 GET.
• Encoding: reuse @workflow/serde (the PG world stores CBOR); bytes ≤ ~400 KB in DynamoDB, larger in S3.

Queue → SQS — queue.ts

Callback-driven: createQueueHandler returns an HTTP handler (req) => Response.

Waits longer than SQS's 15-min DelaySeconds cap → EventBridge Scheduler → enqueue at fire time.

Streamer → the crux — streamer.ts

PG uses LISTEN/NOTIFY for the live readFromStream (a ReadableStream that waits for new chunks). DynamoDB has no pub/sub, so:

• Strategy A (serverless-pure, current): buffered writes reserve indices and batch up to 25 CHUNK#<index> items; live reads poll Query from the cursor every ~100ms until STREAM#meta.done. The unbounded live query reads chunks and the meta row together, avoiding a second point read. Zero standing infra.
• Strategy B (sub-ms): Redis (ElastiCache/MemoryDB) pub/sub. Reintroduces an always-on, in-VPC cluster (Lambda needs VPC attachment). Only if A's latency bites.

getStreamChunks / getStreamInfo are snapshot reads — identical either way.

DynamoDB single-table design

See src/ddb.ts for the key builders. One on-demand table:

RUN#<runId>          EVT#<ulid>            append-only event log (source of truth)
RUN#<runId>          RUN#meta              run materialized view
RUN#<runId>          STEP#<stepId>         step materialized view
HOOK#<token>         HOOK#meta             hook (global token lookup)   GSI1 -> RUN#<runId>
STREAM#<runId>#<nm>  CHUNK#<index>         stream chunk (20-digit zero-padded SK)
STREAM#<runId>#<nm>  STREAM#meta           { tailIndex, done }
GSI1PK = CORR#<correlationId>              events.listByCorrelationId

Host layer (separate from the world)

• eve's Nitro Node output → Lambda via Lambda Web Adapter (container image), function URL in RESPONSE_STREAM mode for token streaming.
• Serves /eve/* and /.well-known/workflow/v1/flow. The SQS-consumer Lambda drives continuations into that callback.
• If a reverse proxy/ingress sits in front, forward both /eve/ and /.well-known/workflow/ — a proxy scoped to /eve/ lets sessions start but stalls runs forever (callbacks never arrive).

Staged plan (de-risk the world before Lambda)

1. Port from PG: read @workflow/world-postgres storage.js/queue.js/streamer.js; implement the methods above. Run pnpm typecheck against the Eve-compatible @workflow/world pin — the type errors are the precise checklist; fill in until the scoped casts in storage.ts are unnecessary upstream.
2. Prove it under plain eve start (local Node, not Lambda) with experimental.workflow.world: "@portfolio/eve-world-aws", pointed at real DynamoDB + SQS + (Strategy A) streamer. Validate durability + mid-turn resume here.
3. Containerize the host → Lambda (LWA + streaming function URL); wire the SQS consumer → flow callback.
4. Optimize the streamer (Redis) only if Strategy A's token latency is poor.

Implementation status (port complete)

storage.ts, queue.ts, streamer.ts, index.ts are implemented and typecheck clean (0 errors) against @workflow/world@5.0.0-beta.16 + @workflow/errors@5.0.0-beta.10 (tsc --noEmit). Notes that supersede the original scaffold's assumptions:

• Interface shapes corrected vs. the scaffold. The Streamer is a nested streams: { write, writeMulti?, close, get, list, getChunks, getInfo } object (arg order (runId, name, …)), not the flat writeToStream/readFromStream/… the scaffold stubbed. Storage.hooks is { get(hookId), getByToken(token), list } with no dispose — hook teardown is implicit on terminal run_* events. events.listByCorrelationId exists; there is no hooks.listByCorrelationId.
• Errors must be the real @workflow/errors classes (added as a dep). The core runtime instanceof-checks EntityConflictError to drive its concurrent-replay dedup/idempotency path; a look-alike class silently breaks replay.
• runs/steps keep a scoped as unknown as Storage[...] cast (see the comment at the bottom of storage.ts). Their get/list are overloaded so resolveData:'none' returns the *WithoutData variant (input/output typed as the literal undefined); a single impl signature can't statically satisfy that even though it does at runtime. world-postgres has the same gap. events and hooks are cast-free (satisfies Storage).
• Required indexes: GSI1 (gsi1pk/gsi1sk) for events.listByCorrelationId (CORR#…) and hooks.list(runId) (RUN#…); GSI2 (gsi2pk/gsi2sk) reused for global run listing (gsi2pk="RUN") and hook-by-id lookup (gsi2pk="HOOK"). Code is robust to a KEYS_ONLY projection (it refetches full items by primary key), but projection ALL is recommended to avoid the extra GET.
• Large payloads (run/step input/output/error, hook metadata, and the matching EVENT_DATA_REF_FIELDS inside eventData) over ~300 KB are offloaded to S3 with a { __s3: key } pointer; resolveData:'none' blanks input/output and skips the GET. Payloads are stored verbatim (Uint8Array) — world-postgres does not use @workflow/serde, so neither do we.

Open verification items (need a live run, not typecheck)

• SQS FIFO vs. DelaySeconds. FIFO queues ignore per-message delay, so for *.fifo we set MessageGroupId/MessageDeduplicationId (ordering+dedup) and drop DelaySeconds; for standard queues we set DelaySeconds. Waits > 15 min exceed the SQS cap entirely — they need EventBridge Scheduler in the host layer (currently clamped to 900 s and left to the runtime's TooEarlyError re-enqueue loop). Confirm the chosen queue type + the long-wait path end-to-end.
• createQueueHandler wire contract. We reproduce world-postgres's x-vqs-* header contract (x-vqs-queue-name/-message-id/-message-attempt) and the Uint8Array-preserving JSON transport. Confirm the SQS-consumer Lambda forwards these (attempt ← ApproximateReceiveCount) and that flow/step routes are both proxied (a /eve/-only ingress stalls runs forever).
• Idempotency/race fidelity. Creation events use DynamoDB attribute_not_exists conditions and terminal-state guards; the deep PG edge cases (orphaned-hook recovery, attr_set 64-key cap enforcement, legacy specVersion routing) are intentionally simplified — validate under real redelivery + concurrent replay.
• specVersion negotiation. We advertise SPEC_VERSION_CURRENT (matching world-postgres). Confirm replay compatibility against Eve 0.22.4 / @workflow/core@5.0.0-beta.28 with pre-upgrade runs.
• Confirm eve's eve build Node output runs under Lambda Web Adapter with RESPONSE_STREAM (host-layer assumption, not yet validated).