Skip to content

ADR-0003: Async Pipeline — Deferred

  • Status: Deferred (not "rejected")
  • Date: 2026-04-18
  • Deciders: Andrew Pappas (autonomous session, post-migration)
  • Supersedes: N/A
  • Superseded by: N/A (may be revived if motivating problem emerges)
  • Migration provenance: fuzzywigg/PikoClaw#187 (abandoned during migration, branch preserved in archived source repo)

Context

During the fuzzywigg → nft2-me migration triage, open PR #187 ("Async pipeline WIP") was classified ABANDON (class: WIP-vague) in migration-abandoned-prs.md. The PR represented a multi-week pivot toward an asynchronous processing pipeline for PST ingest, intended to decouple the extraction worker from the embedding worker and allow retry, backpressure, and per-stage observability.

The work was abandoned not because the direction is wrong, but because:

  1. The motivating production problem ("we are bottlenecked by synchronous ingest") does not yet exist — the current synchronous path handles every demo-grade PST within acceptable latency.
  2. The implementation had diverged across multiple incomplete branches, with unclear contract at the stage boundaries.
  3. The Panathenea demo (May 27–29, 2026) is single-user, on-stage, and does not exercise the scale that would force this architecture.

This ADR exists to record the rationale for not building the async pipeline right now, so that future autonomous sessions do not re-discover the problem, re-scope the rewrite, and re-abandon it without knowing the prior work happened.

Decision

We defer the async pipeline rewrite indefinitely. The current synchronous ingest path remains the default. The idea is preserved here, not in code.

Motivating Conditions for Revival

The async pipeline should be revisited only if at least one of the following is observed:

  1. Measurable ingest backpressure — a PST takes longer than 15 minutes on reference hardware (RTX 5070 / 64GB RAM) OR a user reports the UI "feels stuck" during ingest.
  2. Multi-tenant scope — PikoClaw gets deployed to serve more than one user concurrently, at which point serial ingest becomes head-of-line blocking.
  3. Retry requirement — a production user reports a failed ingest that would have been recoverable mid-pipeline (embedding-stage OOM after successful extraction).

Until one of those conditions is observed and documented in a motivating issue, any PR proposing async rewrites should be closed with a link to this ADR.

Alternatives Considered

  • Accept #187 as-is: Rejected. The branch was incomplete and its stage boundaries were not contract-tested.
  • Cherry-pick the observability pieces only: Rejected. Those pieces don't stand alone without the pipeline they were instrumenting.
  • Rewrite from scratch now: Rejected. No motivating problem; would delay demo readiness.

Consequences

  • Positive: No distraction ahead of Panathenea. Single code path. Reviewable CI.
  • Negative: If we do eventually need async, we start roughly from scratch. The #187 branch in archived fuzzywigg/PikoClaw is available for archaeological reference (SHA preserved in migration-abandoned-prs.md).
  • Neutral: This ADR should be revisited if any of the revival conditions above becomes true. Link the motivating issue back here.

References