Skip to main content

Factory as Supervisor — Architecture Plan, v2.1

Date: 2026-05-02 · Author: Sauna synthesis for Adrian · Status: Ready-to-execute Supersedes: ARCHITECTURE_PLAN_2026-05-02_SUPERVISOR_v2.md (v2), _v1.md (v1) Related: factory_core_architecture.md, AI_CORE_AUDIT.md, 2026-05-01_KANBAN_TO_PROD_FLOW.md, 2026-05-01_MULTI_AGENT_READINESS.md, FACTORY_STRATEGIC_REVIEW.md (Phase 8.5)
Changelog v2 → v2.1:
  • §5.5 Blessed-template threshold locked at 3 successful runs, not 10.
  • §5.7 Verifier tokens given their own scope class (supervisor.verifier-readonly) so the audit trail is complete.
  • §5.8 Lock primitive replaced — true CAS via LockDO Durable Object singleton per app. D1 becomes audit log, not mutex.
  • §5.9 Template quality tracking — new section. template_stats table, revert-detection webhook, quality threshold that demotes templates back to plan-approval on revert-rate > 20%.
  • §6 Long-context fallback decided: Gemini 2.5 Pro via Vertex AI on factory-495015. Grok-4 dropped entirely. Reasoning in §6.
  • §7 Capability schema tightenedmutating flag replaced with graded side_effects levels. String slots require regex / enum / referential check. Dynamic filters must declare their allowlist inline. App-side parameterization at @latimer-woods-tech/admin is the primary SQL-injection defense; lint is the backstop.
  • §10 Credential-leak doc-lint moved from aspiration to concrete regex set.
  • §12 Open questions reduced from 7 to 3 — 4 are now decisions.

TL;DR

The factory can become the supervisor that drives the other projects to completion using AI tokens as the compute unit. You already have ~70% of the substrate. What’s missing is a control plane binding three things you’ve already built: (a) the kanban→prod pipeline, (b) the 19 @latimer-woods-tech/* packages, (c) the docs/service-registry.yml inventory of app capabilities. The supervisor is template-grounded (not generative), security-bounded (not God-mode on /admin), and single-writer (mechanically, via LockDO). It earns autonomy by accumulating templates from real human work. Three planes, clean separation:
  1. Dreamstate — intent becomes structured work items.
  2. Factory Supervisorapps/supervisor worker. Template-matches issues, executes scoped tool calls.
  3. App Plane — 11 repos. Each product AND a bounded capability surface.
Realistic shipping window: 4–6 weeks at factory pace. Selfprime’s broken Stripe funnel is fixed by you this weekend, not by the supervisor in week 8.

1. Where each project actually stands

Out-of-scope: neighbor-aid, ijustus, cypher-healing, the-calling, xpelevator. The leverage argument: every fix in factory packages or reusable workflows simplifies 4–7 downstream apps in one PR. The converse isn’t true.

2. What substrate already exists

Don’t rebuild.

Control surfaces

  • GitHub App factory-cross-repo (App ID 3560471). Currently installed on adrper79-dot; needs second install on Latimer-Woods-Tech.
  • GitHub Project v2 #1 “LatWood Operations” — kanban with Priority, Sprint, Agent, Deploy SHA, Sentry Link, Status.
  • AGENT_PROTOCOL.md merged via factory#52.
  • Reusable workflows_app-ci.yml, _app-deploy.yml, _post-deploy-verify.yml. SYN-2 reliability-gate + SYN-3 prod-canary in flight.
  • docs/service-registry.yml — canonical worker→URL→consumer map. Becomes the supervisor’s tool catalog index.

Shared library

19 @latimer-woods-tech/* packages at 0.2.0. Supervisor-relevant: llm, monitoring, stripe, neon, email, analytics, deploy, admin, compliance.

Runtime primitives (CF Agents Week 2026)

Dynamic Workers + Durable Object Facets, Agent Memory (beta), unified AI Gateway + Workers AI, Artifacts, Cloudflare + Stripe Machine Payments Protocol.

Observability + finance

  • Sentry org latwood-tech (10 worker projects).
  • Stripe live; test-mode key pending.
  • GCP factory-495015service-account key rotation: today, before anything else.
  • Anthropic, Groq, xAI, Gemini (via Vertex AI on factory-495015) all accessible.

3. The Dreamstate plane

Turns “I want X” into structured work items.

Sources

  • documents/factory/dreamstate/<app>/<feature>/spec.yml — Dreamstate schema (intent, capabilities_required, capabilities_exposed, dependencies, acceptance_gates, non_goals).
  • docs/adr/NNNN-slug.md — ADRs.
  • docs/APP_PLANNING_PATTERN.md, docs/APP_SCOPE_REGISTRY.md.

Item entry, priority order

  1. Human-authored — direct on the board. Default forever.
  2. Dreamstate-authored — scheduled workflow diffs spec vs current state. Labels: source:dreamstate, supervisor:approved-source.
  3. Sentry/Stripe-authored — webhook workers file issues. Labels: source:webhook, supervisor:approved-source.
Only items with supervisor:approved-source are eligible for supervisor pickup. Factory is public — anyone can file an issue. Random issues are quarantined until a CODEOWNER triages.

4. The factory supervisor — shape

apps/supervisor worker. One SupervisorDO class with Facets per app. Separate LockDO class for single-writer enforcement.

5. Supervisor design

5.1 Loop

5.2 Trust tiers

Hard-never (inherit docs/AGENTS.md): delete CF resources, change rulesets, write to Neon prod user tables, mutate Stripe products/prices/webhooks, send live email/SMS outside test mode.

5.3 Why Durable Object + Facet

  • Long-running (one run can span hours waiting for CI).
  • Single-writer per DO instance, augmented by LockDO for cross-instance safety (§5.8).
  • Colocated state.
  • Facets handle multiple concurrent app contexts cleanly.

5.5 Planner — template-grounded

Two-function planner:
  • matchTemplate(issue) → Template[]deterministic. No LLM. Pattern-matches labels + title + body.
  • parameterize(template, issue) → Plannarrow LLM call. Fills declared slots.
Zero matches ⇒ Red, supervisor:no-template, queued in no-match-candidates/ for human template authoring. Template schema (docs/supervisor/plans/<slug>.yml, versioned):
Note the acceptance_gate now includes a verifier_query — the verifier runs this deterministically rather than asking an LLM to introspect. LLM only asked for binary pass/fail when the gate is narrative-only (e.g., “docs read clearly”). Five gates before EXEC:
  1. Schema — Plan validates against Plan TS schema.
  2. Capability — every tool exists in loaded capabilities.yml.
  3. Slot validators — type + regex + referential checks.
  4. Tier — static path-analysis; auto-promotes to Red on protected paths.
  5. Plan-approval — required on (first 10 runs of any template) OR Yellow OR Red. Dropped for Green once a template is blessed (≥3 successful runs, zero reverts).
Match-rank tiebreakers: specificity → hit-rate → recency → supervisor:ambiguous-template → human. Bootstrapping (week 3): reverse-engineer 6–8 starter templates from the last ~50 merged PRs. Target coverage: Stripe funnel debug, Sentry migration gap, deploy config drift, capabilities.yml lint fix, reusable workflow rollout, docs/naming PRs. Fewer than 5 viable templates from first pass → add a week before first supervised run. Growth loop: every Red-tier issue closed manually triggers a workflow that drafts a candidate template from the receipt trail, files as Green-tier PR to docs/supervisor/plans/. Rot prevention: tests/supervisor/template-suite.test.ts runs on every capabilities.yml change AND every template change. What this does NOT solve: novel issue types (by design → human). Badly-written templates (defenses: human review on creation, test harness, §5.9 quality tracking).

5.6 Memory — dual-write, fail to D1

Two impls selected by MEMORY_BACKEND=agent|d1:
  • CfAgentMemoryImpl — primary.
  • D1KvMemoryImpl — fallback. D1 for run/step/feedback; KV for last-run-per-app; Vectorize for searchSimilar.
Dual-write day one. ~10% extra write volume. Flip env flag on beta flakiness, zero data loss. Reads primary only. Monthly reconciliation scheduled workflow → issue on drift. Promotion: if Agent Memory hits GA cleanly, stay. If beta past 2026-Q3 or two breaking changes, flip primary to D1 permanently.

5.7 Security model

Confused-deputy attack surface. Designed accordingly. Issue bodies are data, not instructions. Planner system prompt:
Tool calls schema-bounded. No “execute this string” or arbitrary POST tool. Every call must match a declared capabilities.yml entry with validated slots. Three token classes per run, all short-lived (1h TTL), minted at run-start, revoked at run-end, run_id in JWT claims for app-side audit traceability: No long-lived god-tokens. App-side /admin audit logs distinguish EXEC reads from verifier reads from mutations. Source-trust label non-bypassable. Absence of supervisor:approved-source ⇒ unconditional skip. Per-run write-amplification ceiling. ≤25 mutating tool calls per run. ≤5 per app. Hijacked planner can’t pump 1000 calls. Out-of-band confirmation for any /admin mutation regardless of tier. Supervisor posts exact tool-call JSON + args. Requires CODEOWNER ✅ before EXEC. Finer-grained than plan-approval — catches malicious slot values. Audit log. Every tool call → Sentry breadcrumb + D1 row + GitHub comment. Tool-layer rate limits. Token-bucket per app and global. Prevents runaway loops and DDoS-via-supervisor. App-side defenses, explicit. Capabilities.yml lint is a backstop, not the primary defense:
  • @latimer-woods-tech/admin base class enforces parameterized queries + explicit allowlists on every /admin route.
  • No /admin route takes user-supplied string params that flow into SQL without a parameterized binding.
  • Schema-lint (see §7) rejects ambiguous slot types at PR time.

5.8 Single-writer enforcement — LockDO, not D1 CAS

D1 can’t deliver true compare-and-swap across concurrent writers. Durable Objects are single-threaded per instance — that’s the correct primitive. LockDO singleton per app (named env.LOCKS.idFromName(appName)):
True CAS by DO single-threading. D1 is the audit log, not the mutex. Supervisor:
TTL 30 min, renewed every 10 min. Crash → expires → next run reclaims. Visibility: supervisor:lock-status label on a per-app pinned issue updated on claim/release. Informative, not enforcing.

5.9 Template quality tracking

Hit-rate and revert-rate need a data model, not a prayer. template_stats table:
Updates:
  • At run close (§5.1 step 16) — stats.ts increments runs_attempted, conditionally runs_passed_intent_verification, runs_merged.
  • Revert detection — GitHub pull_request webhook worker watches for Revert "<original PR title>" or commit messages referencing supervisor-authored PR SHAs. On match, increment runs_reverted on the template that produced the original PR.
  • Human override — CODEOWNER closes a supervisor-opened PR without merging and without the supervisor:approved reaction → increment runs_human_overridden.
Blessing threshold: runs_merged ≥ 3 AND runs_reverted = 0 AND runs_human_overridden = 0. Template becomes eligible for Green-tier auto-merge without plan-approval. Quality threshold (demotion): runs_reverted / runs_merged > 0.2 over last 20 runs → template auto-labeled supervisor:template-quality-review, blessed_at cleared, demoted_at set. Future runs require plan-approval until a CODEOWNER investigates + updates the template version or explicitly re-blesses. Review workflow: scheduled daily job queries demoted templates, posts summary to Pushover + opens a review issue per demoted template. Initial state: all bootstrap templates start unblessed. Earn blessing through real runs.

6. AI tokens — workload-by-workload, Gemini locked in

Split by workload, not by theology: Grok dropped everywhere. No role left after Gemini takes the long-context slot. xAI account can be cancelled at convenience.

Routing

  • All LLM calls org-wide → env.AI_GATEWAY_URL.
  • Gateway: exact-match cache (never semantic — privacy/quality footgun per AI_CORE_AUDIT), rate-limit, cost log.
  • Prompt-cache Anthropic system blocks > 1024 tokens.
  • Gemini path provisioned via Vertex AI on factory-495015 — enable aiplatform.googleapis.com one-time, authenticate via supervisor’s own service account (new, not reusing factory-sa — principle of least privilege).

Budget — calibration before caps

Phase 1 — Calibration (weeks 2–4): no monthly caps. Per-call ceiling 50k tokens. Per-run hard ceiling $5. Every call metered to D1 keyed (project, actor, run_id, yyyy-mm). Phase 2 — Steady state (week 5+): observed p95 × 1.5 = monthly cap per (project, actor). Email alert 75%. Hard stop 100%. Monthly review. BUDGET_EXCEEDEDsupervisor:budget-paused + human issue. Supervisor does NOT silently degrade tier.

AI tokens as a product primitive (later)

Once the meter is real, expose token-denominated SKUs: HumanDesign deep-synthesis add-on, videoking title-optimizer, xico-city feed curation. One ledger, many front-ends. Ship only after Phase 2 caps are stable.

7. Capability sharing — bounded schema

Every app exposes /admin via @latimer-woods-tech/admin. Supervisor calls via scoped JWTs (§5.7). Contract (docs/CAPABILITY_CONTRACT.md, new): every app ships capabilities.yml:
Schema requirements, strict:
  • No bare string slots — every string slot needs validator regex OR enum OR referential_check.
  • Enum preferred over string whenever the domain is finite.
  • Dynamic filter slots declare allowlist inline: { type: enum, values: [...explicit...] }. A { filter: string } shape = schema error.
  • side_effects levels (replaces mutating):
    • none — pure read.
    • read-external — reads from external service (no side effects there).
    • write-app — mutates app’s own DB/state. Requires idempotency_key + reversibility.
    • write-external — mutates external service (Stripe, Neon external, email). Requires reversibility + extra_guard: requires_codeowner_approval.
  • reversibility: idempotent, reversible-cheap, reversible-with-effort, irreversible. Supervisor refuses to execute irreversible automatically — always Red.
CI lint (.github/workflows/capabilities-lint.yml) validates every capabilities.yml against schema on PR. Blocks merge on violations. App-side enforcement (@latimer-woods-tech/admin base class — requires a minor bump):
  • Every registered route is keyed by its capability id.
  • Route handler receives typed, validated slots — framework refuses to invoke with slots that failed schema.
  • Parameterized queries enforced at the framework level for any handler touching Neon.
service-registry.yml remains the index; capabilities.yml per app is the schema.

8. Dreamstate → factory handoff (rate-budgeted)

Math: 11 apps × ~5 reads × 24/day = 1320 calls — under App rate limits. Behavior still matters.
  1. Webhook-primary. repository.pushed, issues.*, pull_request.* webhooks invalidate per-repo cached state in KV. Hourly cron is safety net.
  2. ETag every read. 304 responses don’t count against secondary rate limits.
  3. Bounded concurrency. Max 5 parallel fetches per cycle.
  4. Partial-failure mode. 3/11 apps failed fetch → process the 8 we got, re-queue 3 with backoff. Never produce misclassification from partial reads.
  5. Decoupled supervisor pickup. Supervisor polls kanban for source:dreamstate + supervisor:approved-source. Doesn’t care about sync timing.
  6. Closure feedback. Every closed issue writes fulfilled_by: <merge SHA> back to the spec.

9. Rollout — 4–6 weeks, phased

Track 0 — this weekend (human-led, parallel)

Selfprime funnel fix. Not blocked by the supervisor. ~1–2 days.
  • Ship 2 migration fixes (psn.shared_at, param-count prepared-statement).
  • Resolve the test-mode Stripe key.
  • Walk funnel with a real test card.
  • Confirm one clean conversion before Monday.

Week 1 — control-plane primitives

  • Close MA-0 gaps (labels, GH_PROJECT_TOKEN, path-scoped CODEOWNERS, concurrency groups, PR queue digest).
  • Ship SYN-2 reliability-gate, SYN-3 prod-canary.
  • Install factory-cross-repo GitHub App on Latimer-Woods-Tech.
  • Rotate the GCP service-account key. Add doc-lint (§10).
  • Enable Vertex AI on factory-495015, mint a dedicated supervisor-sa service account (least privilege).
  • Deliverable: kanban → prod runs 8/9 hands-off for Green-tier.

Week 2 — LLM substrate

  • Ship @latimer-woods-tech/llm@0.3.0: AI Gateway mandatory, workload-split routing (Anthropic primary; Gemini via Vertex fallback on long-context + Anthropic-degraded; Groq for verifier fallback only), prompt caching, tier enum, AbortController, 3-attempt backoff. Fix file:../* deps. Budget: 2 days.
  • Migrate HumanDesign to package; delete workers/src/lib/llm.js. Validate on 20 historical syntheses. Budget: 1 day.
  • Ship @latimer-woods-tech/llm-meter@0.1.0 (D1 ledger + per-run $5 hard cap). Wire HumanDesign. Budget: 1 day.
  • Deliverable: every LLM call org-wide metered + gateway-routed. Calibration begins.

Week 3 — capabilities + template library bootstrap

  • Bump @latimer-woods-tech/admin with side_effects-aware route registration + parameterized-query enforcement. Minor bump (0.3.0).
  • Standardize capabilities.yml in 4 apps (selfprime, videoking, xico-city, factory-admin). Schema-lint workflow on PR.
  • Bootstrap template library — reverse-engineer 6–8 starter templates from last ~50 merged PRs. Human review each.
  • Write docs/CAPABILITY_CONTRACT.md and docs/supervisor/TEMPLATE_SPEC.md.
  • Scaffold apps/supervisorSupervisorDO + LockDO, tools, SupervisorMemory dual-write, auth.ts 3-token minter, stats.ts, ledger, migrations. No production runs.
  • Deliverable: supervisor compiles + runs against a fixture issue end-to-end on a test branch.
If first-pass yields fewer than 5 viable templates, add a week. Supervisor with 2 templates is a toy.

Week 4 — first supervised runs on Green

  • Pick Green-tier issue matching existing template (SYN-7 docs, SYN-8 testing skill).
  • Supervisor matches → parameterizes → plan-approval → EXEC → VERIFY → PR → merge.
  • Repeat on 3 more Green issues. No-match issues go to no-match-candidates/.
  • Deliverable: 4 Green closures. Template library +2–4 from real runs.

Weeks 5–6 — Yellow + Dreamstate + budget steady-state

  • Templates with ≥3 clean runs auto-blessed → skip plan-approval on Green.
  • Template test harness on capabilities.yml PRs.
  • Pull 2 weeks of ledger data → Phase 2 budget caps at p95 × 1.5.
  • Ship scripts/dreamstate-to-issues.mjs with webhook + ETag cache. Pilot: videoking VK-11.
  • Supervisor handles Yellow-tier VK with human PR review.
Honest expectation for month one: supervisor classifies a high fraction of novel issues as Red → humans. Coverage feels thin. That’s the point.

What ships in parallel (human-led, not blocked)

  • VK-7 deploy.yml rewrite — week 1 or 2.
  • xico-city first artist payload end-to-end — weeks 2–3.
  • focusbro AdWords acceptance — when it returns.
  • wordis-bond compliance decision — legal.

Schedule risk

LLM@0.3.0 migration is the likeliest slip. Template bootstrap is the second. If either slips, slip Weeks 3–6 by one week each; don’t compress.

10. What NOT to do

  • Don’t hand the supervisor wordis-bond — enforce mechanically, 3 layers:
    • (a) wordis-bond/.github/CODEOWNERS keeps * @adrper79-dot, no supervisor/bot co-owner.
    • (b) docs/service-registry.yml entry: supervisor_access: denied.
    • (c) apps/supervisor/src/tools/denylist.ts boot-time throws SupervisorDenied on any tool call to Latimer-Woods-Tech/wordis-bond or any worker whose name starts with wordis.
  • Don’t put credentials in docs, memory, or plans. Doc-lint workflow (.github/workflows/credential-scrub.yml) fails CI on PRs to documents/factory/** or memory/** introducing strings matching these regexes:
    • AKIA[0-9A-Z]{16} (AWS key id)
    • "private_key":\s*"-----BEGIN (GCP service account)
    • ghp_[A-Za-z0-9]{36,} (GitHub classic PAT)
    • github_pat_[A-Za-z0-9_]{82} (GitHub fine-grained PAT)
    • sk-ant-api0[0-9]-[A-Za-z0-9_-]{93} (Anthropic)
    • sk-proj-[A-Za-z0-9_-]{48,} (OpenAI project)
    • rk_live_[A-Za-z0-9]{24,} / sk_live_[A-Za-z0-9]{24,} (Stripe live)
    • xai-[A-Za-z0-9]{80,} (xAI)
    • gsk_[A-Za-z0-9]{52} (Groq)
    • Any string matching ^[a-f0-9]{40,}$ in a file named anything containing “key”, “token”, or “cred”. Any key that appears in chat or a doc is treated as compromised — rotate, don’t schedule.
  • Don’t cache semantically in AI Gateway. Synthesis output collisions = privacy + quality footgun.
  • Don’t auto-merge Red tier. Ever.
  • Don’t build a new orchestrator framework. CF Durable Object Facets natively.
  • Don’t merge dreamstate → kanban until MA-0 lands. Race condition.
  • Don’t touch production Neon user tables from supervisor. Read replicas only; writes via app routes.
  • Don’t trust side_effects: none as sufficient for arbitrary param shapes. Schema-lint rejects bare string slots without validator/enum/referential_check.
  • Don’t run the supervisor without intent verification on Green. Runtime-only verification is generative-planner-grade safety.
  • Don’t skip the per-run lock renewal on long CI waits. Lock TTL 30 min; CI can exceed. Renew every 10 min during WATCH.

11. What changes for you day-to-day

Before:
  • You hand-write issues, hand-open PRs, hand-verify curls, manually triage Sentry, manually walk Stripe funnels.
After (~4–6 weeks):
  • You hand-write specs and ADRs. Factory supervisor matches templates, opens PRs on Green/Yellow with template-grounded plans, drives canary + rollback, writes receipts on the issue thread.
  • Morning Pushover digest = “supervisor closed 6, your review queue is 2.” You spend time on Red, novel-issue triage (template authoring), and product decisions.
  • Stripe funnel regression: supervisor matches stripe-funnel-debug, reads sub state + checkout_sessions, files an issue with failure class identified, drafts a migration PR. You read the plan, approve, walk away. No match → human Red.
  • AI cost is on the ledger. Per-project, per-actor, per-run. One command kills a runaway.
  • Sibling apps are callable as tools through scoped JWTs. From xico-city’s onboarding you can call selfprime.user.lookup without writing glue.

12. Decisions made in v2.1 + remaining open questions

Decided

Still open — need your call before Week 3


Appendix A — Requirements summary (for execution planning)

Consolidated list for the execution plan in the next conversation. Packages to ship, workflows to create, migrations to run, decisions to make.

New packages

  • @latimer-woods-tech/llm@0.3.0 — AI Gateway mandatory, 3-provider split routing, prompt caching, tier enum, AbortController.
  • @latimer-woods-tech/llm-meter@0.1.0 — D1 ledger, per-run $5 cap, BUDGET_EXCEEDED.
  • @latimer-woods-tech/admin@0.3.0 — side_effects-aware route registration, parameterized-query enforcement.

New apps

  • apps/supervisor — DO + LockDO + tools + planner + memory + auth + stats + ledger.

New workflows

  • .github/workflows/capabilities-lint.yml — validates capabilities.yml against schema.
  • .github/workflows/template-suite.yml — runs template test harness on capabilities.yml + template PRs.
  • .github/workflows/credential-scrub.yml — blocks PRs with leaked credentials.
  • .github/workflows/revert-detector.yml — watches for supervisor PR reverts, updates template_stats.
  • .github/workflows/template-quality-daily.yml — scheduled demotion alert.
  • .github/workflows/dreamstate-sync.yml — webhook + cron; ETag-cached.
  • .github/workflows/memory-reconciliation-monthly.yml — Agent Memory vs D1 drift check.

New CF resources

  • D1: supervisor-db (runs, steps, feedback, locks_audit, template_stats, llm_ledger, cost_runs).
  • KV: supervisor-state (per-app cached state for dreamstate sync).
  • Vectorize: supervisor-similar-runs (embeddings for fallback memory).
  • Workers AI / AI Gateway binding with project supervisor.
  • Vertex AI service account supervisor-sa@factory-495015 for Gemini fallback.

New docs

  • docs/CAPABILITY_CONTRACT.md
  • docs/supervisor/TEMPLATE_SPEC.md
  • docs/supervisor/plans/*.yml (6–8 starter templates)
  • docs/adr/1002-supervisor-architecture.md (point to this doc as canonical)

Preconditions (Week 0 / this weekend)

  • Rotate GCP SA key 76bc15364b7d…
  • Selfprime migration fixes (2) merged + verified
  • Test-mode Stripe rk_test_* key in place
  • One real selfprime conversion verified end-to-end
  • MA-0 gaps scoped as explicit issues on LatWood Operations board