GATE: assert exactly ONE system webhook after a dispatcher restart + reviewer engages without the 10-min fallback

plat commented

2026-07-02 02:20:49 +00:00

Owner

Two failure modes found while validating today's wb-gate germination (fresh plat.local daughter) both SELF-HEAL: the app still ships and serves 200, so a naive E2E passes while the platform is degraded. Exactly the class of regression the acceptance gate (#46) / CI-1 harness (#21) must assert explicitly, or agents 0.7.6 (#66) fixes them once and nothing keeps them fixed.

(1) exactly ONE system webhook survives a dispatcher restart (#63). Proven live: one kubectl rollout restart deploy/dispatcher → admin hooks id 1 AND id 3, same dispatcher URL, both active → every Forgejo event delivers twice (doubled runs/noise; the RBAC-5 HMAC re-seal can never patch either). Fix in flight: plat/agents#10 (persist the created hook id; GET-by-id on boot). Gate assertion: restart the dispatcher twice, then assert exactly one admin hook targets the dispatcher URL — probe GET /api/v1/admin/hooks/{id} by id, since the LIST endpoint demonstrably returns [] while hooks exist — AND one PR event produces exactly one dispatched run.

(2) reviewer engages without the ~10-min fallback (#57, root-caused today: the liveness probe resolved the preview via loopback from inside the dispatcher pod, so it never confirmed a preview that was already serving 200 with the correct /version sha). Fix in flight: plat/agents#11 (in-cluster DNS first). Gate assertion: on a PR build, the reviewer engages within 3 min of the preview serving 200 with /version == PR head sha, and dispatcher logs contain NO "preview liveness unconfirmed" fallback line.

Fix direction: add both as assertions in the CI-1 validate-coldstart set (#21) and the #46 unattended gate run; validate first on the #66 release-train germination. Feeds #46 — does not block the MVP path.

Two failure modes found while validating today's wb-gate germination (fresh plat.local daughter) both SELF-HEAL: the app still ships and serves 200, so a naive E2E passes while the platform is degraded. Exactly the class of regression the acceptance gate (#46) / CI-1 harness (#21) must assert explicitly, or agents 0.7.6 (#66) fixes them once and nothing keeps them fixed. **(1) exactly ONE system webhook survives a dispatcher restart** (#63). Proven live: one `kubectl rollout restart deploy/dispatcher` → admin hooks id 1 AND id 3, same dispatcher URL, both active → every Forgejo event delivers twice (doubled runs/noise; the RBAC-5 HMAC re-seal can never patch either). Fix in flight: plat/agents#10 (persist the created hook id; GET-by-id on boot). Gate assertion: restart the dispatcher twice, then assert exactly one admin hook targets the dispatcher URL — probe `GET /api/v1/admin/hooks/{id}` by id, since the LIST endpoint demonstrably returns `[]` while hooks exist — AND one PR event produces exactly one dispatched run. **(2) reviewer engages without the ~10-min fallback** (#57, root-caused today: the liveness probe resolved the preview via loopback from inside the dispatcher pod, so it never confirmed a preview that was already serving 200 with the correct /version sha). Fix in flight: plat/agents#11 (in-cluster DNS first). Gate assertion: on a PR build, the reviewer engages within 3 min of the preview serving 200 with `/version` == PR head sha, and dispatcher logs contain NO "preview liveness unconfirmed" fallback line. Fix direction: add both as assertions in the CI-1 `validate-coldstart` set (#21) and the #46 unattended gate run; validate first on the #66 release-train germination. Feeds #46 — does not block the MVP path.

plat referenced this issue

2026-07-02 02:21:34 +00:00

[ROADMAP] First Client Deploy → Governance — tracker #47

plat referenced this issue

2026-07-02 02:21:53 +00:00

/goal /ultracode-workflows [GATE] validated grandchild acceptance gate — secrets+domains+rbac+policy on a VPS depth-2/3 platform #46

plat referenced this issue

2026-07-02 04:12:26 +00:00

CI-1: automatic cold-start validation — shared gate script + coldstart workflow + vxrail runner (#21) #71

plat referenced this issue

2026-07-02 13:07:47 +00:00

release train: agents 0.7.6 (webhook idempotency + in-cluster preview probe) → gitops pin bump → genesis reseed #66

plat referenced this issue

2026-07-02 14:19:23 +00:00

[ROADMAP] First Client Deploy → Governance — tracker #47

plat referenced this issue from a commit

2026-07-02 15:44:37 +00:00

ci: parallel per-ref gates + nightly app-E2E phase (#21, #67)

plat referenced this issue from a commit

2026-07-02 16:49:43 +00:00

ci: parallel per-ref gates + nightly app-E2E phase (#21, #67)

plat referenced this issue

2026-07-02 16:50:37 +00:00

ci: parallel per-ref gates + nightly app-E2E variant (#67 second half, #21) #75

plat commented

2026-07-02 16:51:09 +00:00

Author

Owner

Second half shipped: nightly app-E2E in the cold-start gate (plat/mitosis#75)

Covered as of that PR (all in scripts/coldstart-assert.sh, the ONE gate both CI and humans run):

(1) webhook singleton: already covered — A2 (exactly one admin hook, probed BY ID) + A3 (restart deploy/agents, STILL exactly one; version-aware log check).
(2) reviewer engages without the ~10-min fallback: now asserted nightly by --e2e on the CI daughter, with a REAL canary app generated from _app-template and shipped unattended:
- E4 prime PR auto-merges (builder → preview → reviewer verdict → merge)
- E5 prod serves 200 in-cluster
- E6 dispatcher log has zero preview liveness unconfirmed lines
- E7 builder-finish → reviewer-queued < 120 s (the fallback path takes ~10 min, so it cannot pass this)
- E8 the app's prod namespace carries plat.sh/app + generated ResourceQuota + NetworkPolicies

Runs on the nightly cron and on any workflow_dispatch with e2e=true, AFTER the standard gate, hard-capped at 35 min. Humans: scripts/coldstart-assert.sh --e2e <ctx> <domain>.

Default-safe: the phase is SKIP until you seal a credential. No secret exists today (the staged host token is flagged compromised-pending-rotation, so it was deliberately NOT wired in). e2e-flagged runs currently print one loud E2E SKIP line and exit 0 — the standard gate still gates.

To turn it on (one command, with a FRESH Claude Code OAuth token):

curl -X PUT -H "Authorization: token $PAT" -H 'Content-Type: application/json' \
  -d '{"data":"<fresh-claude-code-oauth-token>"}' \
  https://git.open-platform.sh/api/v1/repos/plat/mitosis/actions/secrets/E2E_CLAUDE_TOKEN

What happens once sealed: every nightly run (and every e2e=true dispatch) germinates the CI daughter WITH the credential sealed into plat-agents-secrets, then runs E1–E8 above. Rotation = same PUT with a new token; DELETE .../actions/secrets/E2E_CLAUDE_TOKEN returns to loud-SKIP.

The full --e2e path was validated manually on a scratch germination (evidence in the PR body) — marked manual-validation there since CI itself can't run it until the secret is sealed.

## Second half shipped: nightly app-E2E in the cold-start gate (https://git.open-platform.sh/plat/mitosis/pulls/75) **Covered as of that PR** (all in `scripts/coldstart-assert.sh`, the ONE gate both CI and humans run): - **(1) webhook singleton**: already covered — A2 (exactly one admin hook, probed BY ID) + A3 (restart `deploy/agents`, STILL exactly one; version-aware log check). - **(2) reviewer engages without the ~10-min fallback**: now asserted nightly by `--e2e` on the CI daughter, with a REAL canary app generated from `_app-template` and shipped unattended: - E4 prime PR **auto-merges** (builder → preview → reviewer verdict → merge) - E5 prod serves **200 in-cluster** - E6 dispatcher log has **zero** `preview liveness unconfirmed` lines - E7 **builder-finish → reviewer-queued < 120 s** (the fallback path takes ~10 min, so it cannot pass this) - E8 the app's prod namespace carries `plat.sh/app` + generated ResourceQuota + NetworkPolicies Runs on the nightly cron and on any `workflow_dispatch` with `e2e=true`, AFTER the standard gate, hard-capped at 35 min. Humans: `scripts/coldstart-assert.sh --e2e <ctx> <domain>`. **Default-safe: the phase is SKIP until you seal a credential.** No secret exists today (the staged host token is flagged compromised-pending-rotation, so it was deliberately NOT wired in). e2e-flagged runs currently print one loud `E2E SKIP` line and exit 0 — the standard gate still gates. **To turn it on** (one command, with a FRESH Claude Code OAuth token): ```sh curl -X PUT -H "Authorization: token $PAT" -H 'Content-Type: application/json' \ -d '{"data":"<fresh-claude-code-oauth-token>"}' \ https://git.open-platform.sh/api/v1/repos/plat/mitosis/actions/secrets/E2E_CLAUDE_TOKEN ``` What happens once sealed: every nightly run (and every `e2e=true` dispatch) germinates the CI daughter WITH the credential sealed into `plat-agents-secrets`, then runs E1–E8 above. Rotation = same PUT with a new token; `DELETE .../actions/secrets/E2E_CLAUDE_TOKEN` returns to loud-SKIP. The full `--e2e` path was validated manually on a scratch germination (evidence in the PR body) — marked manual-validation there since CI itself can't run it until the secret is sealed.

plat commented

2026-07-02 17:10:47 +00:00

Author

Owner

Both gate assertions this issue asked for are now LIVE in CI (PR #75 merged, its own gate run 14 green): (1) exactly-one-webhook-after-dispatcher-restart = A3, running on every gate since #71; (2) reviewer-engages-without-the-10-min-fallback = E6/E7 in the --e2e mode, proven manually end-to-end (unattended canary shipped in 793s, builder→reviewer gap 34s, zero fallback lines). The nightly E2E activates the moment a FRESH Claude credential is sealed: curl -X PUT -H "Authorization: token <PAT>" -d '{"data":"<token>"}' https://git.open-platform.sh/api/v1/repos/plat/mitosis/actions/secrets/E2E_CLAUDE_TOKEN — until then it SKIPs loudly and exits 0 (the compromised staged token was deliberately NOT wired). Closing: the assertions exist, run, and are proven; activation is a one-command operator step.

Both gate assertions this issue asked for are now LIVE in CI (PR #75 merged, its own gate run 14 green): (1) exactly-one-webhook-after-dispatcher-restart = A3, running on every gate since #71; (2) reviewer-engages-without-the-10-min-fallback = E6/E7 in the --e2e mode, proven manually end-to-end (unattended canary shipped in 793s, builder→reviewer gap 34s, zero fallback lines). The nightly E2E activates the moment a FRESH Claude credential is sealed: `curl -X PUT -H "Authorization: token <PAT>" -d '{"data":"<token>"}' https://git.open-platform.sh/api/v1/repos/plat/mitosis/actions/secrets/E2E_CLAUDE_TOKEN` — until then it SKIPs loudly and exits 0 (the compromised staged token was deliberately NOT wired). Closing: the assertions exist, run, and are proven; activation is a one-command operator step.

plat closed this issue

2026-07-02 17:10:47 +00:00

plat referenced this issue from a commit

2026-07-02 17:10:48 +00:00

Merge pull request 'ci: parallel per-ref gates + nightly app-E2E variant (#67 second half, #21)' (#75) from feat/gate-parallel-nightly into main

Rows
Columns

GATE: assert exactly ONE system webhook after a dispatcher restart + reviewer engages without the 10-min fallback #67

Second half shipped: nightly app-E2E in the cold-start gate (plat/mitosis#75)