Cilium adoption prerequisite (blocks the DOM-2 CNI swap; FQDN-egress renderer already merged default-off) #60
Labels
No labels
bug
discussion
duplicate
enhancement
goal
help wanted
horizon:backlog
horizon:governance
horizon:mvp
invalid
operator-decision
question
roadmap
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
open-platform/mitosis#60
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
DOM-2 delivered the per-app CiliumNetworkPolicy toFQDNs egress renderer (mcp, default-off PLAT_FQDN_EGRESS, proven live on Cilium: api.github.com ALLOW / gitlab+pypi BLOCK; std NetworkPolicy still enforces under Cilium). The germination CNI SWAP to Cilium is DEFERRED — two reproducible blockers on k3s/k3d: (A) kube-proxy present → Cilium does not bind the registry-mirror hostPort 127.0.0.1:31100 (only under kube-proxy-replacement) → in-cluster image pulls fail; (B) kpr=true → Flux/metrics-server CrashLoop with x509 apiserver cert-trust failure (known k3s+Cilium-kpr incompatibility; bpf.hostLegacyRouting did not fix). PREREQUISITE (pick one) before adopting Cilium: (a) make the registry mirror not depend on hostPort (NodePort/hostNetwork socat) so kube-proxy can stay, OR (b) resolve the k3s+Cilium-kpr apiserver cert-trust (dedicated apiserver LB SAN/cert). Then flip PLAT_FQDN_EGRESS=1 and the egress feature lights up unchanged. Evidence + the bin/up swap were in the closed PR #59.
Prerequisite (a) implemented, pending live validation: plat/gitops#12 (plat/gitops#12) makes the registry mirror independent of the CNI-managed hostPort, so kube-proxy can stay for the Cilium swap.
Design in one line: the mirror endpoint splits into a
hostNetworkCaddy DaemonSet that does a plain loopback-onlybind 127.0.0.1:31100in the host netns (no CNI portmap, no service-proxy datapath — binds identically under flannel+kube-proxy, Cilium+kube-proxy, or Cilium+kpr, directly clearing blocker A) plus a pod-networkregistry-realm-rewriteDeployment that keeps the Www-Authenticate realm rewrite and the pod-sourced hop to forgejo-http, soallow-ingress-forgejois untouched. The creation-timeregistries.yamladdress127.0.0.1:31100keeps being served as-is — no recipe change, works for already-created platforms.Status: static validation only (kustomize render,
caddy validate/adapt, apiserver--dry-run=serverincl. Kyverno admission). Deliberately NOT rolled on the shared live cluster — it is image-pull-critical. Remaining before flipping the swap: a disposable germination to prove end-to-end pull-through, the rollover port handoff (hostPort pod -> hostNetwork pod), and then a Cilium (kpr off) germination to confirm pulls and light upPLAT_FQDN_EGRESS=1.Prerequisite (a) DELIVERED and on prod + in the seed: plat/gitops#12 (hostNetwork loopback forwarder + pod-network realm rewriter + NET_BIND_SERVICE capfix) merged after live validation on val076 (fresh admission Running, 127.0.0.1:31100 bound in node netns, crictl pull green). The mirror no longer depends on CNI hostPort plumbing — the Cilium swap can proceed against blocker (A); blocker (B) kpr apiserver cert-trust remains open. Issue stays open for the swap itself.