deploy/do: firewall races droplet tag creation, fails first apply on a fresh account #232

New issue

Closed

opened 2026-08-26 19:37:37 +00:00 by benjivers · 0 comments

benjivers commented

2026-08-26 19:37:37 +00:00

Collaborator

What happens

On a fresh DigitalOcean account (no pre-existing mitosis tag), tofu/terraform apply in deploy/do fails the first time with:

Error: Error creating firewall: POST https://api.digitalocean.com/v2/firewalls: 422 ... tag mitosis does not exist

digitalocean_firewall.mitosis attaches to tags = ["mitosis"] / source_tags = ["mitosis"], but the mitosis tag only comes into existence implicitly when digitalocean_droplet.leader (tagged ["mitosis"]) is created. There is no dependency between the firewall and the droplet, so they create in parallel and the firewall loses the race — the droplet takes ~30s to create while the firewall POSTs immediately.

Reproduction

Fresh DO account/token, no mitosis tag present.
cd deploy/do && tofu init && tofu apply -var "ssh_public_key=..."
First apply: firewall errors as above (droplet + ssh key succeed).
Re-running apply succeeds because the droplet (hence the tag) now exists.

Observed today on a single-droplet (ha=false) s-4vcpu-8gb / nyc3 deploy from origin/main (b0ea45d). Took two applies to converge.

Suggested fix

Declare the tag explicitly and depend on it, so it exists before either resource that references it:

resource "digitalocean_tag" "mitosis" {
  name = "mitosis"
}

Then reference digitalocean_tag.mitosis.name in the droplet's tags and the firewall's tags/source_tags (or add depends_on = [digitalocean_droplet.leader] to the firewall as a minimal fix). An explicit digitalocean_tag resource is the cleaner, idempotent option and also makes destroy clean up the tag.

Impact

Cosmetic-but-confusing for first-time sovereign deploys: the apply looks like it failed, and the droplet is briefly live with no cloud firewall until the second apply attaches it. Worth fixing so the documented one-command deploy converges in a single apply.

## What happens On a **fresh DigitalOcean account** (no pre-existing `mitosis` tag), `tofu/terraform apply` in `deploy/do` fails the first time with: ``` Error: Error creating firewall: POST https://api.digitalocean.com/v2/firewalls: 422 ... tag mitosis does not exist ``` `digitalocean_firewall.mitosis` attaches to `tags = ["mitosis"]` / `source_tags = ["mitosis"]`, but the `mitosis` tag only comes into existence implicitly when `digitalocean_droplet.leader` (tagged `["mitosis"]`) is created. There is **no dependency** between the firewall and the droplet, so they create in parallel and the firewall loses the race — the droplet takes ~30s to create while the firewall POSTs immediately. ## Reproduction 1. Fresh DO account/token, no `mitosis` tag present. 2. `cd deploy/do && tofu init && tofu apply -var "ssh_public_key=..."` 3. First apply: firewall errors as above (droplet + ssh key succeed). 4. Re-running `apply` succeeds because the droplet (hence the tag) now exists. Observed today on a single-droplet (`ha=false`) `s-4vcpu-8gb` / `nyc3` deploy from `origin/main` (b0ea45d). Took two applies to converge. ## Suggested fix Declare the tag explicitly and depend on it, so it exists before either resource that references it: ```hcl resource "digitalocean_tag" "mitosis" { name = "mitosis" } ``` Then reference `digitalocean_tag.mitosis.name` in the droplet's `tags` and the firewall's `tags`/`source_tags` (or add `depends_on = [digitalocean_droplet.leader]` to the firewall as a minimal fix). An explicit `digitalocean_tag` resource is the cleaner, idempotent option and also makes `destroy` clean up the tag. ## Impact Cosmetic-but-confusing for first-time sovereign deploys: the apply looks like it failed, and the droplet is briefly live with **no cloud firewall** until the second apply attaches it. Worth fixing so the documented one-command deploy converges in a single `apply`.

benjivers added the

bug

label

2026-08-26 19:37:37 +00:00

benjivers referenced this issue

2026-08-27 19:17:48 +00:00

germination: seeded _app-template is private + owned by the plat user — operators expect a discoverable/public app template #236