I run a small home lab behind a dynamic residential IP, so a handful of subdomains — the ones I use for remote access to stuff at home — depend on a dynamic DNS updater that checks my public IP every few minutes and rewrites an A record when it changes. Nothing exotic. It’s worked fine for years.
Separately, I manage all my actual DNS zones declaratively — every record for every domain I own lives in one config file, applied to the DNS provider by a small tool whenever I run it. For a long time this ran manually, whenever I remembered to. Eventually I moved it to a daily scheduled job, because “whenever I remember” is not a real maintenance strategy.
The day after that schedule went live, my remote-access subdomain stopped resolving to my actual home IP. It kept getting silently reset back to whatever address had been hardcoded in the declarative config file — sometimes hours old, sometimes from a previous ISP lease entirely. The dynamic updater was doing its job correctly, every few minutes, exactly as designed. And every night, the declarative tool would come along and calmly stomp it back to the stale value, because as far as that tool was concerned, the config file was the desired state, full stop.
Neither tool was broken
This is the frustrating part: both systems worked exactly as designed, in isolation. The dynamic updater’s whole job is “the real IP changed, fix the record.” The declarative tool’s whole job is “make the live record match what’s in the file.” Neither one is aware the other exists. Neither one is wrong about its own responsibility. The bug only exists at the intersection — two independent, correct systems both convinced they own the same field.
And critically: this had been true the entire time the declarative tool only ran manually. The conflict was always latent, but it never actually collided with the dynamic updater, because a human running the declarative push once in a while just happened to not do it at the exact moment the IP had drifted. Turning that same tool into a scheduled job didn’t introduce a new bug — it took a pre-existing, dormant conflict and gave it a recurring trigger. The tool didn’t get worse. The bug got a cron schedule.
The fix is an explicit exclusion, not a smarter merge
There’s no clever reconciliation that solves this automatically, because there’s no way for the declarative tool to know, just from looking at its own config file, that some other system is also a legitimate writer for that one record. The fix has to be explicit: tell the declarative tool to leave that specific field alone entirely — an ignore rule, scoped to exactly the record the dynamic updater owns — rather than trying to make the declarative side “smarter” about detecting drift. The declarative tool doesn’t need to understand the dynamic updater exists. It just needs to be told, once, “not this one.”
The generalizable rule
Any time a previously-manual, occasionally-run declarative system (an IaC apply, a config sync, a GitOps push) gets turned into something that runs on a schedule or on every commit, that’s the moment to go audit every field it manages and ask: does anything else — an autoscaler, an operator, a dynamic updater, another pipeline — also write to any of these? If yes, that field needs an explicit exclusion rule before the schedule goes live, not after the first mysterious “why did this revert overnight” ticket.
The uncomfortable part is that this conflict is invisible by construction until the moment it isn’t. There’s no log line warning you two writers exist for the same field — you only find out when their timing finally overlaps. Auditing for this ahead of time is cheap. Debugging it after the fact, when everything downstream just looks like “a value keeps randomly reverting for no reason,” is not.