Cloudflare cuts DNS Zone Builder memory 85% and halves build delay with targeted builds
Cloudflare describes re-engineering its DNS propagation pipeline. The Zone Builder now receives about 700 DNS record changes per second, up from about 250 in 2022, and some zones exceed 20 million records. A new targeted build planner and a unified scheduler replace most full-zone rebuilds triggered by zone setting changes. Peak memory fell 85% (79 GiB to 12 GiB), and the median build completion delay halved from about 7.7 s to about 3.3 s. The largest p99 spike dropped from 14.6 minutes to 1.21 minutes. This is engineering and product news with no vulnerability or threat activity.
Detalhes técnicos
- The planner picks one of five build types: resources only, resources and metadata, apex and metadata, purge, or full zone. Targeted output must be byte-for-byte identical to a full build, and any uncertainty escalates to a full build.
- Over 24 hours, 86.81% of builds were resources only, 2.54% resources and metadata, 2.30% apex and metadata, and 1.30% purges. Full builds were 7.04%, and full builds overall fell by about 40%.
- Above a fixed record-count threshold, very large zones never escalate to a full build.
- Planning happens twice: from the event, and again against a fresh publication snapshot read from database replicas before execution. The shared same-name metadata key is rebuilt from the current candidate set.
- The unified scheduler merges work per zone and tracks Kafka messages separately, using soft committing so no change is lost on restart. It enforces a per-zone rebuild interval that varies by plan.
- Validation used equivalence tests and a mirror Zone Builder writing to a separate Quicksilver environment. A comparer sampled 20% of zones, and results were tracked in the DNS Mirror Dashboard, which runs on Workers, Durable Objects and Access.
- Rollout was by hash of the zone name, free zones first and then paid. Shorter rebuild intervals and a smaller DNS Kafka Aggregator hold time then produced the delay reduction.
Linha do tempo
-
Cloudflare cuts DNS Zone Builder memory 85% and halves build delay with targeted builds
ler a análise desta fonte ocultar
Cloudflare describes re-engineering its DNS propagation pipeline. The Zone Builder now receives about 700 DNS record changes per second, up from about 250 in 2022, and some zones exceed 20 million records. A new targeted build planner and a unified scheduler replace most full-zone rebuilds triggered by zone setting changes. Peak memory fell 85% (79 GiB to 12 GiB), and the median build completion delay halved from about 7.7 s to about 3.3 s. The largest p99 spike dropped from 14.6 minutes to 1.21 minutes. This is engineering and product news with no vulnerability or threat activity.
Technical Details
- The planner picks one of five build types: resources only, resources and metadata, apex and metadata, purge, or full zone. Targeted output must be byte-for-byte identical to a full build, and any uncertainty escalates to a full build.
- Over 24 hours, 86.81% of builds were resources only, 2.54% resources and metadata, 2.30% apex and metadata, and 1.30% purges. Full builds were 7.04%, and full builds overall fell by about 40%.
- Above a fixed record-count threshold, very large zones never escalate to a full build.
- Planning happens twice: from the event, and again against a fresh publication snapshot read from database replicas before execution. The shared same-name metadata key is rebuilt from the current candidate set.
- The unified scheduler merges work per zone and tracks Kafka messages separately, using soft committing so no change is lost on restart. It enforces a per-zone rebuild interval that varies by plan.
- Validation used equivalence tests and a mirror Zone Builder writing to a separate Quicksilver environment. A comparer sampled 20% of zones, and results were tracked in the DNS Mirror Dashboard, which runs on Workers, Durable Objects and Access.
- Rollout was by hash of the zone name, free zones first and then paid. Shorter rebuild intervals and a smaller DNS Kafka Aggregator hold time then produced the delay reduction.