Cloudflare adopts QuePaxa consensus for Meerkat to improve global control-plane availability
Cloudflare announced Meerkat, an experimental distributed consensus service for its global control plane, and that it is powered by a less-common consensus algorithm called QuePaxa. QuePaxa was chosen because it tolerates wide-area, high-variance network conditions better than traditional leader-centric protocols (Raft, Paxos, Multi-Paxos), avoiding long stalls when a leader becomes slow or temporarily unreachable. The algorithm provides a one-round-trip fast path when an optional leader can dominate, but otherwise uses randomized priorities, proposal adoption, and cooperative proposers to make progress without relying on fault detection or slow leader elections. Cloudflare reports Meerkat sustained progress across global deployments and that most decisions (>95%) take the leader fast path; when the fast path fails, decisions continue after a few additional round trips rather than stalling.
Detalhes técnicos
- Roles and goals: QuePaxa separates roles into proposers (submit proposals) and recorders (intermediaries that record proposals). The system agrees an ordered log of events for replicated state (e.g., a key-value store) and guarantees safety via majority intersection.
- Optional leader and fast path: One proposer may be designated leader; the leader always uses the maximum priority and can decide in a single round trip if its proposal reaches a majority of recorders first. Clients may send requests to any proposer; hedging helps exploit the leader fast path without making the leader a single point of failure.
- Randomized priorities and cooperative proposers: Non-leader proposers pick random priorities. Recorders propagate proposals and proposers adopt the highest-priority proposal they learn from a majority of recorders. This design turns destructive interference into constructive convergence rather than mutual blocking.
- Four phases per round (as described by Cloudflare): Phase 0 (Propose & Adopt) — proposers choose priorities, send proposals to recorders and adopt the highest-priority majority proposal; Phase 1 (Spread) — proposers spread adopted proposals and recorders track highest seen; Phase 2 (Check & Decide) — proposers query recorders for the highest-priority proposal seen in Phase 1 and decide if it matches their adopted proposal; Phase 3 (Carry Forward) — proposers that did not decide adopt the best proposal from a majority to prepare the next round.
- Hedging and timeouts: Cloudflare’s implementation hedges client requests to multiple replicas and uses a small, local hedge timeout (they deviate from the paper by having clients wait the hedge timeout), which increases responsiveness to leader changes without relying on slow, network-bound fault detection. Hedging trades extra bandwidth and possible slower convergence for faster recovery from leader unavailability.
- Operational results reported: In Cloudflare’s global tests, Meerkat continued processing requests during leader crashes or disconnections, with most decisions made on the leader fast path and occasional slowdowns (to roughly one-third throughput) when additional rounds were required.
Linha do tempo
-
Cloudflare adopts QuePaxa consensus for Meerkat to improve global control-plane availability
ler a análise desta fonte ocultar
Cloudflare announced Meerkat, an experimental distributed consensus service for its global control plane, and that it is powered by a less-common consensus algorithm called QuePaxa. QuePaxa was chosen because it tolerates wide-area, high-variance network conditions better than traditional leader-centric protocols (Raft, Paxos, Multi-Paxos), avoiding long stalls when a leader becomes slow or temporarily unreachable. The algorithm provides a one-round-trip fast path when an optional leader can dominate, but otherwise uses randomized priorities, proposal adoption, and cooperative proposers to make progress without relying on fault detection or slow leader elections. Cloudflare reports Meerkat sustained progress across global deployments and that most decisions (>95%) take the leader fast path; when the fast path fails, decisions continue after a few additional round trips rather than stalling.
Technical Details
- Roles and goals: QuePaxa separates roles into proposers (submit proposals) and recorders (intermediaries that record proposals). The system agrees an ordered log of events for replicated state (e.g., a key-value store) and guarantees safety via majority intersection.
- Optional leader and fast path: One proposer may be designated leader; the leader always uses the maximum priority and can decide in a single round trip if its proposal reaches a majority of recorders first. Clients may send requests to any proposer; hedging helps exploit the leader fast path without making the leader a single point of failure.
- Randomized priorities and cooperative proposers: Non-leader proposers pick random priorities. Recorders propagate proposals and proposers adopt the highest-priority proposal they learn from a majority of recorders. This design turns destructive interference into constructive convergence rather than mutual blocking.
- Four phases per round (as described by Cloudflare): Phase 0 (Propose & Adopt) — proposers choose priorities, send proposals to recorders and adopt the highest-priority majority proposal; Phase 1 (Spread) — proposers spread adopted proposals and recorders track highest seen; Phase 2 (Check & Decide) — proposers query recorders for the highest-priority proposal seen in Phase 1 and decide if it matches their adopted proposal; Phase 3 (Carry Forward) — proposers that did not decide adopt the best proposal from a majority to prepare the next round.
- Hedging and timeouts: Cloudflare’s implementation hedges client requests to multiple replicas and uses a small, local hedge timeout (they deviate from the paper by having clients wait the hedge timeout), which increases responsiveness to leader changes without relying on slow, network-bound fault detection. Hedging trades extra bandwidth and possible slower convergence for faster recovery from leader unavailability.
- Operational results reported: In Cloudflare’s global tests, Meerkat continued processing requests during leader crashes or disconnections, with most decisions made on the leader fast path and occasional slowdowns (to roughly one-third throughput) when additional rounds were required.