Two Control Planes Are Not a Luxury
The most important question in a mesh upgrade is not how you move forward. It is how you get back when the first wave creaks.
- Date
- Topic
- Lifecycle
- Sources
- 2
- Reading time
- 2 min
- Version
- Translation
The in-place upgrade answers it like this: one control plane gets replaced, all workloads hang on the new version immediately. If something jams, the way back is called restore. That is less a plan than a prayer with a backup.
Figure as text
The canary upgrade with revisions: old and new control plane run in parallel, namespace waves switch to the new revision via the istio.io/rev label, the switch takes effect only at the sidecar restart, wave 3 (customer-facing) still hangs on the old one; a dashed arrow marks the way back (label back, restart, done). Below, istioctl proxy-status as proof of which proxy hangs on which control plane, and the red in-place contrast: everything hangs on the new version immediately, the way back is called restore.
The canary pattern with revisions answers it differently. The new control plane revision is installed alongside the old one, both run at the same time. Which workload belongs to which revision is decided by a namespace label, and the switch happens only at the next sidecar restart. Nothing tips over at once.
The ordering turns this into an orderly process: first a non-critical namespace, then wave by wave towards customer-facing services. After every wave, verify instead of hope. istioctl proxy-status shows at any time which proxy hangs on which control plane, the same command that opened the diagnostic path in report #8. And the old revision stays up until the last wave is through. The way back: label back, restart, done.
Yes, two control planes cost double resources for a while. Compared to one night in which the entire mesh hangs on a freshly replaced control plane, that is simply the price of a way back that deserves the name.
For decision makers, more than a technical nuance gets decided here: whether the most important intervention in the network comes with an orderly retreat or only the flight forward. And nobody pays down the upgrade debt from report #19 as long as every upgrade is a test of courage.
So here is the question for the room: how do control plane upgrades run for you today, in-place, in waves, or is the ticket still waiting? Honest answers explicitly welcome.
