Multi-cluster topology
A Flui installation is one control cluster and any number of workload clusters, operated together. This chapter covers what that arrangement gives you, where the boundaries lie, and what each cluster knows about the others.
The shape of an installation
Every installation has the same skeleton:
- One control cluster. Created first, by the CLI bootstrap, and unique within the installation. It hosts the control plane: dashboard, API, identity, database and observability backend.
- Zero or many workload clusters. Added later, from the dashboard, the CLI or by an agent. They host user applications and the small set of platform components an application cluster needs.
flui env create creates the control cluster;
flui cluster create adds a workload cluster. See
Add a cluster and give it capacity.
One provider per installation, and the exception
Usually every cluster in an installation lives on the same provider as the control cluster. That lets the clusters share one private network and reach each other without going out to the internet. Region and node sizing can still vary per cluster; the constraint is the provider.
Flui checks this before anything is bought. Each provider
carries a static capability flag, crossClusterAllowed, that
decides whether a workload cluster may sit on a different
provider than the control cluster. When it may not, the request
is rejected with the machine-readable code
CROSS_PROVIDER_NOT_ALLOWED. A second flag, vnetRequired,
rejects creation with VNET_REQUIRED when a provider needs a
private network and none was supplied.
The flag that decides is the control cluster’s. Every cloud
provider Flui integrates declares crossClusterAllowed: false.
Your own servers declare crossClusterAllowed: true: a machine
you already own has no API to buy more machines from, so an
installation whose control cluster is one of your servers would
otherwise be limited to the servers you already have.
The flag stands for “there is no private path between two
providers”, and Flui can build one. With
the Flui network
switched on, which is the default, a control cluster on any
provider may create workload clusters on any other provider it
has credentials for, as long as the control cluster has an
address the new cluster can reach. The request is rejected with
CROSS_PROVIDER_NOT_ALLOWED only when the control cluster’s flag
is false and the Flui network cannot bridge the two: it has
been switched off, or the control cluster has no reachable
address. flui env overlay status shows which case you are in,
and flui env overlay enable switches the network back on where
it can work.
A cluster on a different provider cannot join the installation’s private network, because that network lives on the control cluster’s provider. It brings its own private network on its own provider instead, and the Flui network, encrypted and maintained by Flui, runs between the two to carry management traffic, logs and metrics. None of that crosses the public internet in the clear, and the telemetry ports are never opened publicly. The network Flui builds describes that network and the order in which it replaces the public path. What the arrangement costs you is described in How the clusters relate below.
So “one provider per Flui environment” holds only where the Flui network is off and the control cluster sits on a cloud provider. VNets and subnets, abstracted describes the single-provider shape, which is still the common one; The network Flui builds describes what happens when there is no single network to share.
Running a workload somewhere else
There are two ways to run a workload on a different provider.
A cluster in the same installation. With the Flui network on, which is the default, the other provider’s cluster is part of the same installation: one dashboard, one login, one set of grants, one list of applications. This is the better option.
A second installation. When your control cluster is on a cloud provider and the Flui network is off, the other provider gets an installation of its own, with its own control cluster, operated side by side. The CLI supports this with multiple contexts, each a fully isolated installation with its own credentials, and switching between them is one command. See Multi-context.
A workload can leave the cluster it was first put on. An
application with its volumes, a Flui-managed PostgreSQL database,
or both together can be moved to another cluster in one
operation. Flui builds the copy on the destination, keeps it in
step with the source, and swaps the two when you say so. Nothing
on the source changes until the cutover, and you can cancel
before it. The three commands are flui migrate app,
flui migrate db and flui migrate full, described in
Move an application to another cluster.
An application can move this way only if:
- It has a real domain name. An address derived from the source cluster’s IP cannot follow it to another cluster.
- It is self-contained. An application that reads another component’s secrets moves after the components it reads them from.
- Both clusters belong to one installation. Nothing moves between two separate installations.
So the provider that runs an application is not fixed when you create it. You can change it later, as long as the installation has clusters on more than one provider.
Why more than one workload cluster
A single workload cluster is the default and a valid end state. You add more when one of these applies:
- Environment isolation. A cheap staging cluster and a larger production cluster, kept fully separate.
- Project or customer separation. One cluster per tenant, so a noisy workload on one tenant cannot starve another.
- Geographic or regulatory placement. Data that must stay in a region, or workloads that should run close to their users.
- Blast radius. A cluster that ends up in a bad state takes only its own workloads with it.
Add a workload cluster when you need separation, not more capacity. A single application scales on the cluster it already runs on, and giving a cluster more machines is covered in What a cluster may buy for itself.
What every cluster shares
Every cluster in the installation, of either type, is a Flui cluster as defined in Cluster as a concept. They share the same identity, the same ownership labels on every provider resource, the same dashboard views (nodes, network, storage, firewall, monitoring), and the same operations through the same CLI and dashboard.
What only the control cluster carries
The control cluster is the only host of the control plane. Everything that exists exactly once per installation lives on it: the dashboard and API you talk to, the identity provider that authenticates users, the database that holds the installation’s state, and the observability backend that collects metrics and logs from every cluster.
A workload cluster carries only the small set of platform components Flui adds on top of K3s (certificate lifecycle and optional cluster-side authorization), plus the host-level agents that push metrics and logs back to the control cluster.
The control cluster runs Flui itself, and the workload clusters are where your applications are meant to live. You can deploy applications onto the control cluster, which makes sense for development or staging setups where you want the whole installation on a single server. For production, keep application workloads and the control plane on separate clusters.
How the clusters relate
The control plane can reach every workload cluster, but workload clusters cannot reach each other through Flui:
- Control plane to workload cluster. The API holds the credentials to reach every workload cluster it created. Deploys, scaling, lifecycle operations and reads all go this way, over the cluster’s private network. The Kubernetes API of each workload cluster (port 6443) is never exposed publicly: the credentials Flui stores point at the master’s private VNet address, so traffic between the control and workload clusters stays on the VNet and the public firewall keeps 6443 closed to the outside. See Firewalls.
- Workload cluster to control cluster. Every node pushes metrics and logs to the control cluster’s observability backend over the private network.
- Workload cluster to workload cluster. Flui does not connect workload clusters to each other. They share no state and have no cross-cluster pod networking. They do sit on the same private network as the rest of the installation, so they can reach each other at the network level, but Flui does not expose or manage a path between them. Flui does not offer application-to-application communication across clusters today.
When a cluster is on another provider
A workload cluster on a different provider than the control cluster shares no private network with it, so both relationships above travel over the Flui network instead: an encrypted tunnel from the workload cluster to the control cluster, which Flui builds and maintains.
Control plane to workload cluster traffic starts on the public interface and moves onto the tunnel once the tunnel works. Until then the cluster’s API port is open, but only to the control cluster’s own address, and it admits nobody who cannot present the installation’s own certificate, so the open port grants no access by itself. The public opening is withdrawn only after the control cluster has proven it can reach the cluster over the tunnel. This is decided per cluster and only for a workload cluster; a control cluster is never switched to the public address.
Workload cluster to control plane traffic, meaning metrics and logs, travels over the tunnel as well. Flui keeps each node’s push target pointed at the control cluster’s address on the Flui network, and corrects it if it drifts. The ports that receive telemetry are never opened on the control cluster’s public address, so a cross-provider workload cluster’s metrics and logs arrive centrally like any other cluster’s, without a public write endpoint into the control plane.
If the Flui network is switched off, a cross-provider workload cluster has no private path for its telemetry, and its metrics and logs do not arrive centrally.
Failure boundaries
This arrangement determines what fails together:
- A workload cluster failing affects only the workloads on that cluster. The control plane keeps running, the other workload clusters keep serving traffic, and you can drive the recovery from the dashboard.
- The control cluster failing takes the control plane offline (no dashboard, no API, no central log and metric queries), but every workload cluster keeps operating normally. Each cluster runs on its own in steady state; you need the control plane to change the installation, not to keep it running.
Working with more than one cluster
The dashboard and the CLI list every cluster of the installation together. When an operation targets a specific one, you pick which; with a single workload cluster the choice is implicit.
flui cluster list always works: the control cluster is stored
on your own machine and is listed even when the API cannot be
reached, while workload clusters live only in the control
cluster’s database and are read over the API.
Destroying the control cluster is CLI-only. See Creating a cluster from zero.
Where this chapter goes from here
- Nodes and cluster topology: what each cluster looks like from the inside.
- The control cluster: what makes the control cluster the control plane.
- Move an application to another cluster: the operation that makes a cluster choice reversible.