Two schedulers are not high availability
When a cloud control plane places work onto Proxmox capacity, allowing both layers to schedule the same guest creates conflicting authority—not resilience.
Two systems capable of restarting the same workload can look like redundancy.
If both systems are allowed to decide where that workload belongs, they are also two independent writers to the same operational truth.
RS Platform’s home compute design uses two Proxmox hosts, but it does not ask Proxmox HA to schedule the platform’s tasks. Proxmox supplies capacity. A cloud-hosted console owns placement. A restricted worker on each host executes the allocation it receives.
That is not a missing high-availability feature. It is an authority boundary.
Capacity and placement are different responsibilities
The hypervisor knows facts the scheduler needs:
- which templates and storage are available;
- which guests exist and what state they report;
- how much CPU, memory, disk, and device capacity remains; and
- whether a local operation succeeded.
It does not automatically know the platform’s intent. It cannot infer which task revision was approved, whether another host already holds a valid allocation, whether an exclusive writer has been fenced, or whether a generation is stale.
Those decisions belong to rs-console. Stable task definitions live in Git.
The console combines those definitions with live inventory, health evidence,
capacity reservations, drains, leases, and fencing state. It persists an
allocation before asking a host to act.
The worker receives a narrower instruction: run revision X, generation Y, on this host, with these resources, until this lease expires.
A worker should prove execution, not invent intent
One rs-worker instance corresponds to one physical Proxmox host. It runs in a
dedicated management container with a token restricted to an approved pool,
templates, storage, and networks.
It may:
- validate that an allocation is signed and intended for its host;
- reconcile the managed pool toward that allocation;
- report capacity, observed task state, and health; and
- acknowledge the exact allocation digest it applied.
It may not choose a host, move a task, start an unapproved template, clear a fence, modify the hypervisor’s management boundary, or act on an old generation.
This keeps a local compromise or implementation bug from silently promoting an executor into a second control plane.
Why dual scheduling fails strangely
Imagine that rs-console allocates a service to host A. During a WAN failure,
Proxmox decides the guest should restart on host B. The console still holds an
allocation for A, while each local layer has evidence that its own action was
reasonable.
For a stateless service, the result may be duplicate work or an unexpected network identity. For an exclusive writer, it may be concurrent access to state. Adding health checks does not resolve which scheduler is authoritative.
High availability cannot mean “whichever controller acts first is right.” The system needs one durable decision, one generation, and evidence that the previous authority has ended before a conflicting placement begins.
Proxmox clustering or a QDevice may later earn a role in host configuration or
shared-storage locking. That does not require giving Proxmox and rs-console
equal authority over platform placement.
Continuity during cloud loss is not local rescheduling
A sole cloud scheduler does not require healthy local services to stop when the WAN disappears.
Workers retain their last-known-good allocations. They continue local health monitoring and ordinary restart behaviour inside the assigned guest boundary. They refuse new or stale instructions and avoid destructive cleanup until they can reconcile again.
The edge can therefore continue executing a valid decision without making a new placement decision.
This is the same distinction between continuity and availability that appears in replaceable gateways and cached desired state. A dependency can be unavailable without invalidating every decision it previously made.
Use the lightest runtime that satisfies the boundary
The design defaults to unprivileged LXC for trusted native Linux services with a clean packaging contract. A VM is used when the workload needs stronger isolation, a different kernel, sensitive device access, complex mount behaviour, or Docker and OCI images as its native packaging contract.
This is not a contest between containers and virtual machines. The scheduler places a declared runtime class; the worker realizes it using the boundary the workload actually needs.
Nesting an arbitrary Docker stack inside LXC merely to make every task look the same would conceal different kernel and isolation assumptions behind a common label.
Fencing should follow the state model
Stateless and shared-read-only tasks can move after lease expiry, generation advance, and host-loss policy. They do not need the full storage-fencing saga.
Exclusive writers do. Before a replacement starts, the previous writer must be fenced and storage ownership must be unambiguous. Host silence alone is not proof that the old writer stopped.
Narrowing fencing to the tasks that can corrupt state makes the mechanism easier to reason about without weakening the boundary where it matters.
Start with one scheduler and prove it
The accepted architecture deliberately excludes live migration, Proxmox HA placement, GPU scheduling, and a generic EC2 capacity provider from its first slice. The initial target is stateless task scheduling across two independently manageable Proxmox hosts.
This is an accepted design decision, not an operated-system claim. The scheduler still needs implementation and failure testing.
That scope is useful. Reliability begins with being able to name the component allowed to decide. A second scheduler does not strengthen that answer. It makes the answer conditional on the failure.