Datacenters

A datacenter is an app you own, on hardware Xuda runs.

The platform is not renting someone else's control panel any more. proxmox.farm_enabled is true in the shipped config, 12 app types are farm eligible, and 'datacenter' is first in that list. Deploying one is a single declared method, deploy_datacenter, and its declaration decides who may call it: clearance 'admin', min_plan 'pro', private: true. That combination means a dashboard session on a Pro plan or above, and it means no API key of any mode reaches it, because private is the wall. It is one of only 19 methods out of the 1010 in the repo config that carry a plan floor at all. You pick from 14 metro codes and 16 server sizes, from shared_hosting_1 up to large_hosting_32, and what comes back is an app document with app_type 'datacenter' and a freshly generated 20 character CouchDB admin password stored on the doc as couchdb_admin_password. On the farm that VM is a full clone of its own golden Proxmox template, vmid 9101; every other farm eligible non-VPS type clones the shared deployment image instead. One wrinkle to say out loud before the rest of the page: the catalogue description for deploy_datacenter still says it creates a DigitalOcean droplet, and the function it calls is still named create_digitalocean_droplet, even though that function hands the create to Proxmox before DO is ever considered. The name is older than the hardware.

clearance admin, min_plan pro, private
14 metros, 16 server sizes
Golden template vmid 9101
A datacenter is an app you own, on hardware Xuda runs.
The rollout console

What a publish actually prints.

A publish is a rollout, not a file copy. It selects the xuda_servers docs with docType 'regional_server', application_server true and stat 1, then walks them one at a time, writing stat 2 publishing, 3 done or 4 failed with a stat_ts on each. Every region is drained before it is touched and refuses to drain if it is the last healthy one. The destructive part runs detached under setsid from /tmp on the target, so it finishes even if the orchestrator, the broker or the SSH connection dies mid-way, and the rollout then polls a status marker for up to 25 minutes while tailing the remote install log by byte offset. These are its lines.

[publish master] mode=soft
soft unzips over the existing install so node_modules survives. publish_mode hard wipes /var/xuda first
[publish master] drained from rotation; waiting 40s for CF to shift traffic
the drain is one touch of /run/xuda-draining. /health then answers 503 and the edge drops the region on its ~30s refresh
[publish master] copying server.zip... <sent>/<total> MB
a heartbeat every 15 seconds, read with stat -c %s on the target while the copy runs
[publish master] server.zip copied (<total> MB)
[publish master] <every line of the remote install log, tailed by byte offset, no repeats>
[publish master] undrained; rejoins rotation within ~30s
then stat 3 and a stat_ts on the doc, and the next region starts. Never two at once
[publish eu] left DRAINED (out of rotation); fix + redeploy, auto-rejoins in ~15m
the failure path: stat 4, publish_error truncated to 500 characters, and the rollout deliberately keeps going
Two lines carry the whole safety model. The drain line is why a reinstall is invisible to users. The left-DRAINED line is why a broken region stays invisible too, and note what puts it back: not a verdict that it was fixed, but the drain flag ageing out after 15 minutes on a timer.
47

pm2 processes a region boots from one file

4

Publish states a region doc moves through

20%

Free capacity a node needs beyond your ask

25 min

Per box cap before a rollout calls it timed out

Same hardware, two tenants

Xuda runs on this farm the same way your datacenter does.

A region of Xuda and a customer datacenter are two guests on the same Proxmox hosts, and the honest way to describe the product is to put them side by side rather than to describe one and imply the other. Read the four rows across: what arrives on the box, what runs once it boots, who rebuilds it, and how it leaves rotation. Three of the four rows differ completely, and the fourth is where the difference stops being a detail.

What the drawing leaves out on purpose: a second copy of either lane. The rollout is sequential across regions and a failed region simply stays out, and a datacenter is one VM, on one node, in one metro. Replication into or out of it is a CouchDB _replicator document you configure, and configuring it is not the same as having somewhere to fail over to.
Replication and sync

Replication is four fields and two documents, and none of it is failover.

configure_datacenter_replication takes four things: replicate_type, one-time or continuous; replicate_from, master or datacenter; replicate_direction, one_way or two_way; and apply, which when false saves the setup without running it, so re-opening a finished seed does not copy the data twice. Everything downstream is ordinary CouchDB. The platform adds plumbing: it creates the _replicator database if the endpoint has none, writes one document per direction, tears the previous pair down first, and narrows what crosses with a selector. The bullets say what state exists after you press save, because for one of the two modes the answer is none.

  • A continuous choice becomes a document. replicate_datacenter builds replicate_opt with create_target: false, adds continuous: true, and saves it into the _replicator database of the chosen endpoint, creating that database first if it is missing. A one-time choice is not a document at all: it calls replicate_couch_start once, the copy runs, and nothing is left on the cluster describing it. Same dialog, two very different amounts of durable state, at cpi/deploy_module/index.mjs line 7302.
  • Two-way is two one-way jobs pointed at each other. The first document is named for the app id with _1 appended, source and target being the two URLs plus their databases. The two_way flag writes a second document, _2, identical except that source and target are swapped. Teardown assumes that shape: delete_prev_replications removes _1 and _2 from both endpoints and clears replication_stat before anything new is written.
  • The selector decides what crosses, and it is coarse on purpose. Replicating from master sets replicate_opt.selector to a $or over docType workspace_drive and docType database, so drive metadata and table rows move and the rest of the project does not. Scope setup and scope custom narrow only the database branch, adding udfData.udffileid $in a list of table ids read from the studio table docs.
  • What it does not do. Nothing promotes a replica, nothing redirects traffic, and nothing reconciles a split brain beyond CouchDB picking a deterministic winning revision and keeping the loser as a conflict. And CouchDB replicates the drive DOCS, not the files: the on-disk workspace_drive folder is rsynced separately, through /tmp so the two VMs never SSH each other, as a one-shot seed. The timer that would make that folder sync continuous is still a TODO in the source.
What one save actually runs
  1. Save the four fields
    clearance admin, private, owned by app_module. apply: false stops here and writes nothing to CouchDB.
  2. Delete the previous pair
    _1 and _2 removed from both endpoints, replication_stat cleared. A re-save replaces, it never merges.
  3. Build replicate_opt
    create_target: false always, continuous: true only for continuous, the $or selector only from master.
  4. Write, or run once
    Continuous writes one document per direction. One-time posts a single replication, then appends a receipt.
Where it can land

Five macro regions are offered. Three of them resolve to a machine.

The region field carries 14 metro codes, and the picker groups them into five macro regions defined in MACRO_REGIONS at cpi/deploy_module/index.mjs line 9515. Each entry holds two lists: farm_locations, the xuda Farm sites the region owns, and do_slugs, the DigitalOcean slug prefixes it owns. Picking a region picks neither a provider nor a box. It scopes the candidate set to those two lists, and the rest is placement.

Choosing a region does not choose a box. Inside a region, get_farm_server orders the surviving candidates by great-circle distance from that region centre coordinates and takes the nearest that clears the fit test. Two datacenters created a minute apart in one region can land on different hosts.
Three ways to get one

Regional, Global, or one you already run.

The same deploy_datacenter call produces two different machines depending on one field, app_hosting_preference, and a third option produces no machine at all. These are code paths, not tiers, and they answer differently on the four questions that matter when something goes wrong.

Regional (DigitalOcean)
Global (xuda Farm)
One you already run
Image sourceA pre-baked DigitalOcean image: create_digitalocean_droplet is called with image_id set to digitalocean.datacenter_image_id, 198689047 in the repo config.A full clone of a golden Proxmox template, vmid 9101, declared as proxmox.templates.datacenter_vmid. Every other farm eligible non-VPS type clones the shared deployment template. The hosts are standalone boxes with no shared storage, so a host missing 9101 has it moved to it by vzdump into PBS then qmrestore, over the REST API.None. Nothing boots. The deployment is written a config.json carrying datacenter_id and a datacenter DSN, and that is the whole attach.
Backup pathA droplet snapshot. The handle kept on the app doc is snapshot_image_id, and a restore hands that image id to a new droplet.Hypervisor level. vzdump writes into the PBS datastore and the task resolves to a PBS volume id, the handle for both restore and delete. A restore either replaces the same vmid in place or builds a new VM from the archive.Whatever already covers that box. The attached deployment keeps its data in db_<datacenter_id> on the datacenter, so it is protected exactly when the datacenter is and not otherwise.
Price sourceThe slug. Each of the 16 server types is an entry in PRICE_OBJ.server_slugs with fixed cpu, ram, disk and a fixed monthly price, starting at shared_hosting_1 with 1 vCPU, 2 GB and 50 GB.compute_farm_price: cores times the tier cpu rate, plus RAM times the ram rate, plus disk times the disk rate. Each input is clamped to the tier maximum, the total floored at the tier minimum, the result rounded to the nearest 10 cents.Nothing new is computed. The attach creates no server, so neither a slug price nor a farm rate is produced by it.
Capacity refusalNot yours to see. No headroom arithmetic runs here, and a create the provider will not satisfy comes back as the provider error rather than a farm code.get_farm_server multiplies your cores, RAM and disk by 1 plus proxmox.headroom_pct over 100, currently 20 percent, before comparing them to each node free capacity. If nothing clears it and proxmox.do_failover is false, the create is refused with -462 and is not queued. A region with empty lists refuses earlier, with -463.There is no check at all. Placement never runs on an attach, so nothing verifies the box has room for one more database before the DSN is written.
Getting one

Three moves from nothing to a datacenter with deployments on it.

This is not a self-serve API action. deploy_datacenter is declared with clearance admin, min_plan pro and private: true, so it is a dashboard session on a Pro plan or above and no API key of any mode reaches it. It is one of only 19 methods in the repo config carrying a pro floor, out of 1010 declared.

  1. 01

    Declare it

    A name, a region from the 14 metro codes, and a server type from the 16 sizes running shared_hosting_1 up to large_hosting_32. That writes an app doc with app_type datacenter and, on the same document, a freshly generated 20 character CouchDB admin password under couchdb_admin_password. One credential is created here and everything later shares it. The catalogue entry still describes this as creating a DigitalOcean droplet, and the function is still named create_digitalocean_droplet, even though it hands the create to Proxmox first. The name is stale, the behaviour is not.

  2. 02

    The box boots and calls back

    Nothing polls the machine. The VM xuda-firstboot.service curls the digitalocean_webhook method with its own id about ten seconds after the network is up, and the webhook runs the install: 10 named steps from connecting to server through the self signed certificate and the ini files to creating db. Before creating anything it polls the box CouchDB _users endpoint 10 times, 5 seconds apart, and gives up if it never answers. Then it creates db_<app_id>, runs the replication you configured, writes the design docs and indexes, and only then marks the server active.

  3. 03

    Attach deployments by id

    A deployment placed onto a datacenter is written a config.json carrying datacenter_id and a datacenter DSN pointing at db_<datacenter_id> on the datacenter private address, with the same admin credential minted in step one. That is the whole binding: no agent, no registration handshake. The two share one password, and a farm datacenter sits behind a default-deny firewall with ipfilter on: 443 restricted to Cloudflare ranges when the app is fronted by it, CouchDB 5984 only from the project own IPSet, SSH only from the xuda_control set.

Honest limits

Nine things a datacenter does not do, starting with the one everyone assumes.

  • Nothing here scales itself. A node crossing proxmox.capacity_alert_pct, which is 80, raises an alert whose recommended action is a person ordering another box, and it is only pulled out of placement at 95 percent. With proxmox.do_failover false, a deploy that finds no fitting node is refused with code -462. It is not queued, and nothing grows to fit it.
  • Two of the five macro regions are labels with nothing behind them. MACRO_REGIONS gives africa and latam an empty farm_locations list and an empty do_slugs list, so placement returns no node there and the create path answers -463, this region is not available for deployment, choose a different one.
  • There is no cross-region failover for your project. A project is pinned to a home region, and when that region is unreachable a location-sensitive call comes back with a plain message saying the work was not started. It does not quietly run somewhere else.
  • Replication is not failover. You pick one-time or continuous, master or datacenter, one way or two way, and that is the entire feature. Nothing promotes a replica, redirects traffic, or reconciles a split brain for you.
  • The rollout is strictly sequential, one region at a time, capped at 25 minutes per box, and a box that fails is deliberately left out of rotation. It rejoins when the drain flag ages out after 15 minutes, on a timer, not on any verdict about whether it got fixed.
  • The publish guard is a shell check inside the build scripts, not a platform lock. It only looks for stat 1 or 2 on a regional_server doc, and XUDA_SKIP_PUBLISH_GUARD=1 turns it off, which is exactly how a half-finished rollout gets overtaken.
  • Capacity reconciliation and node health polling run on exactly one controller box on purpose, because two sweepers would race each other. That is one process deciding, every five minutes, which nodes the whole farm may place onto.
  • The regional disk self-heal path, clean tmp then grow the volume then declare the region full, is shipped and switched off: server_capacity_monitor.enabled is false in the repo config, and its grow step is a DigitalOcean volume call regardless.
  • Every number on this page was read from the repo copy of config.json and the cpi sources. The live master config is known to drift from that copy and was not read, so treat these as the shape of the thing rather than as today's exact totals.
Before you press deploy

The six questions this page exists to answer

Answered against the shipped code, with the file or the config key that decides each one.

Watch

The dashboard the deploy starts from.

There is no datacenter specific recording yet, so this is the general platform walkthrough. It shows the surface you press the button on, not the placement and install path this page describes.

Pick the region with the capacity rules in front of you.

A datacenter is a Pro plan deploy with two real decisions in it: the region, 14 metro codes, and the server type, 16 sizes from shared_hosting_1 up to large_hosting_32. Decide the region first, because placement never crosses one and nothing grows to fit you. africa and latam hold no nodes at all and answer -463, and a region whose nodes are inside the 20 percent headroom of your request answers -462 rather than queueing you. Once it lands, the install runs its 10 steps and ends by creating db_&lt;app_id&gt; on the box, and every deployment you place there shares that credential.