Beyond One Box: The Honest Scaling Audit

Every chapter since going live has defended the same boundary: one box, understood completely, is this book’s deliberate scope. This chapter does not cross the boundary – it surveys it, with the book’s method applied one last time: enumerate what the second box actually changes, verify what a repository can verify, and price what it cannot. An operator who outgrows this book should leave it knowing which pieces travel, which break, and in what order the road runs. Scaling folklore says “everything changes at two.” The audit says: remarkably little, at first, and each break has a name.

What already works at two – verified, not promised

The strange advantage of auditing this stack is how much of the n=2 story has already run, because the pair-deploy chapter was secretly the rehearsal. Sessions crossed instances under load – sealed in the cookie, decodable by any process holding SESSION_KEY, no session store, no sticky routing: 280 requests over a live handoff, zero failures. The page validator agreed across processes – W/"1035-en-fce9306" from both, because the build id is a build property, not a process one. And two peers served one database concurrently, which is simply the Peer model’s native posture: reads scale by adding peers. A second application box is the pair topology with a network in the middle (DATABASE_URI pointing across the wire, Caddy’s upstream pool naming another host). Datomic’s design means the read path was built for this direction all along.

What breaks, itemized

Each entry names the break, the shape of its fix, and the threshold where the fix stops being optional.

  1. The rate limiter – the first and cleanest break, flagged the day it was built: an in-process sliding window means n boxes grant n budgets. The e2e suite already met this bug’s little cousin, the shared window that starved parallel workers. Fix shape: a shared store (Redis is the conventional spelling) behind the same allow? signature – forced specifically by the per-email limit, since sticky-hash routing or edge throttling would cover the per-IP dimension with no new store, but neither can see an address that lives in the request body. Due: the day box two serves login traffic – this is the first new infrastructure the fleet demands.
  2. Metrics locality – each process answers for itself; two boxes means two scrape targets and questions (“total error rate?”) no single endpoint answers. Fix shape: the Prometheus server the endpoint was always formatted for. Due: when aggregate questions start mattering, which is roughly when the fleet does.
  3. Alert plumbing – the cooldown stamp lives in /run, so n boxes send n copies of correlated failures. Fix shape: alert routing and dedup, which is the actual product a pager service sells; ch.38’s buy threshold, “the day two people share on-call,” is also this one.
  4. Deploys fan out – deploy-pair.sh per box, static tree shipped to each, asset GC on each. Fix shape: first a loop over hosts in the pipeline, much later real orchestration. Due: gradually; the scripts are honest at small n.
  5. Secrets multiply – the 0600-file argument expires by its own stated terms the moment there are two boxes to drift apart. Fix shape: a secrets manager; config management for /etc/myapp generally. Due: box two, immediately, before the first drift incident.
  6. The proxy becomes the SPOF – one Caddy fronting two app boxes has just relocated the single point of failure. Fix shape: the load-balancer tier, DNS failover, all the machinery whose absence defined this book’s scope. Due: this is the boundary itself.

The shared-cache decision, finally

The conditional-GET chapter left one decision explicitly to these chapters: its validators would work for CDNs and shared proxies, if every anonymous page were audited for accidental personalization and public added deliberately. The audit ran, and it found what audits exist to find. The good news: anonymous pages are clean. Owner controls render only on authenticated requests (which are no-store), the locale is in the Vary and the validator both, and anonymous GETs write no cookies. Then the exception: the landing page’s stale-session sweep – a visitor arriving with a cookie for a deleted user gets a response that clears their session. Correct per-visitor; under a shared cache marked public, a cached Set-Cookie-bearing response becomes one visitor’s session surgery served to everybody. It is the class of bug the ch.31 trade-off predicted, found by reading, in the one place the app personalizes an “anonymous” response.

So the decision: public stays off. Not because the audit failed (one when branch is excludable) but because this topology contains no shared cache to serve it, and claiming cacheability nothing exercises is an untested promise wearing a header. The day a CDN fronts this application, the change is one word in wrap-conditional-get, minus the landing page, plus this audit re-run against whatever the schema has grown since. Priced, parked, and no longer an IOU.

The database’s own road

The data tier scales on a different axis than the app tier, and the shape is worth stating cleanly because it is the reason this stack was chosen. Reads: add peers – already proven above, and one properties line away from a shared memcached tier when the peers’ object caches deserve pooling. Storage: the shelf is one table in PostgreSQL, which means the deepest replication and HA tooling in the industry applies to it unmodified, or a managed PostgreSQL takes the whole problem. The transactor is where honesty gets interesting. Datomic Pro’s HA design is a standby pair coordinating through storage, and drilling the takeover – the one claim an earlier draft of this chapter had to leave as ran the pair but could not verify the failover – is what the failover lab (ops/lab/) exists to close. Two transactors against one Postgres, kill the active mid-write, and the standby promotes: the peer rediscovers it through storage and resumes in about twelve seconds, no committed transaction lost and no application code involved. What the lab cannot be is production topology – two machines with a real network between them, a tuned heartbeat, the partition that makes both transactors believe they are alone – so the mechanism is verified while the twelve-second number stays a single-host artifact of default timings, and the two-host failover keeps its split-brain questions for a fleet’s fault-injection lab. The finding survives, sharpened: beyond one box the guarantees stop being things you verify once and file away; they become things you rehearse – and the rehearsal now ships with a committed harness to start from.

And the ceiling, named without flinching: writes do not scale out. One transactor serializes every write; that is not a limitation of the free tier or a missing feature but the serializability bargain chapter 8 signed – the same single-writer design that made versions, the activity log, and the ETag fall out as byproducts. The escape hatches (sharding by database, rearchitecting the write path) are real and drastic, and the operator’s job is merely to know their distance: this catalog’s write rate is humans editing recipes, basis-t advancing a few times a minute at its busiest hour, against a ceiling measured in thousands per second. The cliff exists; it is nowhere in sight; and knowing both halves of that sentence is what an audit is for.

Measured, not just reasoned

The two paragraphs above are the audit’s reasoning; here are the numbers, driven against the same PostgreSQL-backed stack going live assembled, one JVM, warm. As with the render measurement, the machine and the day do not transfer – the shapes do.

Reads confirm the read-scale claim, emphatically. The most expensive public page, the deepest-lineage recipe, served ~16,000 requests per second under fifty concurrent clients, zero errors, 99th-percentile latency under seven milliseconds; pushed to two hundred concurrent, throughput held and latency rose linearly, the textbook shape of a saturated-but-healthy server, not a failing one. That is the peer’s object cache answering from memory with no storage round-trip, exactly the mechanism that makes “add a peer” a real scaling lever. And the conditional-GET fast path turns out to be the headline: presenting a matching If-None-Match, the same endpoint answered ~263,000 requests per second – seventeen times the full render – because the 304 is decided before a byte is rendered. The anonymous, repeat, crawler-and-unfurler traffic the machine-legibility chapter courts is served, in practice, nearly for free.

Writes confirm the ceiling, and quantify it. A single thread committed ~90 transactions per second; eight threads, ~620; thirty-two, ~1,290 – the transactor batches concurrent commits instead of dribbling them out one at a time, so “serialized” means one ordering point, not one at a time. The ceiling is real and it is in the low thousands per second on this modest sandbox storage, three orders of magnitude above a recipe site’s human write rate. The cliff was reasoned; now it is measured, and it is exactly as far away as the audit claimed.

Two concurrency properties load brings to the surface. The first is reassuring: the rate limiter is exact under contention – five hundred concurrent hits against a limit of one hundred admitted a clean one hundred, because its swap! retries on the CAS instead of losing an increment, so the security control does not soften under the concurrent brute-force it exists to stop. The second is a genuine limitation the single-box calm had hidden: recipe edits have no optimistic-concurrency control. update! reads to check ownership, then transacts the submitted fields as a blind overwrite, so two saves to the same recipe race to last-write-wins, and the loser’s edit vanishes with no “someone else changed this” warning. For this app’s single-owner-per-recipe model the collision is rare (one user, two tabs) and last-write-wins is a defensible answer; but it is a property, not an accident. The forms chapter now closes it: a compare-and-swap (:db.fn/cas) on :recipe/updated-at turns the silent overwrite into an announced 409 the form re-renders, scoped to the recipe’s own content clock rather than the global basis-t (which would false-conflict on every unrelated write). Surfaced by this measurement, fixed there.

Where the road ends

This is where the road ends – the scaling road, not the book. Past this line the premise itself changes: everything from here (the second box and its shared stores, the load-balancer tier, the pager rotation, the transactor drill on real hardware) is different work, fleet work, with its own tooling, its own rehearsal discipline, and its own books. What this book hands over at the boundary is what it promised at the first chapter’s wager: a system with no layer taken on faith – now including its operations – and an audit, this one, that turns “we should scale someday” from a mood into a checklist with prices. The chapters that follow stay on this side of the line, and turn back to the application running on the one box.