Lessons (6, study in order)

Lesson 1 - The Kinds of Data Center in One Region

Beginner · about 10 min

Goal: Know where each of the eight facility types sits and what it does — the prerequisite for every relationship that follows.

Three tiers first: core, aggregation, outlying

A region's traffic converges bottom-up:

· Core DCs (AWS Availability Zones) — where compute and storage live; a region usually has 2-3, running active-active
· Aggregation DCs (AWS Transit Centers) — traffic from the core and every outlying site converges here before the backbone or the Internet; they hold backbone routers, almost no application servers
· Outlying sites — business DCs, satellite DCs, PoPs, colocation — all hang off the aggregation layer by one or two uplinks

★ Keep this shape in mind: aggregation is the hub; if it dies, everything below it is cut off.

★ The two outward-facing kinds: interconnect and scrubbing

The interconnect DC is where the region shakes hands with the outside world — a carrier-neutral third-party building with an IX, every carrier, and cloud private-line on-ramps. It's the region's Internet egress; AWS Direct Connect locations live in exactly these buildings.

The scrubbing DC exists to absorb DDoS: nearly idle normally, under attack the targeted prefixes are diverted here, cleaned, and re-injected. It always sits next to the interconnect site — where bandwidth is most plentiful.

★★ Satellite vs PoP vs colocation

All three sit outside the region and uplink back, but they are not the same thing:

· Satellite DC = a branch you built (AWS Local Zone): has compute, but no control plane of its own — it depends entirely on the parent region
· PoP = a traffic on/off ramp (CloudFront edge): a few racks of caches and an access router, no compute hinterland
· Colocation = space you rent: racks in a third-party facility that can hold real compute; the landlord runs power and cooling

Try it: click every facility once

Go to the Topology & Failover page (the us-east-1 anatomy scenario is loaded) and click every facility on the canvas — the panel on the right shows its type, duty, city, and what it hangs off. Get "who is where, doing what" clear before the failover drills.

Key takeaways

• A region = core (compute) + aggregation (hub) + outlying (business/satellite/PoP/colo) + outward-facing (interconnect/scrubbing)
• ★ The aggregation layer is the region's hub — the one place that must never be a single point
• ★★ A satellite is a branch you built, colocation is space you rent, a PoP is only a traffic ramp

Lesson 2 - Location Decides the Relationship: Distance, Latency, Sync

Beginner · about 8 min

Goal: How far apart two facilities are directly decides what relationship they can have.

Light in fiber only does 200 km/ms

Light travels through fiber at about 200,000 km/s — 200 km per millisecond — and a round trip goes twice, so RTT ~= fiber length (km) / 100 ms.

Fiber doesn't run straight: it follows roads, railways and submarine routes, 20-50% longer than the great-circle distance. This module estimates fiber length as x 1.3 throughout.

★★ The boundary for synchronous replication: RTT <= 2ms

Synchronous replication means every write waits for the far end's acknowledgement before it completes. At 1ms RTT, every write costs an extra 1ms; at 50ms RTT the database is essentially unusable.

The industry rule of thumb is RTT <= 2ms, i.e. 100-200 km of fiber — that is the definition of "metro", and why Availability Zones sit tens of km apart: far enough that one fire or outage doesn't reach both, close enough to replicate synchronously.

Look at the Tianjin satellite DC in the Beijing anatomy scenario: 120 km, ~1.5ms RTT — right at the edge.

Across regions it can only be asynchronous

Virginia to Oregon is ~5,000 km of fiber, ~50ms RTT; transatlantic and transpacific are farther still. At these distances only asynchronous replication works: the write completes locally and the data follows later.

The price is that RPO (the slice of data you can lose in a failure) is no longer zero — typically seconds to minutes.

Try it: the distance & latency table

Go to the Distance · Latency · Traffic page — every link shows fiber length and RTT computed from coordinates, and whether sync is feasible. Find which links can replicate synchronously and which can't.

Key takeaways

• RTT ~= fiber length (km) / 100 ms; fiber length ~= great-circle distance x 1.3
• ★★ Synchronous replication needs RTT <= 2ms, i.e. 100-200 km — that is "metro"
• Across regions only asynchronous replication works, and RPO is no longer zero

Lesson 3 - Active-Active and Active-Standby

Advanced · about 12 min

Goal: The two most important relationships between facilities, and what happens to each on failure.

Active-active: both sides serve

Active-active = both facilities serve traffic at once and replicate to each other.

· Metro active-active: synchronous, RPO = 0, the survivor takes over within seconds
· Cross-region active-active: asynchronous only, eventually consistent; on takeover the last few seconds of writes may be lost and the application must tolerate that

★ The price: neither side may normally run above 50% load — or it can't absorb the other's half.

Active-standby: one serves, one waits

Active-standby = one side serves; the other only receives replication and takes no traffic. Cross-region disaster recovery is almost always this shape.

Two key numbers: RPO (how much data the cut-over loses — minutes under async replication) and RTO (how long the cut-over takes — DNS/traffic switch, cache warm-up — minutes to hours).

★★★ Try it: take a core down

Go to the Topology & Failover page (the AWS global scenario is loaded):

(1) Take down CORE-IAD (Virginia, active) — watch Oregon's CORE-PDX get promoted from standby to active; the event spells out the RPO / RTO magnitude
(2) Take down CORE-DUB (Dublin) — watch Frankfurt's CORE-FRA take over all traffic; note the event says "asynchronous, last few seconds may be lost"
(3) Cut the AGG-IAD - AGG-PDX backbone span — neither core is broken, but replication now detours around the other side of the planet; see how much the RTT grows

Failback is not automatic

In this module the standby steps back the moment the active recovers — so you can experiment repeatedly. In a real system cross-region failback is a planned operation: confirm the data has synced back, pick a quiet window, shift traffic gradually. Automatic failback flaps during network jitter and is more dangerous than the outage itself.

Key takeaways

• ★★★ Active-active = both serve, sync in metro, async only across regions; active-standby = one serves, one waits
• ★★ Neither active-active side may exceed 50% load, or the takeover is fiction
• RPO is the data a cut-over loses, RTO is the time it takes; failback is planned, never automatic

Lesson 4 - Uplinks and Single-Homing: How Outlying Sites Attach

Advanced · about 8 min

Goal: The most common redundancy in the topology, and the one most often cut to save money.

Every outlying site lives by its uplink

Business DCs, satellite DCs, PoPs, colocation — even the core DCs — attach to the aggregation layer by uplinks. They exist on the network only while an uplink is up; lose them all and the site is an island, however much compute sits inside.

★ Single-homing vs dual-homing

One uplink is single-homing: one link and one device saved, but one dead parent or cable and the site is isolated.

Two uplinks to different parents is dual-homing — and it has to be different parents: two links into the same aggregation DC still die with that DC; they only protect against a single cable cut.

In the anatomy scenario the cores are dual-homed (to AGG-A and AGG-B); the business and satellite DCs are single-homed (AGG-A only).

★★ Try it: take down aggregation A

Go to the Topology & Failover page (the us-east-1 anatomy scenario is loaded) and take down AGG-A:

· Both cores are fine — they still have AGG-B
· The business DC BIZ and satellite DC SAT are isolated — they only uplink to AGG-A
· The scrubbing DC is isolated too — clean traffic can't be re-injected, so the region loses DDoS protection

Then take down AGG-B: now even the cores are isolated and the whole region is cut off.

Key takeaways

• Outlying sites live by their uplinks; lose them all and the site is an island
• ★ Dual-homing means two links to two different parents — two links to one parent only protect against a cable cut
• ★★ Aggregation is the region's hub: one down isolates every single-homed site, both down cuts off the whole region

Lesson 5 - Egress and Scrubbing: When the Interconnect Site Dies

Advanced · about 10 min

Goal: How a region's Internet egress and DDoS protection back each other up.

The interconnect site is the egress

Every region's aggregation DCs connect to an interconnect DC: a carrier-neutral building with an IX and every carrier. This is where we peer with everyone else's network (peering / private lines) — the region's Internet egress.

Outbound (what we send to the Internet) is usually far larger than inbound.

★ Egress can detour

When the interconnect site dies, the region does not instantly lose the Internet: the aggregation DCs are still on the backbone ring and can exit through another region's interconnect site. The price is latency — Virginia exiting via Dublin travels 5,000+ km more, roughly +50ms RTT.

★ That's why real regions have several interconnect sites (multiple Direct Connect locations); this scenario gives each region just one, precisely so you can watch the detour happen.

Scrubbing: idle normally, diverted under attack

The scrubbing DC carries almost nothing in normal times; under attack, BGP diverts the targeted prefixes there for cleaning, and clean traffic is re-injected at the aggregation layer. Sized at Tbps scale — capacity is the whole point.

Usually only a handful exist worldwide. This scenario has just the one in Virginia — European and Asian regions under attack have their traffic detoured there.

★★ Try it: take down the interconnect and the scrubber

Go to the Topology & Failover page (the AWS global scenario is loaded):

(1) Take down IX-IAD — see which region Virginia's egress moves to and how far the detour is
(2) Take down SCRUB-IAD — all six regions lose DDoS protection at once
(3) Check the region panel on the right — every region's current egress and scrubbing path is listed there

Key takeaways

• The interconnect site = the region's Internet egress, peering / private lines in a carrier-neutral building
• ★ If it dies, egress can detour over the backbone to another region — at a latency cost
• Scrubbing is idle normally and diverted-to under attack; only a few exist worldwide, other regions detour to them

Lesson 6 - Orders of Magnitude: Tbps in Metro, 100s of Gbps Across Continents

Beginner · about 8 min

Goal: Roughly how much traffic flows between two facilities — not exact numbers, but the ability to name the order of magnitude.

Where these numbers come from

Providers like AWS do not publish real inter-facility traffic. This module's figures are order-of-magnitude estimates from public architecture material (the scale of inter-region bandwidth, of east-west traffic between AZs, of edge origin-fetch ratios), and every one carries a "~".

★ The goal isn't to memorize numbers — it's to look at a link and say whether it is Tbps-scale, hundreds-of-Gbps-scale or tens-of-Gbps-scale.

★ The three orders of magnitude

· Between metro cores: Tbps — synchronous replication plus heavy east-west traffic (services calling each other)
· Core to aggregation, aggregation to interconnect: hundreds of Gbps to Tbps
· Cross-region backbone: one to a few hundred Gbps — submarine capacity is leased and expensive
· PoP, satellite, colocation: tens to hundreds of Gbps, and downstream far exceeds upstream (content flows toward users)
· Egress: outbound far exceeds inbound (we send out more than we take in)

East-west vs north-south

East-west traffic = services calling each other, inside and between facilities; north-south traffic = traffic to and from the Internet.

Modern applications have far more east-west than north-south — one user request can fan out into dozens of service calls. That's why the line between two metro cores is thicker than the Internet egress.

Try it: the traffic column

Go to the Distance · Latency · Traffic page, sort by traffic, and check which links are the fattest and thinnest against the orders of magnitude above.

Key takeaways

• Traffic figures are order-of-magnitude estimates, not official data
• ★ Metro cores Tbps > core-to-aggregation hundreds of Gbps > cross-region backbone one to a few hundred Gbps > edge sites tens of Gbps
• East-west far exceeds north-south, downstream far exceeds upstream, outbound far exceeds inbound