Lessons (6, study in order)

① Why show commands are not enough

Beginner · about 8 min

Goal: Understand what BMP actually solves -- not "another way to look at BGP" but the classes of question the device simply cannot answer.

Start with one concrete question

"Why isn't this route in my BGP table?"

★ It has only two possible answers, and they point in opposite directions:
· the neighbour never sent it -> go to them
· they sent it and my inbound policy dropped it -> read my own route-map

★★★ And on the box, show bgp produces identical output for both: "no such route".

★★ What it costs to distinguish them on-box

You need soft-reconfiguration inbound before show bgp neighbor x received-routes works.

★★ And that stores a second copy of the Adj-RIB-In: about 1.4 GB extra for four full-table peers.

★★★ Worse is the timing: it must be enabled before the problem. Configuring it afterwards shows the state after configuration, and the moment you wanted is gone.

★★ And a whole class show cannot answer at all

★★★ show gives a snapshot of the current state, so:

· "how many times did that route flap half an hour ago?" -> no record
· "what was the original MED they sent?" -> overwritten by policy
· "why were we down for five minutes yesterday afternoon?" -> logs rolled over

★ Plus you have to be logged into the right device at the right moment.

What BMP does

It records the device's BGP tables continuously, verbatim, off-box. The key is that it carries more than one table:

· Adj-RIB-In pre-policy -- what the neighbour actually sent (★★★ the valuable one)
· Adj-RIB-In post-policy -- what survived the policy
· Adj-RIB-Out pre/post (RFC 8671) -- what I advertised
· Local-RIB (RFC 9069) -- after best-path selection

★★★ The difference between pre and post is exactly what your policy did.

Key takeaways

• ★★★ "Why isn't this route here" has two answers, and show bgp output is identical for both.
• ★★ Distinguishing them on-box costs soft-reconfiguration memory (~1.4 GB) and must be enabled beforehand.
• ★ BMP is fundamentally a continuous stream while show is a snapshot of now.

② ★★★ pre minus post is what your policy did

Beginner · about 10 min

Goal: Master this module's central operation: subtract post-policy from pre-policy and read off both the route's fate and who to go ask.

Three possible outcomes

★★★ A route's presence across pre and post has only three meaningful combinations:

· in pre, not in post -> your policy dropped it -> look here
· in neither -> the neighbour never sent it -> go to them
· in both but with different attributes -> the policy silently rewrote it (the hardest to find)

★ The fourth combination, "in both and identical", is just a normal permit.

★★★ Compare scenarios 2 and 3 by hand

★★★ These two have identical symptoms (10.20.0.0/16 is missing) with opposite causes:

· ②policy-drop: it is in pre -> your own deny dropped it
· ③never-sent: absent from pre too -> the neighbour never sent it

★★ Read them back to back and it becomes obvious why, without BMP, these two lead to mutual blame: you say "you didn't send it", they say "we did", and neither has evidence.

★ The difference also names the rule

Policy is ordered: the first matching rule decides, and later ones never run.

★ So the diff page points at exactly which route-map line did it -- and "the rules are in the wrong order" is a genuinely common production error: one broad match placed too early shadows every precise rule after it.

Why this matters practically

★★ In peering and upstream disputes, the side with BMP argues from a different position:

· without pre-policy all you can say is "I don't see it here" -- not evidence
· with pre-policy you can say "what you sent at 14:32 had exactly these attributes"

★ Which is why large networks and IXPs have almost universally deployed BMP.

Key takeaways

• ★★★ In pre but not post = you dropped it; in neither = they never sent it. show bgp cannot tell these apart.
• ★★ Policy is first-match-wins, so the diff page identifies which rule decided.
• ★ The pre-policy view is the only presentable evidence in a peering dispute.

③ ★★ A rewritten attribute is harder than a drop

Advanced · about 9 min

Goal: Understand why "the route is there but its attributes changed" is the most insidious class of problem, and why it is entirely un-investigable on the box.

Why it is harder

A drop at least has an unambiguous symptom: the route is missing and you go looking.

★★★ A rewritten attribute has no symptom at all:
· the route is in the table ✓
· the state is fine ✓
· only best-path selection differs from expectation

★ So people suspect "did they change something?" or "do I misunderstand the BGP decision process?" when the truth is their own set local-pref matched too broadly.

★★★ The crux: show displays the post-change value

★★★ The local-pref, MED and AS path in show bgp are all after the inbound policy ran.

★★ So on the box you simply cannot see the original -- you do not know what they sent, and therefore cannot judge whether the difference is your own doing.

★ This class of question falls under "no show command can answer it", not "enable soft-reconfiguration and you're fine".

★★ Typical silent rewrites

· set local-pref -> best-path changes while the route looks perfectly fine
· AS-path prepend -> afterwards you cannot tell which hops are the real path
· set community -> ★★ the effect propagates into other people's networks downstream, and they only see the post-change value
· set MED -> the choice between multiple entries from the same upstream shifts silently

★ That last one is especially hard, because it only manifests with multiple entry points.

Key takeaways

• ★★★ A rewritten attribute has no symptom -- the route is there, the state is fine, only best-path is wrong.
• ★★ show bgp shows post-change values, so the original is invisible on the box.
• ★ Community rewrites propagate downstream, where only the post-change value is visible.

④ ★★ Which RIBs to subscribe, and what the initial sync costs

Advanced · about 9 min

Goal: Work through the subscription trade-off: pre-policy is mandatory, and every extra RIB doubles the full-table dump.

★★★ Post-policy only defeats the point

★★★ The most common production oversight, and it does not error:

BMP is configured, the session is up, data flows, the UI looks fine -- and the one valuable question stays unanswerable.

★★ You paid the full price (full-table dump, bandwidth, station storage) for a remote copy of show bgp.

★ So rule one of subscription design: pre-policy is mandatory.

★ Each extra RIB doubles the volume

Subscribing pre and post together means the same table travels twice. Add Adj-RIB-Out and it is four times.

★ So the sensible trade-off is:
· pre-policy -- mandatory (without it there is no point)
· post-policy -- skippable; show bgp on the box is roughly that
· Adj-RIB-Out -- only for "did I actually advertise it to them" questions
· Local-RIB -- only if the station should see best paths directly

★★ What the initial sync really costs

The moment a BMP session comes up, the device must dump every peer's entire table.

★ Four full-table peers, one RIB: about 3.8 million routes, 540 MB, and 3 minutes at 20k routes/s. Subscribing pre+post doubles it.

★★★ And the control plane is busy throughout. The production risk: enabling BMP on a loaded border router at peak can push the control plane over and affect BGP itself -- monitoring becoming the cause of the outage once again.

How to reduce the risk

★ Three practical measures:

· enable off-peak (the real reason behind that line in the docs)
· add one peer at a time rather than everything at once
· check whether the platform can rate-limit BMP (some cap the generation rate)

★★ And do not forget the station side: the instantaneous rate during a full dump can be tens of times the steady state, and the station has to absorb that peak.

Key takeaways

• ★★★ Post-policy only defeats the point of BMP -- and it does not error, so it can run that way for years.
• ★★ Every additional RIB doubles the full-table dump volume.
• ★ The initial sync occupies the control plane for minutes, so enable off-peak, one peer at a time.

⑤ ★★★ Why the peer went down: attribution and history

Troubleshooting · about 9 min

Goal: Learn to use Peer Down reason codes to locate the investigation directly, and understand why "it flapped and settled" is invisible on the box.

★★★ The reason code hands you the direction

BMP's Peer Down message carries why it went down, and that decides where to look:

· local sent a NOTIFICATION -> we closed it; look here (you hold the error code)
· local FSM event -> usually the peer's keepalives stopped; check the path and their load
· remote sent a NOTIFICATION -> they closed it, and you already have their error code
· remote vanished -> they crashed or rebooted, or a TCP connection was killed in between

★★ The last two are very hard to distinguish on the box, yet they mean entirely different things: "they object" versus "they broke".

★★ Two reasons that are not faults at all

· peer de-configured -> a change, not a fault
· local system closed this monitoring session -> the BGP session is fine; only the monitoring stopped

★★★ The second is especially easy to misread as "the neighbour went down" when nothing happened on the network at all. The reason code settles it instantly.

★★ And flap history is something show cannot give you

★★★ A peer that flapped 14 times and settled looks identical in show bgp summary to one that never flapped: both Established.

★ Some platforms expose uptime and a connection counter in show bgp neighbor, but that is a cumulative number -- when it flapped, for how long, and why are all absent.

★★ And a ticket saying "traffic was down for five minutes yesterday afternoon" is simply unanswerable without a record.

Why this is the essential difference

★★★ BMP is a continuous stream; show is a snapshot of now.

★ A snapshot answers "how are things"; it can never answer "what just happened". And troubleshooting wants the latter.

★★ Which also explains why BMP data goes into a time-series or log system: the value is not "the current table" (show has that) but being able to look back.

Key takeaways

• ★★★ Peer Down reason codes give the direction directly, plus the raw bytes of their NOTIFICATION.
• ★★ Two of the reasons are not faults (peer removed, monitoring stopped) and are easily false-alarmed.
• ★★★ A peer that flapped and settled is indistinguishable in show bgp summary from one that never did.

⑥ ★★ BMP fails silently too

Troubleshooting · about 9 min

Goal: Recognise the failure modes BMP introduces: it is unidirectional, so data quietly disappears; plus the "configured but useless" oversight.

★★ BMP is unidirectional

The station never sends requests to the device; it only receives. So there is no flow control at all to "ask the device to slow down".

★ When the station falls behind, the TCP receive window shrinks and back-pressure reaches the device, which faces a choice: block (possibly affecting BGP) or drop BMP data. Virtually every implementation drops.

★★★ Which gives that familiar shape again

★★★ Data quietly thins out while neither the device nor the station reports an error: the session is established, data is flowing, there is just less of it.

★★ So what you monitor is not "is the BMP session established" but the station's receive rate and sequence continuity -- an established session and lost data coexist perfectly well.

★ Same family as xFlow's dropped export datagrams and telemetry's dead stream: "I sent it" is not "they received it".

★★★ The other kind of "configured but useless"

★★★ More common than data loss is subscribing only post-policy:

every indicator looks healthy (session up, data flowing, table updating) while the one question that matters stays unanswerable. ★★ Right until you actually need to investigate "why isn't this route here" and discover the valuable table was never subscribed.

★ So after deploying BMP, run a capability check: pick any route your policy denies and confirm it really does appear in pre. Skip that and you do not know whether you have the capability at all.

One theme across three modules

★★★ The three observability modules are three faces of the same thing:

· xFlow: traffic cannot be seen completely (sampling erases small flows and scans)
· Telemetry: state cannot be seen accurately (the interval decides what you see)
· BMP: route changes cannot be traced (show only has now)

★★ And their failure modes share a shape too: the dangerous one is not an alarm but silence -- a flat chart, no alerts, and you are already blind.

Key takeaways

• ★★ BMP is unidirectional; when the station falls behind the device drops BMP data rather than block BGP, without erroring.
• ★★★ Monitor the station's receive rate and sequence continuity, not "is the session established".
• ★ After deployment, verify the capability: pick a denied route and confirm it appears in pre-policy.