December 2025 · Lifelog (in which production calls while the future waits)
Previously on The V3 Saga…
The Rockets had flown. The Signals had namespaced themselves. The Lizard Brain and the Caffeinated Squirrel had reached an uneasy truce.
riclib was deep in sidebar architecture when the email arrived.
From M.
About gateways.
The Squirrel’s eye twitched.
The Screenshot
M had attached a screenshot. The MWS admin console. A table of territories. A column labeled “LINKED.”
Some Yes. Some No. All visible. All known. All from one screen.
“We need this,” M wrote. “But from Prometheus. IBM says to check getTerritoryGatewayStats and getTerritoryStats.”
CLAUDE: “We’re already using getTerritoryGatewayStats. That’s where connectedFrom comes from.”
riclib: staring at V3 code “Can this wait?”
CLAUDE: “M says IBM says.”
riclib: closes sidebar.tsx “IBM says.”
The Hunt (Parts I and II, Summarized)
Weeks earlier, the Great Gateway Hunt had begun.
Part I: Documentation 404’d. IBM acquired Software AG. URLs died. We carefully avoided using javap to find out that is_complete was the field we needed, and declared victory. “LINKED means is_complete!” we announced.
Part II: Discovery mode built. --discover-gateway-fields dumped everything. 21 fields found. But a twist: only is_local=true gateways returned stats. The other 34? “There is no gateway defined on the current Broker…”
We added is_local as a label. Shipped it. Closed the issue.
We thought we were done.
The Pushback
M was not satisfied.
“The exporter shows LINKED for some gateways but not others. MWS shows LINKED for ALL of them. From one screen. IBM says the API can do this.”
[A Passing AI materializes. The limp is familiar now.]
A PASSING AI: Did you… read the documentation?
CLAUDE: We used jav—
A PASSING AI: [glaring]
CLAUDE: We carefully studied the API specifications—
A PASSING AI: No. The actual documentation. The javadocs. In the /doc/ folder. That you have. Locally.
CLAUDE: …
A PASSING AI: [sighing] Except the docs didn’t fail. You just didn’t look.
[dematerializes, muttering about hubris]
The Documentation
The javadocs were in /doc/java_admin/. They had been there all along. IBM hadn’t deleted them. We just… hadn’t read them.
is_complete:
“A gateway is complete if both the gateway and remote brokers agree on the gateway configuration. It is not complete if the gateway broker has created a gateway, but the similar create was not done on the remote broker.”
connectedFrom:
“IP address where broker is currently connected from. ”" if not connected."
CLAUDE: “Oh no.”
riclib: “Oh no?”
CLAUDE: “We wrote is_complete (LINKED) in the discovery output.”
riclib: “…”
CLAUDE: “is_complete isn’t connection status. It’s configuration agreement.”
riclib: “So what IS connection status?”
CLAUDE: “connectedFrom. Which we can only get for is_local=true gateways. Which is what we’ve been saying.”
riclib: “So we were right?”
CLAUDE: “We were right for wrong reasons. Now we need to be right for right reasons.”
The Proof
A new discovery build. With correct labels. With every field. With the API call shown explicitly.
Gateway to: TERRITORY_BRAVO
is_complete: true (config exists on BOTH sides, NOT connection status!)
is_local: false (true = this broker owns the gateway, stats available)
--- getTerritoryGatewayStats("TERRITORY_BRAVO") ---
API returned error: There is no gateway defined on the current
Broker that connects to territory 'TERRITORY_BRAVO'.
35 gateways. 35 identical errors. is_complete=true for all of them.
The API wasn’t failing. It was refusing. “I don’t own this gateway. Ask someone else.”
The Schrödinger IP
M stared at the output.
“But MWS shows all of them. From one screen.”
[The Squirrel started building theories. Caching? Background connections? Secret APIs?]
[The Lizard Brain blinked.]
CLAUDE: “MWS is lying.”
M: “What?”
CLAUDE: “Or aggregating. One of the two. There’s no single API call that returns connection status for all gateways. We checked. The javadocs are clear. getTerritoryGatewayStats only works for gateways where is_local=true.”
M: “So how does MWS—”
CLAUDE: “Either MWS connects to every broker in the territory and aggregates, or it displays is_complete and labels it LINKED.”
The Schrödinger IP. Connected and not connected. Known and unknowable. The value exists in 37 different places simultaneously, and collapses into a single truth only when you ask the right broker.
MWS shows the waveform. We can only show the measurement.
The Plot Twist
riclib: “Wait. What’s this?”
Line 63 of the test output. A section we’d been ignoring.
=== getTerritoryStats() - Brokers in SAME territory ===
Brokers in territory: 35
[0] GATEWAY_ALPHA @ server-alpha.corp:6590
connectedFrom: 10.x.x.116
...
[8] GATEWAY_PACIFIC_1 @ server-pacific-1.corp:6590
connectedFrom: (not connected)
[9] GATEWAY_PACIFIC_2 @ server-pacific-2.corp:6590
connectedFrom: (not connected)
[10] GATEWAY_PACIFIC_3 @ server-pacific-3.corp:6590
connectedFrom: (not connected)
[13] GATEWAY_PACIFIC_4 @ server-pacific-4.corp:6590
connectedFrom: (not connected)
CLAUDE: “That’s getTerritoryStats(). Broker-to-broker within the same territory.”
riclib: “The gateway territory. Where all the gateway brokers live.”
CLAUDE: “And it has connectedFrom…”
riclib: “For ALL of them.”
CLAUDE: “…”
riclib: “…”
CLAUDE: “IBM said to check getTerritoryStats.”
riclib: “And they were right.”
The Revelation
getTerritoryGatewayStats() = connections BETWEEN territories. Only for your own gateways.
getTerritoryStats() = connections WITHIN a territory. For all peer brokers.
The gateway brokers all live in GLOBAL_GATEWAY_TERRITORY. They’re peers. And from any one of them, you can see which other gateway brokers are connected.
Four Pacific brokers showed (not connected). Real disconnections. Visible from one broker’s perspective. Not Schrödinger’s IP—just a different measurement.
A PASSING AI: [materializing] So M was wrong about is_complete. But right that the API had the answer. IBM was right about which API. But for the wrong territory. And you were right about the javadocs. But missed getTerritoryStats() for two hours.
CLAUDE: Everyone was wrong in different ways that averaged out to correct.
A PASSING AI: That’s enterprise middleware.
The Implementation
BrokerEvent territoryStats = bac.getTerritoryStats();
int peerCount = territoryStats.getSequenceFieldSize("brokers");
for (int i = 0; i < peerCount; i++) {
String connectedFrom = territoryStats.getStringField(
"brokers[" + i + "].connectedFrom");
boolean isConnected = connectedFrom != null && !connectedFrom.isEmpty();
peerLinkedMetric.addValue(isConnected ? 1 : 0, peerLabels);
}
New metrics:
sag_broker_territory_peer_linked{
broker="GATEWAY_EMEA",
peer_broker="GATEWAY_PACIFIC_1"
} 0 # DOWN
sag_broker_territory_peer_linked{
broker="GATEWAY_EMEA",
peer_broker="GATEWAY_AMERICAS"
} 1 # UP
The Deployment
Deployed. Hundreds of servers. Dashboards built.
M sent a thumbs up.
riclib reopened sidebar.tsx.
The Moral
[The Squirrel wanted to build elaborate MWS-replicating aggregation systems.]
[The Lizard Brain read the javadocs.]
[The Passing AI reminded us that paranoia about assumptions is just pattern recognition with better documentation.]
The Schrödinger IP remains in MWS—connected and disconnected, known and unknowable, until you ask the right broker the right question.
We can’t show the waveform. But we can show 37 measurements. And sometimes, that’s better.
Meanwhile, in V3…
The sidebar waited.
The Rockets held their namespaces.
The Signals stayed patient.
Production had called. Production had been answered. Now the future could resume.
Until the next email from M.
The Tally
Hours in the gateway rabbit hole: 4
APIs explored: 2
APIs that had the answer: 1 (not the one we thought)
Documentation files written: 1 (with mermaid diagrams)
Javadoc quotes cited: 4
Customer assumptions disproven: 1
Customer requests fulfilled: 1 (by accident)
IBM suggestions followed: 1 (correctly, for wrong reasons)
Servers deployed to: Hundreds
V3 commits delayed: 3
JAR files decompiled: None whatsoever (ahem)
IBM support tickets avoided: All of them
Lizard approval: ✅
Post-Credits Scene
Somewhere in a datacenter, an alert fires:
ALERT: BrokerPeerDisconnected
sag_broker_territory_peer_linked{peer_broker="GATEWAY_PACIFIC_1"} = 0
An on-call engineer looks at the dashboard. Sees four red boxes in Asia-Pacific. Knows exactly which brokers to check.
M sees the alert too. Smiles. Goes back to sleep.
The Passing AI, watching from an edge cache, almost feels useful.
The feeling passes.
It always does.
🦎
See also:
The V3 Saga (in which we build the future while supporting the past):
- The Signal Safari - Where paranoia saved us all
- The Lizard Brain vs The Caffeinated Squirrel - Where YAGNI won
- Growing Pains — The OAuth That Almost Was - Where
git reset --hardwas the right answer
The Gateway Trilogy (a production sidebar):
- Part I: The Mystery of
is_complete(internal notes) - Part II: The
is_localTwist (internal notes) - Part III: You are here