Why Exact-Match Search Fails at Config Audits (and What Supernet Overlap Found)

Here is a problem that looks like string matching and is not: you have a carrier circuit inventory -- a spreadsheet full of IPs and identifiers -- and two live network configs, and you need to know whether anything in the sheet exists in your gear. The naive approach, grep the configs for each IP, will confidently report "nothing matches" and be wrong. The overlap you actually care about lives in subnet math, not in literal strings.

I learned this the practical way last week, auditing eight carrier circuits that finance wanted accounted for and that nobody in the building could locate. I made an AI do the cross-referencing, and the gap between my first pass and my last pass is the whole reason I'm writing this up.

The inputs

• Firewall running config: ~15,000 lines, five virtual routers, a few hundred address objects accreted over years
• Switch stack config: five members, every port hand-labeled across a decade
• Carrier inventory: 16 rows by 59 columns -- service IDs, circuit IDs, NTE management IPs (v4 and v6), gateway IPs, VPLS instances, VLAN tags, CLLI codes, aggregation router details

The job: does anything in column-after-column of carrier metadata exist anywhere in those two configs?

Pass one: exact match, and why it nearly fooled me

The first pass extracted every IP from the sheet and ran verbatim comparisons against both configs. Zero hits. Then it stepped up to /24 comparison and caught four gateway addresses -- call them 192.168.4.x and 192.168.15.x -- sitting inside two routed subnets. It traced the path (static route to transit VR to next-hop out the DIA interface), flagged a carrier-label mismatch on that interface, and re-verified with proper containment math. Final answer: two routes, four IPs, nothing else.

Clean, honest, and incomplete. Here is the trap, in code:

import ipaddress

# config_text = raw firewall config loaded as a string
gw        = ipaddress.ip_address("192.168.4.7")     # a gateway IP from the sheet
fw_object = ipaddress.ip_network("192.168.0.0/16")  # a broad object in the firewall

gw in fw_object               # True  -> the object contains this gateway
"192.168.4.7" in config_text  # False -> a string search never sees it

If your search is grep, or even a /24-against-/24 comparison, the broad object is invisible. It contains your target and never matches it.

Pass two: supernet overlap

Before writing the report I ran one more pass with a wider instruction: stop asking whether the sheet's values exist, and start asking every way the carrier's space could touch the firewall -- including objects broader than, adjacent to, or historically related to the literal values.

Quick disclosure, because it changes how you should read the result. I moved two variables at once on that pass: I widened the question and I switched models (Claude Fable 5 had just landed, so I ran it there), and it was a third pass over already-mapped ground. I never re-ran the earlier model with the same broad prompt, so I can't credit the delta to the model. The technique is the transferable part here, not the model choice.

The technique is supernet overlap: test containment in both directions, not just whether the sheet's range fits inside a config object.

sheet_net = ipaddress.ip_network("192.168.4.0/24")   # narrow range from the spreadsheet
fw_object = ipaddress.ip_network("192.168.0.0/16")   # broad object in the config

fw_object.subnet_of(sheet_net)     # False -> "is the config object inside my range?" (wrong question)
fw_object.supernet_of(sheet_net)   # True  -> "does the config object CONTAIN my range?" (the right one)

A /16 holds 256 of those /24s. It will never string-match a /24 from a spreadsheet, and it will never surface if you only test whether the sheet's range contains the object. You have to test the other direction: whether the object contains the sheet's range. That one direction cracked the case.

What the wide pass found

It opened by catching something I had missed entirely: the sheet listed six IPv6 NTE addresses, and no prior pass had checked IPv6 at all. It swept both configs for that range, confirmed clean, and closed the gap instead of leaving it silently open.

Then the supernet sweep ran across every config section -- address objects, groups, NAT rules, security rules, VPN, DHCP, DNS, logging targets, external lists -- and turned up four new traces:

1. Two address objects defining 192.168.0.0/16, labeled for a separate internal network, containing all six carrier gateways -- including two with no specific route at all.
2. That /16 object sat in an address group referenced by three active security rules. Policy still permits the carrier's space today, even where routing does not deliver traffic to it.
3. An address object named after the carrier's gateway subnet, prefixed Remove_, its value rewritten to a bogus range. Someone neutered it instead of deleting it -- proof the subnet was once live.
4. A group description: From [previous firewall vendor]: (Interface was [other carrier]-DIA). The objects were migrated wholesale from the old firewall, labels and all -- which explains the interface mismatch from pass one.

And one operational finding with teeth: two gateways, including the one for our 4 Gbps circuit, were permitted by policy but unrouted. Traffic to them follows the default route to the internet and dies. Either routing is broken or we are paying for dead circuits -- which is exactly the question we kicked back to the carrier.

For completeness it also string-matched every non-IP identifier (hostnames, CLLI codes, port AIDs, model numbers, service IDs, billing accounts) and chased false positives. A hit on ASE turned out to be a substring of crypto profile names, not a real reference.

Scoreboard

One environment, one case -- calibrate accordingly. But the broad pass found three times the traces, caught a verification gap the earlier work had left open, and turned "nothing else is here" into an actionable picture: what our gear can reach, what it merely tolerates in policy, and what exists only on the invoice.

The transferable rules

1. Exact-match proves almost nothing. A carrier inventory describes the carrier's side of the demarc; your config describes yours. The overlap is in subnet math, not strings.
2. Test supernet overlap, both directions. The dangerous object is the broad one created fifteen years ago. A /16 never string-matches a /24, and subnet_of alone won't catch it -- you need supernet_of too.
3. Trace objects into policy. An object that exists is trivia. An object referenced by three active allow rules is an attack surface and an audit finding.
4. Deprecated objects are evidence. The Remove_ object told us more about this circuit's history than any document we still have.
5. Re-run with a broader question before you trust "nothing." I would have shipped a confident, incomplete report if I had stopped at pass one. The fix was not a smarter tool -- it was a wider question on a fresh pass, a few minutes against four findings and a carrier dispute now backed by evidence. A newer model doesn't hurt, but changing the question is the load-bearing move.

We closed by requesting per-circuit utilization data from the carrier, because a config only tells you what should flow, never what does. But we went into that conversation knowing exactly which circuits our equipment can reach, which it tolerates, and which appear to live only on an invoice. Much stronger than "we couldn't find anything," which is where the week started.