Skip to content

tn3w/ASNDB

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

18 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ASNDB

Release License

Compact ASN/BGP lookup DB → single mmap'd binary file.

8 µs load, 22 µs IP lookup
mini 12 / tiny 16 / lite 24 / full 52 MB
one-shot build → 13 artifacts

Build

cargo build --release
./target/release/asndb build --out out

First run fetches ~1 GB of sources to $XDG_CACHE_HOME/asndb-cache/. Re-runs reuse the cache. Peak RAM ~4 GB.

Set PEERINGDB_API_KEY to authenticate PeeringDB requests (avoids anonymous rate limits/403). Optional but recommended for CI.

CLI

asndb build [--out DIR]
asndb stats
asndb ip 8.8.8.8
asndb ip 2606:4700::1
asndb asn 13335
asndb prefixes 13335
asndb neighbors 13335

Artifacts

One build writes every artifact:

file contents
asndb-mini.bin IP→ASN + name
asndb-tiny.bin +company, website, country, kind, info_type, RIR — everything needed for IP enrichment
asndb-lite.bin +org, RIR, alloc, HQ, kind, traffic, scope, flags, RPKI counts, address counts
asndb-full.bin +per-ASN prefixes, neighbors w/ relationship, contact info
ip2asn-v4.tsv / ip2asn-v6.tsv start end asn cc name
ip2asn-v4.csv / ip2asn-v6.csv start,end,asn intervals
asn-info.csv per-ASN metadata (bool→1/empty, zero floats empty)
asn-prefixes.csv asn,prefix,rpki
asn-neighbors.csv asn,neighbor,rel
asn-company.tsv asn<TAB>company sorted by ASN (CAIDA org name, e.g. 13335\tCloudflare, Inc.)
asn-provider.tsv asn<TAB>provider cleaned brand (e.g. 13335\tCloudflare, 24940\tHetzner)
asndb.ndjson one JSON object per ASN; default-valued keys omitted

Sources

  • RIPE asn-names: handle + country
  • RIPE/APNIC/AFRINIC whois: abuse email, phone, address
  • NRO delegated-stats: RIR + allocation date
  • CAIDA as-org2info: org name
  • CAIDA as-rel2: provider/customer/peer
  • PeeringDB (net, org, netixlan): type, traffic, scope, policy, flags, HQ, IX count
  • RPKI VRPs: per-prefix valid/invalid/unknown
  • RIPE RIS RRC00 RIB: prefix→ASN + BGP adjacencies

Binary format

Header (104B)            magic, flavor, counts, section offsets
AsnRec[]   by asn        flavor-sized record; first u32 = ASN
                         mini  8B / tiny 24B / lite 88B / full 152B
Seg4[]     by start       8B: (start_ip,    asn_idx)
Seg6[]     by start      20B: (start_ip128, asn_idx)  big-endian start
Pfx4[]     per-ASN        8B: (start, len, rpki, _pad)        (full only)
Pfx6[]     per-ASN       18B: (start128, len, rpki)           (full only)
Neigh[]    per-ASN       u32 packed: asn_idx<<2 | rel         (full only)
StrPool                  deduped UTF-8: u32 len + bytes; offset 0 = empty

TinyRec (24B): asn, name_off, company_off, website_off, cc, kind, info_type, rir — provider is derived (provider(name, company)), not stored.

  • Sweep-segmented IP space: one partition_point binary search per lookup.
  • ASN indices (not raw ASNs) in prefix/neighbor tables.
  • Packed neighbor+rel: 2 bits carry the relationship.
  • mmap + #[repr(C)] Pod: zero-copy load.

Python parser

Single file asndb.py, stdlib only (mmap, struct, bisect, ipaddress):

from asndb import Db

db = Db("asndb-full.bin")
db.lookup("8.8.8.8")            # → 15169
db.info(13335)                  # → dict (fields vary by flavor)
db.prefixes(13335)              # → ([(v4_pfx, rpki)], [(v6_pfx, rpki)])
db.neighbors(13335)             # → [(asn, rel), ...]

Each info() dict carries derived fields:

  • handle — RPSL as-name only (CLOUDFLARENET)
  • company — CAIDA org or parsed tail (Cloudflare, Inc.)
  • provider — cleaned brand (Cloudflare)

CLI:

python3 asndb.py asndb-full.bin ip 8.8.8.8
python3 asndb.py asndb-full.bin asn 13335

Provider derivation

provider.py / src/provider.rs — heuristic that maps (as-name, org) → short brand name. Needs only two inputs per ASN.

CLOUDFLARENET - Cloudflare, Inc.   →  Cloudflare
HETZNER-AS Hetzner Online GmbH     →  Hetzner
DTAG Deutsche Telekom AG           →  Deutsche Telekom
SOFTLAYER - IBM Cloud              →  IBM
ATT-INTERNET4 - AT&T Enterprises   →  AT&T

Algorithm:

  1. From company: tokenize, drop trailing legal/generic words (Inc, LLC, GmbH, Networks, ...), drop leading articles, strip .com/.net, smart-case.
  2. From name: split first token (on " - " or space), strip RIR suffix (-AS/-AP/-NET), strip trailing net/com/tel if all-caps, smart-case.
  3. Prefer company-derived; if one is a clean word-prefix of the other, keep the shorter.
  4. Smart-case title-cases only alpha ALL-CAPS tokens of length ≥5 → preserves IBM/OVH/NHS/AT&T/M247/DigitalOcean.

CLI:

python3 provider.py asndb-lite.bin           # dump asn<TAB>provider
python3 provider.py asndb-lite.bin 13335     # → Cloudflare

Works on all three flavors. Methods returning prefix/neighbor data require full.

Releases

Daily GitHub release via .github/workflows/build.yml. Latest:

BASE=https://github.com/tn3w/ASNDB/releases/latest/download
curl -fsSLO $BASE/asndb-full.bin
curl -fsSLO $BASE/ip2asn-v4.csv

License

Apache-2.0. See LICENSE.

About

Compact ASN/BGP lookup DB → single mmap'd binary file.

Topics

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors