A pcap/pcapng reader that takes bytes, not filenames (#100)

[EONRaider]

Summary

Tier 4 (#106), item 3 of 3 — the largest and the only one adding public API surface. Closes#100.

What's included

• New netprotocols.pcap: read_captures(buffer: bytes | memoryview) -> Iterator[CapturedFrame] auto-detects classic pcap vs. pcapng from the magic bytes; read_pcap()/read_pcapng() are the same for a caller who already knows the format.
• CapturedFrame: a NamedTuple (timestamp: int, data: bytes) — mirrors the existing FlowKey precedent for "a derived value, not a wire format, is a NamedTuple". Every timestamp format (classic pcap µs/ns from its magic; pcapng's per-interface if_tsresol-scaled Enhanced Packet Block timestamp) is normalized to nanoseconds since the Unix epoch at read time. A pcapng Simple Packet Block carries no timestamp at all (the block format has none) — those frames report 0, not a guess.
• pcapng block coverage: exactly the four types frames can come from — Section Header (SHB), Interface Description (IDB, read only for if_tsresol), Enhanced Packet (EPB), Simple Packet (SPB). Every other block type (Interface Statistics, Name Resolution, Decryption Secrets, vendor/custom) is skipped wholesale. Multiple concatenated sections in one buffer are supported — each section's byte order and interface list are independent, per spec.
• New MalformedCaptureError (ProtocolError family, in utils/exceptions.py): a malformed/truncated capture raises this rather than a bare ValueError. No lax=True mode — a corrupt container is a different failure shape than a malformed header inside an already-extracted frame (decode_frame's lax is for the latter).
• Eager detection, lazy frames: format auto-detection raises immediately on a bad/too-short buffer; producing frames is a generator, so a malformed record/block raises only once iteration reaches it, and a huge capture is never forced into a list nobody asked for. Documented explicitly since it's a deliberate departure from decode_frame's immediate-raise contract.
• tests/conftest.py drops its private classic-pcap reader (acceptance criterion 4) — pcap_frames() is now a thin adapter over the shipped read_pcap(), and all ~10 dependent test files (test_corpus.py, test_dhcp.py, test_dns.py, test_gre.py, test_igmp.py, test_ipv6_ext.py, plus test_checksum.py/test_docs.py/test_fuzz.py/test_walk.py which only needed corpus_frames() and required no changes) are migrated onto it. tests/test_pcap.py keeps its own independent reference reader for its cross-check — deliberately never importing the module under test, the same "standalone, so a shared bug can't cancel itself out" precedent scripts/benchmark.py and scripts/check_fixtures.py already set.
• pcapng test fixtures are hand-built Python, not binary files (per the roadmap's guidance): the real corpus has zero pcapng captures, and exotic if_tsresol values essentially never occur in real traffic, so tests/test_pcap.py includes small little-/big-endian block builders instead of adding non-real binary fixtures to tests/fixtures/, keeping that directory's real-capture-only guarantee (MANIFEST.md) intact. Classic-pcap coverage reuses the real corpus as normal.
• docs/CLAIMS.md: 3.1 already referenced the memoryview-vs-bytes benchmark as motivation for this issue — see the note below on why that specific number needed correcting, not just citing.
• README.md: new "Reading captures" section (non-comparative, no README/CLAIMS embargo implications). ARCHITECTURE.md's layout map gets a one-line entry for the new module.
• CHANGELOG.md entry under ## [Unreleased].

A design idea tried and reverted — read before reviewing pcap.py

docs/CLAIMS.md 5.8 forward-referenced this issue with an unverified "1.8x" figure for slicing frames lazily out of a memoryview over the whole capture buffer (zero-copy) instead of copying the buffer once up front and slicing plain bytes per frame. I implemented the memoryview-passthrough version first and could not reproduce any speedup — measured across synthetic captures from ~6MB to ~140MB, it was 0.91x–0.98x (never faster, sometimes slower). The reason generalizes issue #88's own already-documented finding (a memoryview costs more to build than the copy it saves for one small frame) to many small frames: a real capture is overwhelmingly small frames, not one giant one, and the per-frame memoryview slice overhead outweighs the one-time copy it was meant to avoid.

Reverted to: copy the input once up front (bytes(buffer)), return plain bytes per frame regardless of whether bytes or memoryview was passed in. docs/CLAIMS.md 5.8 is corrected with the real measurement rather than left with the unverified forward-reference — flagging this prominently since it's a case of the PR not matching the issue's own pre-written motivating text, on purpose, with numbers to back it.

Verification

• uv run ruff check and uv run ruff format --check are clean
• uv run mypy is clean (strict)
• uv run pytest passes locally — full suite green, netprotocols/pcap.py and the migrated test files at 100%/full coverage (--cov=netprotocols reports 99.95% overall, gate is 98%)
• CHANGELOG.md has an entry under ## [Unreleased]
• uv run --group bench python scripts/benchmark.py --check --threshold 15 — within threshold (this PR doesn't touch the decode hot path at all; one transient run on the shared sandbox briefly read -15.5%, three clean re-runs afterward landed at -11.6% to -12.8%, consistent with the benchmark script's own documented "a shared runner can lose a slice of CPU to a neighbour" caveat, not a real regression)
• tests/test_pcap.py's classic-pcap tests cross-check every real corpus frame (97 frames, 17 files) against an independent reference reader — zero mismatches
• Manually verified the README's new read_captures() example runs end-to-end against a real fixture

Notes

Q7 (whether to attempt #100 this tier at all) was resolved "yes" up front, along with the rest of the Tier 4 design questions, before any code was written.

Closes#100.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MGDTcK51CWcy6PrNetN213

---

Generated by Claude Code