Naut/plan.md

# Naut-Torrent — Architecture & Implementation Plan

## Context

This is a greenfield project (`Naut-Torrent/` is empty). The goal is a **maintainable, extensible BitTorrent client** capable of **saturating a 10 Gigabit Ethernet link (~1.25 GB/s) in both directions**. The implementation language is C-equivalent, so the plan is expressed in C terms (structs, function-pointer vtables, manual memory management, no exceptions/RAII).

Decisions confirmed with the user:

- **Platform:** Linux-only, optimized hard around **io_uring** (network *and* disk).
- **Protocol:** BitTorrent **v1 + v2 hybrid** (BEP-3 SHA-1 pieces *and* BEP-52 SHA-256 Merkle trees).
- **Workload:** Saturate **both download and upload**, with **MSE/PE encryption (RC4)** in the hot path.
- **Extensibility:** Clean module boundaries + control **RPC** + full **BEP-10 extension protocol** + a versioned **native plugin ABI** + an **embedded scripting** runtime.

The architecture is driven first by the throughput budget below — every structural decision (shared-nothing reactors, work offload, zero-copy buffers) exists to hit 1.25 GB/s on commodity multicore hardware.

---

## 1. Throughput Budget (why the architecture looks the way it does)

At **1.25 GB/s sustained**, the per-byte costs that must be parallelized:

| Work item | Cost (per core, conservative) | Cores @ 1.25 GB/s |
|---|---|---|
| MSE RC4 encrypt/decrypt | ~500 MB/s/core | ~2.5 per active direction |
| SHA-256 verify (v2, SHA-NI) | ~1.5 GB/s/core | ~1 (download) |
| SHA-1 verify (v1, SHA-NI) | ~2.5 GB/s/core | <1 |
| Network recv/send + framing | high, but DMA-bound | spread across reactors |
| Disk write/read (O_DIRECT NVMe) | 5–7 GB/s/device | I/O-bound, not CPU |
| memcpy | **eliminate** via registered buffers | ~0 |

**Conclusions that shape the design:**

1. RC4 is the surprise cost — encryption alone wants several cores. The hot path **must scale linearly across cores** (shared-nothing reactors, no global locks on the data path).
2. Hashing and crypto are **offloadable, embarrassingly parallel** units of work → dedicated **worker pools** fed by lock-free queues, never run inline on a reactor.
3. **memcpy must be designed out**: kernel→userspace via io_uring registered buffers, hash/crypto operate in place, disk writes come straight from the same buffers (O_DIRECT, page-aligned).
4. A ~8–16 core box with one Gen4 NVMe and a 10 GbE NIC should saturate the link; the software just has to not get in the way (syscalls, locks, copies, allocator churn).

---

## 2. High-Level Architecture: Shared-Nothing Thread-Per-Core Reactors + Offload Pools

```
              ┌──────────────────────── Control plane (1 thread) ───────────────────────┐
              │  RPC server · plugin host · script VM · session/torrent registry · stats │
              └───────┬─────────────────────────────────────────────────────────────────┘
                      │ message passing (MPSC command queues, no shared locks on hot path)
   ┌──────────────┬───┴──────────┬──────────────┐
   │ Reactor 0    │ Reactor 1    │ … Reactor N-1 │   (one pinned thread per I/O core)
   │ own io_uring │ own io_uring │ own io_uring  │   network + disk SQ/CQ, shared-nothing
   │ owns a shard │ owns a shard │ owns a shard  │   of peer connections (SO_REUSEPORT)
   └──────┬───────┴──────┬───────┴───────┬───────┘
          │ submit jobs   │               │  job results returned via io_uring msg_ring / eventfd
   ┌──────┴───────────────┴───────────────┴───────┐
   │  Hash worker pool        Crypto-assist pool   │  (pinned to remaining cores,
   │  (SHA-1 / SHA-256 / Merkle, MPMC job queue)   │   pull jobs, post completions)
   └───────────────────────────────────────────────┘
```

**Sharding model (the central decision):**

- **Connections are sharded across reactors.** A `SO_REUSEPORT` listen socket per reactor lets the kernel hash inbound peers across cores; outbound peers are assigned by hash. Each reactor **exclusively owns** its connections → all per-connection state (recv/send buffers, MSE keystream, message parser) is **lock-free**.
- **Torrent piece-state is shared** (peers of one torrent live on many reactors). It is guarded by a **per-torrent lock that protects only bookkeeping** (the picker's rarity counts, request map, have-bitfield). The lock is held for microseconds; all heavy work (decrypt, hash, disk) happens **outside** it. Rationale: the expensive bytes never touch a lock; only the tiny "which block next / mark block received" decisions do.
- **Heavy work is offloaded**, not run on reactors: completed piece buffers go to the **hash pool**; crypto can run inline on the reactor (cheap per-message) or be batched to a crypto-assist pool under load. Results return to the **owning reactor** via `io_uring` `msg_ring` (cross-ring wakeup) or eventfd.
- **Control plane is off the hot path entirely.** RPC, plugins, and scripts run on their own thread(s) and communicate with reactors only via per-reactor MPSC command queues + event fan-out. A misbehaving plugin/script can never stall the data path.

**Single-torrent scaling note:** because crypto/hash/disk for one torrent fan out to all cores via the pools, even a lone large torrent uses the whole machine. Only the per-torrent picker lock is single-point; if it ever becomes hot, upgrade the picker to the lock-free variant (§6, optimization).

**io_uring features exploited:** multishot `accept`/`recv`, registered buffers (`PROVIDE_BUFFERS` ring) for zero-copy recv, registered files, `SEND_ZC` (zero-copy send) for seeding, `SQPOLL` (kernel-side submission polling to cut syscalls under load), `msg_ring` for cross-thread completions, linked SQEs for read→hash chaining. NUMA-aware + hugepage buffer pools per reactor.

---

## 3. Module / Layer Breakdown

Layers are bottom-up; each is independently unit-testable and has explicit extension seams. Suggested repo layout under `src/`.

### Foundation

- **`platform/`** — the *only* code that touches the kernel. io_uring lifecycle (ring setup, SQE build, CQE reaping), socket setup (`SO_REUSEPORT`, `TCP_NODELAY`, large `SO_RCV/SNDBUF`, `TCP_FASTOPEN`), file ops (`O_DIRECT`, `fallocate`, `fadvise`), `eventfd`/`timerfd`, CPU pinning, hugepage/NUMA allocation, monotonic clock. This is the seam that keeps "Linux-only now" from becoming "Linux-only forever."
- **`core/`** — data-structure toolbox, zero dependencies:
  - **Buffer pool**: slab allocator of fixed-size (16 KiB block + piece-sized) page-aligned blocks, per-reactor freelists, refcounted so one buffer flows recv→decrypt→hash→disk without copy.
  - Intrusive doubly-linked lists, open-addressing hash maps, dynamic arrays, **bitfields with popcount** (have/interested/request maps), object pools, **SPSC + MPSC + MPMC lock-free queues**, per-thread lock-free logging ring, config parser, lock-free stats counters.
- **`crypto/`** — SHA-1, SHA-256 with **runtime SHA-NI dispatch** (scalar fallback), **Merkle tree** builder/verifier (v2 piece layers), **RC4** (MSE), **Diffie-Hellman** (MSE handshake, RFC 2631 768-bit group), CSPRNG. Designed for in-place operation on pooled buffers.

### Protocol

- **`bencode/`** — streaming, allocation-light parser producing **zero-copy slices** into the source buffer; encoder. Hardened against malicious input (depth/size limits) — primary fuzz target.
- **`metainfo/`** — `.torrent` parse for v1, v2, and hybrid; file tree + piece layers; **magnet URI** parsing; info-hash computation (both v1 SHA-1 and v2 SHA-256 truncated).
- **`tracker/`** — interface `tracker_backend` with built-in **HTTP(S)** (BEP-3/BEP-23 compact) and **UDP** (BEP-15) backends; announce scheduling, scrape, multi-tier (BEP-12). *Extension seam: register custom backends.*
- **`dht/`** — Kademlia routing table, `get_peers`/`announce_peer`/`find_node`, bootstrap, token management, BEP-32 IPv6, BEP-51 infohash indexing. Runs as its own UDP endpoint on a reactor.
- **`peer/`** — wire codec (handshake, all BEP-3 messages), **MSE/PE layer** (DH handshake, RC4 keystream, plaintext fallback), framing, and a **BEP-10 extension registry** (`extension_handler` vtable) with built-ins: **PEX (BEP-11)**, **ut_metadata (BEP-9)**, **LTEP** negotiation. *Extension seam: register custom extension message handlers.*

### Engine

- **`piece/`** — `piece_picker` interface (vtable) with built-in **rarest-first**, **sequential/streaming**, **priority**, and **endgame** strategies; per-peer **adaptive request pipelining** (depth auto-tuned from observed throughput × RTT, not a fixed window — essential for filling a 10 GbE BDP); block accounting; per-torrent lock holder. *Extension seam: pluggable picker strategy.*
- **`storage/`** — `storage_backend` interface with built-in **file backend**: maps (piece,offset)→(file,offset) across multi-file torrents, **O_DIRECT** aligned read/write via the reactor's io_uring, write coalescing, bounded read cache, configurable fsync policy, **fast-resume** (persisted bitfield + partial-piece state), preallocation. *Extension seam: register custom storage (memory, network, object-store).*
- **`verify/`** — hashing job dispatcher: full-piece SHA-1 (v1) and incremental **Merkle leaf/branch** hashing (v2) submitted to the hash pool; on completion marks piece valid/invalid on the owning reactor.
- **`scheduler/`** — token-bucket **rate limiting** (global + per-torrent + per-peer), **choking** algorithm (tit-for-tat + optimistic unchoke, BEP-3), connection budget, bandwidth fairness across torrents, super-seeding (BEP-16) option.
- **`session/`** — torrent lifecycle state machine, peer-set management, alert/event bus, aggregate stats. The single registry the control plane talks to.

### Surface (the "extensible" layer)

- **`rpc/`** — control protocol over a UNIX domain socket (and optional TCP): command/response + **streaming event subscription**. Versioned, length-prefixed binary frames (compact) with an optional JSON mode for tooling. A stable **command registry** so plugins can add RPC verbs. This is how UIs/automation drive the client.
- **`plugin/`** — **versioned C ABI**: host passes a `naut_host_api` struct of function pointers; the plugin (`.so`) exports `naut_plugin_register(host)`. Plugins hook the same vtables the core uses: `tracker_backend`, `storage_backend`, `piece_picker`, `extension_handler`, RPC commands, and the event bus. ABI version checked at load; plugins run on the control thread, never the data path.
- **`script/`** — embedded **Lua-style VM** bound to the event bus (`on_torrent_added`, `on_piece_complete`, **`on_file_complete`**, `on_torrent_finished`, `on_peer_connected`, `on_alert`) and a sandboxed control API. Runs on a dedicated thread with a bounded work queue; cannot block reactors.
  - **User feature — move files as they finish (libtorrent can't):** the engine seam is already built (Phase 3): `naut_download` fires `on_file_complete(file_index, path)` the moment a file's last covering piece verifies, and `naut_storage_relocate()` moves a single completed file safely (even mid-download). Phase 7 forwards the event to the `on_file_complete` script hook and exposes a `move_file` API; the relocate must be marshalled onto the owning reactor thread via the command queue (scripts run on their own thread).

---

## 4. Concurrency & Memory Model (invariants)

- **Data path is lock-free.** A connection is touched by exactly one reactor. The only data-path lock is the per-torrent picker mutex, held only for O(1)–O(log n) bookkeeping.
- **One buffer, one lifetime, zero copies.** A pooled, refcounted, page-aligned buffer is filled by `recv` (registered buffer), decrypted in place, hashed in place by a worker, then written by `SEND_ZC`/`O_DIRECT` write — same physical pages throughout.
- **Cross-thread communication is message passing**, never shared mutable state: MPSC command queues into reactors, MPMC job queue into the hash pool, `msg_ring`/eventfd for completions back out.
- **No allocation on the hot path.** All buffers, connection objects, request objects come from preallocated per-reactor pools sized from `peers × pipeline_depth × block_size`. Allocator is only touched at torrent add/remove.
- **Backpressure is explicit.** Bounded queues everywhere; when the hash pool or disk falls behind, reactors stop issuing `recv` (flow control) rather than growing memory unboundedly.

---

## 5. Build, Repo & Quality

```
Naut-Torrent/
├── src/{platform,core,crypto,bencode,metainfo,tracker,dht,peer,piece,storage,verify,scheduler,session,rpc,plugin,script}/
├── include/naut/         # public headers incl. versioned plugin ABI (naut_plugin.h)
├── apps/{nautd,nautctl}/  # daemon + CLI client over RPC
├── plugins/example/       # reference plugin against the ABI
├── tests/{unit,fuzz,integration,bench}/
└── build/                 # build-system outputs
```

- **Daemon/CLI split**: `nautd` (engine) + `nautctl` (thin RPC client). UIs are just RPC consumers — keeps the core headless and embeddable.
- **Tooling**: AddressSanitizer/UBSan/ThreadSanitizer builds; `perf`/eBPF-friendly; built-in per-core stats exported over RPC.

---

## 6. Implementation Phases (milestones, each independently demoable)

1. **Foundation** — `platform/` io_uring echo server + `core/` buffer pool, queues, bitfields, hash map. *Gate: loopback echo saturates a core with zero per-op allocation.*
2. **Crypto + parsers** — SHA-1/256 (+SHA-NI), Merkle, RC4, DH; `bencode/`, `metainfo/` (v1/v2/hybrid + magnet). *Gate: parse real torrents; hash throughput benchmark ≥1.5 GB/s/core.*
3. **Single-peer transfer** — `peer/` handshake + messages (plaintext), `piece/` basic picker, `storage/` file backend, `verify/`. Download a real torrent from one peer to disk, verified. *Gate: byte-correct file from a known seed.*
4. **Trackers + swarm** — HTTP/UDP `tracker/`, choking `scheduler/`, multi-peer, rarest-first, endgame. *Gate: download from a public/local swarm; interop with libtorrent/Transmission.*
5. **MSE + DHT + extensions** — RC4 MSE handshake, `dht/`, PEX, ut_metadata, magnet-only start. *Gate: magnet link with no trackers completes via DHT; encrypted peers work.*
6. **Scale to 10 GbE** — adaptive pipelining, `SEND_ZC`, registered buffers, `SQPOLL`, hash/crypto pools, NUMA/hugepages, O_DIRECT tuning. *Gate: two boxes (or two NICs) sustain ≥9.4 Gbit/s both directions.* Optional: lock-free picker if the per-torrent lock shows contention in `perf`.
7. **Extensibility surface** — `rpc/`, `plugin/` ABI + reference plugin, `script/` VM + event hooks, `nautctl`. *Gate: a sample plugin adds a storage backend and a script reacts to `on_torrent_finished`.*

---

## 7. Verification (end-to-end)

- **Microbenchmarks** (`tests/bench/`): hash GB/s/core, RC4 GB/s/core, bencode parse MB/s, buffer-pool alloc/free ns, picker decisions/s. Each has a regression threshold tied to the §1 budget.
- **Correctness**: unit tests per module; **fuzzers** (`tests/fuzz/`) on bencode, peer-message, tracker-response, and metainfo parsers (the attack surface); byte-for-byte file verification after download; resume-from-partial test.
- **Interop**: run against **libtorrent/qBittorrent** and **Transmission** as both seed and leech, plaintext and MSE; magnet + DHT-only bootstrap test against the public DHT.
- **Throughput (the headline test)**:
  1. Baseline the link with `iperf3` to confirm ~9.4 Gbit/s is achievable end-to-end.
  2. Seed a large (≥50 GB) torrent from box A, download on box B over the 10 GbE link (or two NICs on one box via loopback-to-NIC). Measure with the client's own per-core RPC stats + `nstat`/`ifstat`.
  3. Confirm sustained throughput is link-bound (not CPU/disk/lock-bound) via `perf top` — no single thread pinned at 100% on locks/copies, hash & crypto spread across pool cores.
  4. Reverse roles to validate upload saturation (`SEND_ZC` path).
- **Soak**: 24 h multi-torrent run under ASan-off release build; assert flat memory (pools, no leaks), no descriptor growth, stable throughput.

---

## 8. Key Risks & Mitigations

- **Per-torrent picker lock contention** at 10 GbE → keep heavy work outside the lock; escalate to lock-free claim-by-CAS picker (already an interface, so it's a swap not a rewrite).
- **RC4/MSE CPU cost** underestimated → crypto-assist pool + prefer plaintext when both peers allow; measure early in Phase 5.
- **io_uring portability lock-in** → all kernel calls behind `platform/`; a future epoll backend is a new file, not a refactor.
- **O_DIRECT alignment complexity** → enforce page-aligned pooled buffers from day one (Phase 1), so storage never has to bounce-buffer.
- **Hybrid v1/v2 data-model complexity** → model the piece/file/Merkle layout once in `metainfo/` + `storage/` and treat v1 as the degenerate single-layer case.