98 lines
4.4 KiB
Markdown
98 lines
4.4 KiB
Markdown
# torrent-tracker Protocol Plan
|
|
|
|
Goal: build a tracker-side library that can serve the tracker protocols and
|
|
extensions seen in real torrents, with protocol parsing separated from swarm
|
|
storage and serving policy.
|
|
|
|
## Implemented Foundation
|
|
|
|
- BEP-3 HTTP announce parameters: `info_hash`, `peer_id`, `port`, `uploaded`,
|
|
`downloaded`, `left`, `compact`, `no_peer_id`, `event`, `numwant`, `key`,
|
|
`ip`, and `trackerid`.
|
|
- BEP-23 compact peer responses: IPv4 `peers` and BEP-7-style IPv6 `peers6`.
|
|
- BEP-48 HTTP scrape parsing and bencoded scrape responses.
|
|
- BEP-15 UDP connect, announce, scrape, and error packet handling.
|
|
- BEP-41 UDP announce extension parser, including concatenated URLData chunks.
|
|
- Client-side HTTP announce/scrape query builders and bencoded tracker response
|
|
parsers.
|
|
- Client-side UDP connect/announce/scrape packet builders and response parsers
|
|
with transaction-id validation.
|
|
- BEP-5 DHT/KRPC message builders and parser for `ping`, `find_node`,
|
|
`get_peers`, `announce_peer`, response, and error packets.
|
|
- BEP-32 IPv6 DHT compact node support through `nodes6` plus `want` flags.
|
|
- In-memory swarm table keyed by 20-byte tracker infohash.
|
|
- Announce handling for insert/update/stop, source-address-derived endpoints,
|
|
seed/leecher counts, completed-download scrape counts, no-self filtering,
|
|
randomized peer selection, `numwant` clamping, and stale peer pruning.
|
|
|
|
## Tracker-Side Roadmap
|
|
|
|
## Client-Side Roadmap
|
|
|
|
1. DHT node/session manager:
|
|
Add a UDP event loop, node ID generation, transaction table, timeout/retry
|
|
logic, bootstrap nodes, and request dispatch for KRPC packets.
|
|
|
|
2. DHT routing table:
|
|
Implement BEP-5 k-buckets, XOR distance sorting, good/questionable/bad node
|
|
state, bucket refresh, and persistence of known nodes between runs.
|
|
|
|
3. DHT peer discovery:
|
|
Implement iterative `get_peers` lookup, token storage, `announce_peer`, and
|
|
integration with the tracker client result format so callers can consume
|
|
central tracker and DHT peers through one path.
|
|
|
|
4. Tracker session manager:
|
|
Add a small client state machine that stores UDP connection IDs until expiry,
|
|
schedules announces at `interval`, handles started/completed/stopped events,
|
|
retries UDP with exponential backoff, and rotates across announce-list tiers.
|
|
|
|
5. Transport adapters:
|
|
Wrap the protocol helpers with optional HTTP(S) and UDP socket code. Keep TLS
|
|
outside the core ABI, but provide a CLI/harness path that exercises real
|
|
network announces through the C helpers.
|
|
|
|
6. Magnet and v2 metadata:
|
|
Keep accepting 20-byte tracker hashes: v1 SHA-1 infohash and BEP-52 truncated
|
|
SHA-256 for v2/hybrid torrents. Add metadata utilities or bindings so callers
|
|
can compute the right tracker hash from metainfo without duplicating logic.
|
|
|
|
7. Client interop tests:
|
|
Compare C helper output against libtorrent/tracker_probe behavior for HTTP,
|
|
UDP, compact IPv4, `peers6`, scrape, failure responses, BEP-41 URLData, and
|
|
DHT KRPC packets observed from real nodes.
|
|
|
|
## Tracker-Side Roadmap
|
|
|
|
1. Network daemons:
|
|
Add HTTP/HTTPS and UDP listeners around the protocol core. HTTPS should live
|
|
behind a TLS terminator initially; the library API should not require a TLS
|
|
dependency.
|
|
|
|
2. IPv6 and multi-homed behavior:
|
|
The store accepts IPv4 and IPv6 endpoints and prefers source addresses over
|
|
user-supplied `ip`. Add daemon-level tests for announcing the same peer over
|
|
multiple listen interfaces and returning family-appropriate UDP responses.
|
|
|
|
3. Tracker policy:
|
|
Add hooks for private torrents, whitelist/auth tokens in HTTP query strings
|
|
and BEP-41 UDP URLData, per-swarm limits, rate limits, and abuse controls.
|
|
|
|
4. Response selection:
|
|
Add stronger selection policy controls: per-family caps, seed/leecher mix,
|
|
deterministic test hooks for sampling, and configurable `tracker id`
|
|
generation/validation.
|
|
|
|
5. Persistence and admin:
|
|
Add optional durable storage or snapshot/restore for long-running trackers,
|
|
metrics export, registered-torrent management, and scrape-cache policy.
|
|
|
|
6. Compatibility matrix:
|
|
Build interop tests against libtorrent, Transmission, qBittorrent, aria2,
|
|
rtorrent, and opentracker for HTTP, UDP, compact, IPv6, scrape, and private
|
|
torrent behavior.
|
|
|
|
7. Optional adjacent discovery:
|
|
DHT, PEX, and local peer discovery are peer-discovery mechanisms rather than
|
|
central tracker protocols. Keep them as separate modules if this project grows
|
|
into a full discovery stack.
|