Initial commit: multi-peer torrent download engine

Reactor/loop-pool engine with TCP/µTP/MSE transports, per-connection
pipelining, priority-driven piece selection with endgame, and the Python
FFI test harness.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
ookami125 2026-06-21 23:12:32 -04:00
commit d8208685a2
55 changed files with 9989 additions and 0 deletions

32
interop/Dockerfile Normal file
View file

@ -0,0 +1,32 @@
FROM ubuntu:24.04
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends \
aria2 \
build-essential \
ca-certificates \
cmake \
deluge-console \
deluged \
procps \
python3 \
python3-libtorrent \
qbittorrent-nox \
rtorrent \
transmission-cli \
transmission-daemon \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /work
COPY CMakeLists.txt /work/CMakeLists.txt
COPY include /work/include
COPY src /work/src
COPY harness /work/harness
COPY interop /work/interop
RUN cmake -S /work -B /work/build -DCMAKE_BUILD_TYPE=Release -DPEER_NATIVE=OFF \
&& cmake --build /work/build
ENV PYTHONPATH=/work/harness:/work/interop

91
interop/README.md Normal file
View file

@ -0,0 +1,91 @@
# Offline Client Interop
This harness starts several seed-only BitTorrent clients on a Docker Compose
network with `internal: true`, generates one private test torrent,
mounts the same data into every seeder, and drives this engine against each
client independently.
It is meant to catch interoperability failures: bad handshakes, encryption/uTP
negotiation mistakes, malformed requests, or behavior that makes common clients
reject us. It is not a public-swarm or tracker test.
## Clients
Default matrix:
- `libtorrent-plain-tcp`
- `libtorrent-mse-tcp` with RC4 required
- `libtorrent-utp`
- `libtorrent-utp-mse`
- `transmission-tcp`
- `transmission-utp`
- `transmission-mse-tcp` with encryption required
- `aria2-tcp`
- `qbittorrent-tcp`
- `deluge-tcp`
- `rtorrent-tcp`
All clients run with DHT, PEX, local peer discovery, UPnP, and NAT-PMP disabled.
The fixture torrent includes the deterministic dummy announce URL
`http://fixture:9/announce` because rTorrent rejects trackerless torrents, but
the runner still injects peers directly. The Compose network is internal-only,
so containers cannot route to the internet during the test run.
## Run
From the repository root:
```sh
docker compose -f interop/docker-compose.yml up --build \
--abort-on-container-exit --exit-code-from runner
```
Useful overrides:
```sh
FIXTURE_SIZE=128M TEST_TIMEOUT=180 \
docker compose -f interop/docker-compose.yml up --build \
--abort-on-container-exit --exit-code-from runner
```
Results are written to `interop/results/results.json`.
Clean generated containers, networks, and the fixture volume:
```sh
docker compose -f interop/docker-compose.yml down -v
```
## Run One Client
The runner supports `--only`, but Compose still starts all default dependencies.
For focused debugging, run a shell after the stack is up:
```sh
docker compose -f interop/docker-compose.yml run --rm runner \
python3 /work/interop/run_matrix.py \
--fixture /fixture \
--clients /work/interop/clients.json \
--results /results/one.json \
--only transmission-tcp
```
## Adding Clients
Add a seeder service to `docker-compose.yml`, disable all discovery/tracker/NAT
features for that client, expose it only on `torrent_lab`, then add an entry to
`clients.json`:
```json
{
"name": "new-client-tcp",
"host": "seed-new-client",
"port": 6900,
"utp": 0,
"encryption": 0,
"fallback": 0
}
```
The engine currently needs a numeric IP, so the runner resolves the Compose DNS
name to IPv4 before calling `engine_add_peer`.

90
interop/clients.json Normal file
View file

@ -0,0 +1,90 @@
[
{
"name": "libtorrent-plain-tcp",
"host": "seed-libtorrent-plain",
"port": 6881,
"utp": 0,
"encryption": 0,
"fallback": 0
},
{
"name": "libtorrent-mse-tcp",
"host": "seed-libtorrent-mse",
"port": 6882,
"utp": 0,
"encryption": 2,
"fallback": 0
},
{
"name": "libtorrent-utp",
"host": "seed-libtorrent-utp",
"port": 6883,
"utp": 1,
"encryption": 0,
"fallback": 0
},
{
"name": "libtorrent-utp-mse",
"host": "seed-libtorrent-utp-mse",
"port": 6884,
"utp": 1,
"encryption": 1,
"fallback": 0
},
{
"name": "transmission-tcp",
"host": "seed-transmission",
"port": 6891,
"utp": 0,
"encryption": 0,
"fallback": 0
},
{
"name": "transmission-utp",
"host": "seed-transmission-utp",
"port": 6896,
"utp": 1,
"encryption": 0,
"fallback": 0
},
{
"name": "transmission-mse-tcp",
"host": "seed-transmission-mse",
"port": 6897,
"utp": 0,
"encryption": 2,
"fallback": 0
},
{
"name": "aria2-tcp",
"host": "seed-aria2",
"port": 6892,
"utp": 0,
"encryption": 0,
"fallback": 0
},
{
"name": "qbittorrent-tcp",
"host": "seed-qbittorrent",
"port": 6893,
"utp": 0,
"encryption": 0,
"fallback": 0
},
{
"name": "deluge-tcp",
"host": "seed-deluge",
"port": 6894,
"utp": 0,
"encryption": 0,
"fallback": 0
},
{
"name": "rtorrent-tcp",
"host": "seed-rtorrent",
"port": 6895,
"utp": 0,
"encryption": 0,
"fallback": 0
}
]

63
interop/common.py Normal file
View file

@ -0,0 +1,63 @@
from __future__ import annotations
import hashlib
import json
import os
import socket
import time
import sys
ROOT = os.environ.get("TORRENT_PEER_ROOT", "/work")
sys.path.insert(0, os.path.join(ROOT, "harness"))
from torrent_meta import Metadata, load_metadata # noqa: E402
def parse_size(text: str) -> int:
s = text.strip().upper()
mult = 1
if s[-1:] in ("K", "M", "G"):
mult = {"K": 1024, "M": 1024**2, "G": 1024**3}[s[-1]]
s = s[:-1]
return int(s) * mult
def file_sha1(path: str) -> str:
h = hashlib.sha1()
with open(path, "rb") as f:
for chunk in iter(lambda: f.read(1024 * 1024), b""):
h.update(chunk)
return h.hexdigest()
def write_manifest(path: str, **items) -> None:
tmp = path + ".tmp"
with open(tmp, "w", encoding="utf-8") as f:
json.dump(items, f, indent=2, sort_keys=True)
f.write("\n")
os.replace(tmp, path)
def wait_for_tcp(host: str, port: int, timeout: float = 30.0) -> None:
deadline = time.time() + timeout
last_error = None
while time.time() < deadline:
try:
with socket.create_connection((host, port), timeout=1.0):
return
except OSError as exc:
last_error = exc
time.sleep(0.25)
raise TimeoutError(f"timed out waiting for {host}:{port}: {last_error}")
def resolve_ipv4(host: str) -> str:
infos = socket.getaddrinfo(host, None, socket.AF_INET, socket.SOCK_STREAM)
if not infos:
raise OSError(f"no IPv4 address for {host}")
return infos[0][4][0]
def touch_ready(path: str = "/tmp/seed-ready") -> None:
with open(path, "w", encoding="utf-8") as f:
f.write("ready\n")

379
interop/docker-compose.yml Normal file
View file

@ -0,0 +1,379 @@
name: torrent-peer-interop
services:
fixture:
build:
context: ..
dockerfile: interop/Dockerfile
command:
- /bin/sh
- -c
- >
python3 /work/interop/make_fixture.py --out /fixture --size ${FIXTURE_SIZE:-32M}
&& touch /tmp/fixture-ready
&& tail -f /dev/null
volumes:
- fixture:/fixture
networks:
- torrent_lab
healthcheck:
test: ["CMD", "test", "-f", "/tmp/fixture-ready"]
interval: 2s
timeout: 1s
retries: 30
seed-libtorrent-plain:
build:
context: ..
dockerfile: interop/Dockerfile
depends_on:
fixture:
condition: service_healthy
command:
- python3
- /work/interop/seed_libtorrent.py
- --torrent
- /fixture/test.torrent
- --data
- /fixture
- --port
- "6881"
- --mode
- plain
volumes:
- fixture:/fixture:ro
networks:
- torrent_lab
healthcheck:
test: ["CMD", "test", "-f", "/tmp/seed-ready"]
interval: 2s
timeout: 1s
retries: 30
seed-libtorrent-mse:
build:
context: ..
dockerfile: interop/Dockerfile
depends_on:
fixture:
condition: service_healthy
command:
- python3
- /work/interop/seed_libtorrent.py
- --torrent
- /fixture/test.torrent
- --data
- /fixture
- --port
- "6882"
- --mode
- mse
volumes:
- fixture:/fixture:ro
networks:
- torrent_lab
healthcheck:
test: ["CMD", "test", "-f", "/tmp/seed-ready"]
interval: 2s
timeout: 1s
retries: 30
seed-libtorrent-utp:
build:
context: ..
dockerfile: interop/Dockerfile
depends_on:
fixture:
condition: service_healthy
command:
- python3
- /work/interop/seed_libtorrent.py
- --torrent
- /fixture/test.torrent
- --data
- /fixture
- --port
- "6883"
- --mode
- utp
volumes:
- fixture:/fixture:ro
networks:
- torrent_lab
healthcheck:
test: ["CMD", "test", "-f", "/tmp/seed-ready"]
interval: 2s
timeout: 1s
retries: 30
seed-libtorrent-utp-mse:
build:
context: ..
dockerfile: interop/Dockerfile
depends_on:
fixture:
condition: service_healthy
command:
- python3
- /work/interop/seed_libtorrent.py
- --torrent
- /fixture/test.torrent
- --data
- /fixture
- --port
- "6884"
- --mode
- utp-mse
volumes:
- fixture:/fixture:ro
networks:
- torrent_lab
healthcheck:
test: ["CMD", "test", "-f", "/tmp/seed-ready"]
interval: 2s
timeout: 1s
retries: 30
seed-transmission:
build:
context: ..
dockerfile: interop/Dockerfile
depends_on:
fixture:
condition: service_healthy
command:
- python3
- /work/interop/seed_transmission.py
- --torrent
- /fixture/test.torrent
- --data
- /fixture
- --peer-port
- "6891"
- --rpc-port
- "9091"
volumes:
- fixture:/fixture:ro
networks:
- torrent_lab
healthcheck:
test: ["CMD", "test", "-f", "/tmp/seed-ready"]
interval: 2s
timeout: 1s
retries: 45
seed-transmission-utp:
build:
context: ..
dockerfile: interop/Dockerfile
depends_on:
fixture:
condition: service_healthy
command:
- python3
- /work/interop/seed_transmission.py
- --torrent
- /fixture/test.torrent
- --data
- /fixture
- --peer-port
- "6896"
- --rpc-port
- "9092"
- --utp
volumes:
- fixture:/fixture:ro
networks:
- torrent_lab
healthcheck:
test: ["CMD", "test", "-f", "/tmp/seed-ready"]
interval: 2s
timeout: 1s
retries: 45
seed-transmission-mse:
build:
context: ..
dockerfile: interop/Dockerfile
depends_on:
fixture:
condition: service_healthy
command:
- python3
- /work/interop/seed_transmission.py
- --torrent
- /fixture/test.torrent
- --data
- /fixture
- --peer-port
- "6897"
- --rpc-port
- "9093"
- --encryption
- required
volumes:
- fixture:/fixture:ro
networks:
- torrent_lab
healthcheck:
test: ["CMD", "test", "-f", "/tmp/seed-ready"]
interval: 2s
timeout: 1s
retries: 45
seed-aria2:
build:
context: ..
dockerfile: interop/Dockerfile
depends_on:
fixture:
condition: service_healthy
command:
- python3
- /work/interop/seed_aria2.py
- --torrent
- /fixture/test.torrent
- --data
- /fixture
- --port
- "6892"
volumes:
- fixture:/fixture:ro
networks:
- torrent_lab
healthcheck:
test: ["CMD", "test", "-f", "/tmp/seed-ready"]
interval: 2s
timeout: 1s
retries: 45
seed-qbittorrent:
build:
context: ..
dockerfile: interop/Dockerfile
depends_on:
fixture:
condition: service_healthy
command:
- python3
- /work/interop/seed_qbittorrent.py
- --torrent
- /fixture/test.torrent
- --data
- /fixture
- --port
- "6893"
volumes:
- fixture:/fixture:ro
networks:
- torrent_lab
healthcheck:
test: ["CMD", "test", "-f", "/tmp/seed-ready"]
interval: 2s
timeout: 1s
retries: 45
seed-deluge:
build:
context: ..
dockerfile: interop/Dockerfile
depends_on:
fixture:
condition: service_healthy
command:
- python3
- /work/interop/seed_deluge.py
- --torrent
- /fixture/test.torrent
- --data
- /fixture
- --port
- "6894"
- --daemon-port
- "58846"
volumes:
- fixture:/fixture:ro
networks:
- torrent_lab
healthcheck:
test: ["CMD", "test", "-f", "/tmp/seed-ready"]
interval: 2s
timeout: 1s
retries: 45
seed-rtorrent:
build:
context: ..
dockerfile: interop/Dockerfile
depends_on:
fixture:
condition: service_healthy
command:
- python3
- /work/interop/seed_rtorrent.py
- --torrent
- /fixture/test.torrent
- --data
- /fixture
- --port
- "6895"
volumes:
- fixture:/fixture:ro
networks:
- torrent_lab
healthcheck:
test: ["CMD", "test", "-f", "/tmp/seed-ready"]
interval: 2s
timeout: 1s
retries: 45
runner:
build:
context: ..
dockerfile: interop/Dockerfile
depends_on:
seed-libtorrent-plain:
condition: service_healthy
seed-libtorrent-mse:
condition: service_healthy
seed-libtorrent-utp:
condition: service_healthy
seed-libtorrent-utp-mse:
condition: service_healthy
seed-transmission:
condition: service_healthy
seed-transmission-utp:
condition: service_healthy
seed-transmission-mse:
condition: service_healthy
seed-aria2:
condition: service_healthy
seed-qbittorrent:
condition: service_healthy
seed-deluge:
condition: service_healthy
seed-rtorrent:
condition: service_healthy
command:
- python3
- /work/interop/run_matrix.py
- --fixture
- /fixture
- --clients
- /work/interop/clients.json
- --results
- /results/results.json
- --timeout
- ${TEST_TIMEOUT:-90}
volumes:
- fixture:/fixture:ro
- ./results:/results
networks:
- torrent_lab
volumes:
fixture:
networks:
torrent_lab:
internal: true

76
interop/make_fixture.py Normal file
View file

@ -0,0 +1,76 @@
from __future__ import annotations
import argparse
import hashlib
import os
import libtorrent as lt
from common import file_sha1, parse_size, write_manifest
def deterministic_bytes(offset: int, size: int) -> bytes:
out = bytearray()
counter = offset // 32
while len(out) < size:
out.extend(hashlib.sha256(f"torrent-peer-interop:{counter}".encode()).digest())
counter += 1
return bytes(out[:size])
def write_data(path: str, size: int) -> None:
with open(path, "wb") as f:
off = 0
while off < size:
n = min(1024 * 1024, size - off)
f.write(deterministic_bytes(off, n))
off += n
def make_torrent(root: str, size: int, piece_size: int) -> str:
os.makedirs(root, exist_ok=True)
data_path = os.path.join(root, "data.bin")
torrent_path = os.path.join(root, "test.torrent")
write_data(data_path, size)
fs = lt.file_storage()
lt.add_files(fs, data_path)
t = lt.create_torrent(fs, piece_size=piece_size)
t.set_priv(True)
t.add_tracker("http://fixture:9/announce")
lt.set_piece_hashes(t, root)
with open(torrent_path, "wb") as f:
f.write(lt.bencode(t.generate()))
return torrent_path
def main() -> int:
ap = argparse.ArgumentParser(description="Create the offline interop fixture.")
ap.add_argument("--out", required=True)
ap.add_argument("--size", default="32M")
ap.add_argument("--piece-size", default="256K")
args = ap.parse_args()
size = parse_size(args.size)
piece_size = parse_size(args.piece_size)
torrent_path = make_torrent(args.out, size, piece_size)
data_path = os.path.join(args.out, "data.bin")
ti = lt.torrent_info(torrent_path)
write_manifest(
os.path.join(args.out, "manifest.json"),
name=ti.name(),
size=size,
piece_size=piece_size,
num_pieces=ti.num_pieces(),
sha1=file_sha1(data_path),
announce="http://fixture:9/announce",
torrent=os.path.basename(torrent_path),
data=os.path.basename(data_path),
)
print(f"fixture ready: {size} bytes, {ti.num_pieces()} pieces", flush=True)
return 0
if __name__ == "__main__":
raise SystemExit(main())

206
interop/run_matrix.py Normal file
View file

@ -0,0 +1,206 @@
from __future__ import annotations
import argparse
import hashlib
import json
import os
import secrets
import sys
import time
import traceback
ROOT = "/work"
sys.path.insert(0, os.path.join(ROOT, "harness"))
from common import load_metadata, resolve_ipv4, write_manifest # noqa: E402
from engine_ffi import Engine, EngineConfig, ERROR_NAMES, STATE_ERROR, STATE_NAMES # noqa: E402
LIB = "/work/build/libtorrentpeer.so"
def make_peer_id() -> bytes:
return b"-PC0001-" + secrets.token_bytes(12)
def load_clients(path: str, only: set[str] | None) -> list[dict]:
with open(path, "r", encoding="utf-8") as f:
clients = json.load(f)
if only:
clients = [c for c in clients if c["name"] in only]
if not clients:
raise ValueError("no clients selected")
return clients
def download_from_client(meta, client: dict, timeout: float, lib_path: str) -> dict:
ip = resolve_ipv4(client["host"])
port = int(client["port"])
cfg = EngineConfig(
loop_count=1,
slots_per_loop=int(client.get("slots_per_loop", 1024)),
max_pipeline=int(client.get("max_pipeline", 256)),
request_timeout_ms=int(client.get("request_timeout_ms", 10000)),
encryption=int(client.get("encryption", 0)),
utp=int(client.get("utp", 0)),
connect_timeout_ms=int(client.get("connect_timeout_ms", 5000)),
fallback=int(client.get("fallback", 0)),
)
started = time.time()
buffers = [bytearray(meta.piece_len(i)) for i in range(meta.num_pieces)]
received = [0] * meta.num_pieces
done = bytearray(meta.num_pieces)
done_count = 0
last_progress = started
last_bytes = 0
status_snap = None
with Engine(cfg, lib_path=lib_path, poll_batch=2048) as eng:
tid = eng.add_torrent(
meta.info_hash,
make_peer_id(),
meta.piece_length,
meta.total_size,
meta.num_pieces,
)
eng.set_priorities(tid, [1] * meta.num_pieces)
eng.add_peer(tid, ip, port)
while done_count < meta.num_pieces:
status_snap = eng.status(tid)
if status_snap.state == STATE_ERROR:
raise RuntimeError(f"engine error: {ERROR_NAMES[status_snap.error]}")
descs = eng.poll_ready()
if not descs:
eng.wait(200)
now = time.time()
if status_snap.bytes_received != last_bytes:
last_bytes = status_snap.bytes_received
last_progress = now
if now - started > timeout or now - last_progress > timeout:
raise TimeoutError(
f"stalled after {now - started:.1f}s: "
f"{done_count}/{meta.num_pieces} pieces, "
f"state={STATE_NAMES[status_snap.state]}, "
f"connected={status_snap.peers_connected}, "
f"failed={status_snap.peers_failed}, "
f"outstanding={status_snap.outstanding}"
)
continue
for block in descs:
buf = buffers[block.piece]
buf[block.begin:block.begin + block.len] = eng.block_data(
block.loop, block.slot, block.len
)
eng.release(block.loop, block.slot)
received[block.piece] += block.len
if not done[block.piece] and received[block.piece] >= meta.piece_len(block.piece):
digest = hashlib.sha1(bytes(buf)).digest()
if digest != meta.piece_hashes[block.piece]:
raise ValueError(f"piece {block.piece} hash mismatch")
done[block.piece] = 1
done_count += 1
eng.set_priority(tid, block.piece, 0)
status_snap = eng.status(tid)
full = hashlib.sha1()
for buf in buffers:
full.update(buf)
elapsed = time.time() - started
return {
"name": client["name"],
"ok": True,
"host": client["host"],
"ip": ip,
"port": port,
"utp": cfg.utp,
"encryption": cfg.encryption,
"fallback": cfg.fallback,
"pieces": done_count,
"bytes": meta.total_size,
"sha1": full.hexdigest(),
"elapsed_s": elapsed,
"mbps": (meta.total_size / 1e6 / elapsed) if elapsed > 0 else 0.0,
"peers_connected": int(status_snap.peers_connected if status_snap else 0),
"peers_failed": int(status_snap.peers_failed if status_snap else 0),
"state": STATE_NAMES[status_snap.state] if status_snap else "UNKNOWN",
"error": ERROR_NAMES[status_snap.error] if status_snap else "OK",
}
def main() -> int:
ap = argparse.ArgumentParser(description="Run offline client interop matrix.")
ap.add_argument("--fixture", required=True)
ap.add_argument("--clients", required=True)
ap.add_argument("--results", required=True)
ap.add_argument("--timeout", type=float, default=90.0)
ap.add_argument("--lib", default=LIB)
ap.add_argument("--only", action="append",
help="client name to run; may be repeated")
args = ap.parse_args()
torrent_path = os.path.join(args.fixture, "test.torrent")
manifest_path = os.path.join(args.fixture, "manifest.json")
meta = load_metadata(torrent_path)
with open(manifest_path, "r", encoding="utf-8") as f:
manifest = json.load(f)
clients = load_clients(args.clients, set(args.only or []) or None)
print(f"fixture: {meta.name}, {meta.total_size} bytes, "
f"{meta.num_pieces} pieces", flush=True)
print(f"clients: {', '.join(c['name'] for c in clients)}", flush=True)
results = []
for client in clients:
print(f"==> {client['name']} ({client['host']}:{client['port']})", flush=True)
try:
result = download_from_client(meta, client, args.timeout, args.lib)
if result["sha1"] != manifest["sha1"]:
raise ValueError(
f"full-file sha1 mismatch: got {result['sha1']} expected {manifest['sha1']}"
)
print(f" ok: {result['bytes']/1e6:.1f} MB in "
f"{result['elapsed_s']:.2f}s ({result['mbps']:.1f} MB/s)",
flush=True)
except Exception as exc:
result = {
"name": client["name"],
"ok": False,
"host": client.get("host"),
"port": client.get("port"),
"utp": client.get("utp", 0),
"encryption": client.get("encryption", 0),
"fallback": client.get("fallback", 0),
"error": str(exc),
"traceback": traceback.format_exc(),
}
print(f" FAIL: {exc}", flush=True)
results.append(result)
os.makedirs(os.path.dirname(args.results), exist_ok=True)
write_manifest(
args.results,
fixture=manifest,
timeout_s=args.timeout,
results=results,
passed=sum(1 for r in results if r["ok"]),
failed=sum(1 for r in results if not r["ok"]),
)
print("-" * 72)
for r in results:
if r["ok"]:
print(f"PASS {r['name']:<24} {r['mbps']:8.1f} MB/s "
f"{r['state']} enc={r['encryption']} utp={r['utp']}")
else:
print(f"FAIL {r['name']:<24} {r['error']}")
return 0 if all(r["ok"] for r in results) else 1
if __name__ == "__main__":
raise SystemExit(main())

66
interop/seed_aria2.py Normal file
View file

@ -0,0 +1,66 @@
from __future__ import annotations
import argparse
import signal
import subprocess
import time
from common import touch_ready, wait_for_tcp
def main() -> int:
ap = argparse.ArgumentParser(description="Seed a fixture with aria2.")
ap.add_argument("--torrent", required=True)
ap.add_argument("--data", required=True)
ap.add_argument("--port", type=int, required=True)
args = ap.parse_args()
cmd = [
"aria2c",
"--dir", args.data,
"--seed-time=1000000",
"--check-integrity=true",
"--allow-overwrite=false",
"--auto-file-renaming=false",
"--enable-dht=false",
"--enable-dht6=false",
"--enable-peer-exchange=false",
"--bt-enable-lpd=false",
"--listen-port", str(args.port),
"--dht-listen-port", str(args.port),
"--summary-interval=0",
args.torrent,
]
proc = subprocess.Popen(cmd)
stop = False
def _stop(signum, frame):
nonlocal stop
stop = True
proc.terminate()
signal.signal(signal.SIGTERM, _stop)
signal.signal(signal.SIGINT, _stop)
try:
wait_for_tcp("127.0.0.1", args.port, timeout=45)
if proc.poll() is not None:
return proc.returncode or 1
# aria2 may still be checking files; the runner also has retry/timeout,
# so readiness here means the peer port is accepting connections.
touch_ready()
print(f"aria2 seeding on {args.port}", flush=True)
while not stop and proc.poll() is None:
time.sleep(1)
return proc.returncode or 0
finally:
if proc.poll() is None:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
if __name__ == "__main__":
raise SystemExit(main())

82
interop/seed_deluge.py Normal file
View file

@ -0,0 +1,82 @@
from __future__ import annotations
import argparse
import os
import signal
import subprocess
import time
from common import touch_ready, wait_for_tcp
def console(config: str, command: str, check: bool = True) -> subprocess.CompletedProcess:
return subprocess.run(
["deluge-console", "-c", config, command],
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
check=check,
timeout=15,
)
def main() -> int:
ap = argparse.ArgumentParser(description="Seed a fixture with Deluge.")
ap.add_argument("--torrent", required=True)
ap.add_argument("--data", required=True)
ap.add_argument("--port", type=int, required=True)
ap.add_argument("--daemon-port", type=int, default=58846)
args = ap.parse_args()
config = "/tmp/deluge-config"
os.makedirs(config, exist_ok=True)
proc = subprocess.Popen([
"deluged",
"-d",
"-c", config,
"-i", "0.0.0.0",
"-p", str(args.daemon_port),
"-L", "warning",
])
stop = False
def _stop(signum, frame):
nonlocal stop
stop = True
proc.terminate()
signal.signal(signal.SIGTERM, _stop)
signal.signal(signal.SIGINT, _stop)
try:
wait_for_tcp("127.0.0.1", args.daemon_port, timeout=45)
settings = [
("listen_ports", f"({args.port}, {args.port})"),
("random_port", "False"),
("dht", "False"),
("lsd", "False"),
("upnp", "False"),
("natpmp", "False"),
("add_paused", "False"),
("download_location", args.data),
]
for key, value in settings:
console(config, f"config -s {key} {value}")
console(config, f"add -p {args.data} {args.torrent}")
wait_for_tcp("127.0.0.1", args.port, timeout=45)
touch_ready()
print(f"Deluge seeding on {args.port}", flush=True)
while not stop and proc.poll() is None:
time.sleep(1)
return proc.returncode or 0
finally:
if proc.poll() is None:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -0,0 +1,84 @@
from __future__ import annotations
import argparse
import os
import signal
import time
import libtorrent as lt
from common import touch_ready
def settings_for(mode: str, port: int) -> dict:
tcp = mode in ("plain", "mse")
utp = mode in ("utp", "utp-mse")
encrypted = mode in ("mse", "utp-mse")
settings = {
"listen_interfaces": f"0.0.0.0:{port}",
"enable_dht": False,
"enable_lsd": False,
"enable_upnp": False,
"enable_natpmp": False,
"enable_outgoing_tcp": tcp,
"enable_incoming_tcp": tcp,
"enable_outgoing_utp": utp,
"enable_incoming_utp": utp,
"announce_to_all_trackers": False,
"announce_to_all_tiers": False,
"alert_mask": 0,
}
if encrypted:
settings.update({
"in_enc_policy": int(lt.enc_policy.forced),
"out_enc_policy": int(lt.enc_policy.forced),
"allowed_enc_level": int(lt.enc_level.rc4),
"prefer_rc4": True,
})
else:
settings.update({
"in_enc_policy": int(lt.enc_policy.disabled),
"out_enc_policy": int(lt.enc_policy.disabled),
})
return settings
def main() -> int:
ap = argparse.ArgumentParser(description="Seed a fixture with libtorrent.")
ap.add_argument("--torrent", required=True)
ap.add_argument("--data", required=True)
ap.add_argument("--port", required=True, type=int)
ap.add_argument("--mode", required=True, choices=["plain", "mse", "utp", "utp-mse"])
args = ap.parse_args()
stop = False
def _stop(signum, frame):
nonlocal stop
stop = True
signal.signal(signal.SIGTERM, _stop)
signal.signal(signal.SIGINT, _stop)
ses = lt.session(settings_for(args.mode, args.port))
h = ses.add_torrent({
"ti": lt.torrent_info(args.torrent),
"save_path": args.data,
"flags": lt.torrent_flags.seed_mode,
})
deadline = time.time() + 60
while time.time() < deadline and not h.status().is_seeding:
time.sleep(0.25)
if not h.status().is_seeding:
raise TimeoutError(f"libtorrent {args.mode} did not enter seed mode")
touch_ready()
print(f"libtorrent {args.mode} seeding on {args.port}", flush=True)
while not stop:
time.sleep(1)
ses.remove_torrent(h)
return 0
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -0,0 +1,88 @@
from __future__ import annotations
import argparse
import os
import signal
import subprocess
import time
from common import touch_ready, wait_for_tcp
def write_config(profile: str, data_dir: str, port: int) -> None:
cfg_dir = os.path.join(profile, "qBittorrent", "config")
os.makedirs(cfg_dir, exist_ok=True)
# qBittorrent stores settings in an INI-like file with escaped keys.
# These disable internet/discovery paths and fix the peer port.
with open(os.path.join(cfg_dir, "qBittorrent.conf"), "w", encoding="utf-8") as f:
f.write(f"""[BitTorrent]
Session\\AddTorrentPaused=false
Session\\BTProtocol=TCP
Session\\DHTEnabled=false
Session\\DefaultSavePath={data_dir}
Session\\DisableAutoTMMByDefault=true
Session\\LSDEnabled=false
Session\\PeXEnabled=false
Session\\Port={port}
Session\\QueueingSystemEnabled=false
Session\\UPnP=false
[LegalNotice]
Accepted=true
[Preferences]
WebUI\\Enabled=false
""")
def main() -> int:
ap = argparse.ArgumentParser(description="Seed a fixture with qBittorrent-nox.")
ap.add_argument("--torrent", required=True)
ap.add_argument("--data", required=True)
ap.add_argument("--port", type=int, required=True)
args = ap.parse_args()
profile = "/tmp/qbt-profile"
write_config(profile, args.data, args.port)
cmd = [
"qbittorrent-nox",
f"--profile={profile}",
"--configuration=interop",
f"--torrenting-port={args.port}",
f"--save-path={args.data}",
"--add-paused=false",
"--skip-dialog=true",
"--skip-hash-check",
args.torrent,
]
proc = subprocess.Popen(cmd)
stop = False
def _stop(signum, frame):
nonlocal stop
stop = True
proc.terminate()
signal.signal(signal.SIGTERM, _stop)
signal.signal(signal.SIGINT, _stop)
try:
wait_for_tcp("127.0.0.1", args.port, timeout=45)
if proc.poll() is not None:
return proc.returncode or 1
touch_ready()
print(f"qBittorrent seeding on {args.port}", flush=True)
while not stop and proc.poll() is None:
time.sleep(1)
return proc.returncode or 0
finally:
if proc.poll() is None:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
if __name__ == "__main__":
raise SystemExit(main())

80
interop/seed_rtorrent.py Normal file
View file

@ -0,0 +1,80 @@
from __future__ import annotations
import argparse
import os
import signal
import subprocess
import time
from common import touch_ready, wait_for_tcp
def write_rc(path: str, torrent: str, data: str, port: int, session: str) -> None:
os.makedirs(session, exist_ok=True)
with open(path, "w", encoding="utf-8") as f:
f.write(f"""
directory.default.set = {data}
session.path.set = {session}
network.port_range.set = {port}-{port}
network.port_random.set = no
dht.mode.set = disable
protocol.pex.set = no
trackers.use_udp.set = no
network.http.max_open.set = 0
pieces.hash.on_completion.set = no
""")
def main() -> int:
ap = argparse.ArgumentParser(description="Seed a fixture with rTorrent.")
ap.add_argument("--torrent", required=True)
ap.add_argument("--data", required=True)
ap.add_argument("--port", type=int, required=True)
args = ap.parse_args()
rc = "/tmp/rtorrent.rc"
session = "/tmp/rtorrent-session"
write_rc(rc, args.torrent, args.data, args.port, session)
env = os.environ.copy()
env.setdefault("TERM", "xterm")
command = f"rtorrent -n -o import={rc}"
proc = subprocess.Popen(
["script", "-q", "-e", "-c", command, "/dev/null"],
env=env,
stdin=subprocess.PIPE,
)
stop = False
def _stop(signum, frame):
nonlocal stop
stop = True
proc.terminate()
subprocess.run(["pkill", "-TERM", "rtorrent"], check=False)
signal.signal(signal.SIGTERM, _stop)
signal.signal(signal.SIGINT, _stop)
try:
wait_for_tcp("127.0.0.1", args.port, timeout=45)
if proc.stdin:
proc.stdin.write(b"\x7f" + args.torrent.encode("utf-8") + b"\n")
proc.stdin.flush()
time.sleep(2)
if proc.poll() is not None:
return proc.returncode or 1
touch_ready()
print(f"rTorrent seeding on {args.port}", flush=True)
while not stop:
if subprocess.run(["pgrep", "rtorrent"], stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL).returncode != 0:
return 1
time.sleep(1)
return 0
finally:
subprocess.run(["pkill", "-TERM", "rtorrent"], check=False)
if proc.poll() is None:
proc.terminate()
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -0,0 +1,109 @@
from __future__ import annotations
import argparse
import json
import os
import shutil
import signal
import subprocess
import time
from common import touch_ready, wait_for_tcp
def run_remote(rpc_port: int, *args: str, check: bool = True) -> subprocess.CompletedProcess:
return subprocess.run(
["transmission-remote", f"127.0.0.1:{rpc_port}", *args],
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
check=check,
)
def write_settings(config_dir: str, peer_port: int, rpc_port: int, data_dir: str) -> None:
os.makedirs(config_dir, exist_ok=True)
settings = {
"download-dir": data_dir,
"incomplete-dir-enabled": False,
"dht-enabled": False,
"pex-enabled": False,
"lpd-enabled": False,
"utp-enabled": False,
"port-forwarding-enabled": False,
"peer-port": peer_port,
"peer-port-random-on-start": False,
"rpc-enabled": True,
"rpc-bind-address": "127.0.0.1",
"rpc-port": rpc_port,
"rpc-whitelist-enabled": False,
"start-added-torrents": True,
"trash-original-torrent-files": False,
}
with open(os.path.join(config_dir, "settings.json"), "w", encoding="utf-8") as f:
json.dump(settings, f, indent=2, sort_keys=True)
def main() -> int:
ap = argparse.ArgumentParser(description="Seed a fixture with Transmission.")
ap.add_argument("--torrent", required=True)
ap.add_argument("--data", required=True)
ap.add_argument("--peer-port", type=int, required=True)
ap.add_argument("--rpc-port", type=int, required=True)
ap.add_argument("--utp", action="store_true")
ap.add_argument("--encryption", choices=["tolerated", "preferred", "required"],
default="tolerated")
args = ap.parse_args()
config_dir = "/tmp/transmission-config"
shutil.rmtree(config_dir, ignore_errors=True)
write_settings(config_dir, args.peer_port, args.rpc_port, args.data)
proc = subprocess.Popen(["transmission-daemon", "-f", "-g", config_dir])
stop = False
def _stop(signum, frame):
nonlocal stop
stop = True
proc.terminate()
signal.signal(signal.SIGTERM, _stop)
signal.signal(signal.SIGINT, _stop)
try:
wait_for_tcp("127.0.0.1", args.rpc_port, timeout=30)
run_remote(args.rpc_port, "--no-dht", "--no-pex", "--no-lpd", "--no-portmap")
run_remote(args.rpc_port, "--utp" if args.utp else "--no-utp")
run_remote(args.rpc_port, {
"tolerated": "--encryption-tolerated",
"preferred": "--encryption-preferred",
"required": "--encryption-required",
}[args.encryption])
run_remote(args.rpc_port, "-a", args.torrent, "-w", args.data)
run_remote(args.rpc_port, "-t", "all", "--start")
wait_for_tcp("127.0.0.1", args.peer_port, timeout=30)
deadline = time.time() + 60
while time.time() < deadline:
info = run_remote(args.rpc_port, "-t", "all", "-i", check=False).stdout
if "Percent Done: 100%" in info or "Seeding" in info:
touch_ready()
print(f"transmission seeding on {args.peer_port}", flush=True)
break
time.sleep(1)
else:
raise TimeoutError("Transmission did not report a complete seed")
while not stop and proc.poll() is None:
time.sleep(1)
return proc.returncode or 0
finally:
if proc.poll() is None:
proc.terminate()
try:
proc.wait(timeout=10)
except subprocess.TimeoutExpired:
proc.kill()
if __name__ == "__main__":
raise SystemExit(main())