Architecture

dvb-WarpPool is a Rust workspace with 18 library crates + 6 binaries. A clear layering — from pure schema crates without async at the bottom up to the async-orchestrated binaries on top — keeps the system testable and lets sub-crates run standalone (e.g. the translator as a sidecar without the pool daemon).

Crate Overview

Schema + foundation (no async)

Core mining layer

Storage + API

Operational subsystems

Tools

Binaries

Data Flow (happy path)

                            +---------------------+
                            | Bitcoin Core        |
                            | (mainnet / regtest) |
                            +---------------------+
                              | RPC          | ZMQ pub
                              v              v
                            +---------------------+
                            | bitcoin-rpc         |
                            |  - getblocktemplate  |
                            |  - submitblock       |
                            |  - hashblock watch   |
                            +---------------------+
                                   |
                                   v
                            +---------------------+
                            | job-builder         |
                            |  - coinbase splits  |
                            |  - merkle tree      |
                            |  - StratumJob       |
                            +---------------------+
                                   |
                       +-----------+-----------+
                       v                       v
              +------------------+   +------------------+
              | stratum-v1       |   | stratum-v2       |
              |  TCP / TCP+TLS   |   |  TCP+NOISE       |
              +------------------+   +------------------+
                       |                       |
                       v                       v
               V1 miners (Bitaxe,       V2 miners / translator
               NerdNOS, AntminerS23)
                       |                       |
                       v                       v
                      submit (extranonce, nonce, ntime)
                                   |
                                   v
                            +---------------------+
                            | share-validator     |
                            |  - PoW check        |
                            |  - block detection  |
                            +---------------------+
                                   |
                       +-----------+-----------+
                       v                       v
              +------------------+   +------------------+
              | storage          |   | block-submit-loop|
              |  - record_share  |   |  - assemble block|
              |  - vardiff_state |   |  - submitblock   |
              +------------------+   +------------------+
                                              |
                                              v on success
                            +---------------------+
                            | notifier            |
                            |  fire BlockFound    |
                            +---------------------+
                              |    |    |    |    |
                              v    v    v    v    v
                            ntfy tg disc slack email

Daemon Task Topology

dvb-warppool-daemon is a Tokio multi-reactor with some two dozen long-lived tasks plus per-connection tasks. All tasks share one CancellationToken for graceful shutdown (Ctrl-C and, on Unix, SIGTERM).

                  +------- main() async -------+
                  |                            |
                  +- spawn ---------+----------+
                          |         |
                  +-------+-------+ +------------------+
                  | Stratum-V1     | Stratum-V2        |
                  | accept loop    | accept loop       |
                  | (+ TLS)        | (+ NOISE)         |
                  +----------------+ +-----------------+
                          |               |
                  per connection: session task
                          |               |
                          v               v
                  +---------------------------------+
                  | block_found_tx broadcast        |
                  +---------------------------------+
                          |
                  +-------+--------+
                  | block_submit_  |
                  | loop           |
                  +----------------+
                          |
                  Bitcoin Core RPC submitblock
                          |
                  +-------+--------+
                  | notifier.notify(BlockFound)     |

                  +- spawn ---------+----------+
                  | job_refresh_loop                |
                  |  - Box<dyn TemplateSource>:     |
                  |    GBT (poll 30s + ZMQ tip      |
                  |    fast path) or IPC (waitNext) |
                  |  - drain & build job            |
                  |  - tip_to_job timing → snapshot |
                  |  - push to stratum handles      |
                  +---------------------------------+

                  +- spawn ---------+----------+
                  | aggregate_loop (60s tick)       |
                  |  - storage.aggregate_5min       |
                  |  - SharesAccepted SSE event     |
                  +---------------------------------+

                  +- spawn ---------+----------+
                  | health_check_loop (60s tick)    |
                  |  - check_bitcoin_health         |
                  |  - SSE HealthSnapshot           |
                  |  - RpcDown/Recovered notify     |
                  +---------------------------------+

                  +- spawn ---------+----------+
                  | periodic_update_check (24h)     |
                  |  - Forge Releases API           |
                  |  - SSE UpdateAvailable          |
                  +---------------------------------+

                  +- spawn ---------+----------+
                  | miner_poll_loop (vendor probes) |
                  +---------------------------------+

                  +- spawn ---------+----------+
                  | HTTP API (axum on :18334)       |
                  |  - REST endpoints               |
                  |  - SSE /api/events              |
                  |  - static UI from --ui-dir      |
                  +---------------------------------+

(Later phases added further loops on the same pattern: UPnP lease renewal, the Anker-bridge supervisor, rejected_flush_loop (2 s coalesced flush of the RejectedShareBuffer fed by both stratum crates), backup_loop, anomaly_check_loop, push_send_loop, webhook_dispatch_loop, solar_poll_loop, the 30 s network-stats poll and the 10 min inbound-peer scan; the daily energy aggregation runs hourly inside aggregate_loop.)

Template sources. job_refresh_loop is source-neutral: it consumes a Box<dyn TemplateSource> (warppool-bitcoin-rpc::template_source). The GBT source polls getblocktemplate on the refresh tick and refreshes immediately on a ZMQ hashblock event; it also keeps a ZmqLiveness counter of tips it only discovered by polling while ZMQ watchers were configured, which the health tick turns into the bitcoin_zmq_hashblock_silent warning. The IPC source (warppool-bitcoin-ipc) blocks in Core’s waitNext instead; ZMQ watchers are not spawned in that mode. Every TemplateUpdate carries a TemplateTiming (source ready → fetched → mapped); the loop derives tip_to_job_ms for real tip changes and exposes fetch/map/tip-to-job in the snapshot, /api/overview and /metrics.

Shared State

Tokio tasks share Arc<...> handles instead of global statics:

Storage Schema

Core tables from phases 1-15; the schema is created programmatically from a bundled SCHEMA_SQL constant (CREATE TABLE IF NOT EXISTS …, run by init_schema() in crates/storage/src/lib.rs) — there are no sqlx file migrations. Later phases added the device tables (miners, miner-telemetry raw/5min), the per-day energy history (daily_aggregates), solar_excess_minute, the best-share tables (best_shares, best_shares_all_time), wallet_prefs, pool_meta (started_at) and admin_2fa_recovery on the same pattern.

An idle worker can be forgotten from the admin UI (DELETE /api/admin/workers, v1.38): its rows in workers, best_shares, best_shares_all_time and vardiff_state (V1 key and :sv2 snapshot) go in one transaction, together with the in-memory reject breakdown and the Sv2 VarDiff snapshot cache; shares_* and blocks_found stay. A worker with a live V1, TLS or Sv2 session is refused (an all-protocol refcount fed by the connection sinks). The name is matched exactly as the miner authorized it. Retention evicts workers rows but not best shares or VarDiff memory; GET /api/admin/workers/orphans lists those leftover names (union of best_shares, best_shares_all_time and vardiff_state minus workers, the :sv2 key under its bare name) so the admin page can remove them the same way.

Timestamp convention. DATETIME columns are written by SQLite itself (CURRENT_TIMESTAMP → YYYY-MM-DD HH:MM:SS, space separator, UTC); pool_settings string values are written by Rust as RFC3339 (T separator). The two are not lexically comparable ('T' > ' '), so cutoffs against DATETIME columns must be computed inside SQL (datetime('now', '-N seconds')), never bound as Rust-formatted strings — a v1.0.16 bug filtered out every active worker exactly this way. The full rules live in the warppool-storage module docs.

The Sv2 Stack in Detail

Stratum V2 is a binary-framed protocol with a NOISE handshake in front. Two subprotocols matter in the pool context:

1. Mining subprotocol (implemented) — channel-based, extranonce-aware, version rolling per BIP-320, per-channel set-target.
2. Template Distribution Protocol (TDP; foundation in phase 7.6a, wiring in 7.6b deferred) — replaces getblocktemplate with push-driven template updates straight from the Bitcoin node.

Standard vs. extended channels. Both are supported; pick by role. Standard is header-only mining for end devices: the pool computes the merkle root and ships ready-to-hash work — smaller frames, no merkle work on the device. Extended hands the miner coinbase_prefix/suffix plus the merkle path and full extranonce rolling, which is what proxies, translators and farm aggregators need to manage their own search space (our own translator opens an extended channel upstream for exactly that reason). Extended costs a device bigger jobs and per-job merkle work on the ESP32, and a single Bitaxe-class miner does not need the extra search space — nonce + version rolling is ample for its hash rate.

That efficiency argument is real but it is not the deciding one. Extended is also what lets the miner see the coinbase it is hashing, which is the whole point of Sv2 for someone connecting to a pool they do not run. AxeOS says as much in its own UI (“Extended Channels are recommended for external pools — the pool sends the coinbase template so the miner can verify it. Standard Channels should only be used with a locally deployed SV2 node or JD client”), and it is a user-selectable setting there (SV2 Channel Type, shown once the pool is set to V2), not something the firmware decides alone. Not every firmware leaves the choice open, though: NerdQAxe hard-wires extended (m_channelType = SV2_CHANNEL_EXTENDED in main/stratum/stratum_task_v2.cpp, with the config read commented out), and AxeOS disables standard on BM1397. Recommendation for miners connecting to this pool: extended. Standard is the right pick when the Sv2 endpoint is a node the miner operates itself. VarDiff converges per channel either way; a difficulty difference between the two modes is convergence state, not quality.

V1 miner ----TCP plain---->  stratum-v1 server  ----------+
                                                          |
V1 miner ----TCP+TLS------>  stratum-v1 TLS server -------+
                                                          |
                                                          v
                                              +------------------+
                                              |  job-builder     |
                                              |  share-validator |
                                              +------------------+
                                                          ^
                                                          |
V2 miner ----TCP+NOISE----> stratum-v2 server (sv2_listen,  +
                                       reference deployment :3336)
                                                          |
V1 miner ----TCP--+                                       |
                  |                                        |
                  v                                        |
     dvb-warppool-translator (sidecar) --TCP+NOISE--------+
       - SetupConnection
       - OpenExtendedMiningChannel with user_identity
       - SubmitSharesExtended with miner extranonce
       - receives NewExtendedMiningJob + SetNewPrevHash
       - maps to V1 mining.notify + slushpool prev_hash + BIP-320 version rolling

The concrete wire format lives in crates/stratum-v2/src/messages.rs with roundtrip tests. The NOISE handshake uses the SRI noise_sv2 stack (secp256k1 + EllSwift + Schnorr-signed certificates, spec-conformant since the Sv2 modernisation) — pure Rust, no OpenSSL.

Connection Lifecycle Hooks

Both Stratum servers expose an optional ConnectionSink trait. For authenticated workers (V1: mining.authorize, V2: OpenChannel with user_identity) on_authorized fires; on connection end on_disconnect.

The daemon implements a NotifierConnectionSink struct that implements both traits. A per-worker debounce (default 30s) keeps flapping miners from firing a notification on every reconnect. The same hooks feed the live worker sets (TLS/Sv2 badges) and the peer-IP refcount map used by the energy accounting (“Disconnected” devices are excluded from watts/kWh).

   v1.Session::run
         |
   handle_authorize → connection_sink.on_authorized
         |
   ... shares ...
         |
   loop exit → connection_sink.on_disconnect

   v2.handle_connection
         |
   process_frame → channels().iter() → new user_identity?
         |                                    → on_authorized
   ... shares ...
         |
   loop exit → for each notified_user → on_disconnect

Event Bus + SSE

PoolEventBus is a tokio broadcast channel with PoolEvent variants:

The API’s /api/events opens one SSE stream per client with the current subscription set. The UI uses it for HealthBanner + UpdateBanner + toast events.

Configuration Layers

/etc/dvb-warppool/
├── config.toml         # non-sensitive settings
├── secrets.toml        # chmod 600 — admin hash, jwt secret, rpc pass, sv2 key
└── pool.db             # SQLite (path configurable)

Env vars for opt-in subsystems (see the Configuration Reference): WARPPOOL_HEALTH_CHECK_INTERVAL_SECONDS, WARPPOOL_AUTOUPDATE_REPO, WARPPOOL_DISCONNECT_DEBOUNCE_SECS, sink-specific tokens, etc.

Lifecycle (daemon boot)

1. Parse CLI (clap) + ENV
2. Load config.toml + (optional) secrets.toml
3. Validate config (mining.payout_address required, ratelimit constraints, …)
4. Init tracing subscriber (JSON when logging.json)
5. Open SQLite + idempotent schema init (CREATE TABLE IF NOT EXISTS …)
6. Resolve admin profile (persisted value in pool_settings beats the config default)
7. Construct BitcoinRpc with with_metrics(pool_metrics)
8. Probe RPC (warning on failure; the daemon still starts)
9. Construct Notifier from config
10. Construct PoolMetrics (Arc-shared)
11. Build the initial job (or skip when RPC failed)
12. Spawn the Stratum V1 listener (+ TLS listener when cert/key configured)
13. Spawn the Stratum V2 listener when sv2_listen is configured (the authority key is auto-generated into secrets.toml if missing)
14. Spawn job_refresh_loop (poll + optional ZMQ)
15. Spawn block_submit_loop
16. Spawn aggregate_loop (60s)
17. Spawn health_check_loop when WARPPOOL_HEALTH_CHECK_INTERVAL_SECONDS > 0
18. Spawn periodic_update_check when WARPPOOL_AUTOUPDATE_REPO + interval > 0
19. Spawn the HTTP API (axum on status_listen)
20. Install signal handlers (SIGTERM → cancel.cancel() → all tasks shut down)
21. The tokio::main event loop runs until cancel

Crate Dependencies (DAG)

Simplified crate graph (not every edge; no cyclic deps):

profiles --+
hwdetect --+--> config
              |
              +--> storage
              +--> bitcoin-rpc --+
              +--> bitcoin-ipc   |
              +--> job-builder   |
              +--> share-validator
              +--> stratum-v1 -----+
              +--> stratum-v2 -----+
              +--> translator -----+
              +--> telemetry -----+
              +--> notifier ------+
              +--> autoupdate ----+
              +--> health --------+
              +--> uninstall -----+
              +--> api -----------+
              +--> simulator -----+
                                 |
                                 v
                  dvb-warppool-daemon
                  dvb-warppool-cli
                  dvb-warppool-setup
                  dvb-warppool-translator
                  dvb-warppool-sim
                  dvb-warppool-mac-launcher

Crates and their one-directional deps:

• telemetry ← none (foundation for metrics)
• bitcoin-rpc → telemetry (for the RPC latency histogram, phase 16.3)
• api → notifier + autoupdate + telemetry + storage + …
• daemon → all library crates plus runtime deps (tokio, sqlx, …)

Testing Strategy

Three layers, plus one extra operator-driven layer:

As of v1.39.2: 1703 tests + 6 ignored, all green. UI side: pnpm svelte-check → 376 files, 0 errors.

See TESTING.md for details + the simulator workflow.