Architecture

dccd v3 is built as a hexagonal architecture: business logic is fully isolated from I/O and from the interfaces that drive it. Each layer depends only on the ones beneath it, so the domain can be tested in isolation and a new exchange, storage backend or interface plugs in without touching the rest.

Interfaces   CLI  ·  HTTP API  ·  Web UI  ·  Python Client
                               │
Application   backfill · stream · read · inventory · Scheduler · Config
                               │
Domain  ◄──  Sources (7 adapters)  ◄──  Transport (httpx · WS · Paginator)
                               │
                           Storage (Parquet · SQLite runs · rclone)

The layers

Domain

Pure, synchronous value objects and transforms — Symbol, OHLCBar, Trade, Capability — with no I/O. All timestamps are nanoseconds UTC (int64).

API Reference
Transport

Async HTTP (retry/backoff), a reconnecting WebSocket base, a token-bucket rate limiter, and the generic cursor/forward paginator.

API Reference
Sources

One adapter per exchange implementing fine-grained Source protocols, declaring their Capability per (data type × transport × mode).

Exchanges
Storage

ParquetStore (annual/daily files, per-type dedup, provenance, atomic writes), an append-only SQLite run history, and rclone sync.

API Reference
Application

The operations — backfill, stream, read, inventory — plus the async Scheduler and EventBus.

API Reference
Interfaces

CLI (Typer), HTTP API (FastAPI) + SSE, the Jinja2 Web UI (a pure HTTP client of the API), and the async Client.

CLI Reference

Key design rules

  • Domain is pure and synchronous. It never imports transport, sources or storage. This is the only layer under strict mypy.

  • Capabilities are declarative and honoured. An adapter declares what it can do (history depth, page_direction, supported spans, WS channels); the engine resolves against them and raises NoCapability early rather than failing midway or running an empty stream.

  • Nanosecond UTC timestamps everywhere (int64). Legacy v2 frames are normalised on read/merge/migration.

  • The UI is a thin client of the API — no direct calls into the application layer, so the front-end can be replaced without touching business logic.

Data flow — a backfill

  1. An interface builds a JobSpec and calls backfill.

  2. The operation resolves the exchange adapter from the registry and reads its Capability.

  3. The transport paginator drives the adapter’s fetch_*_page — by fixed windows for OHLC, by cursor for trades (draining the full requested window).

  4. Records are flushed in batches to ParquetStore, deduplicated on the natural key per data type, with progress emitted on the EventBus.