backfill¶
Defined in dccd.application.operations
- async backfill(spec, *, registry, store, runs_store=None, events=None, stop_event=None, run_id=None)[source]
Backfill historical data from a source to the Parquet store.
- Parameters:
- specJobSpec
Job specification (
operation='backfill').spec.params.startcontrols the start point:'last': resume from the last stored timestamp. If no data exists yet, defaults to 30 days ago to avoid a multi-decade paginator run from epoch 0.'origin': start from the exchange’s earliest available data (timestamp 0 — many pages will be empty before the exchange’s launch date).An ISO-8601 date string (
'2024-01-01') or a nanosecond integer: explicit start timestamp.
- registrySourceRegistry
- storeParquetStore
- runs_storeRunsStore or None
- eventsRunEvents or None
- stop_eventasyncio.Event or None
Set externally to cancel mid-run cleanly.
- run_idstr or None
Override the auto-generated run ID (used by the API endpoint so the polling URL matches what is stored in RunsStore).
- Returns:
- dict
{'run_id', 'rows_written', 'start_ns', 'end_ns'}on success;{'run_id', 'rows_written', 'error'}on failure.