DownloadBitfinexData

Defined in dccd.continuous_dl.bitfinex

class DownloadBitfinexData(time_step=60, until=3600, checkpoint_dir=None)

Bases: ContinuousDownloader

Basis object to download data from a stream websocket client API.

Parameters:

time_stepint or None, optional

Number of seconds between two snapshots of data, minimum is 1, default is 60 (one minute). Each time_step seconds data will be processed and pushed to the database. Pass None to receive data tick-by-tick without periodic aggregation.

untilint, optional

Number of seconds before stopping, or a future Unix timestamp at which to stop. Default is 3600 (one hour).

Attributes:

hoststr

Address of host to connect.

conn_pardict

Parameters of websocket connection.

wswebsockets.client.WebSocketClientProtocol

Connection with the websocket client.

is_connectbool

True if is connected, False otherwise.

tsint

Number of second between two snapshots of data.

tint

Current timestamp but rounded by ts.

untilint

Timestamp to stop to download data.

Methods

set_process_data(func, **kwargs)	Set processing function.
set_saver(call, **kwargs)	Set saver object to save data or update a database.
__call__(channel, **kwargs)	Open a websocket connection and save/update the database.

__call__(channel, **kwargs)

Open a websocket connection and save/update the database.

Run asynchronously two loops to get data from bitfinex websocket and save/update the database.

Parameters:

channel{‘book’, ‘book_raw’, ‘trades’, ‘trades_raw’}

Channel to get data, by default data will be aggregated (OHLC for ‘trades’ and reconstructed orderbook for ‘book’), add ‘_raw’ to the channel to get raw data (trade tick by tick or each orders).

**kwargs

Any revelevant keyword arguments will be passed to the websocket connector, see API documentation [1] for more details.

Warning

‘book_raw’ and ‘trades_raw’ can be very memory expensive.

References

[1]

https://docs.bitfinex.com/v2/docs/ws-public

get_parser(key)

Get allowed data parser.

Parameters:

keystr

Name code of data to parse.

Returns:

function

The allowed function to parse this kind of data.

Raises:

KeyError

If key is not in the allowed parser keys.

on_close()

On websocket close print and fire event.

async on_error(error, *args)

On websocket error print and fire event.

async on_message(data)

Route an incoming websocket message to the appropriate parser.

on_open(**kwargs)

On websocket open.

Parameters:

**kwargs

Any relevant keyword arguments to set connection.

parser_book(data)

Parse market depth of order book.

Parameters:

datalist

Order data.

parser_raw_book(data)

Parse raw order book, each timestep set in a list all orders.

Parameters:

datalist

Order data.

parser_raw_trades(data)

Parse raw trade data tick-by-tick.

Parameters:

datalist

Trade data.

parser_trades(data)

Parse trade data and aggregate into OHLCV snapshots.

Parameters:

datalist

Trade data.

set_book_saver(saver, process_func=<function set_marketdepth>, **kwargs)

Set saver for the order-book channel.

Parameters:

savercallable

Callable to persist the processed DataFrame (e.g. IODataBase).

process_funccallable, optional

Function to convert the book dict to a DataFrame, default is dccd.process_data.set_marketdepth.

**kwargs

Extra keyword arguments forwarded to saver on each call.

set_process_data(func, **kwargs)

Set processing function.

Parameters:

funccallable

Function to process and clean data before to be saved. Must take data in arguments and can take any optional keywords arguments, cf exemples in dccd.process_data.

**kwargs

Any keyword arguments to be passed to func.

set_saver(call, **kwargs)

Set saver object to save data or update a database.

Parameters:

callcallable

Callable object to save data or update a database. Must take data in arguments and can take any optional keywords arguments, cf exemples in dccd.io_tools.

**kwargs

Any keyword arguments to be passed to call.

set_trades_saver(saver, process_func=<function set_trades>, **kwargs)

Set saver for the trades channel.

Parameters:

savercallable

Callable to persist the processed DataFrame (e.g. IODataBase).

process_funccallable, optional

Function to convert raw trade list to a DataFrame, default is dccd.process_data.set_trades.

**kwargs

Extra keyword arguments forwarded to saver on each call.

async wait_that(is_true)

Wait before running.