Bitmex Continuous Downloader (dccd.continuous_dl.bitmex)

Objects and functions to download data from Bitmex exchange.

These functions and objects allow you to continuously download data and update your database.

High level API

get_data_bitmex(process_func, *args, time_step=60, until=None, path=None, save_method='dataframe', io_params={}, **kwargs)

Download data from Bitmex exchange and update the database.

Parameters:

process_funccallable

Function to process and clean data before saving. Must accept data as its first argument plus optional keyword arguments; see dccd.process_data for examples.

*argsstr

Channel and optional instrument, e.g. 'trade', 'XBTUSD'. Passed directly to DownloadBitmexData.__call__.

time_stepint, optional

Number of seconds between snapshots, default 60 (1 minute).

untilint, optional

Seconds to run, or a future Unix timestamp to stop at. None or 0 means run indefinitely.

pathstr, optional

Directory for the database. Defaults to 'database/bitmex/{channel}'.

save_method{‘DataFrame’, ‘SQLite’, ‘CSV’, ‘Excel’, ‘PostgreSQL’, ‘Oracle’, ‘MSSQL’, ‘MySQL’}, optional

Storage format for IODataBase, default 'dataframe'.

io_paramsdict, optional

Extra keyword arguments forwarded to the IODataBase callable.

**kwargs

Additional keyword arguments forwarded to the websocket connector.

Warning

‘_raw’ option not yet working for Bitmex.

See also

process_data

helper functions to transform raw payloads.

tools.io.IODataBase

persistence layer.

References

[1]

https://www.bitmex.com/api/

get_orderbook_bitmex(*args, time_step=60, until=None, path=None, save_method='dataframe', io_params={})

Download reconstructed order book from Bitmex exchange.

get_trades_bitmex(*args, time_step=60, until=None, path=None, save_method='dataframe', io_params={})

Download trades tick by tick from Bitmex exchange.

Low level API

class DownloadBitmexData(time_step=60, until=3600)

Bases: ContinuousDownloader

Basis object to download data from a stream websocket client API.

Parameters:

time_stepint or None, optional

Number of seconds between two snapshots of data, minimum is 1, default is 60 (one minute). Each time_step seconds data will be processed and pushed to the database. Pass None to receive data tick-by-tick without periodic aggregation.

untilint, optional

Number of seconds before stopping, or a future Unix timestamp at which to stop. Default is 3600 (one hour).

Attributes:

hoststr

Address of host to connect.

conn_pardict

Parameters of websocket connection.

wswebsockets.client.WebSocketClientProtocol

Connection with the websocket client.

is_connectbool

True if is connected, False otherwise.

tsint

Number of second between two snapshots of data.

tint

Current timestamp but rounded by ts.

untilint

Timestamp to stop to download data.

Methods

set_process_data(func, **kwargs)	Set processing function.
set_saver(call, **kwargs)	Set saver object to save data or update a database.
__call__(*args)	Open a websocket connection and save/update the database.

__call__(*args)

Open a websocket connection and save/update the database.

Run asynchronously two loops to get data from Bitmex websocket and save/update the database.

Parameters:

*argsstr

Positional arguments joined with ':' and passed as the args subscribe parameter. The first element should be the channel name (e.g. 'orderBookL2_25' or 'trade') followed by any instrument symbol (e.g. 'XBTUSD').

Warning

‘_raw’ option not yet working for Bitmex.

References

[1]

https://www.bitmex.com/api/

set_process_data(func, **kwargs)

Set processing function.

Parameters:

funccallable

Function to process and clean data before to be saved. Must take data in arguments and can take any optional keywords arguments, cf exemples in dccd.process_data.

**kwargs

Any keyword arguments to be passed to func.

set_saver(call, **kwargs)

Set saver object to save data or update a database.

Parameters:

callcallable

Callable object to save data or update a database. Must take data in arguments and can take any optional keywords arguments, cf exemples in dccd.io_tools.

**kwargs

Any keyword arguments to be passed to call.

DownloadBitmexData streams real-time trades and order book data from BitMEX via WebSocket. No authentication is required for public channels.

Note

BitMEX does not provide a native OHLCV WebSocket channel via this client. Only trade and order book streams are available.

Exchange specifics

WebSocket URL	wss://www.bitmex.com/realtime
Available channels	Trades and order book (L2).
OHLCV	Not available via this client.
Authentication	Not required for public streams.

Quick example

from dccd.continuous_dl import DownloadBitmexData, get_data_bitmex
from dccd.tools.io import IODataBase

# Convenience function
get_data_bitmex('/data/crypto/', pair='XBTUSD',
                time_step=60, until=3600, form='parquet')

# Class-based API
dl = DownloadBitmexData(pair='XBTUSD', time_step=60, until=3600)
dl.set_trades_saver(IODataBase('/data/crypto/trades', method='parquet'))
dl.set_book_saver(IODataBase('/data/crypto/book', method='parquet'))
dl.run()