Welcome to Download Crypto Currencies Data project

This is the documentation of dccd package, package to download crypto-currencies data from Binance, Bitfinex, Bitmex, GDax (Coinbase), Kraken and Poloniex.

Installation

From pip:

$ pip install dccd

From source:

$ git clone https://github.com/ArthurBernard/Download_Crypto_Currencies_Data.git

$ cd Download_Crypto_Currencies_Data

$ python setup.py install –user

Presentation

The dccd package allow you two main methods to download data. The first one is recommended to download data at high frequency (minutely or tick by tick), and the second one is recommended to download data at a lower frequency (hourly or daily):

• Continuous Downloader dccd.continuous_dl:

  Download and update continuously data (orderbook, trades tick by tick, ohlc, etc) and save it in a database. Currently only support Bitfinex and Bitmex exchanges.

• Historical Downloader dccd.histo_dl:

  Download historical data (ohlc, trades, etc.) and save it. Currently only support Binance, GDax, Kraken and Poloniex exchanges.

Contents

Continuous Downloader (dccd.continuous_dl)

Module to download continuously data.

Module to download continuously data (orderbook, trades, etc.) and update automatically the database. Currently only supports Bitfinex and Bitmex exchanges.

Bitfinex Continuous Downloader (dccd.continuous_dl.bitfinex)

Objects and functions to download data from Bitfinex exchange.

These functions and objects allow you to continuously download data and update your database.

High level API

dccd.continuous_dl.bitfinex.get_data_bitfinex(channel, process_func, process_params={}, save_method='dataframe', io_params={}, time_step=60, until=None, path=None, **kwargs)

Download data from Bitfinex exchange and update the database.

Parameters:

channel : str, {‘book’, ‘book_raw’, ‘trades’, ‘trades_raw’}

Websocket channel to get data, by default data will be aggregated (OHLC for ‘trades’ and reconstructed orderbook for ‘book’), add ‘_raw’ to the channel to get raw data (trade tick by tick or each orders).

process_func : callable

Function to process and clean data before to be saved. Must take data in arguments and can take any optional keywords arguments, cf function exemples in dccd.process_data.

process_params : dict, optional

Dictionary of the keyword arguments available to process_func, cf documentation into dccd.process_data.

save_method : {‘DataFrame’, ‘SQLite’, ‘CSV’, ‘Excel’, ‘PostgreSQL’, ‘Oracle’, ‘MSSQL’, ‘MySQL’},

It will create an IODataBase object to save/update the database in the specified format save_method, default is ‘DataFrame’ it save as binary pd.DataFrame object. More informations are available into dccd.tools.io.

io_params : dict, optional

Dictionary of the keyword arguments available to the dccd.tools.io.IODataBase callable method. Note: With SQL format some parameters are compulsory, see details into dccd.tools.io.

time_step : int, optional

Number of second between two snapshots of data, default 60 (1 minute).

until : int, optional

Number of seconds before stoping to download and update, default is None. If until equal 0 or None it means it never stop.

path : str, optional

Path to save/update the database, default is None. If path is None, database is saved at the relative path ‘./database/bitfinex/channel’.

**kwargs

Any revelevant keyword arguments will be passed to the websocket connector, see Bitfinex API documentation [2] for more details.

Warning

‘book_raw’ and ‘trades_raw’ can be very memory expensive.

See also

process_data

function to process/clean data (set_marketdepth, set_ohlc, set_orders, set_marketdepth).

tools.io.IODataBase

object to save/update the database with respect to specified format.

References

[2]	https://docs.bitfinex.com/v2/docs/ws-public

dccd.continuous_dl.bitfinex.get_orderbook_bitfinex(symbol, precision='P0', frequency='F0', lenght='25', time_step=60, until=None, path=None, save_method='dataframe', io_params={})

Download orderbook from Bitfinex exchange.

dccd.continuous_dl.bitfinex.get_trades_bitfinex(symbol, time_step=60, until=None, path=None, save_method='dataframe', io_params={})

Download trades tick by tick from Bitfinex exchange.

Low level API

class dccd.continuous_dl.bitfinex.DownloadBitfinexData(time_step=60, until=3600)

Bases: dccd.continuous_dl.exchange.ContinuousDownloader

Basis object to download data from a stream websocket client API.

Parameters:

time_step : int, optional

Number of seconds between two snapshots of data, minimum is 1, default is 60 (one minute). Each time_step data will be processed and updated to the database.

until : int, optional

Number of seconds before stoping or timestamp of when stoping, default is 3600 (one hour).

Attributes:

host : str

Adress of host to connect.

conn_par : dict

Parameters of websocket connection.

ws : websockets.client.WebSocketClientProtocol

Connection with the websocket client.

is_connect : bool

True if is connected, False otherwise.

ts : int

Number of second between two snapshots of data.

t : int

Current timestamp but rounded by ts.

until : int

Timestamp to stop to download data.

Methods

set_process_data(self, func, \*\*kwargs)	Set processing function.
set_saver(self, call, \*\*kwargs)	Set saver object to save data or update a database.
__call__(self, channel, \*\*kwargs)	Open a websocket connection and save/update the database.

__call__(self, channel, **kwargs)

Open a websocket connection and save/update the database.

Run asynchronously two loops to get data from bitfinex websocket and save/update the database.

Parameters:

channel : {‘book’, ‘book_raw’, ‘trades’, ‘trades_raw’}

Channel to get data, by default data will be aggregated (OHLC for ‘trades’ and reconstructed orderbook for ‘book’), add ‘_raw’ to the channel to get raw data (trade tick by tick or each orders).

**kwargs

Any revelevant keyword arguments will be passed to the websocket connector, see API documentation [1] for more details.

Warning

‘book_raw’ and ‘trades_raw’ can be very memory expensive.

References

[1]	https://docs.bitfinex.com/v2/docs/ws-public

get_parser(self, key)

Get allowed data parser.

Parameters:

key : str

Name code of data to parse. If key is not allowed then return a debug_parser which will display data structure.

Returns:

function

The allowed function to parse this kind of data.

on_close(self)

On websocket close print and fire event.

on_error(self, error, *args)

On websocket error print and fire event.

on_message(self, data)

Set data to order book.

on_open(self, **kwargs)

On websocket open.

Parameters:

**kwargs

Any relevant keyword arguments to set connection.

parser_book(self, data)

Parse market depth of order book.

Parameters:

data : list

Order data.

parser_raw_book(self, data)

Parse raw order book, each timestep set in a list all orders.

Parameters:

data : list

Order data.

parser_raw_trades(self, data)

Parse trade data.

Parameters:

data : list

Trade data.

parser_trades(self, data)

Parse OHLC data.

Parameters:

data : list

Trade data.

set_process_data(self, func, **kwargs)

Set processing function.

Parameters:

func : callable

Function to process and clean data before to be saved. Must take data in arguments and can take any optional keywords arguments, cf exemples in dccd.process_data.

**kwargs

Any keyword arguments to be passed to func.

set_saver(self, call, **kwargs)

Set saver object to save data or update a database.

Parameters:

call : callable

Callable object to save data or update a database. Must take data in arguments and can take any optional keywords arguments, cf exemples in dccd.io_tools.

**kwargs

Any keyword arguments to be passed to call.

wait_that(self, is_true)

Wait before running.

Bitmex Continuous Downloader (dccd.continuous_dl.bitmex)

Objects and functions to download data from Bitmex exchange.

These functions and objects allow you to continuously download data and update your database.

High level API

dccd.continuous_dl.bitmex.get_data_bitmex(process_func, *args, time_step=60, until=None, path=None, save_method='dataframe', io_params={}, **kwargs)

Download data from Bitmex exchange and update the database.

Parameters:

channel : str, {‘book’, ‘book_raw’, ‘trades’, ‘trades_raw’}

Websocket channel to get data, by default data will be aggregated (OHLC for ‘trades’ and reconstructed orderbook for ‘book’), add ‘_raw’ to the channel to get raw data (trade tick by tick or each orders).

process_func : callable

Function to process and clean data before to be saved. Must take data in arguments and can take any optional keywords arguments, cf function exemples in dccd.process_data.

process_params : dict, optional

Dictionary of the keyword arguments available to process_func, cf documentation into dccd.process_data.

save_method : {‘DataFrame’, ‘SQLite’, ‘CSV’, ‘Excel’, ‘PostgreSQL’, ‘Oracle’, ‘MSSQL’, ‘MySQL’},

It will create an IODataBase object to save/update the database in the specified format save_method, default is ‘DataFrame’ it save as binary pd.DataFrame object. More informations are available into dccd.tools.io.

io_params : dict, optional

Dictionary of the keyword arguments available to the dccd.tools.io.IODataBase callable method. Note: With SQL format some parameters are compulsory, seed details into dccd.tools.io.

time_step : int, optional

Number of second between two snapshots of data, default 60 (1 minute).

until : int, optional

Number of seconds before stoping to download and update, default is None. If until equal 0 or None it means it never stop.

path : str, optional

Path to save/update the database, default is None. If path is None, database is saved at the relative path ‘./database/bitmex/channel’.

**kwargs

Any revelevant keyword arguments will be passed to the websocket connector, see Bitmex API documentation [2] for more details.

Warning

‘_raw’ option not yet working for bitmex.

See also

process_data

function to process/clean data (set_marketdepth, set_ohlc, set_orders, set_marketdepth).

tools.io.IODataBase

object to save/update the database with respect to specified format.

References

[2]	https://www.bitmex.com/api/

dccd.continuous_dl.bitmex.get_orderbook_bitmex(*args, time_step=60, until=None, path=None, save_method='dataframe', io_params={})

Download orderbook from Bitmex exchange.

dccd.continuous_dl.bitmex.get_trades_bitmex(*args, time_step=60, until=None, path=None, save_method='dataframe', io_params={})

Download trades tick by tick from Bitmex exchange.

Low level API

class dccd.continuous_dl.bitmex.DownloadBitmexData(time_step=60, until=3600)

Bases: dccd.continuous_dl.exchange.ContinuousDownloader

Basis object to download data from a stream websocket client API.

Parameters:

time_step : int, optional

Number of seconds between two snapshots of data, minimum is 1, default is 60 (one minute). Each time_step data will be processed and updated to the database.

until : int, optional

Number of seconds before stoping or timestamp of when stoping, default is 3600 (one hour).

Attributes:

host : str

Adress of host to connect.

conn_par : dict

Parameters of websocket connection.

ws : websockets.client.WebSocketClientProtocol

Connection with the websocket client.

is_connect : bool

True if is connected, False otherwise.

ts : int

Number of second between two snapshots of data.

t : int

Current timestamp but rounded by ts.

until : int

Timestamp to stop to download data.

Methods

set_process_data(self, func, \*\*kwargs)	Set processing function.
set_saver(self, call, \*\*kwargs)	Set saver object to save data or update a database.
__call__(self, \*args)	Open a websocket connection and save/update the database.

__call__(self, *args)

Open a websocket connection and save/update the database.

Run asynchronously two loops to get data from Bitmex websocket and save/update the database.

Parameters:

channel : {‘orderBookL2_25’, ‘trade’}

Channel to get data, by default data will be aggregated (OHLC for ‘trades’ and reconstructed orderbook for ‘orderBookL2_25’), add ‘_raw’ to the channel to get raw data (trade tick by tick or each orders).

**kwargs

Any revelevant keyword arguments will be passed to the websocket connector, see API documentation [1] for more details.

Warning

‘_raw’ option not yet working for Bitmex.

References

[1]	https://www.bitmex.com/api/

get_parser(self, key)

Get allowed data parser.

Parameters:

key : str

Name code of data to parse. If key is not allowed then return a debug_parser which will display data structure.

Returns:

function

The allowed function to parse this kind of data.

on_close(self)

On websocket close print and fire event.

on_error(self, error, *args)

On websocket error print and fire event.

on_message(self, data)

Set data to order book.

on_open(self, **kwargs)

On websocket open.

Parameters:

**kwargs

Any relevant keyword arguments to set connection.

parser_book(self, data)

Parse data of order book.

Parameters:

data : dict

Order data from the ws API.

parser_trades(self, data)

Parse data of trades.

Parameters:

data : dict

Order data from the ws API.

set_process_data(self, func, **kwargs)

Set processing function.

Parameters:

func : callable

Function to process and clean data before to be saved. Must take data in arguments and can take any optional keywords arguments, cf exemples in dccd.process_data.

**kwargs

Any keyword arguments to be passed to func.

set_saver(self, call, **kwargs)

Set saver object to save data or update a database.

Parameters:

call : callable

Callable object to save data or update a database. Must take data in arguments and can take any optional keywords arguments, cf exemples in dccd.io_tools.

**kwargs

Any keyword arguments to be passed to call.

wait_that(self, is_true)

Wait before running.

High level API

bitfinex.get_data_bitfinex(channel, process_func)	Download data from Bitfinex exchange and update the database.
bitfinex.get_orderbook_bitfinex(symbol[, …])	Download orderbook from Bitfinex exchange.
bitfinex.get_trades_bitfinex(symbol[, …])	Download trades tick by tick from Bitfinex exchange.
bitmex.get_data_bitmex(process_func, \*args)	Download data from Bitmex exchange and update the database.
bitmex.get_orderbook_bitmex(\*args[, …])	Download orderbook from Bitmex exchange.
bitmex.get_trades_bitmex(\*args[, …])	Download trades tick by tick from Bitmex exchange.

Low level API

bitfinex.DownloadBitfinexData([time_step, until])	Basis object to download data from a stream websocket client API.
bitmex.DownloadBitmexData([time_step, until])	Basis object to download data from a stream websocket client API.

Historical Downloader (dccd.histo_dl)

Module to download historical data.

Module to download historical data (ohlc, trades, etc.) and automatically update the database. Currently only supports Binance, GDax, Kraken and Poloniex exchanges.

Only Poloniex allow you to download old historical data.

The ‘histo_dl’ module contains a main class and four classes to download and update data for each exchange.

The four classes to download data are FromBinance, FromGDax, FromKraken and FromPoloniex. All have the same methods and almost the same parameters:

• __init__(path, crypto, span, fiat(optional), form(optional)):

  Initialisation with path is the path where save the data (string), crypto is a crypto currency (string) and span is the interval time between each observation in seconds (integer) or can be a string as ‘hourly’, ‘daily’, etc. (see details on the doc string). The optional parameters are fiat the second currency (default is ‘USD’ and ‘USDT’ for poloniex and binance) and form the format to save the data (default is ‘xlsx’).

• import_data(start, end):

  Download data with start and end the timestamp (integer) or the date and time (string as ‘yyyy-mm-dd hh:mm:ss’), respectively of the first observation and the last observation (default are special parameters start=’last’ allow the last data saved and end=’now’ allow the last observation available). Exclusion: Kraken don’t allow the end parameter and provide only the thousand last observations.

• save(form(optional), by(optional)):

  Save the data with form the format of the saved data (default is ‘xlsx’) and by is the “size” of each saved file (default is ‘Y’ as an entire year). Exclusion: This optional parameters are in progress, let the default parameter for the moment, other are not allow.

• get_data():

  returns the data frame without any parameter.

Method chaining is available for these classes.

Binance Historical Downloader (dccd.histo_dl.binance)

Objects to download historical data from Binance exchange.

class dccd.histo_dl.binance.FromBinance(path, crypto, span, fiat='USD', form='xlsx')

Bases: dccd.histo_dl.exchange.ImportDataCryptoCurrencies

Class to import crypto-currencies data from the Binance exchange.

Parameters:

path : str

The path where data will be save.

crypto : str

The abreviation of the crypto-currency.

span : {int, ‘weekly’, ‘daily’, ‘hourly’}

• If str, periodicity of observation.
• If int, number of the seconds between each observation, minimal span is 60 seconds.

fiat : str

A fiat currency or a crypto-currency. Binance don’t allow fiat currencies, but USD theter.

form : {‘xlsx’, ‘csv’}

Your favorit format. Only ‘xlsx’ and ‘csv’ for the moment.

See also

FromGDax, FromKraken, FromPoloniex

Notes

See Binance API documentation [Rbab00e26f0b6-1] for more details on parameters.

References

[Rbab00e26f0b6-1]

https://github.com/binance-exchange/binance-official-api-docs

Attributes:

pair : str

Pair symbol, crypto + fiat.

start, end : int

Timestamp to starting and ending download data.

span : int

Number of seconds between observations.

full_path : str

Path to save data.

form : str

Format to save data.

Methods

import_data(self[, start, end])	Download data from Binance for specific time interval.
save(self[, form, by_period])	Save data by period (default is year) in the corresponding format and file.
get_data(self)	Print the dataframe.

get_data(self)

Print the dataframe.

Returns:

Data : pd.DataFrame

Current data.

import_data(self, start='last', end='now')

Download data from Binance for specific time interval.

Parameters:

start : int or str

Timestamp of the first observation of you want as int or date format ‘yyyy-mm-dd hh:mm:ss’ as string.

end : int or str

Timestamp of the last observation of you want as int or date format ‘yyyy-mm-dd hh:mm:ss’ as string.

Returns:

data : pd.DataFrame

Data sorted and cleaned in a data frame.

save(self, form='xlsx', by_period='Y')

Save data by period (default is year) in the corresponding format and file.

TODO : to finish

Parameters:

form : {‘xlsx’, ‘csv’}

Format to save data.

by_period : {‘Y’, ‘M’, ‘D’}

• If ‘Y’ group data by year.
• If ‘M’ group data by month.
• If ‘D’ group data by day.

set_hierarchy(self, liste)

Set the specific hierarchy of the files where will save your data.

TODO : to finish

GDax Historical Downloader (dccd.histo_dl.gdax)

Objects to download historical data from GDax exchange.

class dccd.histo_dl.gdax.FromGDax(path, crypto, span, fiat='USD', form='xlsx')

Bases: dccd.histo_dl.exchange.ImportDataCryptoCurrencies

Class to import crypto-currencies data from the GDax exchange.

Parameters:

path : str

The path where data will be save.

crypto : str

The abreviation of the crypto-currency.

span : {int, ‘weekly’, ‘daily’, ‘hourly’}

• If str, periodicity of observation.
• If int, number of the seconds between each observation, minimal span is 60 seconds.

fiat : str

A fiat currency or a crypto-currency.

form : {‘xlsx’, ‘csv’}

Your favorit format. Only ‘xlsx’ and ‘csv’ for the moment.

See also

FromBinance, FromKraken, FromPoloniex

Notes

See GDax API documentation [Rfd0c943f5f8a-1] for more details on parameters.

References

[Rfd0c943f5f8a-1]

https://docs.pro.coinbase.com/

Attributes:

pair : str

Pair symbol, crypto + fiat.

start, end : int

Timestamp to starting and ending download data.

span : int

Number of seconds between observations.

full_path : str

Path to save data.

form : str

Format to save data.

Methods

import_data(self[, start, end])	Download data from GDax for specific time interval.
save(self[, form, by_period])	Save data by period (default is year) in the corresponding format and file.
get_data(self)	Print the dataframe.

get_data(self)

Print the dataframe.

Returns:

Data : pd.DataFrame

Current data.

import_data(self, start='last', end='now')

Download data from GDax for specific time interval.

Parameters:

start : int or str

Timestamp of the first observation of you want as int or date format ‘yyyy-mm-dd hh:mm:ss’ as string.

end : int or str

Timestamp of the last observation of you want as int or date format ‘yyyy-mm-dd hh:mm:ss’ as string.

Returns:

data : pd.DataFrame

Data sorted and cleaned in a data frame.

save(self, form='xlsx', by_period='Y')

Save data by period (default is year) in the corresponding format and file.

TODO : to finish

Parameters:

form : {‘xlsx’, ‘csv’}

Format to save data.

by_period : {‘Y’, ‘M’, ‘D’}

• If ‘Y’ group data by year.
• If ‘M’ group data by month.
• If ‘D’ group data by day.

set_hierarchy(self, liste)

Set the specific hierarchy of the files where will save your data.

TODO : to finish

Kraken Historical Downloader (dccd.histo_dl.kraken)

Objects to download historical data from Kraken exchange.

class dccd.histo_dl.kraken.FromKraken(path, crypto, span, fiat='USD', form='xlsx')

Bases: dccd.histo_dl.exchange.ImportDataCryptoCurrencies

Class to import crypto-currencies data from the Kraken exchange.

Parameters:

path : str

The path where data will be save.

crypto : str

The abreviation of the crypto-currency.

span : {int, ‘weekly’, ‘daily’, ‘hourly’}

• If str, periodicity of observation.
• If int, number of the seconds between each observation, minimal span is 60 seconds.

fiat : str

A fiat currency or a crypto-currency.

form : {‘xlsx’, ‘csv’}

Your favorit format. Only ‘xlsx’ and ‘csv’ for the moment.

See also

FromBinance, FromGDax, FromPoloniex

Notes

See Kraken API documentation [R1ab1d422d7d0-1] for more details on parameters.

References

[R1ab1d422d7d0-1]

https://www.kraken.com/features/api

Attributes:

pair : str

Pair symbol, crypto + fiat.

start, end : int

Timestamp to starting and ending download data.

span : int

Number of seconds between observations.

full_path : str

Path to save data.

form : str

Format to save data.

Methods

import_data(self[, start, end])	Download data from Kraken since a specific time until now.
save(self[, form, by_period])	Save data by period (default is year) in the corresponding format and file.
get_data(self)	Print the dataframe.

get_data(self)

Print the dataframe.

Returns:

Data : pd.DataFrame

Current data.

import_data(self, start='last', end=None)

Download data from Kraken since a specific time until now.

Parameters:

start : int or str

Timestamp of the first observation of you want as int or date format ‘yyyy-mm-dd hh:mm:ss’ as string.

Returns:

data : pd.DataFrame

Data sorted and cleaned in a data frame.

save(self, form='xlsx', by_period='Y')

Save data by period (default is year) in the corresponding format and file.

TODO : to finish

Parameters:

form : {‘xlsx’, ‘csv’}

Format to save data.

by_period : {‘Y’, ‘M’, ‘D’}

• If ‘Y’ group data by year.
• If ‘M’ group data by month.
• If ‘D’ group data by day.

set_hierarchy(self, liste)

Set the specific hierarchy of the files where will save your data.

TODO : to finish

Poloniex Historical Downloader (dccd.histo_dl.poloniex)

Objects to download historical data from Poloniex exchange.

class dccd.histo_dl.poloniex.FromPoloniex(path, crypto, span, fiat='USD', form='xlsx')

Bases: dccd.histo_dl.exchange.ImportDataCryptoCurrencies

Class to import crypto-currencies data from the Poloniex exchange.

Parameters:

path : str

The path where data will be save.

crypto : str

The abreviation of the crypto-currency.

span : {int, ‘weekly’, ‘daily’, ‘hourly’}

• If str, periodicity of observation.
• If int, number of the seconds between each observation, minimal span is 300 seconds.

fiat : str

A fiat currency or a crypto-currency. Poloniex don’t allow fiat currencies, but USD theter.

form : {‘xlsx’, ‘csv’}

Your favorit format. Only ‘xlsx’ and ‘csv’ for the moment.

See also

FromBinance, FromGDax, FromKraken

Notes

See Poloniex API documentation [R5770b2672718-1] for more details on parameters.

References

[R5770b2672718-1]

https://docs.poloniex.com/#introduction

Attributes:

pair : str

Pair symbol, crypto + fiat.

start, end : int

Timestamp to starting and ending download data.

span : int

Number of seconds between observations.

full_path : str

Path to save data.

form : str

Format to save data.

Methods

import_data(self[, start, end])	Download data from Poloniex for specific time interval.
save(self[, form, by_period])	Save data by period (default is year) in the corresponding format and file.
get_data(self)	Print the dataframe.

get_data(self)

Print the dataframe.

Returns:

Data : pd.DataFrame

Current data.

import_data(self, start='last', end='now')

Download data from Poloniex for specific time interval.

Parameters:

start : int or str

Timestamp of the first observation of you want as int or date format ‘yyyy-mm-dd hh:mm:ss’ as string.

end : int or str

Timestamp of the last observation of you want as int or date format ‘yyyy-mm-dd hh:mm:ss’ as string.

Returns:

data : pd.DataFrame

Data sorted and cleaned in a data frame.

save(self, form='xlsx', by_period='Y')

Save data by period (default is year) in the corresponding format and file.

TODO : to finish

Parameters:

form : {‘xlsx’, ‘csv’}

Format to save data.

by_period : {‘Y’, ‘M’, ‘D’}

• If ‘Y’ group data by year.
• If ‘M’ group data by month.
• If ‘D’ group data by day.

set_hierarchy(self, liste)

Set the specific hierarchy of the files where will save your data.

TODO : to finish

Processing Data (dccd.process_data)

Functions to clean, sort and other process data.

dccd.process_data.set_marketdepth(book, t=None)

Set a market depth dataframe with list of order books.

Parameters:

book : dict

Orderbook as dict, where keys is the price and value is the amount.

Returns:

pd.DataFrame

Order book as dataframe.

dccd.process_data.set_ohlc(trades, ts=60)

Aggregate and set a dataframe with list of trades.

Parameters:

trades : list

Historical trades tick by tick as list.

ts : int, optional

Timestep in seconds to aggregate data, default is 60.

Returns:

pd.DataFrame

Aggregated trades as OHLC, dataframe is indexed by timestamp and columns contains ‘open’, ‘high’, ‘low’, ‘close’, and ‘volume’.

dccd.process_data.set_orders(orders, t=None)

Set a dataframe with list of each order.

Parameters:

orders : list

Each orders in a list.

Returns:

pd.DataFrame

List of orders as dataframe.

dccd.process_data.set_trades(trades)

Set a dataframe with list of trades.

Parameters:

trades : list

Historical trades tick by tick as list.

Returns:

pd.DataFrame

Historical trades tick by tick as dataframe.

Tools (dccd.tools)

Module with I/O, date/time and websocket tools.

Time and date tools (dccd.tools.date_time)

Tools to manage some time functions with respect to specific API exchanges.

dccd.tools.date_time.TS_to_date(TS, form='%Y-%m-%d %H:%M:%S', local=True)

Convert timestamp to date in specified format.

Parameters:

TS : int

A timestamp to convert.

form : str (default ‘%Y-%m-%d %H:%M:%S’)

Time format.

local : bool (default is True)

Local time is used if true else return UTC time.

Returns:

date : str

Date as specified format.

Examples

>>> TS_to_date(1548432099, form='%y-%m-%d %H:%M:%S', local=False)
'19-01-25 16:01:39'

dccd.tools.date_time.date_to_TS(date, form='%Y-%m-%d %H:%M:%S')

Use your local time-zone to convert date in specific format to timestamp.

Parameters:

date : str

A date to convert.

form : str (default ‘%Y-%m-%d %H:%M:%S’)

Time format.

Returns:

TS : int

Timestamp of specified date.

Examples

# >>> date_to_TS(‘19-01-25 16:01:39’, form=’%y-%m-%d %H:%M:%S’) # 1548428499

dccd.tools.date_time.str_to_span(string)

Return the equivalent interval time in seconds.

Parameters:

string : str

Time periodicity

Returns:

span : int

Number of seconds in time interval.

Examples

>>> str_to_span('minutely')
60

dccd.tools.date_time.span_to_str(span)

Return the time periodicity.

Parameters:

span : int

Time interval in second.

Returns:

date : str

Time periodicity.

Examples

>>> span_to_str(3600)
'Hourly'

dccd.tools.date_time.binance_interval(interval)

Return the time interval in the specific format allowed by Binance.

Parameters:

interval : int

Must be in seconds as 60, 180, 300, 900, 1800, 3600, 7200, 14400, 21600, 28800, 43200, 86400, 259200, 604800, 2592000.

Returns:

form : str

Specific format allowed by Binance.

Examples

>>> binance_interval(7200)
'2h'

Input/Output tools (dccd.tools.io)

Tools and object to load, append and save differnet kind of database.

class dccd.tools.io.IODataBase(path='./', method='csv')

Object to save a pd.DataFrame into different kind/format of database.

Parameters:

path : str, optional

Path of the database, default is ‘./’ (current directory).

method : {‘DataFrame’, ‘SQLite’, ‘CSV’, ‘Excel’, ‘PostgreSQL’, ‘Oracle’, ‘MSSQL’, ‘MySQL’}

Format of database, default is CSV.

Attributes:

path : str

Path of the database.

method : {‘DataFrame’, ‘SQLite’, ‘CSV’, ‘Excel’, ‘PostgreSQL’, ‘Oracle’, ‘MSSQL’, ‘MySQL’}

Kind/format of the database.

parser : dict

Values are function to corresponding to method.

Methods

save_as_dataframe(self, new_data[, name, ext])	Append and save new_data as pd.DataFrame binary object.
save_as_sql(self, new_data[, table, name, …])	Append and save new_data in SQL database.
save_as_sqlite(self, new_data[, table, …])	Append and save new_data in SQLite database.
save_as_csv(self, new_data[, name, ext, …])	Append and save new_data in database as CSV format.
save_as_excel(self, new_data[, name, …])	Append and save new_data in database as Excel format.
__call__(self, new_data, \*\*kwargs)	Append and save new_data in database as method format.

get_from_dataframe(self, name, ext='.dat')

Get data from pd.DataFrame binary object.

With pickle get as binary pd.DataFrame object.

Parameters:

name : str

Name of the database.

ext : str, optional

Extension of the database, default is ‘.dat’.

get_from_sqlite(self, name, table='main_table', ext='.db')

Get data from SQLite database.

Parameters:

name : str, optional

Name of the database, default is the current year.

table : str, optional

Name of the table, default is ‘main_table’.

ext : str, optional

Extension of the database, default is ‘.db’.

index : bool, optional

Write pd.DataFrame index as a column. Uses index_label as the column name in the table. Default is True.

index_label : string or sequence, optional

Column label for index column(s). If None is given (default) and index is True, then the index names are used. A sequence should be given if the pd.DataFrame uses pd.MultiIndex.

save_as_csv(self, new_data, name=None, ext='.csv', index=True, index_label=None)

Append and save new_data in database as CSV format.

With pickle save as binary pd.DataFrame object, if name database already exists append new_data, otherwise create a new file.

Parameters:

new_data : pd.DataFrame

Data to append to the database.

name : str, optional

Name of the database, default is the current year.

ext : str, optional

Extension of the database, default is ‘.csv’.

index : bool, optional

Write row names (index), default is True.

index_label : str or sequence, optional

Column label for index column(s) if desired. If not specified (default is None), and index are True, then the index names are used. A sequence should be given if the DataFrame uses MultiIndex. If False do not print fields for index names. Use index_label=False for easier importing in R.

save_as_dataframe(self, new_data, name=None, ext='.dat')

Append and save new_data as pd.DataFrame binary object.

With pickle save as binary pd.DataFrame object, if name database already exists, load it, append new_data and save it, else create a new database.

Parameters:

new_data : pd.DataFrame

Data to append to the database.

name : str, optional

Name of the database, default is the current date.

ext : str, optional

Extension of the database, default is ‘.dat’.

save_as_excel(self, new_data, name=None, sheet_name='Sheet1', ext='.xlsx', index=True, index_label=None)

Append and save new_data in database as Excel format.

With pickle save as binary pd.DataFrame object, if name database already exists append new_data, else create a new file.

Parameters:

new_data : pd.DataFrame

Data to append to the database.

name : str, optional

Name of the database, default is the current date.

sheet_name : str, optional

Name of sheet which will contain new_data, default is ‘Sheet1’.

ext : str, optional

Extension of the database, default is ‘.xlsx’.

index : bool, optional

Write row names (index), default is True.

index_label : str or sequence, optional

Column label for index column(s) if desired. If not specified (default is None), and index are True, then the index names are used. A sequence should be given if the DataFrame uses MultiIndex.

Warning

Slow method, not recommanded for large database.

save_as_sql(self, new_data, table='main_table', name=None, ext='', index=True, index_label=None, driver=None, username=None, password=None, host=None, port=None, **kwargs)

Append and save new_data in SQL database.

SQL database as method={‘PostgreSQL’, ‘Oracle’, ‘MSSQL’, ‘MySQL’}. If name already exists append new_data, else create a new database. See SQLAlchemy documentation for more details [1].

Parameters:

new_data : pd.DataFrame

Data to append to the database.

table : str, optional

Name of the table, default is ‘main_table’.

name : str, optional

Name of the database, default is the current year.

ext : str, optional

Extension of the database, default is ‘.db’.

index : bool, optional

Write pd.DataFrame index as a column. Uses index_label as the column name in the table. Default is True.

index_label : string or sequence, optional

Column label for index column(s). If None is given (default) and index is True, then the index names are used. A sequence should be given if the pd.DataFrame uses pd.MultiIndex.

driver : {‘psycopg2’, ‘pg8000’, ‘mysqlclient’, pymysql’, ‘cx_oracle’, ‘pyodbc’, ‘pymssql’}

The name of the DBAPI to be used to connect to the database using all lowercase letters. If not specified, a default DBAPI will be imported if available - this default is typically the most widely known driver available for that backend.

username, password : str

Username and password to connect to the SQL database.

host : str, optional

Host to connect, default is ‘localhost’.

port : str, optional

The port number, default is None.

kwargs : dict, optional

A dictionary of options to be passed to the dialect and/or the DBAPI upon connect.

References

[1]	https://docs.sqlalchemy.org/en/13/core/engines.html

save_as_sqlite(self, new_data, table='main_table', name=None, ext='.db', index=True, index_label=None)

Append and save new_data in SQLite database.

With sqlite, if name database already exists append new_data, else create a new data base.

Parameters:

new_data : pd.DataFrame

Data to append to the database.

table : str, optional

Name of the table, default is ‘main_table’.

name : str, optional

Name of the database, default is the current year.

ext : str, optional

Extension of the database, default is ‘.db’.

index : bool, optional

Write pd.DataFrame index as a column. Uses index_label as the column name in the table. Default is True.

index_label : string or sequence, optional

Column label for index column(s). If None is given (default) and index is True, then the index names are used. A sequence should be given if the pd.DataFrame uses pd.MultiIndex.

dccd.tools.io.get_df(path, name, ext='')

Load a dataframe as binnary file.

Parameters:

path, name, ext : str

Path to the file, name of the file and the extension of the file.

Returns:

pandas.DataFrame

A dataframe, if file not find return an empty dataframe.

dccd.tools.io.save_df(df, path, name, ext='')

Save a dataframe as a binnary file.

Parameters:

df : pandas.DataFrame

A dataframe to save as binnary file.

path, name, ext : str

Path to the file, name of the file and the extension of the file.

Websocket Tools (dccd.tools.websocket)

Connector objects to WebSockets API client to download data.

class dccd.tools.websocket.BasisWebSocket(host, conn={}, subs={})

Basis object to connect at a specified stream to websocket client API.

Parameters:

host : str

Adress of host to connect.

conn : dict

Parameters to connection setting.

subs : dict

Data to subscribe to a stream.

Attributes:

host : str

Adress of host to connect.

conn_para : dict

Parameters of websocket connection.

ws : websockets.client.WebSocketClientProtocol

Connection with the websocket client.

is_connect : bool

• True if connected.
• False`otherwise.

Methods

on_open(self, \*\*kwargs)

On websocket open.

on_close(self)

On websocket close print and fire event.

on_error(self, error, *args)

On websocket error print and fire event.

on_message(self, message)

On websocket display message.

on_open(self, **kwargs)

On websocket open.

Parameters:

**kwargs

Any relevant keyword arguments to set connection.

wait_that(self, is_true)

Wait before running.