The DataFrame serialisation is slower than in v1

[benb92]

Using python pandas. Version 1 i used this:

def dbpop_influx(data, dbname, measurement, columns):
    ## constants:
    dbclient = DataFrameClient(host='localhost', port=8086, username='root', password='root', database=dbname)
    n_import_chunks = math.ceil(len(data) / 10000)
    data_chunks = np.array_split(data, n_import_chunks)
    for d in data_chunks:
        dbclient.write_points(d, measurement, tag_columns = columns, protocol = 'line')

Takes 29 seconds (was looking to improve that speed with multiprocessing)

Version 2 i used this:

_client = InfluxDBClient(url="http://localhost:9999", token=token, org="org")
_write_client = _client.write_api(write_options=WriteOptions(batch_size=10000,
                                                             flush_interval=10_000,
                                                             jitter_interval=0,
                                                             retry_interval=5_000))


start = time.time()
_write_client.write('data', record=imp_dat[0], data_frame_measurement_name='coinmarketcap_ohlcv',
                    data_frame_tag_columns=['quote_asset','base_asset'])
print(time.time() - start)

this takes 118 seconds...

data looks like:

@bednar

[bednar]

Randomized data to testing: https://we.tl/t-QBtxHlvfvx

Time format:

[bednar]

Hi @bburden,

The issue is fixed in 1.8.0 milestone.

If you would like to use a dev version then install client via:

pip install git+https://github.com/influxdata/influxdb-client-python.git@master

Regards