expobrain / pyexasol Goto Github PK

PyEXASOL is a custom Python driver for Exasol created in Badoo. It helps us to handle massive volumes of data commonly associated with this database.

You may expect significant performance improvement over existing ODBC / JDBC solutions in single process scenario involving pandas. It is also possible to split data set across multiple processes and servers to achieve linear scalability. With PyEXASOL you are not limited by single CPU core.

Quick links

• Getting started
• Reference
• Examples
• Best practices
• Local config (.ini file)
• SQL formatting
• HTTP Transport
• HTTP Transport (multiprocessing)
• SSL encryption
• Performance tests
• UDF scripts output
• DB-API 2.0 compatibility
• Optional dependencies
• Changelog

PyEXASOL main concepts

• Based on WebSocket protocol;
• Optimized for minimum overhead;
• Easy integration with pandas via HTTP transport;
• Compression to reduce network bottleneck;

System requirements

• Exasol >= 6
• Python >= 3.6

Getting started

Install PyEXASOL:

pip install pyexasol[pandas]

Run basic query:

import pyexasol

C = pyexasol.connect(dsn='<host:port>', user='sys', password='exasol')

stmt = C.execute("SELECT * FROM EXA_ALL_USERS")

for row in stmt:
    print(row)

Load data into pandas.DataFrame:

import pyexasol

C = pyexasol.connect(dsn='<host:port>', user='sys', password='exasol', compression=True)

df = C.export_to_pandas("SELECT * FROM EXA_ALL_USERS")
print(df.head())

You may set up local config to store your personal Exasol credentials and connection options:

import pyexasol

C = pyexasol.connect_local_config('my_config')

stmt = C.execute("SELECT CURRENT_TIMESTAMP")
print(stmt.fetchone())

Created by

Vitaly Markov, 2018

Enjoy!