housepower / clickhouse-native-jdbc Goto Github PK

View Code? Open in Web Editor NEW

520.0 520.0 147.0 2.11 MB

ClickHouse Native Protocol JDBC implementation

Home Page: https://housepower.github.io/ClickHouse-Native-JDBC/

License: Apache License 2.0

Java 97.42% Scala 2.32% Shell 0.27%

analytics clickhouse clickhouse-client database jdbc spark tcp-protocol

clickhouse-native-jdbc's Issues

java.sql.SQLException occurs when using nullIf() with DateTime type

When using the driver in Scala with the following code, a java.sql.SQLException is thrown:

val stmt = connection.createStatement()
stmt.executeQuery("SELECT nullIf(toDateTime('2018-06-01 00:00:00'), toDateTime(0))")

How to access the MySQL directly?

ClickHouse is pretty cool.

I want to use the SQL with ClickHouse-Native-JDBC to connect to the MySQL by using ClickHouse.

How to do? Any idea about ClickHouse and MySQL.

Maybe is it slow for ClickHouse with ODBC?

I want to use JDBC directly with ClickHouse and MySQL.

Batch insert into CK but get more records.

JDBC version in pom.xml

com.github.housepower clickhouse-native-jdbc 1.6-stable net.jpountz.lz4 lz4

ClickHouse version

ru.yandex.clickhouse clickhouse-jdbc 0.1.52 net.jpountz.lz4 lz4 guava com.google.guava

Error logs

no error logs

Steps to reproduce

I use Java language.
First, I use spark to query Hive table to produce a DataSet.Which count is 2w record.
Then, I use spark.forearchPartition ,and use iterator to insert into ck which is preparestatement ,and I count the number I have inserted . It's also 20000 record.
Finally, when the job have completed, there is no error logs. It seems every thing is ok (Because I have catch all exception and print messages but get nothing error). But when I query clickhouse use client , it show it has 360736 record in db.

Other descriptions

At first, I use distribute table , and I thought it is the reason which cause this problem . Then I alter the target table as a normal table,but it has no use. And I have try to insert about 17,000,000 record to a distribute table ,but I query about 200,000,000+ records in clickhouse client.

Add README doc

Error in sending request.

Class.forName("com.github.housepower.jdbc.ClickHouseDriver");
    Connection connection = DriverManager.getConnection("jdbc:clickhouse://115.29.37.214:9000");
    Statement stmt = connection.createStatement();
    ResultSet rs = stmt.executeQuery("SELECT (number % 3 + 1) as n, sum(number) FROM numbers(10000000) GROUP BY n");

    while (rs.next()) {
      System.out.println(rs.getInt(1) + "\t" + rs.getLong(2));
    }

Exception in thread "main" java.sql.SQLException: Broken pipe (Write failed)
	at com.github.housepower.jdbc.connect.PhysicalConnection.disPhysicalConnection(PhysicalConnection.java:103)
	at com.github.housepower.jdbc.ClickHouseConnection.getHealthyPhysicalConnection(ClickHouseConnection.java:146)

Support ResultSetMetaData

Use travis for CI

Date type is wrong when insert into clickhouse

when I insert into clickhouse table , I met a problem：
code：

`
Statement stmt = connection.createStatement();
stmt.executeQuery("drop table if exists test_jdbc_example");
stmt.executeQuery("create table test_jdbc_example(day Date, name String, age UInt8) Engine=Log");

    PreparedStatement pstmt = connection.prepareStatement("INSERT INTO test_jdbc_example VALUES(?, ?, ?)");

    for (int i = 0; i < 10; i++) {
        long time = 1530374400;
        System.out.println(new Date((time) * 1000));
        pstmt.setDate(1, new Date(time*1000));
        pstmt.setString(2, "Zhang San" + i);
        pstmt.setByte(3, (byte)i);
        pstmt.addBatch();
    }
    pstmt.executeBatch();

1530374400 convert to Date is ‘2018-07-01’， but in clickhouse Table，it show ‘2018-06-30’，I use it the wrong way？thank you.

SELECT *
FROM test_jdbc_example

┌────────day─┬─name───────┬─age─┐
│ 2018-06-30 │ Zhang San0 │ 0 │
│ 2018-06-30 │ Zhang San1 │ 1 │
│ 2018-06-30 │ Zhang San2 │ 2 │

connect_timeout exception

Connection connection = DriverManager.getConnection("jdbc:clickhouse://127.0.0.1:9000?connect_timeout=10");

use connect_timeout throw exception:
java.lang.ClassCastException: java.lang.Long cannot be cast to java.lang.Integer

Port 9000 is for clickhouse-client program

I user ClickHouse-Native-JDBC to send data into clickhouse,but get expection like this:
java.lang.RuntimeException: ru.yandex.clickhouse.except.ClickHouseUnknownException: ClickHouse exception, code: 1002, host: 172.18.164.98, port: 9000; Port 9000 is for clickhouse-client program.
You must use port 8123 for HTTP.

I don't know that i should add Clickhouse-Jdbc dependency into pom.xml or not .But when i don't add Clickhouse-Jdbc dependency,there is a exception that is No suitable driver found for jdbc url.

The version of clickhouse is 18.1.0

Error "Unknown compression method: 108" when doing arrayMap() with tuple() over two distinctly-typed arrays.

Reproduced on the version 1.5-stable with:

Class.forName("com.github.housepower.jdbc.ClickHouseDriver");
Connection connection = DriverManager.getConnection("jdbc:clickhouse://localhost:9000","default","");
Statement stmt = connection.createStatement();
val rs = stmt.executeQuery("select distinct arrayMap((x,y) -> tuple(x,y), nums, names) from (select arrayJoin([1,2]) as id, id = 1 ? [1,2,3] : [10,20,30] as nums, id = 1 ? ['one', 'two', 'three'] : ['a','b','c'] as names)")

Freezing on Read

Hi there,
I'm unable to get this lib to work. It is always freezing for me. Do you have any idea why?

Support PreparedStatement

java.io.EOFException: Attempt to read after eof.

Hello!

I'm getting an error with insert batch query once in a while:

Exception in thread "main" java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.spark.deploy.worker.DriverWrapper$.main(DriverWrapper.scala:65)
	at org.apache.spark.deploy.worker.DriverWrapper.main(DriverWrapper.scala)
Caused by: org.apache.spark.SparkException: Job aborted due to stage failure: Task 118 in stage 5221.0 failed 4 times, most recent failure: Lost task 118.3 in stage 5221.0 (TID 626642, 10.111.65.213, executor 35): java.sql.SQLException: Attempt to read after eof.
	at com.github.housepower.jdbc.connect.PhysicalConnection.receiveResponse(PhysicalConnection.java:88)
	at com.github.housepower.jdbc.connect.PhysicalConnection.receiveEndOfStream(PhysicalConnection.java:78)
	at com.github.housepower.jdbc.ClickHouseConnection.sendInsertRequest(ClickHouseConnection.java:138)
	at com.github.housepower.jdbc.statement.ClickHousePreparedInsertStatement.executeBatch(ClickHousePreparedInsertStatement.java:61)
	at com.hellobike.SoaToClickHouse$$anonfun$main$1$$anonfun$apply$1.apply(SoaToClickHouse.scala:107)
	at com.hellobike.SoaToClickHouse$$anonfun$main$1$$anonfun$apply$1.apply(SoaToClickHouse.scala:75)
	at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:929)
	at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:929)
	at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2074)
	at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2074)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
	at org.apache.spark.scheduler.Task.run(Task.scala:109)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.EOFException: Attempt to read after eof.
	at com.github.housepower.jdbc.buffer.SocketBuffedReader.refill(SocketBuffedReader.java:67)
	at com.github.housepower.jdbc.buffer.SocketBuffedReader.readBinary(SocketBuffedReader.java:34)
	at com.github.housepower.jdbc.serializer.BinaryDeserializer.readVarInt(BinaryDeserializer.java:23)
	at com.github.housepower.jdbc.protocol.RequestOrResponse.readFrom(RequestOrResponse.java:32)
	at com.github.housepower.jdbc.connect.PhysicalConnection.receiveResponse(PhysicalConnection.java:86)
	... 15 more

FixedString with Nullable broken

When retrieving a string from the ResultSet which is a Nullable(FixedString(n)) Column it returns ???????????????????????????????????? (probably something different and just encoded to ?) instead of null.

when batch insert FixedString type ,it can't work!

Add integration tests with MyBatis

Does it support read data in terms for stream?

like mysql setting prepareStatement.setFetchSize(Integer.MIN_VALUE);

Release the first preview

Support isValid for Connection

Support SQLConnection.getCatalog API

we try to integrate this clickhouse JDBC connection with c3p0 data source pool. But we encounter the below issue

java.sql.SQLFeatureNotSupportedException
	at com.github.housepower.jdbc.wrapper.SQLConnection.getCatalog(SQLConnection.java:88)
	at com.mchange.v2.c3p0.impl.NewPooledConnection.<init>(NewPooledConnection.java:108)
	at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:159)
	at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:132)
	at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:137)
	at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1014)
	at com.mchange.v2.resourcepool.BasicResourcePool.access$800(BasicResourcePool.java:32)
	at com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask.run(BasicResourcePool.java:1810)
	at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:547)

related source code

 private final ComboPooledDataSource cpds = new ComboPooledDataSource();

  public Connection getConnection() throws SQLException {
    return cpds.getConnection();
  }

  C3poDataSource() {
    try {
      cpds.setDriverClass("com.github.housepower.jdbc.ClickHouseDriver");
      cpds.setJdbcUrl("jdbc:clickhouse://10.8.121.33:9000");
      cpds.setMinPoolSize(10);
      cpds.setMaxPoolSize(10);

    } catch (PropertyVetoException e) {
      // handle the exception
    }
  }

How to set the database in the JDBC connection url.

Class.forName("com.github.housepower.jdbc.ClickHouseDriver");
Connection connection = 
       DriverManager.getConnection("jdbc:clickhouse://127.0.0.1:9001/test");

Error, Can not the database named test.

No support for unsigned int values.

Hi, unfortunately, the driver doesn't support UINT.

i.e. UINT16 - value 36199 will be return as -29337.

Looks like the getObject->Number makes wrong byte shift i.e. ("36199 - 65535 (max uin16))) and as result we got wrong value. IMHO the best way implement getUINT8/16/32 in the ResultSet, because the beauty of CH is unsigned numbers ;-)

Support INSERT INTO with TCP Protocol

server 1.1.54289 jdbc 1.3 insert data

Exception in thread "main" com.github.housepower.jdbc.ClickHouseSQLException: DB::ExceptionDB::Exception: Unknown compression method: 226. Stack trace:

clickhouse-server(StackTrace::StackTrace()+0x16) [0x2fe5936]
clickhouse-server(DB::Exception::Exception(std::string const&, int)+0x1f) [0x1293c2f]
clickhouse-server(DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&)+0x7ca) [0x3003cea]
clickhouse-server(DB::CompressedReadBuffer::nextImpl()+0x24) [0x3002f54]
clickhouse-server() [0x2963ff0]
clickhouse-server(DB::NativeBlockInputStream::readData(DB::IDataType const&, DB::IColumn&, DB::ReadBuffer&, unsigned long, double)+0x125) [0x29287d5]
clickhouse-server(DB::NativeBlockInputStream::readImpl()+0x7b6) [0x2929276]
clickhouse-server(DB::IProfilingBlockInputStream::read()+0x1e6) [0x2925116]
clickhouse-server(DB::TCPHandler::receiveData()+0x58) [0x129fc78]
clickhouse-server(DB::TCPHandler::receivePacket()+0x1b5) [0x12a0335]
clickhouse-server(DB::TCPHandler::readData(DB::Settings const&)+0x142) [0x12a06a2]
clickhouse-server(DB::TCPHandler::processInsertQuery(DB::Settings const&)+0x241) [0x12a1311]
clickhouse-server(DB::TCPHandler::runImpl()+0x647) [0x12a1ab7]
clickhouse-server(DB::TCPHandler::run()+0x1c) [0x12a263c]
clickhouse-server(Poco::Net::TCPServerConnection::start()+0xf) [0x384ddef]
clickhouse-server(Poco::Net::TCPServerDispatcher::run()+0x10b) [0x384f5fb]
clickhouse-server(Poco::PooledThread::run()+0x87) [0x39559b7]
clickhouse-server(Poco::ThreadImpl::runnableEntry(void*)+0x96) [0x3950546]
/lib64/libpthread.so.0(+0x7dc5) [0x7f2889116dc5]
/lib64/libc.so.6(clone+0x6d) [0x7f2886bf373d]

at com.github.housepower.jdbc.protocol.ExceptionResponse.readExceptionFrom(ExceptionResponse.java:32)
at com.github.housepower.jdbc.protocol.RequestOrResponse.readFrom(RequestOrResponse.java:38)
at com.github.housepower.jdbc.connect.PhysicalConnection.receiveResponse(PhysicalConnection.java:86)
at com.github.housepower.jdbc.connect.PhysicalConnection.receiveEndOfStream(PhysicalConnection.java:78)
at com.github.housepower.jdbc.ClickHouseConnection.sendInsertRequest(ClickHouseConnection.java:132)
at com.github.housepower.jdbc.statement.ClickHousePreparedInsertStatement.executeBatch(ClickHousePreparedInsertStatement.java:61)
at lyd.ai.ck.NativeC2CK.generateRowsList(NativeC2CK.java:59)
at lyd.ai.ck.NativeC2CK.main(NativeC2CK.java:68)

java.lang.ClassCastException: class [B cannot be cast to class [C ([B and [C are in module java.base of loader 'bootstrap')

Hello!

Not sure why, but I'm getting an error with insert batch query:

java.lang.ClassCastException: class [B cannot be cast to class [C ([B and [C are in module java.base of loader 'bootstrap')
	at com.github.housepower.jdbc.misc.StringViewCoding.getValues(StringViewCoding.java:25)
	at com.github.housepower.jdbc.misc.SQLLexer.<init>(SQLLexer.java:11)
	at com.github.housepower.jdbc.stream.ValuesWithParametersInputFormat.<init>(ValuesWithParametersInputFormat.java:27)
	at com.github.housepower.jdbc.statement.ClickHousePreparedInsertStatement.executeBatch(ClickHousePreparedInsertStatement.java:61)
	....

Date type is wrong when insert into clickhouse

How to read arrays, tuples, etc.

Maybe this is what you mean by Non-values format, but how do I access array, tuple, etc. values? This makes me think that it just hasn't been implemented yet?

https://github.com/housepower/ClickHouse-Native-JDBC/blob/master/src/main/java/com/github/housepower/jdbc/wrapper/SQLArray.java

UUID support

Hi,

when using clickhouses UUID conversion with a fixed String I'm ending up with this error (for write and read):

[04:55:41] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[04:55:41] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at java.lang.Thread.run(Unknown Source)
[04:55:42] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]: java.sql.SQLException
[04:55:42] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at com.github.housepower.jdbc.misc.Validate.isTrue(Validate.java:14)
[04:55:42] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at com.github.housepower.jdbc.misc.Validate.isTrue(Validate.java:8)
[04:55:42] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at com.github.housepower.jdbc.misc.SQLLexer.stringLiteral(SQLLexer.java:72)
[04:55:42] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at com.github.housepower.jdbc.data.type.complex.DataTypeFixedString.deserializeTextQuoted(DataTypeFixedString.java:60)
[04:55:42] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at com.github.housepower.jdbc.stream.ValuesWithParametersInputFormat.next(ValuesWithParametersInputFormat.java:53)
[04:55:42] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at com.github.housepower.jdbc.ClickHouseConnection.sendInsertRequest(ClickHouseConnection.java:134)
[04:55:42] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at com.github.housepower.jdbc.statement.ClickHousePreparedInsertStatement.executeBatch(ClickHousePreparedInsertStatement.java:61)
[04:55:42] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at com.helion3.prism.storage.clickhouse.ClickhouseRecords.write(ClickhouseRecords.java:65)
[04:55:42] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at com.helion3.prism.queues.RecordingQueueManager.run(RecordingQueueManager.java:49)
[04:55:42] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at org.spongepowered.api.scheduler.Task$Builder.lambda$execute$0(Task.java:139)
[04:55:42] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at org.spongepowered.common.scheduler.SchedulerBase.lambda$startTask$0(SchedulerBase.java:192)
[04:55:42] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
[04:55:42] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at java.util.concurrent.FutureTask.run(Unknown Source)
[04:55:42] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[04:55:42] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[04:55:42] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at java.lang.Thread.run(Unknown Source)

When doing the conversion my self, I'm ending up with setBytes not being supported:

[05:03:05] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]: java.sql.SQLFeatureNotSupportedException
[05:03:05] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at com.github.housepower.jdbc.wrapper.SQLStatement.setBytes(SQLStatement.java:295)
[05:03:05] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at com.helion3.prism.storage.clickhouse.ClickhouseRecords.write(ClickhouseRecords.java:54)
[05:03:05] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at com.helion3.prism.queues.RecordingQueueManager.run(RecordingQueueManager.java:49)
[05:03:05] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at org.spongepowered.api.scheduler.Task$Builder.lambda$execute$0(Task.java:139)
[05:03:05] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at org.spongepowered.common.scheduler.SchedulerBase.lambda$startTask$0(SchedulerBase.java:192)
[05:03:05] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
[05:03:05] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at java.util.concurrent.FutureTask.run(Unknown Source)
[05:03:05] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[05:03:05] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[05:03:05] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.storage.clickhouse.ClickhouseRecords:write:68]:  at java.lang.Thread.run(Unknown Source)

Can this be supported or is there a workaround?

Support SQLStatement FetchSize Injection

Can we support SQLStatement FetchSize Injection?

ClickHouse-Native-JDBC/src/main/java/com/github/housepower/jdbc/wrapper/SQLStatement.java

Lines 113 to 115 in 7ece148

 public void setFetchSize(int rows) throws SQLException { 

 throw new SQLFeatureNotSupportedException(); 

 }

  @Override
    public void setFetchSize(int rows) throws SQLException {
        throw new SQLFeatureNotSupportedException();
    }

Pentaho PDI use Native JDBC TO ClickHouse net/jpountz/lz4/LZ4Factory

Error connecting to database [ClickHouse] :org.pentaho.di.core.exception.KettleDatabaseException:
Error occurred while trying to connect to the database

Error connecting to database: (using class com.github.housepower.jdbc.ClickHouseDriver)
net/jpountz/lz4/LZ4Factory

org.pentaho.di.core.exception.KettleDatabaseException:
Error occurred while trying to connect to the database

Error connecting to database: (using class com.github.housepower.jdbc.ClickHouseDriver)
net/jpountz/lz4/LZ4Factory

at org.pentaho.di.core.database.Database.normalConnect(Database.java:472)
at org.pentaho.di.core.database.Database.connect(Database.java:370)
at org.pentaho.di.core.database.Database.connect(Database.java:341)
at org.pentaho.di.core.database.Database.connect(Database.java:331)
at org.pentaho.di.core.database.DatabaseFactory.getConnectionTestReport(DatabaseFactory.java:80)
at org.pentaho.di.core.database.DatabaseMeta.testConnection(DatabaseMeta.java:2786)
at org.pentaho.ui.database.event.DataHandler.testDatabaseConnection(DataHandler.java:619)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.pentaho.ui.xul.impl.AbstractXulDomContainer.invoke(AbstractXulDomContainer.java:313)
at org.pentaho.ui.xul.impl.AbstractXulComponent.invoke(AbstractXulComponent.java:157)
at org.pentaho.ui.xul.impl.AbstractXulComponent.invoke(AbstractXulComponent.java:141)
at org.pentaho.ui.xul.swt.tags.SwtButton.access$500(SwtButton.java:43)
at org.pentaho.ui.xul.swt.tags.SwtButton$4.widgetSelected(SwtButton.java:137)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at org.eclipse.jface.window.Window.runEventLoop(Window.java:820)
at org.eclipse.jface.window.Window.open(Window.java:796)
at org.pentaho.di.ui.xul.KettleDialog.show(KettleDialog.java:80)
at org.pentaho.di.ui.xul.KettleDialog.show(KettleDialog.java:47)
at org.pentaho.di.ui.core.database.dialog.XulDatabaseDialog.open(XulDatabaseDialog.java:118)
at org.pentaho.di.ui.core.database.dialog.DatabaseDialog.open(DatabaseDialog.java:60)
at org.pentaho.di.ui.spoon.delegates.SpoonDBDelegate.editConnection(SpoonDBDelegate.java:95)
at org.pentaho.di.ui.spoon.Spoon.doubleClickedInTree(Spoon.java:3202)
at org.pentaho.di.ui.spoon.Spoon.doubleClickedInTree(Spoon.java:3137)
at org.pentaho.di.ui.spoon.Spoon.access$2200(Spoon.java:370)
at org.pentaho.di.ui.spoon.Spoon$29.widgetDefaultSelected(Spoon.java:6286)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at org.pentaho.di.ui.spoon.Spoon.readAndDispatch(Spoon.java:1376)
at org.pentaho.di.ui.spoon.Spoon.waitForDispose(Spoon.java:8161)
at org.pentaho.di.ui.spoon.Spoon.start(Spoon.java:9523)
at org.pentaho.di.ui.spoon.Spoon.main(Spoon.java:702)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.pentaho.commons.launcher.Launcher.main(Launcher.java:92)

Caused by: org.pentaho.di.core.exception.KettleDatabaseException:
Error connecting to database: (using class com.github.housepower.jdbc.ClickHouseDriver)
net/jpountz/lz4/LZ4Factory

at org.pentaho.di.core.database.Database.connectUsingClass(Database.java:585)
at org.pentaho.di.core.database.Database.normalConnect(Database.java:456)
... 47 more

Caused by: java.lang.NoClassDefFoundError: net/jpountz/lz4/LZ4Factory
at com.github.housepower.jdbc.buffer.CompressedBuffedWriter.(CompressedBuffedWriter.java:16)
at com.github.housepower.jdbc.serializer.BinarySerializer.(BinarySerializer.java:21)
at com.github.housepower.jdbc.connect.PhysicalConnection.openPhysicalConnection(PhysicalConnection.java:131)
at com.github.housepower.jdbc.ClickHouseConnection.createPhysicalInfo(ClickHouseConnection.java:173)
at com.github.housepower.jdbc.ClickHouseConnection.createClickHouseConnection(ClickHouseConnection.java:169)
at com.github.housepower.jdbc.NonRegisterDriver.connect(NonRegisterDriver.java:23)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at org.pentaho.di.core.database.Database.connectUsingClass(Database.java:567)
... 48 more
Caused by: java.lang.ClassNotFoundException: net.jpountz.lz4.LZ4Factory
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 57 more

Custom URL :jdbc:clickhouse://IP:9000/default
Custom Driver Class:com.github.housepower.jdbc.ClickHouseDriver

Support Failover Connection

Support timezone for DateTime datatype

Better identification of `INSERT`

What about: select * from test where name = 'VALUES (a,b,c,d)' ?
It's not an insert query, but matches the Pattern, should use QuotedLexer to check it's an insert query.

reference: cf51d65

[bug] Host not found

Server resolve client host error

Support complex clickhouse data types

ClickHouseResultSet.getObject() works incorrectly for Nullable data types.

ResultSet incorrectly returns 0.0 instead of null for my Nullable(Float32) column. Looking into the sources I have found that getObject() returns default column value, which is taken from nested data type in case of Nullable.

ClickHouse-Native-JDBC/src/main/java/com/github/housepower/jdbc/ClickHouseResultSet.java

Line 185 in fa6b1d3

return rowData == null ? column.type().defaultValue() : rowData;

Update readme

Add travis logo
Add batch examples

Support Simple JDBC Interface

JDBC Driver
JDBC Connection
JDBC Statement
JDBC ResultSet

Support SQLStatement Batch

Support Enum Data Type

Add JDK6 7 integration testing in traivis

java.lang.ArrayIndexOutOfBoundsException

批量插入数据有异常：

java.lang.ArrayIndexOutOfBoundsException: 0
	at com.github.housepower.jdbc.stream.ValuesWithParametersInputFormat.quotedParameter(ValuesWithParametersInputFormat.java:62)
	at com.github.housepower.jdbc.stream.ValuesWithParametersInputFormat.next(ValuesWithParametersInputFormat.java:52)
	at com.github.housepower.jdbc.ClickHouseConnection.sendInsertRequest(ClickHouseConnection.java:144)
	at com.github.housepower.jdbc.statement.ClickHousePreparedInsertStatement.executeBatch(ClickHousePreparedInsertStatement.java:61)
	at com.dp.kuafu.sdk.SdkDau$$anonfun$1.apply(SdkDau.scala:140)
	at com.dp.kuafu.sdk.SdkDau$$anonfun$1.apply(SdkDau.scala:79)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:797)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:797)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
	at org.apache.spark.scheduler.Task.run(Task.scala:99)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:322)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

代码如下：

val sql = "INSERT INTO mk_dw.thirdappclientfunc(dhid, uhid, appId, productId, verCode, chanId, origChanId, " +
                              "lang, ip, createDt, imei, mac, capSsid, capBssid, capApRefId, longi, lati, mapSP, netModel, ts, " +
                              "funid, ext, cts, manuf, model, seq, vcode, vname, chid, dur, sid, sdk, province, pt, hour, minute) " +
                              "VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"

                            println("sql:" + sql)

                            val pstmt = connection.prepareStatement(sql);

                            iter.foreach(sdkDauReq => {
                                pstmt.setString(1, sdkDauReq.dhid)
                                pstmt.setString(2, sdkDauReq.uhid)
                                pstmt.setString(3, sdkDauReq.appId)
                                pstmt.setInt(4, sdkDauReq.productId)
                                pstmt.setString(5, sdkDauReq.verCode)
                                pstmt.setString(6, sdkDauReq.chanId)
                                pstmt.setString(7, sdkDauReq.origChanId)
                                pstmt.setString(8, sdkDauReq.lang)
                                pstmt.setString(9, sdkDauReq.ip)
                                pstmt.setString(10, sdkDauReq.createDt)
                                pstmt.setString(11, sdkDauReq.imei)
                                pstmt.setString(12, sdkDauReq.mac)
                                pstmt.setString(13, sdkDauReq.capSsid)
                                pstmt.setString(14, sdkDauReq.capBssid)
                                pstmt.setString(15, sdkDauReq.capApRefId)
                                pstmt.setString(16, sdkDauReq.longi)
                                pstmt.setString(17, sdkDauReq.lati)
                                pstmt.setString(18, sdkDauReq.mapSP)
                                pstmt.setString(19, sdkDauReq.netModel)
                                pstmt.setString(20, sdkDauReq.ts)
                                pstmt.setString(21, sdkDauReq.funid)
                                pstmt.setString(22, sdkDauReq.ext)
                                pstmt.setString(23, sdkDauReq.cts)
                                pstmt.setString(24, sdkDauReq.manuf)
                                pstmt.setString(25, sdkDauReq.model)
                                pstmt.setString(26, sdkDauReq.seq)
                                pstmt.setString(27, sdkDauReq.vcode)
                                pstmt.setString(28, sdkDauReq.vname)
                                pstmt.setString(29, sdkDauReq.chid)
                                pstmt.setString(30, sdkDauReq.dur)
                                pstmt.setString(31, sdkDauReq.sid)
                                pstmt.setString(32, sdkDauReq.sdk)
                                pstmt.setString(33, sdkDauReq.province)
                                pstmt.setDate(34, sdkDauReq.pt)
                                pstmt.setString(35, sdkDauReq.hour)
                                pstmt.setString(36, sdkDauReq.minute)

                                pstmt.addBatch()
                            })
                            pstmt.executeBatch();

Pentaho PDI use JDBC to ClickHouse，Could not initialize class ru.yandex.clickhouse.response.ClickHouseLZ4Stream

Error connecting to database [ClickHouse] :org.pentaho.di.core.exception.KettleDatabaseException:
Error occurred while trying to connect to the database

Error connecting to database: (using class ru.yandex.clickhouse.ClickHouseDriver)
Could not initialize class ru.yandex.clickhouse.response.ClickHouseLZ4Stream

org.pentaho.di.core.exception.KettleDatabaseException:
Error occurred while trying to connect to the database

Error connecting to database: (using class ru.yandex.clickhouse.ClickHouseDriver)
Could not initialize class ru.yandex.clickhouse.response.ClickHouseLZ4Stream

at org.pentaho.di.core.database.Database.normalConnect(Database.java:472)
at org.pentaho.di.core.database.Database.connect(Database.java:370)
at org.pentaho.di.core.database.Database.connect(Database.java:341)
at org.pentaho.di.core.database.Database.connect(Database.java:331)
at org.pentaho.di.core.database.DatabaseFactory.getConnectionTestReport(DatabaseFactory.java:80)
at org.pentaho.di.core.database.DatabaseMeta.testConnection(DatabaseMeta.java:2786)
at org.pentaho.ui.database.event.DataHandler.testDatabaseConnection(DataHandler.java:619)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.pentaho.ui.xul.impl.AbstractXulDomContainer.invoke(AbstractXulDomContainer.java:313)
at org.pentaho.ui.xul.impl.AbstractXulComponent.invoke(AbstractXulComponent.java:157)
at org.pentaho.ui.xul.impl.AbstractXulComponent.invoke(AbstractXulComponent.java:141)
at org.pentaho.ui.xul.swt.tags.SwtButton.access$500(SwtButton.java:43)
at org.pentaho.ui.xul.swt.tags.SwtButton$4.widgetSelected(SwtButton.java:137)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at org.eclipse.jface.window.Window.runEventLoop(Window.java:820)
at org.eclipse.jface.window.Window.open(Window.java:796)
at org.pentaho.di.ui.xul.KettleDialog.show(KettleDialog.java:80)
at org.pentaho.di.ui.xul.KettleDialog.show(KettleDialog.java:47)
at org.pentaho.di.ui.core.database.dialog.XulDatabaseDialog.open(XulDatabaseDialog.java:118)
at org.pentaho.di.ui.core.database.dialog.DatabaseDialog.open(DatabaseDialog.java:60)
at org.pentaho.di.ui.spoon.delegates.SpoonDBDelegate.editConnection(SpoonDBDelegate.java:95)
at org.pentaho.di.ui.spoon.Spoon.editConnection(Spoon.java:2787)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.pentaho.ui.xul.impl.AbstractXulDomContainer.invoke(AbstractXulDomContainer.java:313)
at org.pentaho.ui.xul.impl.AbstractXulComponent.invoke(AbstractXulComponent.java:157)
at org.pentaho.ui.xul.impl.AbstractXulComponent.invoke(AbstractXulComponent.java:141)
at org.pentaho.ui.xul.jface.tags.JfaceMenuitem.access$100(JfaceMenuitem.java:43)
at org.pentaho.ui.xul.jface.tags.JfaceMenuitem$1.run(JfaceMenuitem.java:106)
at org.eclipse.jface.action.Action.runWithEvent(Action.java:498)
at org.eclipse.jface.action.ActionContributionItem.handleWidgetSelection(ActionContributionItem.java:545)
at org.eclipse.jface.action.ActionContributionItem.access$2(ActionContributionItem.java:490)
at org.eclipse.jface.action.ActionContributionItem$5.handleEvent(ActionContributionItem.java:402)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at org.pentaho.di.ui.spoon.Spoon.readAndDispatch(Spoon.java:1376)
at org.pentaho.di.ui.spoon.Spoon.waitForDispose(Spoon.java:8161)
at org.pentaho.di.ui.spoon.Spoon.start(Spoon.java:9523)
at org.pentaho.di.ui.spoon.Spoon.main(Spoon.java:702)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.pentaho.commons.launcher.Launcher.main(Launcher.java:92)

Caused by: org.pentaho.di.core.exception.KettleDatabaseException:
Error connecting to database: (using class ru.yandex.clickhouse.ClickHouseDriver)
Could not initialize class ru.yandex.clickhouse.response.ClickHouseLZ4Stream

at org.pentaho.di.core.database.Database.connectUsingClass(Database.java:585)
at org.pentaho.di.core.database.Database.normalConnect(Database.java:456)
... 56 more

Caused by: java.lang.NoClassDefFoundError: Could not initialize class ru.yandex.clickhouse.response.ClickHouseLZ4Stream
at ru.yandex.clickhouse.ClickHouseStatementImpl.checkForErrorAndThrow(ClickHouseStatementImpl.java:728)
at ru.yandex.clickhouse.ClickHouseStatementImpl.getInputStream(ClickHouseStatementImpl.java:551)
at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:114)
at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:97)
at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:92)
at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:88)
at ru.yandex.clickhouse.ClickHouseConnectionImpl.initTimeZone(ClickHouseConnectionImpl.java:86)
at ru.yandex.clickhouse.ClickHouseConnectionImpl.(ClickHouseConnectionImpl.java:75)
at ru.yandex.clickhouse.ClickHouseDriver.connect(ClickHouseDriver.java:58)
at ru.yandex.clickhouse.ClickHouseDriver.connect(ClickHouseDriver.java:50)
at ru.yandex.clickhouse.ClickHouseDriver.connect(ClickHouseDriver.java:32)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at org.pentaho.di.core.database.Database.connectUsingClass(Database.java:567)
... 57 more

Custom URL :jdbc:clickhouse://IP:9000/default
Custom Driver Class:ru.yandex.clickhouse.ClickHouseDriver

Add DatabaseMetadata Support

SQLDatabaseMetadata

java.sql.SQLFeatureNotSupportedException
:at [com.github.housepower.jdbc.wrapper.SQLConnection getMetaData SQLConnection.java

Cannot use syntax for insert with multiple values via jdbc driver, an error occurs

Version 1.5-stable, a way to reproduce:

Class.forName("com.github.housepower.jdbc.ClickHouseDriver");
Connection connection = DriverManager.getConnection("jdbc:clickhouse://localhost:9000","default","");
Statement stmt = connection.createStatement();
stmt.execute("CREATE TEMPORARY TABLE tmp_debug_jdbc(id Int32) ENGINE = Memory")
stmt.execute("INSERT INTO tmp_debug_jdbc values (1), (2)")

Question: clickhouse-jdbc comparison?

Hi. How is this different from https://github.com/yandex/clickhouse-jdbc ?

java.sql.SQLFeatureNotSupportedException is thrown for ResultSetMetaData.getColumnTypeName() method

val stmt = con.createStatement()
val rs = stmt.executeQuery("select 1")
val meta = rs.getMetaData
meta.getColumnTypeName(1))

Wildcard query throwing exception

A simple query like this is throwing the following exception.

SELECT * FROM prism_records ORDER BY created DESC

[05:45:49] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.util.AsyncUtil:lambda$async$1:110]: java.sql.SQLException
[05:45:49] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.util.AsyncUtil:lambda$async$1:110]:  at com.github.housepower.jdbc.misc.Validate.isTrue(Validate.java:14)
[05:45:49] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.util.AsyncUtil:lambda$async$1:110]:  at com.github.housepower.jdbc.misc.Validate.isTrue(Validate.java:8)
[05:45:49] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.util.AsyncUtil:lambda$async$1:110]:  at com.github.housepower.jdbc.data.Block.getPositionByName(Block.java:68)
[05:45:49] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.util.AsyncUtil:lambda$async$1:110]:  at com.github.housepower.jdbc.ClickHouseResultSet.findColumn(ClickHouseResultSet.java:235)
[05:45:49] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.util.AsyncUtil:lambda$async$1:110]:  at com.github.housepower.jdbc.ClickHouseResultSet.getString(ClickHouseResultSet.java:88)
[05:45:49] [pool-3-thread-1/INFO] [STDERR]: [com.helion3.prism.util.AsyncUtil:lambda$async$1:110]:  at com.helion3.prism.storage.clickhouse.ClickhouseRecords.query(ClickhouseRecords.java:106)

Performance and memory usage issues

I tried to do some simple performance tests to compare clickhouse-native-jdbc and official clickhouse-jdbc drivers in terms of speed and memory usage. Below are two examples for making a simple select query from CH numbers function.

clickhouse-jdbc:

Register http driver..
Executing sql: SELECT number as n1 from numbers(100000000)
  0% Total memory (MB):   960, used memory (MB):    20, time elapsed:    354(+   354 ms)
 10% Total memory (MB):  1211, used memory (MB):   386, time elapsed:   1425(+  1071 ms)
 20% Total memory (MB):  1676, used memory (MB):   777, time elapsed:   2433(+  1008 ms)
 30% Total memory (MB):  1595, used memory (MB):   674, time elapsed:   3313(+   880 ms)
 40% Total memory (MB):  1525, used memory (MB):   740, time elapsed:   4197(+   884 ms)
 50% Total memory (MB):  1430, used memory (MB):   140, time elapsed:   5038(+   841 ms)
 60% Total memory (MB):  1375, used memory (MB):   524, time elapsed:   5868(+   830 ms)
 70% Total memory (MB):  1301, used memory (MB):   358, time elapsed:   6698(+   830 ms)
 80% Total memory (MB):  1236, used memory (MB):   401, time elapsed:   7529(+   831 ms)
 90% Total memory (MB):  1164, used memory (MB):   121, time elapsed:   8358(+   829 ms)
100% Total memory (MB):  1103, used memory (MB):   116, time elapsed:   9193(+   835 ms)
4999999950000000

real    0m9.652s
user    0m9.728s
sys     0m0.388s

clickhouse-native-jdbc:

Register native driver..
Executing sql: SELECT number as n1 from numbers(100000000)
  0% Total memory (MB):  9092, used memory (MB):  6802, time elapsed:  21052(+ 21052 ms)
 10% Total memory (MB): 12213, used memory (MB):  6598, time elapsed:  30561(+  9509 ms)
 20% Total memory (MB): 12282, used memory (MB):  6706, time elapsed:  30930(+   369 ms)
 30% Total memory (MB): 12390, used memory (MB):  6786, time elapsed:  31252(+   322 ms)
 40% Total memory (MB): 12412, used memory (MB):  6757, time elapsed:  31576(+   324 ms)
 50% Total memory (MB): 12510, used memory (MB):  6713, time elapsed:  31897(+   321 ms)
 60% Total memory (MB): 12523, used memory (MB):  6585, time elapsed:  32222(+   325 ms)
 70% Total memory (MB): 12523, used memory (MB):  8020, time elapsed:  32521(+   299 ms)
 80% Total memory (MB): 12608, used memory (MB):  7876, time elapsed:  32836(+   315 ms)
 90% Total memory (MB): 12619, used memory (MB):  7639, time elapsed:  33166(+   330 ms)
100% Total memory (MB): 12530, used memory (MB):  7413, time elapsed:  33472(+   306 ms)
4999999950000000

real    0m33.958s
user    3m8.092s
sys     0m3.816s

The first lines in the above examples (marked as 0%) mark the moment when the driver has returned a ResultSet object for the sql (ResultSet ClickHouseStatement.executeQuery(String query)) and it is ready to be iterated.

In these particular examples it can be seen that the official http jdbc driver returns the result set almost instantly (~300ms). Next lines with increasing percentages show the processing of the result set by iterating it via boolean ClickHouseResultSet.next() and long ClickHouseResultSet.getLong(int columnIndex).

For clickhouse-native-jdbc driver there is a considerable delay before the ResultSet object is ready (also, what is not seen here, since these tests are only for one particular result size, but this delay grows as the amount of rows returned by the query increases). By the time the ResultSet is ready a significant amount of memory is used (particularly in comparison to the http driver).

Overall, on this particular example the clickhouse-native-jdbc driver is 3 times slower in total and uses an order of magnitude more memory.

I tried to dig into sources and found that while preparing the result set in ClickHouseStatement.executeUpdate(String query) the QueryResponse object is being built by receiving all responses from PhysicalConnection.receiveResponse(..) and storing the results in ArrayList<RequestOrResponse>. So it looks like all the data is being read from the DB in advance prior to returning the ResultSet object which takes time and memory. As a result I wonder why it is implemented like this. Maybe I'm missing something or I do not understand some details or maybe it was just not a priority to make the driver memory-efficient for large data sets and it is planned to be improved upon in the future.

Would be grateful for any comments and clarifications.

	public void setFetchSize(int rows) throws SQLException {
	throw new SQLFeatureNotSupportedException();
	}