Giter Club home page Giter Club logo

mysql's Introduction

mysql - MySQL connector for Tarantool

fast-testing status publish status

Getting Started

Prerequisites

  • Tarantool 1.6.5+ with header files (tarantool && tarantool-dev / tarantool-devel packages).
  • MySQL 5.1 header files (libmysqlclient-dev package).
  • OpenSSL development package.
  • libucontext (only for Alpine).

If you prefer to install the connector using a system package manager you don't need to manually install dependencies.

Installation

Build from sources

Clone repository and then build it using CMake:

git clone https://github.com/tarantool/mysql.git tarantool-mysql
cd tarantool-mysql && cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo
make
make install

Run tests

To run the tests, the following preparatory steps must be completed:

  • Run mysql instance.
  • Connect a client to the instance and execute the commands:
    • CREATE USER 'test_user' IDENTIFIED WITH mysql_native_password BY 'pass';
    • CREATE DATABASE tarantool_mysql_test;
    • GRANT ALL PRIVILEGES ON *.* TO 'test_user';
  • Define MYSQL environment variable using the following format:
    ip:port:user:user_pass:db_name:
    Example:
    export MYSQL=127.0.0.1:3306:test_user:pass:tarantool_mysql_test:

The tests can now be run by make check.

tt rocks

You can also use tt rocks:

tt rocks install mysql

Install a package

Enable tarantool repository and install tarantool-mysql package:

apt-get install tarantool-mysql # Debian or Ubuntu
yum install tarantool-mysql     # CentOS
dnf install tarantool-mysql     # Fedora

Usage

local mysql = require('mysql')
local pool = mysql.pool_create({ host = '127.0.0.1', user = 'user', password = 'password', db = 'db', size = 5 })
local conn = pool:get()
local tuples, status  = conn:execute("SELECT ? AS a, 'xx' AS b, NULL as c", 42))
conn:begin()
conn:execute("INSERT INTO test VALUES(1, 2, 3)")
conn:commit()
pool:put(conn)

API Documentation

conn = mysql.connect(opts)

Connect to a database.

Options:

  • host - hostname to connect to
  • port - port number to connect to
  • user - username
  • password - password
  • db - database name
  • use_numeric_result - provide result of the "conn:execute" as ordered list (true/false); default value: false
  • keep_null - provide printing null fields in the result of the "conn:execute" (true/false); default value: false

Throws an error on failure.

Returns:

  • connection ~= nil on success

conn:execute(statement, ...)

Execute a statement with arguments in the current transaction.

Throws an error on failure.

Returns:

  • results, true on success, where results is in the following form:

(when use_numeric_result = false or is not set on a pool/connection creation)

{
    { -- result set
        {column1 = r1c1val, column2 = r1c2val, ...}, -- row
        {column1 = r2c1val, column2 = r2c2val, ...}, -- row
        ...
    },
    ...
}

(when use_numeric_result = true on a pool/connection creation)

{
    { -- result set
        rows = {
            {r1c1val, r1c2val, ...}, -- row
            {r2c1val, r2c2val, ...}, -- row
            ...
        },
        metadata = {
            {type = 'long', name = 'col1'}, -- column meta
            {type = 'long', name = 'col2'}, -- column meta
            ...
        },
    },
    ...
}

Example:

(when keep_null = false or is not set on a pool/connection creation)

tarantool> conn:execute("SELECT ? AS a, 'xx' AS b", NULL AS c , 42)
---
- - - a: 42
      b: xx
- true
...

(when keep_null = true on a pool/connection creation)

tarantool> conn:execute("SELECT ? AS a, 'xx' AS b", NULL AS c, 42)
---
- - - a: 42
      b: xx
      c: null
- true
...

conn:begin()

Begin a transaction.

Returns: true

conn:commit()

Commit current transaction.

Returns: true

conn:rollback()

Rollback current transaction.

Returns: true

conn:ping()

Execute a dummy statement to check that connection is alive.

Returns:

  • true on success
  • false on failure

conn:quote()

Quote a query string.

Throws an error on failure.

Returns:

  • quoted_string on success

conn:reset(user, pass, db)

Update the connection authentication settings.

Options:

  • user - username
  • pass - password
  • db - database name

Throws an error on failure.

conn:close()

Close the individual connection or return it to a pool.

Throws an error on failure.

Returns: true

pool = mysql.pool_create(opts)

Create a connection pool with count of size established connections.

Options:

  • host - hostname to connect to
  • port - port number to connect to
  • user - username
  • password - password
  • db - database name
  • size - count of connections in pool
  • use_numeric_result - provide result of the "conn:execute" as ordered list (true/false); default value: false
  • keep_null - provide printing null fields in the result of the "conn:execute" (true/false); default value: false

Throws an error on failure.

Returns

  • pool ~= nil on success

conn = pool:get(opts)

Get a connection from pool. Reset connection before returning it. If connection is broken then it will be reestablished. If there is no free connections and timeout is not specified then calling fiber will sleep until another fiber returns some connection to pool. If timeout is specified, and there is no free connections for the duration of the timeout, then the return value is nil.

Options:

  • timeout - maximum number of seconds to wait for a connection

Returns:

  • conn ~= nil on success
  • conn == nil on there is no free connections when timeout option is specified

pool:put(conn)

Return a connection to connection pool.

Options

  • conn - a connection

pool:close()

Close all connections in pool.

Returns: true

Comments

All calls to connections api will be serialized, so it should to be safe to use one connection from some count of fibers. But you should understand, that you can have some unwanted behavior across db calls, for example if another fiber 'injects' some sql between two your calls.

See Also

mysql's People

Contributors

17e avatar 9e0r9 avatar 9eor9 avatar amdrozdov avatar andreyaksenov avatar avtikhon avatar better0fdead avatar bigbes avatar darkwrat avatar differentialorange avatar ericherman avatar georgykirichenko avatar grooverdan avatar kostja avatar lawrinn avatar leonidvas avatar opomuc avatar romanhabibov avatar rtsisyk avatar sanja-byelkin avatar totktonada avatar vaintroub avatar ylobankov avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mysql's Issues

'Lost connection to MySQL server during query' while executing query in pool

Problem is that when trying to execute query using pool (created by mysql.pool_create) the following error is raised: error: Lost connection to MySQL server during query.
Doing the exact same query inside a connection created by mysql.connect goes fine.
This is a log of what I see:

 $ tarantool
tarantool: version 1.7.2-282-g45423ce
type 'help' for interactive help
tarantool> mysql = require('mysql')
---
...

tarantool> pool = mysql.pool_create({ host = '127.0.0.1', user = '<user>', password = '<pass>', db = '<db>', size = 2 })
---
...

tarantool> pool
---
- host: 127.0.0.1
  db: <db>
  size: 2
  pass: '<pass>'
  user: <user>
  queue: 'channel: 2'
  usable: true
...

tarantool> conn = pool:get()
---
...

tarantool> conn
---
- queue: 'channel: 1'
  __gc_hook: null
  usable: true
  conn: 'MYSQL: 0x01cb5070'
...

tarantool> conn:execute("SELECT ? AS a, 'xx' AS b, NULL as c", 42)
---
- error: Lost connection to MySQL server during query
...

TIMESTAMP type is handled incorrectly

Repro case:

tarantool> conn:execute("CREATE TABLE x(y TIMESTAMP)")

---
- []
- 0
...

tarantool> conn:execute("INSERT INTO x VALUES('2016-03-19 22:37:42')")

---
- []
- 1
...

tarantool> conn:execute("SELECT * FROM x")

---
- - y: 2016
- 1
...

Error processing queries with equal sign

local result = connection:execute(
"SELECT Users.ID, Users.Repeater, Users.Slot, Repeaters.Network " ..
"FROM Users JOIN Repeaters ON Repeaters.ID = Users.Repeater")

/usr/share/tarantool/mysql/init.lua:72: attempt to compare 'number' with 'struct tuple'

test: fix tests on broken connections

Just observed that my test cases for broken connections actually act on healthy connections. I mean the following cases:

mysql/test/mysql.test.lua

Lines 96 to 114 in 2d22046

-- Case: the same, but for broken connection.
test:test('loss a broken connection', function(test)
test:plan(2)
assert(pool.size >= 1, 'test case precondition fails')
-- Get a connection, make a bad query and loss the
-- connection.
local conn = pool:get()
local ok = pcall(conn.execute, conn, 'bad query')
test:ok(not ok, 'a query actually fails')
conn = nil -- luacheck: no unused
-- Collect the lost connection.
collectgarbage('collect')
-- Verify that a pool is aware of collected connections.
test:is(pool.queue:count(), pool.size, 'all connections are put back')
end)

mysql/test/mysql.test.lua

Lines 312 to 354 in 2d22046

-- Case: get a connection, broke it and put back.
test:test('get, broke and put a connection', function(test)
test:plan(2)
assert(pool.size >= 1, 'test case precondition fails')
-- Get a connection and make a bad query.
local conn = pool:get()
local ok = pcall(conn.execute, conn, 'bad query')
test:ok(not ok, 'a query actually fails')
-- Put the connection back and verify that the pool is full.
pool:put(conn)
test:ok(pool.queue:is_full(), 'a broken connection was given back')
end)
-- Case: the same, but loss and collect a connection after
-- put.
test:test('get, broke, put and loss a connection', function(test)
test:plan(3)
assert(pool.size >= 1, 'test case precondition fails')
-- Get a connection and make a bad query.
local conn = pool:get()
local ok = pcall(conn.execute, conn, 'bad query')
test:ok(not ok, 'a query actually fails')
-- Put the connection back, loss it and trigger GC.
pool:put(conn)
conn = nil -- luacheck: no unused
collectgarbage('collect')
-- Verify that the pool is full
test:ok(pool.queue:is_full(), 'a broken connection was given back')
-- Verify the pool will not be populated by a connection's
-- GC callback. Otherwise :put() will hang.
local item = pool.queue:get()
pool.queue:put(item)
test:ok(true, 'GC callback does not put back a connection that was ' ..
'put manually')
end)

My initial assumption was that any SQL error (including syntax error) will lead to marking the connection as broken, however it is not so. Only CR_SERVER_LOST or CR_SERVER_GONE_ERROR errors matter here:

mysql/mysql/driver.c

Lines 149 to 157 in 2d22046

int err = mysql_errno(raw_conn);
switch (err) {
case CR_SERVER_LOST:
case CR_SERVER_GONE_ERROR:
lua_pushnumber(L, -1);
break;
default:
lua_pushnumber(L, 1);
}

mysql/mysql/init.lua

Lines 71 to 74 in 2d22046

if status ~= 0 then
self.queue:put(status > 0)
return error(datas)
end

It seems we should find a way to broke a connection from a test. It seems that support of an error injection is needed for Debug build.

Can't build with openssl 1.0.1k on Alpine 3.5

/rocks/mysql/mariadb-connector-c/libmariadb/secure/openssl.c: In function 'ma_tls_start':
/rocks/mysql/mariadb-connector-c/libmariadb/secure/openssl.c:291:8: warning: implicit declaration of function 'OPENSSL_init_ssl' [-Wimplicit-function-declaration]
   if (!OPENSSL_init_ssl(OPENSSL_INIT_LOAD_CONFIG, NULL))
        ^~~~~~~~~~~~~~~~
/rocks/mysql/mariadb-connector-c/libmariadb/secure/openssl.c:291:25: error: 'OPENSSL_INIT_LOAD_CONFIG' undeclared (first use in this function)
   if (!OPENSSL_init_ssl(OPENSSL_INIT_LOAD_CONFIG, NULL))
                         ^~~~~~~~~~~~~~~~~~~~~~~~
/rocks/mysql/mariadb-connector-c/libmariadb/secure/openssl.c:291:25: note: each undeclared identifier is reported only once for each function it appears in
/rocks/mysql/mariadb-connector-c/libmariadb/secure/openssl.c: In function 'ma_tls_verify_server_cert':
/rocks/mysql/mariadb-connector-c/libmariadb/secure/openssl.c:675:20: warning: implicit declaration of function 'ASN1_STRING_get0_data' [-Wimplicit-function-declaration]
   cn_str = (char *)ASN1_STRING_get0_data(cn_asn1);
                    ^~~~~~~~~~~~~~~~~~~~~
/rocks/mysql/mariadb-connector-c/libmariadb/secure/openssl.c:675:12: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
   cn_str = (char *)ASN1_STRING_get0_data(cn_asn1);
            ^
make[2]: *** [mariadb-connector-c/libmariadb/CMakeFiles/mariadb_obj.dir/build.make:735: mariadb-connector-c/libmariadb/CMakeFiles/mariadb_obj.dir/secure/openssl.c.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:237: mariadb-connector-c/libmariadb/CMakeFiles/mariadb_obj.dir/all] Error 2
make: *** [Makefile:150: all] Error 2

Error: Build error: Failed building.

Handle or forbid putting of closed connections into a pool

Case: a connection is acquired from a pool, then it is closed with <connection object>:close() and then it is put back to the pool.

Note: the code uses <connection object>.usable to mark closed connections or ones that was put back to a pool.

The documentation does not state that putting of closed connections is forbidden. The code don't give an error in the case and even correctly calculates that one more connection can be acquired (<pool object>.queue:put(nil) under hood). However this connection has GC callback that will close underlying raw connection and will allow to acquire one more from the pool: here the math fails.

(1) We can remove the GC callback and close the underlying connection at putting a closed connection to a pool.

(2) We however can give an error in the case: when a user attempt to put a connection with <connection object>.usable == false field, do nothing and give an error. We do not distingush between closed and released (put to a pool) connections: both have <connection object>.usable set to false. If we'll forbid to put closed connections, then we'll also forbid to put a connection twice. This looks as good protection from mistakes.

If we really need to put a manually closed connection to a pool, let's introduce one more flag to distinguish closed and released (put to a pool) connections.

I propose to implement the approach (2).

NB: Don't forget to add a test to verify the new behaviour.

Add Mac OS into CI

Travis-CI supports Mac OS, so I guess it should be not-so-complex to run tests in CI on the OS.

Destroy lost connection objects acquired from mysql.connect()

When a connection is acquired from a pool, lost and collected by Lua GC, its underlying connection will be closed from a GC callback. A connection that is created directly w/o a pool using mysql.connect() has no such GC callback, so its underlying connection will not be properly closed if a user does not call <connection object>:close() manually.

A fiber may store a connection in its local variable. If this fiber does not handle abnormal exit properly and, say, does not close the connection when is hit by fiber.cancel(), the underlying connection will leak.

We should test than the underlying connection is closed in the case.

tarantool-mysql missing from 2.4 live?

I tried doing yum install tarantool-mysql right after installing tarantool from the live repo, but I am getting No package tarantool-mysql available.

yum list doesn't show any presensce of tarantool-mysql, only things I am getting is:

tarantool-authman.noarch                 1.4.35-1.el7.centos      tarantool_2_5
tarantool-avro-schema.x86_64             3.0.5.2-1.el7            tarantool_2_5
tarantool-avro-schema-debuginfo.x86_64   3.0.5.2-1.el7            tarantool_2_5
tarantool-c.x86_64                       1.0.87-1.el7             tarantool_2_5
tarantool-c-debuginfo.x86_64             1.0.87-1.el7             tarantool_2_5
tarantool-c-devel.x86_64                 1.0.87-1.el7             tarantool_2_5
tarantool-checks.noarch                  3.0.1.2-1.el7            tarantool_2_5
tarantool-debuginfo.x86_64               1.10.3.104-1.el7         epel-debuginfo
tarantool-devel.x86_64                   2.4.2.39-1.el7           tarantool_2_5
tarantool-dump.noarch                    1.0.20-1.el7             tarantool_2_5
tarantool-expirationd.noarch             1.0.25-1.el7             tarantool_2_5
tarantool-graphql.noarch                 0.0.1.89-1.el7           tarantool_2_5
tarantool-luacheck.noarch                0.24.0.6-1.el7           tarantool_2_5
tarantool-lulpeg.noarch                  0.1.2.2-1.el7            tarantool_2_5
tarantool-memcached.x86_64               1.0.1.1-1.el7            tarantool_2_5
tarantool-memcached-debuginfo.x86_64     1.0.1.1-1.el7            tarantool_2_5
tarantool-python.noarch                  0.6.6-5.el7              tarantool_2_5
tarantool-queue.noarch                   1.0.7.0-1.el7            tarantool_2_5
tarantool-smtp.x86_64                    0.0.2.11-1.el7           tarantool_2_5
tarantool-smtp-debuginfo.x86_64          0.0.2.11-1.el7           tarantool_2_5
tarantool-try.noarch                     1.0.75-1.el7.centos      tarantool_2_5
tarantool-vshard.noarch                  0.1.16.0-1.el7           tarantool_2_5

Destroy broken connection when it is put back to a pool

The case: a connection is acquired from a pool, a request is made with this connection and fails, the connection is put back to the pool.

The connection is marked as broken in the case and will be discarded: a pool will not actually cache it, just count (<pool object>.queue:put(nil) under hood) that it may give (and create if needed) one more connection when it will be acquired. The math here looks correct. However an underlying raw connection of this 'broken' connection will not be closed neither when it put back to the pool, nor at GC (because the GC callback is removed at <pool object>:put() for both broken and healthy connections).

This behaviour looks as a bug. I propose to destroy the underlying connection when it is discarded at <pool object>:put(): i. e. close the underlying connection if it is broken.

We should test than the underlying connection is closed in the case.

Handle connection error in pool:get()

See the code:

mysql/mysql/init.lua

Lines 31 to 46 in 9ca113b

-- get connection from pool
local function conn_get(pool, timeout)
local mysql_conn = pool.queue:get(timeout)
-- A timeout was reached.
if mysql_conn == nil then return nil end
local status
if mysql_conn == POOL_EMPTY_SLOT then
status, mysql_conn = driver.connect(pool.host, pool.port or 0,
pool.user, pool.pass,
pool.db, pool.use_numeric_result)
if status < 0 then
return error(mysql_conn)
end
end

When we got POOL_EMPTY_SLOT from the pool and a connection error occurs, we don't return the slot back to the pool.

Improve the code readability

In brief: there are a lot of details that neither documented, nor obvious from the code. Any change is painful so. We should provide those details as comments in the code and make appropriate renaming.

I'll share my observations below and hopefully will come back later to actually make proper changes.


<connection object>.queue is not queue, it is a fiber channel with one item at max. It is used for synchronization of using a connection betweeb fiber: :get() is acquiring a lock, :put() is releasing it. Aside of that, a value in this channel is an indicator whether a connection is healthy: :put(true) is used to mark it as healthy and :put(false) to mark it as broken.

A connection is marked as broken when any request fails due to any reason (except when :ping() fails, but it looks more as a mistake).

<connection object>.usable is another marker. When a connection is closed or is put back to a pool it marked as unusable. It is not the same that broken connection.

<pool object>.queue is also not a queue, it is also a fiber channel, which works as fixed-size pool of prespawned raw connection objects (see below about their naming), but also can contain nil markers. This channel contains <pool object>.size elements at max: when the channel is full we can acquire this amount of connections, when it is empty <pool object>:get() will wait until someone will do <pool object>:put(). When a connection is put back to a pool we'll add its raw connection to the pool if it is not broken and is not unusable. Otherwise we'll add nil marker to the pool's and discard the raw connection. We handle this marker at <pool object>:get(): the new raw connection will be created at this moment.

<pool object>.usable is just whether the pool is closed.

About raw connections mentioned above. A connection contains conn field that holds underlying connection object from driver.c. Local variables that hold this field (acquired from driver.connect(), <connection object>.conn or via <pool object>.queue:get()) has no strict naming: they are conn, mysqlconn or mysql_conn. Let's use one name for one term. We should name the field and all local variables in the same way.

The connector handles the case when a connection was not put back to a pool and a user lost it at all: say a fiber, which did acquire the connection was cancelled. In this case the raw connection will be properly closed and nil marker will be added to the pool: a kind of implicit <pool object>:put() if a connection was lost.

Failed to install using both from git and luarocks

Hi,
I'm using ec2 instance with el6 OS and trying to install tarantool-mysql using both guide (git & luarocks), but i got these; any help will be greatly appreciated. :)

[root@ tarantool-mysql]# cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo
-- The C compiler identification is GNU 4.8.3
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:108 (message):
Could NOT find TARANTOOL (missing: TARANTOOL_INCLUDE_DIR)
Call Stack (most recent call first):
/usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:315 (_FPHSA_FAILURE_MESSAGE)
cmake/FindTarantool.cmake:26 (find_package_handle_standard_args)
CMakeLists.txt:15 (find_package)

-- Configuring incomplete, errors occurred!
See also "/usr/share/tarantool/tarantool-mysql/CMakeFiles/CMakeOutput.log".
[root@ tarantool-mysql]# luarocks install https://raw.githubusercontent.com/tarantool/mysql/master/mysql-scm-1.rockspec --local
Using https://raw.githubusercontent.com/tarantool/mysql/master/mysql-scm-1.rockspec... switching to 'build' mode
Cloning into 'mysql'...
remote: Counting objects: 31, done.
remote: Compressing objects: 100% (24/24), done.
remote: Total 31 (delta 1), reused 14 (delta 0), pack-reused 0
Receiving objects: 100% (31/31), 15.07 KiB | 7.53 MiB/s, done.
Resolving deltas: 100% (1/1), done.
-- The C compiler identification is GNU 4.8.3
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:108 (message):
Could NOT find TARANTOOL (missing: TARANTOOL_INCLUDE_DIR)
Call Stack (most recent call first):
/usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:315 (_FPHSA_FAILURE_MESSAGE)
cmake/FindTarantool.cmake:26 (find_package_handle_standard_args)
CMakeLists.txt:15 (find_package)

-- Configuring incomplete, errors occurred!
See also "/tmp/luarocks_mysql-scm-1-1886/mysql/CMakeFiles/CMakeOutput.log".

Error: Build error: Failed cmake.

mysql_init() is not called in worker thread

Sunday, October 4, 2015 11:25 AM -06:00 from Peter Gulutzan < [email protected] >:
Regarding the crash described earlier:
I looked at the file driver.c in your repository
https://github.com/tarantool/mysql/blob/master/mysql/driver.c
line 316 which looks like this:
"
mysql_real_connect(mysql, host, user, password, db, iport, usocket,
CLIENT_MULTI_STATEMENTS | CLIENT_MULTI_RESULTS);
"
I changed it to this:
"
mysql_init(mysql);
mysql_real_connect(mysql, host, user, password, db, iport, usocket,
CLIENT_MULTI_STATEMENTS | CLIENT_MULTI_RESULTS);

coio_call() uses random threads for each call. mysql_init() is only called once in the first thread.
Probably we should use the single thread for all mysql operations.

Prepared statements with fields longtext and longblob cause segmentation fault

Problem: prepared statements with fields longtext and longblob cause segmentation fault if fields value is not equal NULL.

mysql dump

CREATE TABLE `mytable` (
  `pri` int(11) unsigned NOT NULL AUTO_INCREMENT,
  `f_longtext` longtext NOT NULL,
  `f_longtext_null` longtext DEFAULT NULL,
  `f_longblob` longblob NOT NULL,
  `f_longblob_null` longblob DEFAULT NULL,
  PRIMARY KEY (`pri`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

INSERT INTO `mytable` (`pri`, `f_longtext`, `f_longtext_null`, `f_longblob`, `f_longblob_null`)
VALUES
	(1, '', NULL, '', NULL),
	(2, 'qwe', 'qwe', 'qwe', 'qwe');
tmlapp        | Segmentation fault
tmlapp        |   code: SEGV_MAPERR
tmlapp        |   addr: 0
tmlapp        |   context: 0x7f8cb387f6c0
tmlapp        |   siginfo: 0x7f8cb387f7f0
tmlapp        |   rax      0x0                0
tmlapp        |   rbx      0x5601c2a2c2f8     94565560402680
tmlapp        |   rcx      0x0                0
tmlapp        |   rdx      0x3                3
tmlapp        |   rsi      0x5601c2a07063     94565560250467
tmlapp        |   rdi      0x0                0
tmlapp        |   rsp      0x7f8cb387fd78     140242284182904
tmlapp        |   rbp      0x3                3
tmlapp        |   r8       0x1                1
tmlapp        |   r9       0x0                0
tmlapp        |   r10      0x40dcb718         1088206616
tmlapp        |   r11      0x9                9
tmlapp        |   r12      0x3                3
tmlapp        |   r13      0x5601c2a2a600     94565560395264
tmlapp        |   r14      0x0                0
tmlapp        |   r15      0x5601c2a07061     94565560250465
tmlapp        |   rip      0x7f8d1ba4b0a0     140244030894240
tmlapp        |   eflags   0x10206            66054
tmlapp        |   cs       0x33               51
tmlapp        |   gs       0x0                0
tmlapp        |   fs       0x0                0
tmlapp        |   cr2      0x0                0
tmlapp        |   err      0x6                6
tmlapp        |   oldmask  0x0                0
tmlapp        |   trapno   0xe                14
tmlapp        | Current time: 1574495001
tmlapp        | Please file a bug at http://github.com/tarantool/tarantool/issues
tmlapp        | Attempting backtrace... Note: since the server has already crashed,
tmlapp        | this may fail as well
tmlapp        | #0  0x5601c1c7d219 in print_backtrace+9
tmlapp        | #1  0x5601c1b68dda in _ZL12sig_fatal_cbiP9siginfo_tPv+ca
tmlapp        | #2  0x7f8d1ba40e17 in sigwaitinfo+8

Reproduce case:

tarantool> mysql = require("mysql")
---
...
tarantool> pool = mysql.pool_create({host = 'localhost', db = 'test', size = 5})
---
...
tarantool> conn = pool:get()
---
...
tarantool> conn:execute("select `f_longtext` from mytable where pri = ?", 1)
---
- error: null  <-- crash
...
tarantool> conn:execute("select `f_longblob` from mytable where pri = ?", 1)
---
- error: null  <-- crash
...
tarantool> conn:execute("select `f_longtext_null` from mytable where pri = ?", 2)
---
- error: null  <-- crash
...
tarantool> conn:execute("select `f_longblob_null` from mytable where pri = ?", 2)
---
- error: null  <-- crash
...

working cases

tarantool> conn:execute("select `f_longtext` from mytable where pri = 1") <-- without prepare
tarantool> conn:execute("select `f_longtext_null` from mytable where pri = ?", 1) <-- if NULL
tarantool> conn:execute("select `f_longblob_null` from mytable where pri = ?", 1) <-- if NULL

my working example with mariadb-connector-c https://github.com/iMega/tarantool-mysql-learning/blob/master/mysqltest/main.c

app crashes here https://github.com/tarantool/mariadb-connector-c/blob/c5dfd891257f4dca3d1f6aa7645d27f6b491c6a6/libmariadb/mariadb_stmt.c#L1253

Fix packaging on CentOS 6

First, several facts

  1. GCC from the base repository on CentOS 6 is GCC 4.4.7.
  2. We recently switched from devtoolset-6 to devtoolset-7, because the former seems to be gone.
  3. devtoolset-7-gcc on CentOS 6 does not provide gcc (it seems, devtoolset-6-gcc did, but I don't know how to verify).
  4. The tarantool-mysql package depends on gcc >= 4.5.

The problem

Now packaging for CentOS 6 fails with the following messages:

Getting requirements for tarantool-mysql-0.0.1-1.el6.src
 --> Already installed : cmake-2.8.12.2-4.el6.x86_64
Error: No Package found for gcc >= 4.5

Proposed solution

Eliminate the dependency on gcc on CentOS 6, just how it is implemented tarantool.

Fix luacheck warnings

If you run luacheck, it will display a certain number of warnings. Good practice - no linter warnings.

Drop Ubuntu Eoan (19.10) support

It is EOL since 2019-10-17. Now packaging for Ubuntu Eoan fails with the following messages:

sudo apt-get update > /dev/null
E: The repository 'http://security.ubuntu.com/ubuntu eoan-security Release' does not have a Release file.
E: The repository 'http://archive.ubuntu.com/ubuntu eoan Release' does not have a Release file.
E: The repository 'http://archive.ubuntu.com/ubuntu eoan-updates Release' does not have a Release file.
E: The repository 'http://archive.ubuntu.com/ubuntu eoan-backports Release' does not have a Release file.

It seems, the reposiroties were disabled. Maybe there are some archives (I don't know), but I would not bother with supporting old non-LTS Ubuntu distributions, if it requires some effort.

LIKE operator not working properly with prepared statemets

I have code like this
local query = [[
SELECT ID, NAME FROM table
WHERE NAME LIKE '%?%'
]]
local data, err = conn:execute(query, name)
Instead of receiving a record i got an empty table
The same query works when using dbeaver(for example)

Ensure <connection object>:execute() don't block a fiber infinitely

Case: a connection is acquired from a pool, then three fibers use it: one calls <connection object>:execute() and acquires a lock (let's consider a long request), another one perform other calls to :execute() and third one put the connection back to the pool.

Say, when the long request will be done, the third fiber will acquire the lock and put the connection to the pool. The old connection object will be marked as unusable, however the lock will not be released (see conn_put). The second fiber will be blocked infinitely.

Even if we'll release the lock in <pool object>:put() and the second fiber will wake up in :execute(), it will at the point after self.usable check and will try to execute a request. It will even succeed, but it looks as a side effect of #32.

It is maybe not quite correct usage, but we should handle it appropriately: wait for necessary events, give an error for an incorrect usage, don't leak resources and surely don't block a fiber infinitely.

It seems the similar problem with passed <connection object>.usable check may appears for a connection that is created with mysql.connect() w/o a pool if we'll try the similar case, but with <connection object>:close() instead of <pool object>:put().

NB: Add test cases for both described situations.

Unable to build mysql module when cmake 3.20+ is used

OS: Ubuntu 20.04.

When cmake 3.20+ is installed in your system, it turns out that it's not possible to build the mysql module because the following error occurs:

CMake Error at mariadb-connector-c/cmake/ConnectorName.cmake:30 (ENDMACRO):
  Flow control statements are not properly nested.
Call Stack (most recent call first):
  mariadb-connector-c/CMakeLists.txt:391 (INCLUDE)


-- Configuring incomplete, errors occurred!
See also "/home/runner/work/tarantool/tarantool/CMakeFiles/CMakeOutput.log".
See also "/home/runner/work/tarantool/tarantool/CMakeFiles/CMakeError.log".

It looks like the issue is in the mariadb-connector-c https://jira.mariadb.org/browse/CONCPP-79. So in order to fix the problem, we need to port the fix from the mentioned issue to our mariadb-connector-c fork and update the submodule version in the mysql project.

Tarantool+ClickHouse via mysql protocol

I have docker running ClickHouse 22.1.3.7 with revision 54458 (OS name: Linux, version: 5.16.0-kali7-amd64, architecture: x86_64)
I have Tarantool 2.10.4-1-g02908e625 running in docker
I connect to mysql db from Tarantool. Everything works.
I wanted to connect from a Tarantool to a ClickHouse. Reconfigured the ClickHouse to work using the mysql protocol.
Connect to ClickHouse mysql --protocol tcp -u default -P 9004 -p default. Everything works.

Trying to connect from Tarantool
local mysql = require('mysql') local pool, err = mysql.pool_create({ host = 'ch', port = '9004', user = 'default', password = 'TcPfSfIS', db='default' })
Does not work

2022-12-22 07:41:08.635 [1] main/103/tarantool-entrypoint.lua I> [INFO] Start init.lua
2022-12-22 07:41:08.638 [1] main utils.c:489 E> LuajitError: /.rocks/share/tarantool/mysql/init.lua:166: Can't connect to MySQL server on 'ch' (115)
2022-12-22 07:41:08.638 [1] main F> fatal error, exiting the event loop

Alpine on aarch64 error "swapcontext: symbol not found"

How to reproduce:

docker run --rm -it tarantool/tarantool:2.10.0-aarch64

then

tarantool> mysql = require('mysql')
---
- error: "/.rocks/share/tarantool/mysql/init.lua:4: module 'mysql.driver' not found:\n\tno
    field package.preload['mysql.driver']\n\tno file '/opt/tarantool/mysql/driver.lua'\n\tno
    file '/opt/tarantool/mysql/driver/init.lua'\n\tno file '/opt/tarantool/mysql/driver.so'\n\tno
    file '/opt/tarantool/.rocks/share/tarantool/mysql/driver.lua'\n\tno file '/opt/tarantool/.rocks/share/tarantool/mysql/driver/init.lua'\n\tno
    file '/opt/.rocks/share/tarantool/mysql/driver.lua'\n\tno file '/opt/.rocks/share/tarantool/mysql/driver/init.lua'\n\tno
    file '/.rocks/share/tarantool/mysql/driver.lua'\n\tno file '/.rocks/share/tarantool/mysql/driver/init.lua'Error
    relocating /.rocks/lib/tarantool/mysql/driver.so: swapcontext: symbol not found\n\tno
    file '/root/.luarocks/share/lua/5.1/mysql/driver.lua'\n\tno file '/root/.luarocks/share/lua/5.1/mysql/driver/init.lua'\n\tno
    file '/root/.luarocks/share/lua/mysql/driver.lua'\n\tno file '/root/.luarocks/share/lua/mysql/driver/init.lua'\n\tno
    file '/usr/local/share/tarantool/mysql/driver.lua'\n\tno file '/usr/local/share/tarantool/mysql/driver/init.lua'\n\tno
    file '/usr/share/tarantool/mysql/driver.lua'\n\tno file '/usr/share/tarantool/mysql/driver/init.lua'\n\tno
    file '/usr/local/share/lua/5.1/mysql/driver.lua'\n\tno file '/usr/local/share/lua/5.1/mysql/driver/init.lua'\n\tno
    file '/usr/share/lua/5.1/mysql/driver.lua'\n\tno file '/usr/share/lua/5.1/mysql/driver/init.lua'\n\tno
    file '/root/.luarocks/lib/lua/5.1/mysql/driver.so'\n\tno file '/root/.luarocks/lib/lua/mysql/driver.so'\n\tno
    file '/usr/local/lib64/tarantool/mysql/driver.so'\n\tno file '/usr/local/lib/tarantool/mysql/driver.so'\n\tno
    file '/usr/lib64/tarantool/mysql/driver.so'\n\tno file '/usr/local/lib64/lua/5.1/mysql/driver.so'\n\tno
    file '/usr/local/lib/lua/5.1/mysql/driver.so'\n\tno file '/usr/lib64/lua/5.1/mysql/driver.so'\n\tno
    file '/root/.luarocks/lib/lua/5.1/mysql.so'\n\tno file '/root/.luarocks/lib/lua/mysql.so'\n\tno
    file '/usr/local/lib64/tarantool/mysql.so'\n\tno file '/usr/local/lib/tarantool/mysql.so'\n\tno
    file '/usr/lib64/tarantool/mysql.so'\n\tno file '/usr/local/lib64/lua/5.1/mysql.so'\n\tno
    file '/usr/local/lib/lua/5.1/mysql.so'\n\tno file '/usr/lib64/lua/5.1/mysql.so'"

mysql connection handling

May i ask a theoretical noob questions here? Or there is another channel for these?

I'd like to group mysql functions into a module. But if they are in a module, how could i call these from other modules. Should i declare the 'conn' (= mysql_connect()) variable as a global? Or should i pass it to all functions as parameter?
Generally the mysql connection is permanent in tarantool? Need to be permanent (relate to performance)? Or need to be open a new connection for every operation?
Thank you.

raise = false how's it working?

I tried raise = false with a non-existent server, but i get "Unknown MySQL server host" exactly as with raise = true. How's it working? If it's the normal operation, how could i catch that error?
Thx.

MariaDB connector fails to build with OpenSSL 1.1

https://travis-ci.org/tarantool/pg/builds/180318571

[ 38%] Building C object libmariadb/CMakeFiles/mariadb_obj.dir/__/zlib/adler32.c.o

/build/tarantool-mysql-2.0.3/mariadb-connector-c/libmariadb/secure/openssl.c: In function ‘ma_tls_start’:

/build/tarantool-mysql-2.0.3/mariadb-connector-c/libmariadb/secure/openssl.c:258:3: warning: ‘OPENSSL_config’ is deprecated [-Wdeprecated-declarations]

   OPENSSL_config(NULL);

   ^~~~~~~~~~~~~~

In file included from /usr/include/openssl/ct.h:13:0,

                 from /usr/include/openssl/ssl.h:61,

                 from /build/tarantool-mysql-2.0.3/mariadb-connector-c/libmariadb/secure/openssl.c:28:

/usr/include/openssl/conf.h:92:1: note: declared here

 DEPRECATEDIN_1_1_0(void OPENSSL_config(const char *config_name))

 ^

/build/tarantool-mysql-2.0.3/mariadb-connector-c/libmariadb/secure/openssl.c: In function ‘ma_tls_end’:

/build/tarantool-mysql-2.0.3/mariadb-connector-c/libmariadb/secure/openssl.c:320:7: warning: ‘ERR_remove_state’ is deprecated [-Wdeprecated-declarations]

       ERR_remove_state(0);

       ^~~~~~~~~~~~~~~~

In file included from /usr/include/openssl/ct.h:13:0,

                 from /usr/include/openssl/ssl.h:61,

                 from /build/tarantool-mysql-2.0.3/mariadb-connector-c/libmariadb/secure/openssl.c:28:

/usr/include/openssl/err.h:247:1: note: declared here

 DEPRECATEDIN_1_0_0(void ERR_remove_state(unsigned long pid))

 ^

/build/tarantool-mysql-2.0.3/mariadb-connector-c/libmariadb/secure/openssl.c: In function ‘ma_tls_verify_server_cert’:

/build/tarantool-mysql-2.0.3/mariadb-connector-c/libmariadb/secure/openssl.c:616:3: warning: ‘ASN1_STRING_data’ is deprecated [-Wdeprecated-declarations]

   cn_str = (char *)ASN1_STRING_data(cn_asn1);

   ^~~~~~

Drop extra options -lssl -lcrypto

These options are superfluous, because dependencies mariadbclient passed to the linker when linking driver.so/dylib.
From make VERBOSE=1

[100%] Linking C shared library driver.dylib
cd /Users/r.khabibov/tarantool-mysql/mysql && /usr/local/Cellar/cmake/3.14.5/bin/cmake -E cmake_link_script CMakeFiles/driver.dir/link.txt --verbose=1
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -g -Wall -Wextra -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -dynamiclib -Wl,-headerpad_max_install_names  -undefined suppress -flat_namespace -o driver.dylib -install_name /Users/r.khabibov/tarantool-mysql/mysql/driver.dylib CMakeFiles/driver.dir/driver.c.o ../mariadb-connector-c/libmariadb/libmariadbclient.a -lpthread -ldl -lm /usr/lib/libiconv.dylib /usr/local/opt/[email protected]/lib/libssl.dylib /usr/local/opt/[email protected]/lib/libcrypto.dylib

target_link_libraries(driver mariadbclient -lssl -lcrypto)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.