Giter Club home page Giter Club logo

mysql's Introduction

amphp/mysql

AMPHP is a collection of event-driven libraries for PHP designed with fibers and concurrency in mind. amphp/mysql is an asynchronous MySQL client. The library implements concurrent querying by transparently distributing queries across a scalable pool of available connections. The client transparently distributes these queries across a scalable pool of available connections and does so using 100% userland PHP; there are no external extension dependencies (e.g. ext/mysqli, ext/pdo, etc.).

Features

  • Exposes a non-blocking API for issuing multiple MySQL queries concurrently
  • Transparent connection pooling to overcome MySQL's fundamentally synchronous connection protocol
  • MySQL transfer encoding support (gzip, TLS encryption)
  • Support for parameterized prepared statements
  • Nested transactions with commit and rollback event hooks
  • Unbuffered results to reduce memory usage for large result sets
  • Full MySQL protocol support including allโ€  available commands asynchronously

โ€  As documented in official Mysql Internals Manual

Installation

This package can be installed as a Composer dependency.

composer require amphp/mysql

Requirements

  • PHP 8.1+

Usage

More extensive code examples reside in the examples directory.

use Amp\Mysql\MysqlConfig;
use Amp\Mysql\MysqlConnectionPool;

$config = MysqlConfig::fromString(
    "host=localhost user=username password=password db=test"
);

$pool = new MysqlConnectionPool($config);

$statement = $pool->prepare("SELECT * FROM table_name WHERE id = :id");

$result = $statement->execute(['id' => 1337]);
foreach ($result as $row) {
    // $row is an associative-array of column values, e.g.: $row['column_name']
}

Versioning

amphp/mysql follows the semver semantic versioning specification like all other amphp packages.

Security

If you discover any security related issues, please use the private security issue reporter instead of using the public issue tracker.

License

The MIT License (MIT). Please see LICENSE for more information.

mysql's People

Contributors

bennnjamin avatar brstgt avatar bwoebi avatar crtlib avatar danack avatar dbalabka avatar dickson-michael avatar gemorroj avatar grooverdan avatar harikt avatar kelunik avatar krlv avatar marcioalmada avatar optiman avatar prolic avatar rdlowrey avatar ruifil avatar sagara- avatar sevavietl avatar shtse8 avatar staabm avatar trowski avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mysql's Issues

Invalid watcher exception

We updated the amphp/mysql from f077c40 to 419279e. Now we get the following exception from time to time.
I guess it could be related to connections closed by mysql server due to an idle timeout.

And it would be really cool if you tag your releases so one can use composer version deps.

Cannot reference an invalid watcher identifier: 'pk' (File: /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Loop/Driver.php:476) (Class: Amp\Loop\InvalidWatcherError)
[#1] Amp\Loop\Driver->reference in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/lib/Sensphere/Amp/Loop/DecoratableDriver.php on line 141
[#2] Sensphere\Amp\Loop\DecoratableDriver->reference in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Loop.php on line 288
[#3] Amp\Loop::reference in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/byte-stream/lib/ResourceInputStream.php on line 204
[#4] Amp\ByteStream\ResourceInputStream->reference in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/socket/src/Socket.php on line 88
[#5] Amp\Socket\Socket->reference in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/mysql/src/Internal/Processor.php on line 184
[#6] Amp\Mysql\Internal\Processor->addDeferred in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/mysql/src/Internal/Processor.php on line 300
[#7] Amp\Mysql\Internal\Processor->Amp\Mysql\Internal\{closure} in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/mysql/src/Internal/Processor.php on line 276
[#8] Amp\Mysql\Internal\Processor->appendTask in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/mysql/src/Internal/Processor.php on line 302
[#9] Amp\Mysql\Internal\Processor->startCommand in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/mysql/src/Internal/Processor.php on line 337
[#10] Amp\Mysql\Internal\Processor->query in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/mysql/src/Connection.php on line 98
[#11] Amp\Mysql\Connection->Amp\Mysql\{closure} in  on line 
[#12] Generator->current in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Coroutine.php on line 39
[#13] Amp\Coroutine->__construct in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/functions.php on line 61
[#14] Amp\call in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/mysql/src/Connection.php on line 109
[#15] Amp\Mysql\Connection->query in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/mysql/src/Pool.php on line 268
[#16] Amp\Mysql\Pool->Amp\Mysql\{closure} in  on line 
[#17] Generator->current in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Coroutine.php on line 39
[#18] Amp\Coroutine->__construct in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/functions.php on line 61
[#19] Amp\call in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/mysql/src/Pool.php on line 283
[#20] Amp\Mysql\Pool->query in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/lib/Sensphere/DB/MySql/AsyncPool.php on line 58
[#21] Sensphere\DB\MySql\AsyncPool->Sensphere\DB\MySql\{closure} in  on line 
[#22] Generator->throw in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Coroutine.php on line 71
[#23] Amp\Coroutine->Amp\{closure} in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Failure.php on line 26
[#24] Amp\Failure->onResolve in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Internal/Placeholder.php on line 122
[#25] Amp\Coroutine->resolve in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Internal/Placeholder.php on line 151
[#26] Amp\Coroutine->fail in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Coroutine.php on line 93
[#27] Amp\Coroutine->Amp\{closure} in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Failure.php on line 26
[#28] Amp\Failure->onResolve in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Internal/Placeholder.php on line 122
[#29] Amp\Coroutine->resolve in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Internal/Placeholder.php on line 151
[#30] Amp\Coroutine->fail in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Coroutine.php on line 93
[#31] Amp\Coroutine->Amp\{closure} in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Failure.php on line 26
[#32] Amp\Failure->onResolve in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Internal/Placeholder.php on line 122
[#33] class@anonymous/home/ci-php

Connection pool creates unnecessary connections

ConnectionPool creates spare connections, even if I don't use a single one.

  1. on constructor
  2. In ::getReadyConnection()

If 0 connections are used, at least 1 is created
If 1 conneciton is used, 3 are created

If addConnection in c'tor and

if (count($this->ready) < 2) {
      $this->addConnection();
}

are removed, the number of connection seems to be correct.

Insert JSON value is failed

When i try to insert json value in my table

Amp\Loop::run(function () {

    $db = Amp\Mysql\pool("...");

    yield $db->query("CREATE TABLE IF NOT EXISTS amp_json (a INT(10), b JSON)");

    /** @var \Amp\Mysql\Statement $statement */
    $statement = yield $db->prepare("INSERT INTO amp_json (a, b) VALUES (?, ?)");
    foreach (range(1, 10) as $num) {
        $json = json_encode(['num' => $num]);
        yield $statement->execute([$num, $json]);
    }

    $db->close();
});

I get an error:

Fatal error: Uncaught Amp\Mysql\QueryError: MySQL error (3144): #22032 Cannot create a JSON value from a string with CHARACTER SET 'binary'. Current query was INSERT INTO amp_json (a, b) VALUES (?, ?) in /vendor/amphp/mysql/src/Internal/Processor.php:617

I was able to insert a row with json value only by concatenating, but it's not good solution.

yield $db->execute("INSERT INTO amp_json (a, b) VALUES ($num, '" . json_encode(['num'=>$num]) . "')");

May be you have some ideas about it?

Better testability

@trowski
This issue is more a reminder or discussion than a perfectly investigated thesis.

I had in mind, you moved a lot of logic from Connection to Processor. From what I can see, the processor was meant to be a kind of state-machine to process server responses / connection states.
While moving connection logic (and socket stuff) into the processor its testability suffers.
With a processor that is not coupled to the IO method (connection, socket, whatever) it would be much easier to test the state machine and the protocol.

I really appreciate what you achieved with the mysql module (totally!) but I got the notion it is not really well tested or even thoroughly testable in edge cases with the current architecture.
Some of the edge cases I ran into are either not easily or not at all reliably testable with a real server instance - what you obviously to in your tests.
Using a module like that can be really mission critical for guys like me and if there is a problem with a release in can cause REAL trouble.

Edge case examples:

  • Server goes away in different stages (in query, when connection is idle)
  • Response comes fragmented

What do you think?

Connection went away, after wait_timeout expired

mysql has a setting to kill idle connections after a while: wait_timeout - https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_wait_timeout - with a default of 8 hours.

This regularly causes exceptions on very low-frequency ends, while it would be easily avoidable if we restarted connections from time to time.

We might desire to request that timeout from the server via SHOW VARIABLES LIKE 'wait_timeout' on pool startup and restart (or just kill them as long as more than 2 connections are available) the idling connections if their last use gets dangerously close to wait_timeout.

Amazon MySql db connection issue

I am connecting to an amazon DB from windows php 7.2.4 with following code:
https://gist.github.com/plankes-projects/d941f49bf74353f0b29700261192c639

I get DNS timeout exception when I am running it on my windows developer machine.
https://gist.github.com/plankes-projects/c3cfd2a6aead806adc7a9efaf545ba48

Everthing works fine if I am using this config with eloquent or if I am connecting to a local DB:
https://gist.github.com/plankes-projects/afbe1d66eb08262a0f1484cf9fbaf402

Following amphp versions are installed with composer:
amphp/amp dev-master 9b2fb76 A non-blocking concurrency framework for PHP applications.
amphp/byte-stream v1.3.1 A stream abstraction to make working with non-blocking I/O...
amphp/cache v1.2.0 A promise-aware caching API for Amp.
amphp/dns v0.9.12 Async DNS resolution for Amp.
amphp/file v0.3.1 Allows non-blocking access to the filesystem for Amp.
amphp/mysql dev-master aec58f4 Asynchronous parallel Mysql client built on the Amp concur...
amphp/parallel v0.2.5 Parallel processing component for Amp.
amphp/parser v1.0.0 A generator parser to make streaming parsers simple.
amphp/process v0.3.3 Asynchronous process manager.
amphp/socket v0.10.8 Async socket connection / server tools for Amp.
amphp/sync v1.0.1 Mutex, Semaphore, and other synchronization tools for Amp.
amphp/uri v0.1.3 Uri Parser and Resolver.
amphp/windows-registry v0.3.1 Windows Registry Reader.

Add LICENSE

There should be a LICENSE with the license in it.

ConnectionPool is not restoring connection when database restarts.

Using 21801b0 Currently a connection pool owned by an Aerys application is not able to recover when the database restarts. Error:

"ArgumentCountError: Too few arguments to function Amp\\Mysql\\Connection::Amp\\Mysql\\{closure}(), 0 passed in ./vendor/vendor/amphp/mysql/lib/Stmt.php on line 42 and exactly 1 expected in ./vendor/vendor/amphp/mysql/lib/Connection.php:40
Stack trace:
#0 ./vendor/vendor/amphp/mysql/lib/Stmt.php(42): Amp\\Mysql\\Connection::Amp\\Mysql\\{closure}()
#1 ./vendor/vendor/amphp/mysql/lib/Stmt.php(139): Amp\\Mysql\\Stmt->conn()
#2 ./vendor/src/DataStorage/MysqlAsyncStorage.php(228): Amp\\Mysql\\Stmt->execute(Array)
#3 ./vendor/src/DataStorage/MysqlAsyncStorage.php(131): Fs\\Blacklist\\DataStorage\\MysqlAsyncStorage->do('msisdn_permissi...', Array)
#4 ./vendor/src/RouteHandler/V3/MsisdnStatusHandler.php(53): Fs\\Blacklist\\DataStorage\\MysqlAsyncStorage->getMsisdnPermissionHistory('vivo', '5511981349170', NULL)
#5 ./vendor/src/AbstractRestHandler.php(73): Fs\\Blacklist\\RouteHandler\\V3\\MsisdnStatusHandler->get(Object(Aerys\\StandardRequest), Object(Aerys\\StandardResponse), Object(stdClass))
#6 ./vendor/vendor/amphp/aerys/lib/Bootstrapper.php(212): Fs\\Blacklist\\AbstractRestHandler->__invoke(Object(Aerys\\StandardRequest), Object(Aerys\\StandardResponse), Array)
#7 [internal function]: class@anonymous->__invoke(Object(Aerys\\StandardRequest), Object(Aerys\\StandardResponse))
#8 ./vendor/vendor/amphp/amp/lib/functions.php(791): Generator->current()
#9 ./vendor/vendor/amphp/amp/lib/functions.php(779): Amp\\__coroutineAdvance(Object(Amp\\CoroutineState))
#10 ./vendor/vendor/amphp/aerys/lib/Server.php(765): Amp\\resolve(Object(Generator))
#11 ./vendor/vendor/amphp/aerys/lib/Server.php(716): Aerys\\Server->tryApplication(Object(Aerys\\InternalRequest), Object(class@anonymous), Array)
#12 ./vendor/vendor/amphp/aerys/lib/Server.php(555): Aerys\\Server->respond(Object(Aerys\\InternalRequest))
#13 ./vendor/vendor/amphp/aerys/lib/Server.php(530): Aerys\\Server->onParsedMessageWithoutEntity(Object(Aerys\\Client), Array)
#14 ./vendor/vendor/amphp/aerys/lib/Http1Driver.php(318): Aerys\\Server->onParseEmit(Object(Aerys\\Client), 2, Array, NULL)
#15 [internal function]: Aerys\\Http1Driver->parser(Object(Aerys\\Client))
#16 ./vendor/vendor/amphp/aerys/lib/Server.php(524): Generator->send('GET /v3/vivo/ms...')
#17 ./vendor/vendor/amphp/amp/lib/UvReactor.php(424): Aerys\\Server->onReadable('000000004153540...', Resource id #268, Object(Aerys\\Client))
#18 ./vendor/vendor/amphp/amp/lib/UvReactor.php(401): Amp\\UvReactor->invokePollWatcher(Object(stdClass))
#19 [internal function]: Amp\\UvReactor->Amp\\{closure}(Resource id #269, 0, 1, Resource id #268)
#20 ./vendor/vendor/amphp/amp/lib/UvReactor.php(93): uv_run(Resource id #60, 1)
#21 ./vendor/vendor/amphp/amp/lib/functions.php(46): Amp\\UvReactor->run(Object(Closure))
#22 ./vendor/vendor/amphp/aerys/bin/aerys(106): Amp\\run(Object(Closure))
#23 {main}

ConnectionPool does not close idle connections

There should be a mechanism like in \Amp\Socket\BasicSocketPool that closes idle connection after a defined amount of time.
Currently it is not possible to release idle connections gracefully. You can only dispense / replace the whole pool to get rid of idle connections which IMHO is not a viable solution.
In production peaks can created. E.g. by a slightly overloaded server. Not releasing idle connection can create a failure cascade e.g. when the server reaches max connections and starts dropping connection attempts.
This is unlikely to be an issue with a hand full of clients but if say 32 aerys workers are running on 30 machines each, each of them creating 10, 20, 50 connections to the same server AND not releasing them - this could become an issue.

Calling Connection::close always produces a warning

Using latest master, calling Connection::close always triggers a warning:

fclose(): supplied resource is not a valid stream resource in ../vendor/amphp/mysql/lib/Connection.php:1072

Debugger shows that $connection->close() is being called twice but turns out in user space domain I'm only calling ->close() once:

class MyAsyncClientClass
{
    private
        $pool
    ;

    function __construct($pool)
    {
        $this->pool = $pool;
    }

    function transaction(\Closure $fn)
    {
        $connection = yield $this->pool->getConnection();

        try {
            yield $connection->query('START TRANSACTION;');
            $result = yield from $fn($connection);
            yield $connection->query('COMMIT;');
            return $result;

        } catch (\Exception $e) {
            $connection->query('ROLLBACK;');
            throw $e;
        }
        finally {
            $connection->close(); // always triggers fclose() warning
        }
    }
}

If close is supposed to be called implicitly even when the connection was acquired, this may not be a bug. Anyway it would be nice to know when the Pool will close the connection even when the acquired connection was unmapped.

If close is supposed to do not be called implicitly when the connection was acquired, this may be a bug.

I've tried the following patch and, even though it makes sense, it caused connections to never be closed:

	private function closeSocket() {
		$this->disableRead();
		$this->disableWrite();
+                if (is_resource($this->socket)) {
-		   @fclose($this->socket);
+		   fclose($this->socket);
+                }
		$this->connectionState = self::CLOSED;
	}

How to get insert id properly?

I am trying to get the last insert id. But I am using it in pool mode, will it affect the result by other inserts?

$resultSet = yield $pool->prepare("INSERT INTO `table` (`data`) VALUE (:data)", ['data' => $data]);
$insertId= $resultSet->getConnInfo()->insertId;

Is it the correct code to get the insertId?

Call to a member function succeed() on null

This error happens fairly often.

$pool = new Pool();

$promises[] = $pool->query('<query>');
$promises[] = $pool->query('<query>');
$promises[] = $pool->query('<query>'));
$promises[] = $pool->query('<query>'));
$promises[] = $pool->query('<query>'));
$promises[] = $pool->query'<query>'));

foreach ($promises as $promise) {
    // Retrieve the result set
    \Amp\wait($promise);
}
#21 {main}  
[2017-05-31 14:00:38] lumen.ERROR: Symfony\Component\Debug\Exception\FatalThrowableError: Call to a member function succeed() on null in /home/vagrant/fras/source/lumen/vendor/amphp/mysql/lib/Processor.php:458
Stack trace:
#0 /home/vagrant/fras/source/lumen/vendor/amphp/mysql/lib/Processor.php(1086): Amp\Mysql\Processor->handleOk('\x00\x00\x00\n\x00\x00\x00')
#1 /home/vagrant/fras/source/lumen/vendor/amphp/mysql/lib/Processor.php(926): Amp\Mysql\Processor->parsePayload('\x00\x00\x00\n\x00\x00\x00')
#2 /home/vagrant/fras/source/lumen/vendor/amphp/amp/lib/NativeReactor.php(265): Amp\Mysql\Processor->onRead('000000003d7266f...', Resource id #511, NULL)
#3 /home/vagrant/fras/source/lumen/vendor/amphp/amp/lib/NativeReactor.php(249): Amp\NativeReactor->doIoCallback('000000003d7266f...', Object(stdClass), Resource id #511)
#4 /home/vagrant/fras/source/lumen/vendor/amphp/amp/lib/NativeReactor.php(214): Amp\NativeReactor->selectActionableStreams(0.5947)
#5 /home/vagrant/fras/source/lumen/vendor/amphp/amp/lib/NativeReactor.php(156): Amp\NativeReactor->doTick(false)

"Call to a member function updateState() on null" while traversing result set

First of all, thank you for a great library.

In Amp\Mysql\Internal\Processor->successfulResultsetFetch() method, there can be a case where $this->result is null, but a method is executed on it:

private function successfulResultsetFetch() {
    $deferred = &$this->result->next;
    if ($this->connInfo->statusFlags & StatusFlags::SERVER_MORE_RESULTS_EXISTS) {
        $this->parseCallback = [$this, "handleQuery"];
        $this->addDeferred($deferred ?: $deferred = new Deferred);
    } else {
        if (!$deferred) {
            $deferred = new Deferred;
        }
        $deferred->resolve();
        $this->parseCallback = null;
        $this->query = null;
        $this->result = null;
        $this->ready();
    }
    $this->result->updateState(ResultProxy::ROWS_FETCHED);
}

In Amp\Mysql\Internal\Processor->prepareFields() there is a similar situation, but $this->result is stored in the temporary variable. When I use this approach in the situtaion above, the code goes fine, until $db->close(): "Connection went away... unable to fulfil this deferred ... It's unknown whether the query was executed..." (somehow there are still some elements in the $this->deferreds array during closing).

By the way I do not know why, but the script fails silently, and the only way I was able to see exception is to alter Amp\Internal\Placeholder:

/**
 * @param \Throwable $reason Failure reason.
 */
private function fail(\Throwable $reason) {
    var_dump($reason->getMessage());
    var_dump($reason->getTraceAsString());

    $this->resolve(new Failure($reason));
}

Thank you in advance.

Problem with receiving all rows

Hello
My table has 3504 rows.
When I use this code:

	function resultToArray($result) {
    $rows = array();
    while($row = mysqli_fetch_assoc($result)) {
        $rows[] = $row;
    }
    return $rows;
	}
	
	function GetMyConnectionGroup() {
		$servername = 'localhost';
		$username = 'root';
		$password = '******';
		$dbname = 'dbn';
		global $g_link;
		if ($g_link) return $g_link;
		$g_link = new mysqli($servername, $username, $password, $dbname) or die('Could not connect to server.');
		return $g_link;
		}
	
		$res = mysqli_query(GetMyConnectionGroup() , "SELECT * FROM groups ORDER BY id");
		$rows = resultToArray($res);
		mysqli_free_result($res);

I can receive all of 3504 rows
But when I use this code:

function MGSC($code='', $m=0) {
	global $row;
	global $db;
\Amp\Loop::run(function() use ($code, $row, $db, $m) {
	global $row;
	global $db;
	if($code) {
    if(!$db) $db = Amp\Mysql\pool("host=localhost user=root password=***** db=dbn");
    
	$type = explode(' ', $code, 2);
	$type[0] = strtolower($type[0]);
    $result = yield $db->query($code);
	if($type[0] == 'select') {
	$row = [];
    if($m === 0) {
    while (yield $result->advance()) {
        $row = $result->getCurrent();
    }
	}
	elseif($m === 1) {
    while (yield $result->advance()) {
        $row[] = $result->getCurrent();
    }
	}
	return $row;
	}
	else {
	return $result;
	}
	}
	else {
    if($db) $db->close();
	}
});
	if($code) {if($row) return $row; elseif(isset($result)) return $result;}
		}

$rows = MGSC("SELECT * FROM groups ORDER BY id", 1);

I mostly receive just 4 rows and some times 25 rows.
Also these errors:

2018-02-07T16:34:01.942293Z 215 [Note] Aborted connection 215 to db: 'dbn' user: 'root' host: 'localhost' (Got an error writing communication packets)

2018-02-07T15:48:02.011128Z 169 [Note] Aborted connection 169 to db: 'dbn' user: 'root' host: 'localhost' (Got an error reading communication packets)

How can I receive all rows ?
What is the problem ?

Notice when decoding OK packets

Looking at server logs the following notice happens quite frequently:

notice Uninitialized string offset: 9 in ../vendor/amphp/mysql/lib/DataTypes.php on line 202

Database interaction works fine, but this impedes usage of a more picky error handler.

Change connection string to URI

I think it would be good to use URIs to configure connections, e.g. mysql://user:password@host:port/database?timeout=abc. URIs provide everything we need: User + password on host / port, a database name as path and options via URL parameters.

Buffer overflow running in Centos guest.

I'm attempting to run example 003_generic_with_yield inside a vagrant box. The guest OS is Centos 6.4, the host is OSX.

Using a valid IP address the code appears to cause a buffer overflow. This happens whether I use the IP address of '10.0.2.2' which is the correct one that has MySQL running on it, or '127.0.0.1' which obviously exists, but doesn't have MySQL running on it.

There error isn't shown if I attempt to use an IP address that is unreachable.

[root@rpm examples]# strace /usr/local/bin/php 003_generic_with_yield.php > AmpMySQLOverflow.txt 2>&1
*** buffer overflow detected ***: strace terminated
======= Backtrace: =========
/lib64/libc.so.6(__fortify_fail+0x37)[0x7fdb95bb7507]
/lib64/libc.so.6(+0x1003f0)[0x7fdb95bb53f0]
/lib64/libc.so.6(+0xff2a7)[0x7fdb95bb42a7]
strace[0x409c1c]
strace[0x40592f]
strace[0x403045]
strace[0x404661]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x7fdb95ad3cdd]
strace[0x401fd9]
======= Memory map: ========
00400000-0044a000 r-xp 00000000 fd:00 4460201                            /usr/bin/strace
0064a000-0064b000 rw-p 0004a000 fd:00 4460201                            /usr/bin/strace
0064b000-00659000 rw-p 00000000 00:00 0 
00671000-00692000 rw-p 00000000 00:00 0                                  [heap]
7fdb9589f000-7fdb958b5000 r-xp 00000000 fd:00 28836039                   /lib64/libgcc_s-4.4.7-20120601.so.1
7fdb958b5000-7fdb95ab4000 ---p 00016000 fd:00 28836039                   /lib64/libgcc_s-4.4.7-20120601.so.1
7fdb95ab4000-7fdb95ab5000 rw-p 00015000 fd:00 28836039                   /lib64/libgcc_s-4.4.7-20120601.so.1
7fdb95ab5000-7fdb95c3f000 r-xp 00000000 fd:00 28835854                   /lib64/libc-2.12.so
7fdb95c3f000-7fdb95e3e000 ---p 0018a000 fd:00 28835854                   /lib64/libc-2.12.so
7fdb95e3e000-7fdb95e42000 r--p 00189000 fd:00 28835854                   /lib64/libc-2.12.so
7fdb95e42000-7fdb95e43000 rw-p 0018d000 fd:00 28835854                   /lib64/libc-2.12.so
7fdb95e43000-7fdb95e48000 rw-p 00000000 00:00 0 
7fdb95e48000-7fdb95e68000 r-xp 00000000 fd:00 28835847                   /lib64/ld-2.12.so
7fdb9605b000-7fdb9605e000 rw-p 00000000 00:00 0 
7fdb96065000-7fdb96067000 rw-p 00000000 00:00 0 
7fdb96067000-7fdb96068000 r--p 0001f000 fd:00 28835847                   /lib64/ld-2.12.so
7fdb96068000-7fdb96069000 rw-p 00020000 fd:00 28835847                   /lib64/ld-2.12.so
7fdb96069000-7fdb9606a000 rw-p 00000000 00:00 0 
7fffcbe85000-7fffcbe9a000 rw-p 00000000 00:00 0                          [stack]
7fffcbf03000-7fffcbf04000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]

The full output of the strace is too large to post here; it is on S3:
https://s3.amazonaws.com/static.basereality.com/AmpMySQLOverflow.txt

Retry if connection has been stale

If a connection isn't used for a long time, it still remains open. If the next query is issued a long time after then, it fails, because the connection went away. This shouldn't happen, it should be handled automatically internally.

Perf issues with prepared statements ?

I'm seeing a large perf impact on prepared queries vs regular queries :

\Amp\Promise\wait(
    \Amp\call(function() {
        $db = yield \Amp\Mysql\connect('host=127.0.0.1;user=root;pass=foobar;db=my_db');
        for ($i = 0; $i < 10; $i++) {
            $time = microtime(true);
            $results = yield $db->execute("SELECT :rand", [ 'rand' => rand() ]);
            yield $results->advance();
            echo round((microtime(true) - $time) * 1000, 2) . " ms" . PHP_EOL;
        }
    })
);

outputs :
2.73 ms
40.8 ms
39.99 ms
40.22 ms
40.02 ms
39.91 ms
40 ms
39.98 ms
40 ms
39.57 ms

Same code using a regular query :

\Amp\Promise\wait(
    \Amp\call(function() {
        $db = yield \Amp\Mysql\connect('host=127.0.0.1;user=root;pass=foobar;db=my_db');
        for ($i = 0; $i < 10; $i++) {
            $time = microtime(true);
            $results = yield $db->query(sprintf("SELECT %d", rand()));
            yield $results->advance();
            echo round((microtime(true) - $time) * 1000, 2) . " ms" . PHP_EOL;
        }
    })
);

outputs :
1.89 ms
0.37 ms
0.37 ms
0.36 ms
0.36 ms
0.36 ms
0.36 ms
0.36 ms
0.41 ms
0.32 ms

40ms overhead per query seems fairly taxing for the confort of prepared queries ?

getConnection returns NULL

I am using getConnection for my transaction.

$mysqlPool = new Mysql\Pool("host=".DB_HOST.";user=".DB_USER.";pass=".DB_PWD.";db=".DB_NAME, CONN_LIMIT);
var_dump($mysqlPool->getConnection());

Result:

NULL

I expect returning a promise, but null is returned.

Memory leak in \Amp\Mysql\ResultSet

I have a table with 100000 rows with some test data. When i try to select all rows from table i have very large the memory usage. I thought that this value will not increase depending on the number of rows. Is this expected behavior or is it a memory leak?

My example:

$db = Amp\Mysql\pool("...");

$start = microtime(true);

/** @var \Amp\Mysql\ResultSet $result */
$result = yield $db->query("SELECT * FROM `amp_memory`;");

while (yield $result->advance()) {
    // nothing
}

$end = microtime(true);

echo 'Selected 100000 rows: ' . ($end - $start) . PHP_EOL;

echo memory_get_peak_usage(true)/1024/1024 . ' MB' . PHP_EOL;

$db->close();

Output:

Selected 100000 rows: 2.8150889873505
192.00390625 MB

How to release back my connection to the pool?

I got a connection from the pool for doing transaction.
How can I release it back to the pool after commit?

for the current design, it seems I need to close that connection myself. because the connection I got is removed from the connection pool.

It seems the development of this project is being inactive. But I have to say, this is an important project for making PHP better to compare with node.js. Thanks for amphp, I can code event-driven now. But the supporting library is so few. Cheer up. Make amphp better. yield syntax is far much better than evil then.

Fail to connect code is bogus

The code here seems quite wrong.

It's not checking that $socket is not null, and so I'm getting the error fclose() expects parameter 1 to be resource, null given in ...amp-mysql/lib/Connection.php on line 138.

Also, I think it should be checking for the error first and calling the $future->fail regardless of connection state.

Check Stmt::bind() types

If you will run following code:

$stmt->bind('id', [1,2,3,4,5]);

you will get following error:

Notice: Array to string conversion

As quick solution you can use Doctrine helper function to convert params to flat array:

list($sql, $params, $paramsType) = SQLParserUtils::expandListParameters($query->getSQL(), $query->getParameters(), $query->getParameterTypes());

Warning that lead to process hung

After following warning process hung:

Warning: unpack(): Type V: not enough input, need 4, have 1 in ./vendor/amphp/mysql/lib/DataTypes.php on line 293

Currenlty I'm not able to find steps to reproduce. It appears randomly.

Add README

There should be a README with some examples / introduction that also points to /examples.

zlib affects queries performance

After some benchmarking I have figure out that zlib compresion affects query performance a lot.

w/ zlib:

+------------+----------------------------+--------+--------+------+-----+------------+-----------+------------+------------+-------------+------------+---------+------------+
| benchmark  | subject                    | groups | params | revs | its | mem_peak   | best      | mean       | mode       | worst       | stdev      | rstdev  | diff       |
+------------+----------------------------+--------+--------+------+-----+------------+-----------+------------+------------+-------------+------------+---------+------------+
| QueryBench | benchPdoQueries            |        | []     | 1    | 10  | 4,439,176b | 1.17000ms | 1.87690ms  | 1.68573ms  | 3.83200ms   | 0.68067ms  | 36.27%  | 0.00%      |
| QueryBench | benchSyncQueries           |        | []     | 1    | 10  | 4,439,184b | 5.76100ms | 46.13990ms | 10.12466ms | 228.74400ms | 74.52826ms | 161.53% | +2,358.30% |
| QueryBench | benchAsyncQueries          |        | []     | 1    | 10  | 4,439,184b | 4.63700ms | 9.31360ms  | 6.30906ms  | 20.17200ms  | 4.89743ms  | 52.58%  | +396.22%   |
| QueryBench | benchAsyncQueriesUsingPool |        | []     | 1    | 10  | 6,633,522b | 3.53700ms | 4.33780ms  | 3.87706ms  | 6.75500ms   | 0.98273ms  | 22.65%  | +131.12%   |
+------------+----------------------------+--------+--------+------+-----+------------+-----------+------------+------------+-------------+------------+---------+------------+

w/o zlib:

+------------+----------------------------+--------+--------+------+-----+------------+-----------+-----------+-----------+------------+-----------+--------+----------+
| benchmark  | subject                    | groups | params | revs | its | mem_peak   | best      | mean      | mode      | worst      | stdev     | rstdev | diff     |
+------------+----------------------------+--------+--------+------+-----+------------+-----------+-----------+-----------+------------+-----------+--------+----------+
| QueryBench | benchPdoQueries            |        | []     | 1    | 10  | 4,433,944b | 1.28300ms | 1.50140ms | 1.52774ms | 1.76400ms  | 0.13752ms | 9.16%  | 0.00%    |
| QueryBench | benchSyncQueries           |        | []     | 1    | 10  | 4,433,952b | 4.29400ms | 7.52330ms | 4.86931ms | 15.60100ms | 4.11548ms | 54.70% | +401.09% |
| QueryBench | benchAsyncQueries          |        | []     | 1    | 10  | 4,433,952b | 3.94700ms | 4.96600ms | 4.40386ms | 9.93300ms  | 1.68206ms | 33.87% | +230.76% |
| QueryBench | benchAsyncQueriesUsingPool |        | []     | 1    | 10  | 6,592,336b | 2.97000ms | 3.20040ms | 3.27979ms | 3.35800ms  | 0.12980ms | 4.06%  | +113.16% |
+------------+----------------------------+--------+--------+------+-----+------------+-----------+-----------+-----------+------------+-----------+--------+----------+

Probably it must not be enabled by default when zlib is available.

Benchmark source: https://github.com/torinaki/mysql/blob/benchmarks/benchmarks/QueryBench.php

Notice Uninitialized string offset in DataTypes.php

This notice happens when querying insert statements or calling $pool->ping(). Looks like a minor bug as database interactions still works as expected but it's a bummer when using a more strict error handler that transforms all levels of errors into exceptions:

set_error_handler(function($errno, $errstr, $file, $line) {
   throw new \Exception("{$errstr} in {$file}:{$line}", $errno);
});

Error is:

notice Uninitialized string offset: 7 in /.../vendor/amphp/mysql/lib/DataTypes.php on line 202

Request for fetchAssoc()

At the moment, amphp/mysql provides fetchRow, fetchObject and fetch, but no fetchAssoc.
It's very common in mysqli and PDO. I recommend to add this method to provide string index result without combining with numeric index.

Cannot require amphp/mysql

./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Installation request for amphp/mysql ^0.1.6 -> satisfiable by amphp/mysql[v0.1.6].
    - amphp/mysql v0.1.6 requires amphp/amp ^1 -> no matching package found.

Potential causes:
 - A typo in the package name
 - The package is not available in a stable-enough version according to your minimum-stability setting
   see <https://getcomposer.org/doc/04-schema.md#minimum-stability> for more details.

Read <https://getcomposer.org/doc/articles/troubleshooting.md> for further common problems.

Installation failed, reverting ./composer.json to its original content.

It seems amphp/mysql is trying to require amphp/amp v1, but the latest version of amphp/amp is v2 now. when will it support to it?

Important: Unresolved query promise

When a server goes away (which is ok when wait_timeout is set), there must be a way to retry the query with a different connection.

I implemented that outside of amp/mysql but this leads to unresolved query promises in many cases.

See script to reproduce:
https://gist.github.com/brstgt/f7b23ef4a174e950587e0aed4e0bd245

Output looks like:

Try
Try
MySQL went away on query SELECT ...
Try
Got response
[Script ends]

Should look like

Try
Try
MySQL went away on query SELECT ...
Try
Got response
Resolve
DONE

Mysql\Pool::setCharset() runs forever

// 001_connect.php
Amp\run(function() {
    $db = new \Mysql\Pool(DSN);

    // getConfig() added to my source: return Pool::$config;
    echo $db->getConfig()->charset , "\n";

    $db->setCharset("latin1_general_ci");

    echo $db->getConfig()->charset , "\n";

    $db->close();
});

// php 001_connect.php

awkwardly it runs forever.

If I yield $db->setCharset("latin1_general_ci"); it's runs, but I still get
an exception with message 'Connection not ready, cannot send any packets' in
Connection.php:1647'.

In other way if I yield $db->query("SET NAMES 'latin1' COLLATE 'latin1_general_ci'");
(replacing setCharset()) it runs flawlessly.

Looks like Pool::$connections is never filled upon __construct, only if you use
$db = (yield new Mysql\Pool(DSN)); then Pool::$connectios has 1 item (as expected?),
but still can't use $db->setCharset directly, I needed to
yield $db->setCharset("latin1_general_ci");.

// finally...
<?php
require './example_bootstrap.php';

Amp\run(function() {
    $db = (yield new Mysql\Pool(DSN));

    echo $db->getConfig()->charset , "\n"; // utf8mb4

    yield $db->setCharset("latin1_general_ci");

    echo $db->getConfig()->charset , "\n"; // latin1
    $db->close();
});

Thoughts?

$ php -v
PHP 5.6.5-1 (cli) (built: Jan 27 2015 15:59:26) 

Not possible to set limit on Pool.

Connection::parseConnStr($connStr, $sslOptions) fails with " Undefined offset: 1"

Connection string

limit=2;host=<host>;user=<user>;pass=<pass>;db=<database>

Query Promise does not resolve (again)

I was not able yet to find the cause but I am able to reproduce it. In some situations queries never finish (neither failing nor succeeding).
What I found out is that the pool has only a single connection which is idle before the query is sent. So I guess the problem lies in the Processor, not in the pool.

SegFault during unittest (maybe also normal code)

When I am starting my unit test I get to this location which, if I step over it, produces a seg fault. Sadly I cannot share my unittest because I was not able to reproduce this in an easier environment. But I attached a screenshot of the stacktrace.

I think this is because I run into an infinity loop, visible in the screenshot.
The line of code I am breaking has jumped to Coroutine.php:93 on the previous 3 skips as visible in the screenshot.

I can reproduce this issue every time, so feel free to ask for more information.

amphp was installed with composer and has following versions:

amphp/amp dev-master c632e40 A non-blocking concurrency framework for PHP applications.
amphp/byte-stream v1.3.1 A stream abstraction to make working with non-blocking I/...
amphp/cache v1.2.0 A promise-aware caching API for Amp.
amphp/dns v0.9.12 Async DNS resolution for Amp.
amphp/file v0.3.1 Allows non-blocking access to the filesystem for Amp.
amphp/mysql dev-master f3e1f15 Asynchronous parallel Mysql client built on the Amp concu...
amphp/parallel v0.2.5 Parallel processing component for Amp.
amphp/parser v1.0.0 A generator parser to make streaming parsers simple.
amphp/process v0.3.2 Asynchronous process manager.
amphp/socket v0.10.8 Async socket connection / server tools for Amp.
amphp/sync v1.0.1 Mutex, Semaphore, and other synchronization tools for Amp.
amphp/uri v0.1.3 Uri Parser and Resolver.
amphp/windows-registry v0.3.1 Windows Registry Reader.

image

PHP 7.2 deprecations

PHP deprecates each and (unset), both are used in this package and should be removed.

Transaction. Deadlock

Hello!
I get a fatal error, when the Deadlock happens
I use Transaction class, with Connection class such problem is not present

I use 0.3.1 release

PHP Fatal error:  Uncaught TypeError: Argument 1 passed to Amp\Mysql\DataTypes::decodeBinary() must be of the type integer, null given, called in /application/vendor/amphp/mysql/lib/Internal/Processor.php on line 979 and defined in /application/vendor/amphp/mysql/lib/DataTypes.php:80
Stack trace:
#0 /application/vendor/amphp/mysql/lib/Internal/Processor.php(979): Amp\Mysql\DataTypes::decodeBinary(NULL, '\x04#40001Deadlock...', NULL)
#1 /application/vendor/amphp/mysql/lib/Internal/Processor.php(1253): Amp\Mysql\Internal\Processor->handleBinaryResultsetRow('\xFF\xBD\x04#40001Deadlo...')
#2 /application/vendor/amphp/mysql/lib/Internal/Processor.php(240): Amp\Mysql\Internal\Processor->parsePayload('\xFF\xBD\x04#40001Deadlo...')
#3 /application/vendor/amphp/mysql/lib/Internal/Processor.php(216): Amp\Mysql\Internal\Processor->processData(Array)
#4 [internal function]: Amp\Mysql\Internal\Processor->read()
#5 /application/vendor/amphp/amp/lib/Coroutine.php(74): Generator->send('\x01\x00\x00\x01\x04C\x00\x00\x02\x03def\x11m...')
#6 /ap in /application/vendor/amphp/mysql/lib/DataTypes.php on line 80

amphp\mysql not returning all the records

Hi,
Since I raised this concern in stack overflow as well but couldn't find the answer, I have put it here: https://stackoverflow.com/questions/52058338/amphp-mysql-not-returning-all-the-records

We have started using amphp\mysql to run pool of connections for threading concept. We have a table "content" which has 100k of records. Trying to list out for further processing but fetching only 42 records. Also, doesn't have any LIMIT statement.

What am I missing in my code?

Below is the code:

require "vendor/autoload.php";
use Amp\Loop;
use Amp\Artax\Request;
use Amp\Artax\Response;

$db = '';
Amp\Loop::run(function () {
$client = new Amp\Artax\DefaultClient;
$db = Amp\Mysql\pool("host=".$this->conn_prop['db_host']."; user=".$this->conn_prop['db_user']."; pass=".$this->conn_prop['db_passwd']."; db=".$this->conn_prop['db_name']);

    $statement = (yield $db->query("SELECT registration_id,w_id,ip,device_type FROM content WHERE active_status='1' AND  ip !=''"));
    $cnt = 0;
            while (yield $statement->advance()) {
            echo $cnt++;
            $row = $statement->getCurrent();
            if(!empty($row)){
            var_dump($row);
            .................
            ................
            }   
            }   

});

Waiting for the suggestion.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.