05bit / peewee-async Goto Github PK

View Code? Open in Web Editor NEW

720.0 720.0 98.0 470 KB

Asynchronous interface for peewee ORM powered by asyncio

Home Page: http://peewee-async-lib.readthedocs.io

License: MIT License

Python 100.00%

asyncio mysql orm peewee postgresql python

peewee-async's People

Contributors

Stargazers

Watchers

Forkers

ezc netpie horpto fudong1127 jbjuin excpt0 eduard90 fireball2018 mrbox cypreess steinerr bright-pan quantumdark zouchao2010 strogo spumer dgelvin iffius apopov90 kleschenko juntatalor insolite jnosal heysion guser0512 roy2220 vdt dmonroy speedor jerevia serathius thedrow torsdag xnyo consensys-utility-user serozd alexander3771 oopman aloe47 safarihostel linw1995 avostap smagafurov leshy corcoran opn-oss cash111 koos85 iurii-skorniakov bmwant perrornet andr-04 kapucko yropb fanjindong hakanaku1234 cesbit lttzzlll fuzctc zonnesoft msusik talent-source-code leigeng2014 gridl hatedog savvarost 365055066 akatuy enzzc mike96265 acracker jwattik aeasringnar isgasho iksteen jiyeonseo quarkness mafeicnnui kalombos gladioluz janokruta clotyxf pylakey funnor codelvl sk0fa gshmu jackthemico gdassori notsobot mishaga guitvcer yeatry despitedeath artdokxxx akerlay f1int0m

peewee-async's Issues

how about replacing aiopg with asyncpg?

Seems faster?

Support Models without specified database

Currently when you define a model and don't define the database meta on models (but still pass the proper database instance to peewee_async.Manager), peewee-async crashes when executing a query:

"Error, models's database and manager's are different: %s" % model

This can be hacked around like this (assuming the database is postgres):

class MyModel(peewee.Model):
    class Meta:
        # peewee-async insists on "compatible" database
        database = peewee.PostgresqlDatabase(None)

But why the limitation in the first place? I understand that if the model database meta was specified and it was different from what is then passed to peewee_async.Manager, the compatibility check would make sense. But if a user will never use the non-async way, and never specified the database meta, this check is not needed. Please remove it for that case.

Having the "global" database object is an anti-pattern anyway.

Transactions support

Hi, it's a great project - finally I can use normal DB with asyncio. However it seems that transactions doesn't work:

    object = await peewee_async.create_object(TestModel, text="FOO")
    obj_id = object.id
    try:
        with database.atomic() as txn:
            object.text = "BAR"
            print(await peewee_async.update_object(object))
            raise AttributeError("AAAA")

    except AttributeError as e:
        object = await peewee_async.get_object(TestModel, TestModel.id==obj_id)
        print("TEXT:", object.text)

gives

1
TEXT: BAR

At the moment I have no clue, how could I fix this by myself- if you could give me some hints, I'll be more than happy to contribute.

Question - peewee-async & tornado & py.test

Hi! I'm trying to configure testing for my tornado web application with py.test and awesome peewee-async.
When I run application generally - everything is ok.
The problem is that I get my tests stuck and then fail with AssertionError: Async operation timed out after 5 seconds. I think that peewee async run on
I take code from https://github.com/05bit/peewee-async/blob/master/examples/tornado_sample.py, and remove the loop.run_forever().

Than try something like (with https://pypi.python.org/pypi/pytest-tornado):

from tornado_sample import app as application
import pytest

@pytest.fixture
def app():
    return application

@pytest.mark.gen_test
def test_hello_world(http_client, base_url):
    response = yield http_client.fetch(base_url + '?id=1')
    assert response.code == 200

And it fails on timeout.

I tried to reassign application.objects in app fixture. Tried to use tornado AsyncHTTPTestCase. Nothing helps - the await self.application.objects.get(TestNameModel, id=obj_id) (and any other query) never ends. I think, that the problem is in running query on stopped (or not launched) io loop.
Standart torando testing (AsyncHTTPTestCase) fails too.
Also, i tried to run tests using https://pypi.python.org/pypi/pytest-asyncio. It works (not very convenient - all calls to AsyncHTTPClient.fetch() should be bridged with http://www.tornadoweb.org/en/stable/asyncio.html#tornado.platform.asyncio.to_asyncio_future), but event loop closes after first test, and i did not find the way to start it.
Could you please provide the working testable sample?

Compatibility with peewee 3.0

There are some features I wanted to use in 3.0a branch, but It is not compatible with peewee-async. Can it be supported?

Support playhouse.ManyToManyField asynchronously

Currently, using a ManyToManyField from the playhouse package triggers a sync_query. It would be awesome if this could be handled asynchronously.

aiopg/aiomysql as optional dependecies

Right now when installing peewee-async, aiopg/aiomysql has to be added as my project dependency, which can be confusing. I have to wonder if someone added it and forgot to use it or some external (peewee-async) lib uses it. I would expect I can just:

install_requires=[
        'peewee-async[aiopg]==0.5.7',
],

which I feel is the correct way to go with optional dependencies.

Event loop is closed & 'playhouse._speedups._ModelQueryResultWrapper' object has no attribute 'initialize'

code:

import asyncio
import peewee
import peewee_async

database = peewee_async.**PooledMySQLDatabase**('test')

class TestModel(peewee.Model):
    text = peewee.CharField()

    class Meta:
        database = database

TestModel.create_table(True)
TestModel.create(text="Yo, I can do it sync!")
database.close()

objects = peewee_async.Manager(database)
database.set_allow_sync(False)

async def handler():
    await objects.create(TestModel, text="Not bad. Watch this, I'm async!")
    all_objects = await objects.execute(TestModel.select())
    for obj in all_objects:
        print(obj.text)

loop = asyncio.get_event_loop()
loop.run_until_complete(handler())
loop.close()

with objects.allow_sync():
    TestModel.drop_table(True)

Win7

python35
peewee-2.8.5
peewee-async-0.5.6

RuntimeError: Event loop is closed

Ubuntu16.04

python35
peewee-2.8.5
peewee-async-0.5.6

AttributeError: 'playhouse._speedups._ModelQueryResultWrapper' object has no attribute 'initialize'

how to dynamic assgin meta database?

In example there is code:

database = peewee_async.PostgresqlDatabase('test')
class TestModel(peewee.Model):
    text = peewee.CharField()

    class Meta:
        database = database

How can I dynamic assgin the Meta database? database may be different at different env.

No way to pass timeout to database

There is no way to pass timeout to database, without explicitly creating connection with that timeout, and then implicitly guessing that this connection will be used for queries in that asyncio task.

Simple way to do it would be by passing timeout to database for e. g. peewee_async.PostgresDatabase, but that has two problems.

Firstly timeout will be passed to psycopg2.connect that does not handle timeout, but connect_timeout which applies to database, not asyncio socket.

Secondly it will be used correctly in peewee_async.AsyncDatabase.connect_async, but there timeout is already provided from argument.

Any Idea how to go around it?
I need that feature so I will be forking peewee_async with simple fix, if it's acceptable I will make PR.

Manager for each model

May I create Manager for each my model?
Or that would work not good?

For example:

class Model(peewee.Model):
  _objects = None

  @property
  def objects(self):
    if self._objects:
      return self._objects
    self._objects = peewee_async.Manager(self._meta.database)
    return self._objects

So if I wanna do something like this, should I use Model.objects.close() explicity for ever query to database?

Peewee Proxy - "Cannot set attribute on proxy."

I am trying to use a Peewee db proxy but having some issues when fetching objects.

Here is a minimal example:

import peewee
import peewee_async
import asyncio


db_proxy = peewee.Proxy()


class BaseModel(peewee.Model):
    class Meta:
        database = db_proxy


class KeyValue(BaseModel):
    key = peewee.CharField(max_length=40, unique=True)
    value = peewee.TextField(default='')


def init_database(loop=None):
    database = peewee_async.PooledPostgresqlDatabase('<snip>',
                                        port=5432,
                                        host='<snip>',
                                        user='<snip>',
                                        password='<snip>')
    db_proxy.initialize(database)

    # create table synchronously
    KeyValue.create_table(True)

    manager = peewee_async.Manager(db_proxy, loop=loop)
    return manager


def test():
    loop = asyncio.new_event_loop()
    manager = init_database(loop=loop)

    async def handler(key='qwe', value='qwe'):
        obj = await manager.get(KeyValue, id=1)
        return obj

    r = loop.run_until_complete(handler())


test()

The error I get is from peewee (not peewee-async):

AttributeError: Cannot set attribute on proxy.

Any help would be greatly appreciated. Thanks!

Joins by back refs does not working

I try to make query like this

query = ItemModel.select(ItemModel, ManyToManyItemModel)
        .join(VideoItemScreenshotModel, JOIN.LEFT_OUTER,
              on=(ItemModel.id == ManyToManyItemModel.item_id)
              .alias('m2m_item_set')).aggregate_rows()

I expected to have results list of objects at m2m_item_set and i have it if i call this synchronously.
But when i call it async for example await ItemModel.db_manager.execute(query) i'v got ManyToManyItemModel in the m2m_item_set. Am I doing something wrong?
I understand how joined data adds in peewee and i did not find code which should do this in peewee_async.
In peewee https://github.com/coleifer/peewee/blob/master/peewee.py#L2720 this place do exactly what i want.
I can provide small app for showing the problem if you interested in.

Add support for prefetch() new peewee feature

https://github.com/coleifer/peewee/blob/2.7.4/peewee.py#L4754 - this line triggers sync query, seems like that we should have a separate copy-paste async version of prefetch helper.

More robust integration with peewee

The code of aiopeewee relies strongly on some peewee internals, so things may be fragile. That's not actually a single issue, but here's a start point for discussion.

Insert doesn't return the UUID primary key from the database

Hi, I'm using peewee_async in a project where the primary ID (a non auto-incrementing UUID) is initialized by the database. If I use the synchronous insert function then the insert is performed and the return value is the id of the inserted row but using the asynchronous calls the insert works but the id comes back as None. This means the return value of create_object is also None.

Invoking logic from model's .save() method

Hi! I have peewee model with custom save method:

        def save(self, *args, **kwargs):
            if not self.slug:
                self.slug = slugify(self.title.lower())
            return super().save(*args, **kwargs)

And If I call await self.manager.create(...) the save method will not call.
It may little confuse.

Playhouse support

Now peewee_async.PostgresqlDatabase inherits from peewee.PostgresqlDatabase so we cannot use async-peewee and playhouse extensions for PostgreSql (it requires PostgresqlExtDatabase from playhouse.postgres_ext) together.

Error: local variable 'can_swap' referenced before assignment

I try to use peewee async with peewee proxy:

import peewee
from peewee_async import Manager, PostgresqlDatabase

_database = peewee.Proxy()


class BaseModel(peewee.Model):
    class Meta:
        database = _database


class KeyValue(BaseModel):
    key = peewee.CharField(max_length=40, unique=True)
    value = peewee.TextField(default='')


def init_database(database=None, loop=None):
    if not database:
        database = PostgresqlDatabase('test')

    _database.initialize(database)

    # create table synchronously
    KeyValue.create_table(True)
    database.close()

    manager = Manager(database, loop=loop)
    # disable any future syncronous calls
    # raise AssertionError on ANY sync call
    manager.database.allow_sync = False

    return manager, KeyValue

Test:

from asyncio import new_event_loop
from models import init_database


def test_sync():
    loop = new_event_loop()
    models, KeyValue = init_database(loop=loop)

    async def handler(key='qwe', value='qwe'):
        obj = await models.create(KeyValue, key=key, value=value)
        return obj

    r = loop.run_until_complete(handler())

Result:

$ python -m pytest
====================================================================================== test session starts ======================================================================================
platform darwin -- Python 3.5.1, pytest-3.0.3, py-1.4.31, pluggy-0.4.0
rootdir: /Users/pahaz/PycharmProjects/ructfe-2016/services/cross-social-science, inifile: 
collected 3 items 

test_models.py F

=========================================================================================== FAILURES ============================================================================================
___________________________________________________________________________________________ test_sync ___________________________________________________________________________________________

    def test_sync():
        loop = new_event_loop()
        models, KeyValue = init_database(loop=loop)
    
        async def handler(key='qwe', value='qwe'):
            obj = await models.create(KeyValue, key=key, value=value)
            return obj
    
>       r = loop.run_until_complete(handler())

test_models.py:18: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/base_events.py:337: in run_until_complete
    return future.result()
/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/futures.py:274: in result
    raise self._exception
/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/tasks.py:239: in _step
    result = coro.send(None)
test_models.py:15: in handler
    obj = await models.create(KeyValue, key=key, value=value)
../../../../.virtualenvs/cross-social-science/lib/python3.5/site-packages/peewee_async.py:177: in create
    pk = yield from self.execute(query)
../../../../.virtualenvs/cross-social-science/lib/python3.5/site-packages/peewee_async.py:264: in execute
    query = self._swap_database(query)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <peewee_async.Manager object at 0x11002b1d0>, query = <class 'models.KeyValue'> INSERT INTO "keyvalue" ("key", "value") VALUES (%s, %s) RETURNING "id" ['qwe', 'qwe']

    def _swap_database(self, query):
        """Swap database for query if swappable. Return **new query**
            with swapped database.
    
            Check manager database and model database match. If model
            database is `auto`, manager's one will be used.
    
            If query database can't be swapped and differs from manager's
            database, it's **WRONG AND DANGEROUS**, so assertion is raised.
            """
        if query.database == self.database:
            return query
        elif self._subclassed(peewee.PostgresqlDatabase,
                              query.database,
                              self.database):
            can_swap = True
        elif self._subclassed(peewee.MySQLDatabase,
                              query.database,
                              self.database):
            can_swap = True
    
>       if can_swap:
E       UnboundLocalError: local variable 'can_swap' referenced before assignment

../../../../.virtualenvs/cross-social-science/lib/python3.5/site-packages/peewee_async.py:367: UnboundLocalError

AttributeError: 'playhouse._speedups._ModelQueryResultWrapper' object has no attribute 'initialize'

I got an error after I installed peewee=2.8.2

Traceback (most recent call last):
  File "/home/user/project/env/lib/python3.5/site-packages/aiohttp/server.py", line 266, in start
    yield from self.handle_request(message, payload)
  File "/home/user/project/env/lib/python3.5/site-packages/aiohttp/web.py", line 90, in handle_request
    resp = yield from handler(request)
  File "/home/user/project/env/lib/python3.5/site-packages/aiohttp_session/__init__.py", line 129, in middleware
    response = yield from handler(request)
  File "/home/user/project/src/account/middleware.py", line 19, in middleware
    request.user = await User.objects.get(User, id=user_id)
  File "/home/user/project/env/lib/python3.5/site-packages/peewee_async.py", line 151, in get
    result = yield from self.execute(query.limit(1))
  File "/home/user/project/env/lib/python3.5/site-packages/peewee_async.py", line 251, in execute
    return (yield from execute(query))
  File "/home/user/project/env/lib/python3.5/site-packages/peewee_async.py", line 418, in execute
    return (yield from coroutine(query))
  File "/home/user/project/env/lib/python3.5/site-packages/peewee_async.py", line 565, in select
    yield from result.fetchone()
  File "/home/user/project/env/lib/python3.5/site-packages/peewee_async.py", line 789, in fetchone
    self._result_wrapper.initialize(self._cursor.description)
AttributeError: 'playhouse._speedups._ModelQueryResultWrapper' object has no attribute 'initialize'

peewee.InterfaceError: (0, '') | pymysql.err.InterfaceError: (0, '')

I am encountering this exception (using high-level api) then my asyncio based daemon is idle (about 15 minutes or so), after that time, this exception occurs, I have tried to close and reopen connection, and was surprised that Manager (high-level api) did not track its state (close/reopen is done using its methods).

after reopening a connection I've got stuck with a bunch of coroutines waiting for resolution which obviously will never come. =(

can it be a bug in pymysql/peewee
propose add connection status tracking to high-level Manager

any thoughts on how to fix that?

Traceback (most recent call last):
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/peewee.py", line 3748, in execute_sql
warehouse_1        |     cursor.execute(sql, params or ())
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/pymysql/cursors.py", line 166, in execute
warehouse_1        |     result = self._query(query)
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/pymysql/cursors.py", line 322, in _query
warehouse_1        |     conn.query(q)
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/pymysql/connections.py", line 851, in query
warehouse_1        |     self._execute_command(COMMAND.COM_QUERY, sql)
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/pymysql/connections.py", line 1067, in _execute_command
warehouse_1        |     raise err.InterfaceError("(0, '')")
warehouse_1        | pymysql.err.InterfaceError: (0, '')
warehouse_1        | 
warehouse_1        | During handling of the above exception, another exception occurred:
warehouse_1        | 
warehouse_1        | Traceback (most recent call last):
warehouse_1        |   File "/opt/app/storage/sql.py", line 357, in _wrapper_catch_db_excetions
warehouse_1        |     return await fn(*args, **kwargs)
warehouse_1        |   File "/opt/app/storage/sql.py", line 415, in update
warehouse_1        |     await self._manager.create(Item2Source, item_id=existing.id, source_id=source.id)
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/peewee_async.py", line 183, in create
warehouse_1        |     pk = inst._get_pk_value()
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/peewee.py", line 5017, in _get_pk_value
warehouse_1        |     return getattr(self, self._meta.primary_key.name)
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/peewee.py", line 1553, in __get__
warehouse_1        |     for field_name in self.field_names])
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/peewee.py", line 1553, in <listcomp>
warehouse_1        |     for field_name in self.field_names])
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/peewee.py", line 1363, in __get__
warehouse_1        |     return self.get_object_or_id(instance)
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/peewee.py", line 1354, in get_object_or_id
warehouse_1        |     obj = self.rel_model.get(self.field.to_field == rel_id)
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/peewee.py", line 4900, in get
warehouse_1        |     return sq.get()
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/peewee.py", line 3161, in get
warehouse_1        |     return next(clone.execute())
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/peewee.py", line 3213, in execute
warehouse_1        |     self._qr = ResultWrapper(model_class, self._execute(), query_meta)
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/peewee.py", line 2892, in _execute
warehouse_1        |     return self.database.execute_sql(sql, params, self.require_commit)
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/peewee_async.py", line 1024, in execute_sql
warehouse_1        |     return super().execute_sql(*args, **kwargs)
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/peewee.py", line 3755, in execute_sql
warehouse_1        |     self.commit()
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/peewee.py", line 3578, in __exit__
warehouse_1        |     reraise(new_type, new_type(*exc_args), traceback)
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/peewee.py", line 135, in reraise
warehouse_1        |     raise value.with_traceback(tb)
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/peewee.py", line 3748, in execute_sql
warehouse_1        |     cursor.execute(sql, params or ())
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/pymysql/cursors.py", line 166, in execute
warehouse_1        |     result = self._query(query)
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/pymysql/cursors.py", line 322, in _query
warehouse_1        |     conn.query(q)
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/pymysql/connections.py", line 851, in query
warehouse_1        |     self._execute_command(COMMAND.COM_QUERY, sql)
warehouse_1        |   File "/usr/local/lib/python3.5/site-packages/pymysql/connections.py", line 1067, in _execute_command
warehouse_1        |     raise err.InterfaceError("(0, '')")
warehouse_1        | peewee.InterfaceError: (0, '')

Cursor never released for MySQLDatabase

When we bind cursor.release function in AsyncMysqlConnection class we expect this is not a coroutine.

def release():
    cursor.close()
    self.pool.release(conn)
cursor.release = release

# OR 

cursor.release = lambda: cursor.close()

aiomysql.cursors.Cursor.close defined like a coroutine: https://github.com/aio-libs/aiomysql/blob/master/aiomysql/cursors.py#L151

So, for solve this issue we can detour .close():

def release():
    self.loop.call_soon(cursor.close())
    self.pool.release(conn)
cursor.release = release

# OR 

cursor.release = lambda: self.loop.call_soon(cursor.close())

or we can return Future: by asyncio.async(cursor.close())
This will guarantees execution and allow yielding from.

I preffer first solution. Let's talk about that. Можно на нашем :)

the field can't named 'model' and it is normal with peewee...

I use peewee-async Model to related with my postgresql, and there is a field named model in my table
When I create a data,It happens an error.

models.py:

import peewee
class Test(peewee.Model):
    model = peewee.CharField(max_length=64,  null=True)
    class Meta:
        database = DATABASE
        db_table = 'test'
        schema = 'opr'

The place where I create data:

insert_sql = {
    'model':'android2'
}
yield from self.db.create(Test,**insert_sql)

The errors:

File "/home/mwh/py3workplace/workplace/ad_tornado/handlers/opr.py", line 78, in post yield from 

    self.db.create(Test,model='android2') TypeError: create() got multiple values for argument ‘model’

I thought It may some problems in peewee ,But I test create data only use peewee,It runs normally

import peewee
db = peewee.PostgresqlDatabase(database='postgres',
    user='postgres',
)
class Test(peewee.Model):
    id = peewee.IntegerField(primary_key=True)
    model = peewee.CharField(db_column='model',max_length=64,  null=True)
    class Meta:
        database = db
        db_table = 'test'
        schema = 'opr'
db.connect()
Test.create(model='android')

what's the problem in peewee-async?

Add support for aggregate_rows

From peewee documentation (http://docs.peewee-orm.com/en/latest/peewee/querying.html#using-aggregate-rows):

query = (User
         .select(User, Tweet)  # As in the previous example, we select both tables.
         .join(Tweet, JOIN.LEFT_OUTER)
         .order_by(User.username)  # We need to specify an ordering here.
         .aggregate_rows())  # Tell peewee to de-dupe and aggregate results.

for user in query:
    print user.username
    for tweet in user.tweets:
        print '  ', tweet.message

Imagine, we have 2 users: "A" and "B". "A" have 1 tweet and "B" have 2 tweets.
The example above would result in something like this:

A
  tweet1
B
  tweet2
  tweet3

Now, I'm trying to implement the same using peewee_async:

query = yield from peewee_async.execute(
         User
         .select(User, Tweet)  # As in the previous example, we select both tables.
         .join(Tweet, JOIN.LEFT_OUTER)
         .order_by(User.username)  # We need to specify an ordering here.
         .aggregate_rows())  # Tell peewee to de-dupe and aggregate results.

for user in query:
    print user.username
    for tweet in user.tweets:
        print '  ', tweet.message

The result is different:

A
  tweet1
B
  tweet2
B
  tweet3

In pure peewee this happens when aggregate_rows() is not used.
Am I doing something wrong or there is no support for aggregate_rows() in peewee_async?

Log every executed query

Peewee have a nice feature which logs on debug level every executed query. Would be awesome if peewee-async had same feature.

Unexpected keyword when updating

While using peewee async, got an error when trying to update an object:

Traceback (most recent call last):
  File "./src/models/module.py", line 74, in run
    await self._stage_alert_attach(target_alert)
  File "./src/models/module.py", line 293, in _stage_alert_attach
    await target_alert.attach(unattached_issues)
  File "./src/models/alert.py", line 127, in attach
    await issue.attach(self)
  File "./src/models/issue.py", line 81, in attach
    await self.async_save()
  File "./src/models/base.py", line 40, in async_save
    return await self._manager.update(self, only=self.dirty_fields)
  File "/usr/local/lib/python3.6/site-packages/peewee_async.py", line 227, in update
    query = obj.update(**field_dict).where(obj._pk_expr())
TypeError: update() got an unexpected keyword argument 'alert_id'

I can't figure out what's happening because the field is defined in the base class:

class BaseIssue(BaseModel):
    # Database fields
    id = peewee.PrimaryKeyField()
    module_id = peewee.IntegerField()
    model = peewee.CharField(index=True)
    model_id = peewee.CharField(index=True)
    status = peewee.CharField(index=True)
    metadata = JSONField()
    alert_id = peewee.IntegerField(null=True)
    created_at = peewee.DateTimeField(default=datetime.datetime.now)
    solved_at = peewee.DateTimeField(null=True)
    expired_at = peewee.DateTimeField(null=True)

    class Meta:
        def db_table_func(model_cls): return "Issues"

Also tried to use alert instead of alert_id and using it as a foreign key, but got the same error (now with the alert keyword).

I think there might be a problem with inheritance, as this base class is inherited to create an DefaultIssue class that will be inherited again. For some reason the update() method is not expecting this parameter.

Here's a print of the object __dict__ and fields right before the self.async_save() call:

>>> obj.__dict__

{
  "_data": {
    "created_at": datetime.datetime(2017, 10, 10, 17, 6, 8, 47075),
    "id": 2,
    "module_id": 3,
    "model": "transaction",
    "model_id": "23765771",
    "status": "active",
    "metadata": {
      "transaction_id": 23765771,
      "boleto_url": None,
      "boleto_barcode": None
    },
    "alert_id": None,
    "solved_at": None,
    "expired_at": None
  },
  "_dirty": set(),
  "_obj_cache": {},
  "_logger": <Logger Issue 2 (INFO)>
}

>>> obj._meta.fields

{
  "id": <peewee.PrimaryKeyField object at 0x7f49fdb2e198>,
  "module_id": <peewee.IntegerField object at 0x7f49fdb2e3c8>,
  "model": <peewee.CharField object at 0x7f49fdb2e710>,
  "model_id": <peewee.CharField object at 0x7f49fdb2e7f0>,
  "status": <peewee.CharField object at 0x7f49fdb2e860>,
  "metadata": <playhouse.postgres_ext.JSONField object at 0x7f49fdb2e8d0>,
  "alert_id": <peewee.IntegerField object at 0x7f49fdb2e940>,
  "created_at": <peewee.DateTimeField object at 0x7f49fdb2e9b0>,
  "solved_at": <peewee.DateTimeField object at 0x7f49fdb2ea20>,
  "expired_at": <peewee.DateTimeField object at 0x7f49fdb2ea90>
}

If there's any more information I can provide, ask please. I don't know what to do anymore.

Stack Overflow question:
https://stackoverflow.com/questions/46677256/peewee-async-unexpected-keyword-when-updating

MySQL encoding issues

When performing async queries using peewee-async some UTF-8 data is fetched as sequence of question marks while peewee's sync queries return this data as is. This issue doesn't occur with PostgreSQL database.

How to reproduce

Creating database

DROP DATABASE IF EXISTS `test`;

CREATE DATABASE IF NOT EXISTS `test`
    DEFAULT CHARACTER SET utf8
    DEFAULT COLLATE utf8_general_ci;

Code

import peewee
import peewee_async
import asyncio


texts = ['[en] The quick brown fox jumps over the lazy dog', 
        '[ru] Быстрая коричневая лиса прыгает через ленивую собаку',
        '[pl] Szybki brązowy lis przeskoczył nad leniwym psem',
        '[de] Der schnelle braune Fuchs springt über den faulen Hund',
        '[ko] 빠른 갈색 여우가 게으른 개 점프']

database = peewee_async.PooledMySQLDatabase(host='localhost', user='root', 
        password='', database='test')

class TestModel(peewee.Model):
    text = peewee.CharField()

    class Meta:
        database = database


TestModel.create_table(True)

for t in texts:
    obj = TestModel.create(text=t)
    print('(%3s) Created:\t %s' % (('ok' if obj.text in texts else 'err'), obj.text))

print()
for obj in TestModel.select():
    print('(%3s) Sync:\t %s' % (('ok' if obj.text in texts else 'err'), obj.text))


database.close()
objects = peewee_async.Manager(database)
database.allow_sync = False

async def handler():
    all_objects = await objects.execute(TestModel.select())
    for obj in all_objects:
        print('(%3s) Async:\t %s' % (('ok' if obj.text in texts else 'err'), obj.text))

print()
loop = asyncio.get_event_loop()
loop.run_until_complete(handler())
loop.close()

with objects.allow_sync():
    TestModel.drop_table(True)

Result

( ok) Created:   [en] The quick brown fox jumps over the lazy dog
( ok) Created:   [ru] Быстрая коричневая лиса прыгает через ленивую собаку
( ok) Created:   [pl] Szybki brązowy lis przeskoczył nad leniwym psem
( ok) Created:   [de] Der schnelle braune Fuchs springt über den faulen Hund
( ok) Created:   [ko] 빠른 갈색 여우가 게으른 개 점프

( ok) Sync:      [en] The quick brown fox jumps over the lazy dog
( ok) Sync:      [ru] Быстрая коричневая лиса прыгает через ленивую собаку
( ok) Sync:      [pl] Szybki brązowy lis przeskoczył nad leniwym psem
( ok) Sync:      [de] Der schnelle braune Fuchs springt über den faulen Hund
( ok) Sync:      [ko] 빠른 갈색 여우가 게으른 개 점프

( ok) Async:     [en] The quick brown fox jumps over the lazy dog
(err) Async:     [ru] ??????? ?????????? ???? ??????? ????? ??????? ??????
(err) Async:     [pl] Szybki br?zowy lis przeskoczy? nad leniwym psem
( ok) Async:     [de] Der schnelle braune Fuchs springt über den faulen Hund
(err) Async:     [ko] ?? ?? ??? ??? ? ??

Additional info

Mysql server: mariadb 10.1.17-1;
Table engine: InnoDB;
Python version: 3.5.2;
peewee-async: latest from git.

Unexpected "No task is currently running" exception

When a query is run outside of task, we get No task is currently running error. This is not intended behavior! We don't have to wrap our coroutines into tasks. It's only necessary for transactions. All tests are passing because they are running by wrapping into tasks.

RuntimeError: Event loop is closed (aiomysql)

I tried many ways to fix this but I couldn't. I used concurrent, loop.run_until_complete(asyncio.sleep(1)), etc..

import asyncio
import peewee
import peewee_async

database = peewee_async.MySQLDatabase('test')

class test(peewee.Model):
    name = peewee.CharField()

    class Meta:
        database = database


objects = peewee_async.Manager(database)

database.set_allow_sync(False)

async def func():
    await objects.create(test, name='Name')

    all_objects = await objects.execute(test.select())
    for obj in all_objects:
        print(obj.name)

try:
    loop = asyncio.get_event_loop()
    loop.run_until_complete(func())
finally:
    loop.close()

Also, how to use objects_create() to do multi insert?

Manager._swap_database: NameError: name 'model' is not defined

The following code in Manager._swap_database refers to a local variable model that is not defined above:

            assert False, ("Error, models's database and manager's "
                           "database are different: %s" % model)

It seems not support docker container?

my test envirment is:

Docker
ubuntu 16.10
virtualenv Python3.5.2
sanic web app

My code is:

    async def get_user(self, uid: str) -> UserInfo:
        query = UserInfo.select().where(UserInfo.uid == uid)
        result = await self.execute(query)
        return result[0] if len(result) > 0 else None

and the caller code is:

async def valid_access(request: Request):
    uid = request.cookies.get('uid')
    user = await database.get_user(uid)
    if not user:
        raise InvalidUsage('Invalid user.', status_code=403)
    await database.log_access(uid, request.url, dict(request.headers, **{'ip': request.ip}))

# api_v3 is sanic blueprints
@api_v3.middleware('request')
async def on_request(request: Request):
    url = request.url
    if url.startswith(api_v3.url_prefix):
        await valid_access(request)

if not in docker, this code work fine, if in docker container, error is like this:

TypeError
'ExceptionWrapper' object is not callable

Traceback (most recent call last):
File /env/sanic/lib/python3.5/site-packages/sanic/app.py, line 371, in handle_request
response = await response
File /usr/lib/python3.5/asyncio/coroutines.py, line 105, in __next__
return self.gen.send(None)
File /prj/v3/web/main.py, line 54, in on_request
await valid_access(request)
File /usr/lib/python3.5/asyncio/coroutines.py, line 105, in __next__
return self.gen.send(None)
File /prj/v3/web/main.py, line 43, in valid_access
user = await database.get_user(uid)
File /usr/lib/python3.5/asyncio/coroutines.py, line 105, in __next__
return self.gen.send(None)
File /prj/v3/web/database/database.py, line 143, in get_user
result = await self.execute(query)
File /env/sanic/lib/python3.5/site-packages/peewee_async.py, line 269, in execute
return (yield from execute(query))
File /env/sanic/lib/python3.5/site-packages/peewee_async.py, line 428, in execute
return (yield from coroutine(query))
File /env/sanic/lib/python3.5/site-packages/peewee_async.py, line 569, in select
cursor = yield from _execute_query_async(query)
File /env/sanic/lib/python3.5/site-packages/peewee_async.py, line 1491, in _execute_query_async
return (yield from _run_sql(query.database, *query.sql()))
File /env/sanic/lib/python3.5/site-packages/peewee_async.py, line 1475, in _run_sql
with database.exception_wrapper():
TypeError: 'ExceptionWrapper' object is not callable while handling uri /api/v3/user

Unable to use Proxy database

I'm still running into issues trying to wrap the Manager around a peewee Proxy. I've created a repo that shows my use case here:

https://github.com/dgelvin/peewee-async-test

I am using asynctest which uses a new event loop for each test to ensure that tests are completely independent.

Async ForeignKeys

At the moment if you have model

class Model(peewee.Model):
    other_model = peewee.ForeignKey(OrderModel)

using it like this

model_instance.other_model

will trigger sync query.

I've already have this working so this issue is just to keep contribution in mind ;)

something wrong in my app with error "close cannot be used while an asynchronous query is underway"

Renctly I tried to redeploy my project used with sanic frame ,some errors happened when I do a test under high-concurrency.
info in the log:

close cannot be used while an asynchronous query is underway

and the traceback is below:

Traceback (most recent call last):
  File "/home/mwh/ad_sanic/handlers/oprs.py", line 17, in post
    _parsed_req = await ReqParser(_req_body,'parse',req_ip=(request.headers['Remote-Addr']).split(':')[0]).parser()
  File "/home/mwh/ad_sanic/utils/req_parse.py", line 49, in parser
    _mapped['sim_country_iso'] = await self.geo()
  File "/home/mwh/ad_sanic/utils/req_parse.py", line 75, in geo
    db_country = (await self.db.get(DbIp.select(DbIp.country).where((DbIp.ip_from<=self._req_ip)&(DbIp.ip_to>=self._req_ip)))).country
  File "/home/mwh/py3.5_virtuanlenv/lib/python3.5/site-packages/peewee_async.py", line 169, in get
    result = yield from self.execute(query.limit(1))
  File "/home/mwh/py3.5_virtuanlenv/lib/python3.5/site-packages/peewee_async.py", line 269, in execute
    return (yield from execute(query))
  File "/home/mwh/py3.5_virtuanlenv/lib/python3.5/site-packages/peewee_async.py", line 428, in execute
    return (yield from coroutine(query))
  File "/home/mwh/py3.5_virtuanlenv/lib/python3.5/site-packages/peewee_async.py", line 569, in select
    cursor = yield from _execute_query_async(query)
  File "/home/mwh/py3.5_virtuanlenv/lib/python3.5/site-packages/peewee_async.py", line 1491, in _execute_query_async
    return (yield from _run_sql(query.database, *query.sql()))
  File "/home/mwh/py3.5_virtuanlenv/lib/python3.5/site-packages/peewee_async.py", line 1476, in _run_sql
    cursor = yield from database.cursor_async()
  File "/home/mwh/py3.5_virtuanlenv/lib/python3.5/site-packages/peewee_async.py", line 902, in cursor_async
    return (yield from self._async_conn.cursor(conn=conn))
  File "/home/mwh/py3.5_virtuanlenv/lib/python3.5/site-packages/peewee_async.py", line 1067, in cursor
    conn = yield from self.acquire()
  File "/home/mwh/py3.5_virtuanlenv/lib/python3.5/site-packages/peewee_async.py", line 1036, in acquire
    return (yield from self.pool.acquire())
  File "/home/mwh/py3.5_virtuanlenv/lib/python3.5/site-packages/aiopg/utils.py", line 67, in __iter__
    resp = yield from self._coro
  File "/home/mwh/py3.5_virtuanlenv/lib/python3.5/site-packages/aiopg/pool.py", line 180, in _acquire
    yield from self._cond.wait()
  File "/usr/local/lib/python3.5/asyncio/locks.py", line 326, in wait
    yield from fut
  File "/usr/local/lib/python3.5/asyncio/futures.py", line 385, in __iter__
    yield self  # This tells Task to wait for completion.
  File "uvloop/future.pyx", line 434, in uvloop.loop.BaseTask._fast_wakeup (uvloop/loop.c:112710)
  File "/usr/local/lib/python3.5/asyncio/futures.py", line 266, in result
    raise CancelledError
concurrent.futures._base.CancelledError

what caused this problem?

Help! KeyError: 'A@>' when using ArrayField

Query

        lobbies = await db.execute(Lobby.select().where(
            (Lobby.members.contains(user['steam_id'])) & (Lobby.id != lobby_id))
        )

Model

class Lobby(BaseModel):
    members = ArrayField(CharField, default=list)

Error

2017-03-14 00:10:22,513: ERROR: Traceback (most recent call last):                                                                                 W5pYy9saWIvcHl0aG9uMy42L3NpdGUtcGFja2FnZ  File "/home/qwexvf/.pyenv/versions/sanic/lib/python3.6/site-packages/sanic/app.py", line 391, in handle_request                                  L3B1Zy9hcHAvdXRpbHMucHkiLCBsaW5lIDk2LCBp    response = await response                                                                                                                      2aWV3cy9sb2JieS5weSIsIGxpbmUgNTUsIGluIGF  File "/home/qwexvf/Documents/work/pug/app/utils.py", line 96, in decorated_function                                                              dleHZmLy5weWVudi92ZXJzaW9ucy9zYW5pYy9saW    response = await f(req, *args, **kwargs)                                                                                                       CIvaG9tZS9xd2V4dmYvLnB5ZW52L3ZlcnNpb25zL  File "/home/qwexvf/Documents/work/pug/app/views/lobby.py", line 55, in api_lobby_info                                                            eSkpCiAgRmlsZSAiL2hvbWUvcXdleHZmLy5weWVu    lobbies = await db.execute(Lobby.select().where((Lobby.members.contains())))                                                                   lY3V0ZV9xdWVyeV9hc3luYyhxdWVyeSkKICBGaWx  File "/home/qwexvf/.pyenv/versions/sanic/lib/python3.6/site-packages/peewee_async.py", line 269, in execute                                      J5X2FzeW5jCiAgICByZXR1cm4gKHlpZWxkIGZyb2    return (yield from execute(query))                                                                                                             XMvcGVld2VlLnB5IiwgbGluZSAzMTc0LCBpbiBzc  File "/home/qwexvf/.pyenv/versions/sanic/lib/python3.6/site-packages/peewee_async.py", line 433, in execute                                      dGUtcGFja2FnZXMvcGVld2VlLnB5IiwgbGluZSAx    return (yield from coroutine(query))                                                                                                           hbmljL2xpYi9weXRob24zLjYvc2l0ZS1wYWNrYWd  File "/home/qwexvf/.pyenv/versions/sanic/lib/python3.6/site-packages/peewee_async.py", line 574, in select                                       9xd2V4dmYvLnB5ZW52L3ZlcnNpb25zL3NhbmljL2    cursor = yield from _execute_query_async(query)                                                                                                WxpYXNfbWFwLCBjb252KQogIEZpbGUgIi9ob21lL  File "/home/qwexvf/.pyenv/versions/sanic/lib/python3.6/site-packages/peewee_async.py", line 1518, in _execute_query_async                        bGYuX3BhcnNlX21hcFtub2RlX3R5cGVdKG5vZGUs    return (yield from _run_sql(query.database, *query.sql()))                                                                                     sIGluIF9wYXJzZV9jbGF1c2UKICAgIG5vZGUubm9  File "/home/qwexvf/.pyenv/versions/sanic/lib/python3.6/site-packages/peewee.py", line 3174, in sql                                               5weSIsIGxpbmUgMTg2MywgaW4gcGFyc2Vfbm9kZV    return self.compiler().generate_select(self)                                                                                                   2FuaWMvbGliL3B5dGhvbjMuNi9zaXRlLXBhY2thZ  File "/home/qwexvf/.pyenv/versions/sanic/lib/python3.6/site-packages/peewee.py", line 1999, in generate_select                                   L2hvbWUvcXdleHZmLy5weWVudi92ZXJzaW9ucy9z    return self.build_query(clauses, alias_map)                                                                                                    obm9kZSwgYWxpYXNfbWFwLCBjb252KQogIEZpbGU  File "/home/qwexvf/.pyenv/versions/sanic/lib/python3.6/site-packages/peewee.py", line 1885, in build_query                                       AgIHNxbCA9IHRlbXBsYXRlICUgKGxocywgc2VsZi    return self.parse_node(Clause(*clauses), alias_map)                                                                                            DE2OTQsIGluIGdldF9vcAogICAgcmV0dXJuIHNlb  File "/home/qwexvf/.pyenv/versions/sanic/lib/python3.6/site-packages/peewee.py", line 1838, in parse_node                                        │                                       
    sql, params, unknown = self._parse(node, alias_map, conv)                                                                                      │                                       
  File "/home/qwexvf/.pyenv/versions/sanic/lib/python3.6/site-packages/peewee.py", line 1804, in _parse                                            │                                       
    sql, params = self._parse_map[node_type](node, alias_map, conv)                                                                                │                                       
  File "/home/qwexvf/.pyenv/versions/sanic/lib/python3.6/site-packages/peewee.py", line 1735, in _parse_clause                                     │                                       
    node.nodes, alias_map, conv, node.glue)                                                                                                        │                                       
  File "/home/qwexvf/.pyenv/versions/sanic/lib/python3.6/site-packages/peewee.py", line 1863, in parse_node_list                                   │                                       
    node_sql, node_params = self.parse_node(node, alias_map, conv)                                                                                 │                                       
  File "/home/qwexvf/.pyenv/versions/sanic/lib/python3.6/site-packages/peewee.py", line 1838, in parse_node                                        │                                       
    sql, params, unknown = self._parse(node, alias_map, conv)                                                                                      │                                       
  File "/home/qwexvf/.pyenv/versions/sanic/lib/python3.6/site-packages/peewee.py", line 1804, in _parse                                            │                                       
    sql, params = self._parse_map[node_type](node, alias_map, conv)                                                                                │                                       
  File "/home/qwexvf/.pyenv/versions/sanic/lib/python3.6/site-packages/peewee.py", line 1710, in _parse_expression                                 │                                       
    sql = template % (lhs, self.get_op(node.op), rhs)                                                                                              │                                       
  File "/home/qwexvf/.pyenv/versions/sanic/lib/python3.6/site-packages/peewee.py", line 1694, in get_op                                            │                                       
    return self._op_map[q]                                                                                                                         │                                       
KeyError: 'A@>'

i think the problem is with (Lobby.members.contains(user['steam_id'])
if so how can i deal with arrayfield?

Create data dynamically happened error...

I create a Model below

class Student(peewee.Model):
    score = peewee.IntegerField()
    name = peewee.CharField(max_length=64,  null=True)
    class Meta:
        database = DATABASE
        db_table = 'student'
        schema = 'biz'

when I create a data with the usual method:

_create_sql = {
            'name':'jack',
            'score':75,
        }
 yield from self.db.create(Student,**_create_sql)

It worked fine actually.But when I create the data in a dynamic way like this:

_create_sql = {
            'name':'jack',
            'score':75,
        }
 yield from exec('%s.create(%s,**%s)'%(self.db,Student,_create_sql))

It happened a error:

  File "<string>", line 1
        yield from <peewee_async.Manager object at 0x7f1728e74da0>.create(<class 'models.Student'>,**{'name': 'jack', 'score': 75})
                   ^
    SyntaxError: invalid syntax

How can I solve the problem?

An example about using peewee-async with Tornado

I spent several hours to find how can peewee-async and Tornado work together.
And finally I got this:
https://gist.github.com/anjianshi/0cfb8bee7d7e8a7dd9c6

It seems working. But is it the right way to use them?
May be you can add an example in documents, to help people that not familiar with asyncio or Tornado?

Thank you!

AttributeError: 'Manager' object has no attribute 'commit_select'

What I believe is simple code is returning an opaque error. The call is

async def _get_acct_by_userid(self,user_id):
    ah = await db.execute(AccountHolder.select().where(AccountHolder.userid == user_id))
    if ah.exists():
       return ah.get()
    else:
        return None

but the output I get is AttributeError: 'Manager' object has no attribute 'commit_select'

Am I doing something wrong here?

Transactions are broken with connections pool

There's a problem when using transactions with connections pool, for some reason when transactions connection just becomes unresponsible.

See AsyncPostgresTransactionsTestCase test methods, with single connection everything is fine, but with connection pool test_atomic_failed() stays frozen on line 65 get_object()

create_obj return obj without id (the generated int primary key) or some column using constraints

with

peewee==2.10.2
peewee-async==0.5.7
aiomysql==0.0.9
aiopg==0.13.1

The create_obj has this behavior:
with aiopg: the obj can return generated id.
with aiomysql: the obj can't return the generated id.
all the init code are same.

model like

class User(peewee.Model):
    id = peewee.IntegerField(primary_key=True)
    name = peewee.CharField(max_length=100)
    value = peewee.IntegerField()

    class Meta:
        database = misc.mydb
        db_table = "user"

create operation like:

async def create_album(data):

    obj = await misc.objects.create(
        User, name=data.name, value=data.value,
    )   # mysql is misc.my_object
    return obj

Add example for tornado

Hi, I saw yor comment on stackoverflow.com about supporting tornado framework on asyncio loop. Could you add simple example for working with peewee_async on tornado framework?

MySQL support

asyncio powered library for MySQL is available https://github.com/aio-libs/aiomysql

concurent queries

Peewee recomends to conect() just before request and close() after executing user code.

i.e. from documentation:
http://docs.peewee-orm.com/en/latest/peewee/database.html#creating-a-database-connection-and-tables

database = SqliteDatabase('my_app.db')

def before_request_handler():
    database.connect()

def after_request_handler():
    database.close()

On async app handlers are coroitines and requests can be processed while another one waiting bigger data.

Is there way to get another connection from pool and do requsts on it instead of using shared database connection?

Context manager maybe?

related #53

Adding sugar via patching model classes - a discussion topic

Peewee's design is based on maximum simplicity concept, there is not so much of magic in implementation and no "black" magic at all, as I can see. And peewee provides the exceptionally clean and powerful interface -- it's far cleaner than the one for sqlAlchemy and more powerful that the Django's one -- having probably the best balance between simplicity and power.

The design goal for the peewee-async project is to play nice with peewee core conforming its approach and add extra facilities for async programming. That is why the current implementation does not apply any real black magic as well. Yes we rely on some internal logic, but NO, we don't perform any kind of patching!

I like this approach, but probably it would be acceptable to add some more sugar so we could get less verbose async calls.

For now, we have a high-level API via the Manager class and can write code like that:

# `objects` is the `Manager` class instance
user = await objects.get(User, id=user_id)
posts = await objects.execute(Post.select().where(
    Post.is_published == True,
    Post.author == user))

Looks like OK and actually, we can implement nicer interface at the application level, for example:

class UsersManager:
   # ...

   async def get(cls, user_id):
        return (await cls.objects.get(User, id=user_id))

and so on.

So I don't think it's crucial to add more interface methods to the peewee-async, but I'm considering to add some sugar via Manager.extend() method: https://github.com/05bit/peewee-async/blob/extend/peewee_async.py#L115 It will allow to leave the Manager under the hood in the most cases and write like that:

user = await User.get_async(id=user_id)
posts = await Post.execute_async(Post.select().where(
    Post.is_published == True,
    Post.author == user))

-- well, not ideal but simpler as we don't need to pass around objects and keep in mind that we need it every time.

I hope to hear some critics and suggestions on that, so we could refine the idea or left it out.

Bulk insert

I would like make few bulk insert at same time. Can I do that with peewee-async?

Swap database condition bug

Traceback:

  File "/Users/user/.virtualenvs/peewee/lib/python3.5/site-packages/peewee_async.py", line 292, in scalar
    query = self._swap_database(query)
  File "/Users/user/.virtualenvs/peewee/lib/python3.5/site-packages/peewee_async.py", line 367, in _swap_database
    if can_swap:
UnboundLocalError: local variable 'can_swap' referenced before assignment

Code:

def _swap_database(self, query):
        """Swap database for query if swappable. Return **new query**
        with swapped database.
        Check manager database and model database match. If model
        database is `auto`, manager's one will be used.
        If query database can't be swapped and differs from manager's
        database, it's **WRONG AND DANGEROUS**, so assertion is raised.
        """
        if query.database == self.database:
            return query
        elif self._subclassed(peewee.PostgresqlDatabase,
                              query.database,
                              self.database):
            can_swap = True
        elif self._subclassed(peewee.MySQLDatabase,
                              query.database,
                              self.database):
            can_swap = True
        """
        else???
        """
        if can_swap:

Starting transaction before connecting raise `AttributeError` exception about `_task_data`

We need to check if _task_data is present and not None.

'AsyncQueryWrapper' object does not support indexing (since 0.5.8)

Since upgrading to 0.5.8 (and as well 0.5.9) I get the following error:

ERROR    [test2][pyplanet.core.instance] 'AsyncQueryWrapper' object does not support indexing
Traceback (most recent call last):
  File "/home/tom/IdeaProjects/maniaplanet/pyplanet-core/pyplanet/core/instance.py", line 121, in start
    self.loop.run_until_complete(self._start())
  File "/home/tom/.pyenv/versions/3.6.3/lib/python3.6/asyncio/base_events.py", line 467, in run_until_complete
    return future.result()
  File "/home/tom/IdeaProjects/maniaplanet/pyplanet-core/pyplanet/core/instance.py", line 182, in _start
    await self.apps.start()
  File "/home/tom/IdeaProjects/maniaplanet/pyplanet-core/pyplanet/apps/apps.py", line 187, in start
    await app.on_start()
  File "/home/tom/IdeaProjects/maniaplanet/pyplanet-core/pyplanet/apps/contrib/local_records/__init__.py", line 61, in on_start
    await self.load_map_locals()
  File "/home/tom/IdeaProjects/maniaplanet/pyplanet-core/pyplanet/apps/contrib/local_records/__init__.py", line 65, in load_map_locals
    map.local = await self.get_map_record(map)
  File "/home/tom/IdeaProjects/maniaplanet/pyplanet-core/pyplanet/apps/contrib/local_records/__init__.py", line 77, in get_map_record
    'first_record': record_list[0] if len(record_list) > 0 else None
TypeError: 'AsyncQueryWrapper' object does not support indexing

Project can be found here: https://github.com/PyPlanet/PyPlanet

Is this intentional or a bug?

Thanks :-)

Some queries always result in a stack overflow

Some queries always result in a stack overflow. The below code works using playhouse.postgres_ext.PostgresqlExtDatabase but doesn't if I switch to peewee_async.PostgresqlDatabase.

from peewee import Model, ForeignKeyField, CharField, BooleanField, DoubleField, IntegerField, \
    SmallIntegerField, CompositeKey, PrimaryKeyField, FloatField
from peewee_async import Manager, PostgresqlDatabase
from playhouse.postgres_ext import JSONField, PostgresqlExtDatabase

# database = PostgresqlDatabase(
#     database=DATABASE['name'],
#     host=DATABASE['host'],
#     port=DATABASE['port'],
#     user=DATABASE['user'],
#     password=DATABASE['password'],
#     # register_hstore=False,
# )
# database.set_allow_sync(True)

database = PostgresqlExtDatabase(
    database=DATABASE['name'],
    host=DATABASE['host'],
    port=DATABASE['port'],
    user=DATABASE['user'],
    password=DATABASE['password'],
    register_hstore=False,
)


class AsyncBaseModel(Model):
    class Meta:
        database = database


class AlarmHistory(AsyncBaseModel):
    id = PrimaryKeyField()
    alarm_type = CharField(max_length=MAX_ID_LABEL_LENGTH, null=False)
    activation = ForeignKeyField(FeatureHistory, null=False, related_name='activations')
    normalization = ForeignKeyField(FeatureHistory, default=None, null=True,  
                                    related_name='normalizations')
    ack_username = CharField(max_length=MAX_USERNAME_LENGTH, null=True)
    ack_notes = CharField(default=None, null=True)
    ack_timestamp = TimestampField(null=True)


class FeatureHistory(AsyncBaseModel):
    id = PrimaryKeyField()
    timestamp = TimestampField(null=False, default=time)
    identifier = CharField(max_length=MAX_IDENTIFIER_LENGTH, null=False)
    message_format_short = CharField(null=True)
    message_format_long = CharField(null=True)
    user_data = JSONField(null=True)
    system_data = JSONField(null=True)
    filter = CharField(max_length=MAX_ID_LABEL_LENGTH, null=False)
    icon_state = CharField(max_length=MAX_ID_LABEL_LENGTH, null=False)
    tile_set = CharField(max_length=MAX_ID_LABEL_LENGTH, null=True)
    longitude = DoubleField(null=True)
    lattitude = DoubleField(null=True)
    zone_name = CharField(max_length=MAX_ZONE_NAME_LENGTH, null=True)
    battery_percentage = IntegerField(default=None, null=True)

This method will cause a stack overflow if I use peewee_async.PostgresqlDatabase:

def get_alarm_history(from_ts: float, to_ts: float) -> Iterable[dict]:
    q = AlarmHistory.select(
        AlarmHistory.activation,
        AlarmHistory.ack_username,
        AlarmHistory.ack_notes,
        AlarmHistory.ack_timestamp,
        FeatureHistory.identifier,
        FeatureHistory.timestamp,
        FeatureHistory.user_data['owner'].alias('owner'),
        FeatureHistory.user_data['tag#'].alias('tag'),
        FeatureHistory.system_data['zone_name'].alias('zone_name'),
    ).join(
        FeatureHistory, on=(AlarmHistory.activation == FeatureHistory.id)
    ).where(
        FeatureHistory.icon_state == IconState.values.REGULAR_ACTIVE,
        AlarmHistory.alarm_type == AlarmType.values.FEATURE_STATE_ALARM,
        FeatureHistory.timestamp >= from_ts,
        FeatureHistory.timestamp <= to_ts,
    ).order_by(FeatureHistory.timestamp)
    q = tuple(q.dicts())
    return q

update with returning

Currenlty peewee-async always returns affected rows count, ignoring returning:

query = models.Node.update(path=new_path).where(path=old_path).returning()
res = await objects.execute(query)
# res is instance of `int`

We can repeat peewee UpdateQuery.execute() behaviour in peewee_async.update function.

What do you think? PR will be apprecated?