tarantool / queue-python Goto Github PK

View Code? Open in Web Editor NEW

14.0 42.0 11.0 64 KB

Python Bindings for Tarantool Queue (https://github.com/tarantool/queue/)

License: MIT License

Makefile 0.26% Python 99.74%

queue-python's Introduction

queue-python

Python Bindings for Tarantool Queue.

Library depends on:

msgpack-python
tarantool

Basic usage can be found in tests. Description on every command is in source code.

Big thanks to Dmitriy Shveenkov and Alexandr (FZambia) Emelin.

For install of latest "stable" version type:

# using pip
$ sudo pip install tarantool-queue
# or using easy_install
$ sudo easy_install tarantool-queue
# or using python
$ wget http://bit.ly/tarantool_queue -O tarantool_queue.tar.gz
$ tar xzf tarantool_queue.tar.gz
$ cd tarantool-queue-{version}
$ sudo python setup.py install

For install bleeding edge type:

$ sudo pip install git+https://github.com/tarantool/queue-python.git

For configuring Queue in Tarantool read manual Here.

Then just import it, create Queue, create Tube, put and take some elements:

>>> from tarantool_queue import Queue
>>> queue = Queue("localhost", 33013, 0)
>>> tube = queue.tube("name_of_tube")
>>> tube.put([1, 2, 3])
Not taken task instance
>>> task = tube.take()
>>> task.data # take task and read data from it
[1, 2, 3]
>>> task.ack() # move this task into state DONE
True

That's all, folks!

queue-python's People

Stargazers

Watchers

Forkers

pombredanne dreadatour bp72 vladimirbright abrade malikahaidary elbroom digger3d halub3

queue-python's Issues

ZeroTupleException при вызове tube.statistics()

Из нашей sentry:

Типы:   ZeroTupleException
Значение:   stats:                             error when parsing respons
Расположение:   tarantool_queue/tarantool_queue.py in statistics , line 608

ER_INVALID_MSGPACK: Invalid MsgPack - packet header

Hi,

Error occurred during execution of the following code from the README file, using following environment:
Please check, any help would be appreciated.

Error:
iproto xrow.c:141 E> ER_INVALID_MSGPACK: Invalid MsgPack - packet header

Code
from tarantool_queue import Queue
queue = Queue("localhost", 3301, 0)
tube = queue.tube("q1")
tube.put([1, 2, 3])
task = tube.take()
task.ack()

Environment

Ubuntu 18.04.5 LTS (Bionic Beaver)
Tarantool 2.6.2-0-g34d504d7d
Python 2.7.17
Queue-python, v 0.1.4

Thanks in advance.

Update for tarantool 1.6

Please update for use latest python driver which works with tarantool 1.6.

task.meta возвращает время ttl и ttr в микросекундах

При этом установливаются они с точностью до секунды.
Логично возвращать в том же виде.

Install error

tarantool-queue package requires msgpack to be installed.

(tnt-queue-msgpack)[e.iskandarov@win91 ~]$ pip install tarantool-queue
Downloading/unpacking tarantool-queue
  Using download cache from /home/e.iskandarov/.cache/pip/http%3A%2F%2Fpypi.mail.ru%2Froot%2Fpypi%2Ff%2Fhttps%2Fpypi.python.org%2Fpackages%2Fsource%2Ft%2Ftarantool-queue%2Ftarantool-queue-0.1.1.zip
  Running setup.py egg_info for package tarantool-queue
    Traceback (most recent call last):
      File "<string>", line 16, in <module>
      File "/home/e.iskandarov/.virtualenvs/tnt-queue-msgpack/build/tarantool-queue/setup.py", line 6, in <module>
        from tarantool_queue import __version__
      File "tarantool_queue/__init__.py", line 3, in <module>
        from tarantool_queue import Task, Tube, Queue
      File "tarantool_queue/tarantool_queue.py", line 4, in <module>
        import msgpack
    ImportError: No module named msgpack
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 16, in <module>

  File "/home/e.iskandarov/.virtualenvs/tnt-queue-msgpack/build/tarantool-queue/setup.py", line 6, in <module>

    from tarantool_queue import __version__

  File "tarantool_queue/__init__.py", line 3, in <module>

    from tarantool_queue import Task, Tube, Queue

  File "tarantool_queue/tarantool_queue.py", line 4, in <module>

    import msgpack

ImportError: No module named msgpack

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /home/e.iskandarov/.virtualenvs/tnt-queue-msgpack/build/tarantool-queue
Storing complete log in /home/e.iskandarov/.pip/pip.log

Serialization should be pluggable

msgpack serialization dependance can be declared with extras_require

Add tarantool-queue-python to Tarantool buildbot

Write RPM specs/DEB rules and push they to the repository. Please use package name that confirms official distributive naming conventions (e.g. tarantool-python-queue or python-tarantool-queue)
Add this project to build bot (http://build.tarantool.org/)
Check that module build and can be installed from tarantool.org repository

Add Travis configuration

Add a Travis configuration file:
- Module must be build without errors
- Script should install tarantool and tarantool-python from our repos
- Script also should run tests
Update README to add Travis badge

https://github.com/tarantool/tarantool/blob/master/.travis.yml and https://github.com/tarantool/http/blob/master/.travis.yml can be used as examples

Исключение при попытке записать большой объем данных в очередь

Пример кода для воспроизведения проблемы

from tarantool_queue import Queue

queue = Queue("localhost", 33013, 0)
tube = queue.tube("name_of_tube")
tube.put({'x': 'x' * 10000000})

Получаем ошибку:

Traceback (most recent call last):
  File "ex.py", line 6, in <module>
    tube.put({'x': 'x' * 10000000})
  File "/usr/local/lib/python2.7/site-packages/tarantool_queue/tarantool_queue.py", line 260, in put
    self.serialize(data))
  File "/usr/local/lib/python2.7/site-packages/tarantool/connection.py", line 283, in call
    default_type=default_type)
  File "/usr/local/lib/python2.7/site-packages/tarantool/connection.py", line 236, in _send_request
    request, space_name, field_defs, default_type)
  File "/usr/local/lib/python2.7/site-packages/tarantool/connection.py", line 175, in _send_request_wo_reconnect
    self, header, body, space_name, field_defs, default_type)
  File "/usr/local/lib/python2.7/site-packages/tarantool/response.py", line 138, in __init__
    self._unpack_body(body)
  File "/usr/local/lib/python2.7/site-packages/tarantool/response.py", line 237, in _unpack_body
    "<%ds" % (tuple_size), buff, offset + 4)[0]
struct.error: unpack_from requires a buffer of at least 1948279913 bytes

Хотелось бы, чтобы записи подобного размера поддерживались. Была реальная проблема на бою.

Версии библиотек:

tarantool==0.3.6
tarantool-queue==0.1.4

tarantool / queue-python Goto Github PK

queue-python's Introduction

queue-python

See Also

queue-python's People

Stargazers

Watchers

Forkers

queue-python's Issues

ZeroTupleException при вызове tube.statistics()

ER_INVALID_MSGPACK: Invalid MsgPack - packet header

Update for tarantool 1.6

task.meta возвращает время ttl и ttr в микросекундах

Install error

Serialization should be pluggable

Add tarantool-queue-python to Tarantool buildbot

Add Travis configuration

Исключение при попытке записать большой объем данных в очередь

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent