couchbaselabs / couchbase-python-cffi Goto Github PK

Implementation of the Python Couchbase API using cffi

Python 92.65% C 4.08% C++ 3.28%

couchbase-python-cffi's Introduction

Couchbase FFI

https://travis-ci.org/couchbaselabs/couchbase-python-cffi.png

This provides the FFI backend for the couchbase module. This project is mainly of use to those who wish to use the couchbase module with pypy. Normal CPython extensions do not work very well on pypy, and thus the recommended way of interfacing via pypy in a C library is to use the cffi module.

See the _1.x_ branch for the version of this module that uses the 1.x client and API (though ffi module in that branch is no longer maintained)

Using

You will need the cffi Python module (the newer, the better), the libcouchbase development files, and the actual couchbase module (see its instructions for how to build it).

Each version of the CFFI library has a strict dependency on an exact version of the couchbase Python client, due to the tight coupling of internals.

The current version depends on:

libcouchbase > 2.4.7
couchbase = 2.0.0-beta2 (or 2.0.0b2)

Once you've built all that, you must load this module first before any others; thus:

import couchbase_ffi
from couchbase.bucket import Bucket
# ...

This module will inject itself as the couchbase._libcouchbase module, allowing you to use anything that depends on the C extension normally.

This module does not monkey-patch the couchbase Bucket class, but rather the underlying C internals; therefore the ability of a drop-in increases.

The first time you use the module, the couchbase_ffi._cinit module will attempt to generate the appropriate stubs for the library. Regeneration may also take place if you have upgraded your libcouchbase installation.

Because regenerating the header might involve some hacks in dynamically patching the header file to make it acceptable to pycparser, it may fail at times.

Ennvironment variables affecting how the headers are built:

PYCBC_CFFI_REGENERATE forces the header to be rebuilt
PYCBC_CFFI_PREFIX sets the installation prefix for libcouchbase

Implemented Features

Everything the normal C extension supports. This has been tested using the normal couchbase.tests module and the twisted tests.

I may have missed something here and there, but I can't think of anything that isn't supported.

TODO

Better tests.

Performance

Here are some examples using the bench.py script in the main Couchbase Python distribution. These were all executed against a locally running memcached like so

Example:

./examples/txbench.py -U memcached://localhost -t 1 -C 1
#....

Here are the results:


Implementation	Threads	Ops/Sec
FFI (PyPy)	0	9440
FFI (CPython)	0	4400
Ext (CPython)	0	11200
FFI (PyPy)	4	9169
FFI (CPython)	4	4190
Ext (CPython)	4	21685

For twisted:


Implementation	C/T	Ops/Sec
FFI (PyPy)	1/1	7583
FFI (CPython)	1/1	3502
Ext (CPython)	1/1	9895
FFI (PyPy)	4/4	26000
FFI (CPython)	4/4	7500
Ext (CPython)	4/4	33600
FFI (PyPy)	10/10	36277
FFI (CPython)	10/10	9273
Ext (CPython)	10/10	57072

The T and C values show how many total clients were spawned, and how many independent logical sequences (threads) of operations were being performed.

couchbase-python-cffi's People

Contributors

Stargazers

Watchers

Forkers

mnunberg handloomweaver ardydedase yonglehou chippiewill what-studio pauldx guillaumecrowdcomfort

couchbase-python-cffi's Issues

AssertionError: backend.version == version[:3])

Assigning the issue here as suggested by Burhan Khalid on Stackoverflow: http://stackoverflow.com/questions/32063227/python-couchbase-cffi-assertionerror-backend-version-version-3

Full error text:

No module named couchbase._libcouchbase
Traceback (most recent call last):
  File "app_main.py", line 75, in run_toplevel
  File "runtests.py", line 12, in <module>
    mod = __import__(t, globals(), locals(), ['suite'])
  File "/home/travis/build/ardydedase/pycouchbase/tests/test_pycouchbase.py", line 15, in <module>
    from pycouchbase.utils import *
  File "/home/travis/build/ardydedase/pycouchbase/pycouchbase/__init__.py", line 11, in <module>
    import couchbase_ffi as couchase
  File "/home/travis/virtualenv/pypy-2.5.0/site-packages/couchbase_ffi-0.2.0.0-py2.7-linux-x86_64.egg/couchbase_ffi/__init__.py", line 50, in <module>
    import couchbase_ffi._libcouchbase as _libcouchbase
  File "/home/travis/virtualenv/pypy-2.5.0/site-packages/couchbase_ffi-0.2.0.0-py2.7-linux-x86_64.egg/couchbase_ffi/_libcouchbase.py", line 3, in <module>
    from couchbase_ffi._cinit import get_handle
  File "/home/travis/virtualenv/pypy-2.5.0/site-packages/couchbase_ffi-0.2.0.0-py2.7-linux-x86_64.egg/couchbase_ffi/_cinit.py", line 13, in <module>
    ffi = FFI()
  File "/home/travis/virtualenv/pypy-2.5.0/site-packages/cffi/api.py", line 59, in __init__
    backend.__version__ == __version__[:3])
AssertionError

This only happens when I try to import couchbase_ffi in the PyPy build on Travis: https://travis-ci.org/ardydedase/pycouchbase/jobs/76043402#L1663

Details:

I have this in my requirements.txt for couchbase-python-cffi:

cffi==0.8.0
couchbase==2.0

Looks like an incompatibility issue with the packages. But not sure where it's coming from.

N1QL Queries do not work with the CFFI

File "<file>", line 64, in <function>
    for row in bkt.n1ql_query(query):
  File "/usr/local/Cellar/pypy/2.6.1/libexec/site-packages/couchbase/n1ql.py", line 372, in __iter__
    self._start()
  File "/usr/local/Cellar/pypy/2.6.1/libexec/site-packages/couchbase/n1ql.py", line 312, in _start
    self._mres = self._parent._n1ql_query(self._params.encoded,
AttributeError: 'Bucket' object has no attribute '_n1ql_query'

Cannot install due to AttributeError: LCB_SUBDOC_MULTI_FAILURE

I tried to install couchbase-2.2.2 and [email protected] in my PyPy environment. But the installation was failed due to an error:

$ pip install couchbase
Collecting couchbase
  Using cached couchbase-2.2.2.tar.gz
Installing collected packages: couchbase
  Running setup.py install for couchbase ... done
Successfully installed couchbase-2.2.2
You are using pip version 8.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

$ cd ~/couchbase-python-cffi
$ python setup.py install
Traceback (most recent call last):
  File "setup.py", line 2, in <module>
    import couchbase_ffi._cinit
  File "/home/sub/works/couchbase-python-cffi/couchbase_ffi/__init__.py", line 51, in <module>
    _libcouchbase._stage2_bootstrap()
  File "/home/sub/works/couchbase-python-cffi/couchbase_ffi/_libcouchbase.py", line 78, in _stage2_bootstrap
    from couchbase_ffi.bucket import Bucket
  File "/home/sub/works/couchbase-python-cffi/couchbase_ffi/bucket.py", line 18, in <module>
    import couchbase_ffi.executors as executors
  File "/home/sub/works/couchbase-python-cffi/couchbase_ffi/executors.py", line 4, in <module>
    from couchbase.exceptions import ValueFormatError, ArgumentError, CouchbaseError
  File "/home/sub/env-pypy/site-packages/couchbase/exceptions.py", line 507, in <module>
    class SubdocMultipleErrors(CouchbaseError):
  File "/home/sub/env-pypy/site-packages/couchbase/exceptions.py", line 509, in SubdocMultipleErrors
    CODE = C.LCB_SUBDOC_MULTI_FAILURE
  File "/home/sub/works/couchbase-python-cffi/couchbase_ffi/__init__.py", line 46, in __getattr__
    raise AttributeError(item)
AttributeError: LCB_SUBDOC_MULTI_FAILURE

My PyPy environment is:

Python 2.7.10 (5.0.1+dfsg-4, Apr 08 2016, 00:22:14)
[PyPy 5.0.1 with GCC 5.3.1 20160407]

How do I get this working?

Why not Twisted?

Why Twisted will not be supported by this module?
Will installing this module break Twisted?

PyPy supports Twisted.

couchbaselabs / couchbase-python-cffi Goto Github PK

couchbase-python-cffi's Introduction

Couchbase FFI

Using

Implemented Features

TODO

Performance

couchbase-python-cffi's People

Contributors

Stargazers

Watchers

Forkers

couchbase-python-cffi's Issues

AssertionError: backend.version == version[:3])

N1QL Queries do not work with the CFFI

Cannot install due to AttributeError: LCB_SUBDOC_MULTI_FAILURE

How do I get this working?

Why not Twisted?

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent