aerospike / aerospike-client-python Goto Github PK
View Code? Open in Web Editor NEWAerospike Python Client
License: Apache License 2.0
Aerospike Python Client
License: Apache License 2.0
When I try the following
client.put(key, {'a': {u'aa': u'11'}, 'b': {u'bb': u'22'}})
print(client.get(key)[2])
i get
{'a': {u'22': u'22'}, 'b': {u'22': u'22'}}
I don't have any problem storing the string version, but if the keys or values are unicode it comes back with unusual results. Saving the maps separately with 2 put calls also comes back with the wrong value, although it's often different than the one I posted above.
I'm having the issue in python 2.7.8, osx mavericks, aerospike client 1.0.31
Any reason why Ubuntu 14.04 is supported and Linux Mint 17(based on ubuntu 14.04 ) is not ?
It seems the test suite has a large amount of errors when run against the community edition, with the majority of the failure cases having the following exception:
E Exception: (51L, 'AEROSPIKE_SECURITY_NOT_SUPPORTED', 'src/main/aerospike/as_admin.c', 214)
Looking over the test cases, there doesn't appear to be any way to skip them. Perhaps add an option in config.conf
that will enable some skipif
directives? Otherwise development will be limited to people with access to the enterprise version.
Hi,
The below file not found in te repository. Please check.
https://github.com/aerospike/aerospike-client-python/blob/master/doc/increment.md
Thanks,
Dhanasekaran
When UDF executed at the server side record object won't be found if key passed as a unicode string
import aerospike
client = aerospike.client({'hosts': [('localhost', 3000)], 'policies': {'timeout': 1000}})
client.connect()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
Exception: (-1L, 'Failed to seed cluster', 'src/main/aerospike/as_cluster.c', 260)
Whereas, immediately calling connect()
works,
import aerospike
client = aerospike.client({'hosts': [('localhost', 3000)], 'policies': {'timeout': 1000}}).connect()
Thanks for the awesome work! I'm running into a problem when storing unicode values (the bins are strings). Values from one bin might show up in other bins. I'll try and figure out a dataset that consistently replicates the bug, but I CAN tell you it works great in .12 and .13, but starts breaking in .14
The first time I saw it, I had a list of unique values, which when stored became a list of duplicates, hope that helps.
We'd like to use aerospike at a high rate of inserting with our asynchronous code but currently it doesn't work with gevent well.
I have a set with 4 bins - key, email, reg_user, invite_code.
key - type string
email - type string
reg_user - type list
invite_code - type string.
When I insert the values for the first time the data is inserted correctly. But when I update only the reg_user
bin by adding values to the list, the email field gets updated with the key of the record.
Here is the code that inserts/updates the records:
"""
self.objects is a dict type which contains the above mentioned bins.
"""
def save(self, pk):
if 'key' not in self.objects:
self.objects['key'] = str(pk)
record = (self.namespace, self.set, self.objects['key'])
self.client.put(record, self.objects)
This problem occurs in a machine where Aerospike is running on a non-SSD machine with 64 bit Ubuntu and I am using the python client. Python version is 2.7.
This problem does not occur on a SSD machine.
Is there an explanation for this?
Hello, I'm installing Aerospike community 3.5 on Ubuntu 12.04. I then edit /etc/aerospike/aerospike.conf
to add a namespace configuration, restart the server, then run aql
(or aql localhost
) and I'm getting the following error:
2015-06-04 16:04:18 WARN AEROSPIKE_ERR_CLIENT Socket write error: 111
Error -1: Failed to seed cluster
I also get this when reverting aerospike.conf back to its original state and restarting Aerospike.
Any ideas? Thank you.
I have written a toy web service in Python that returns the contents of an Aerospike set on response to a certain kind of query. In my test arrangement, the Aerospike cluster just has one node, and the node is down when the query arrives to the web service daemon.
Here's how the connection object is initialized:
asclient = aerospike.client({
'hosts': [ ('127.0.0.1', 3000) ]
}).connect()
Here's the code that attempts to scan the Aerospike set:
results = []
def cb(result):
print "append mac " + result[2]['mac']
results.append(result[2]['mac'])
print "d"
try:
q = asclient.scan('bar','routers')
q.select(['mac'])
q.foreach(cb)
except:
flask.abort(500)
response = {'macs': results}
return flask.jsonify(response)
When the Aerospike cluster is down, this causes a segfault within the 'q.foreach(cb)' line:
Aug 11 20:07:29 {snip} kernel: python[5623]: segfault at c ip 00007fd18367a4d8 sp 00007fd182a4f220 error 4 in aerospike.so[7fd18365b000+56000]
This is probably an attempt to dereference a null pointer from within the Aerospike client C code:
(gdb) where
#0 process_node_response (v=0x0, err=0x7fd182a4f810)
at src/main/aerospike/aerospike_scan.c:104
#1 0x00007fd18367a9b9 in aerospike_scan_generic (as=0x21b2900,
err=0x7fd182a4f810, policy=<value optimized out>, node=0x0,
scan=0x7fd174014798, callback=0x7fd183674e88 <each_result>,
udata=0x7fd182a4f3e0) at src/main/aerospike/aerospike_scan.c:241
#2 0x00007fd18367aa56 in aerospike_scan_foreach (as=0x21b2900,
err=0x7fd182a4f810, policy=<value optimized out>, scan=0x7fd174014798,
callback=0x7fd183674e88 <each_result>, udata=0x7fd182a4f3e0)
at src/main/aerospike/aerospike_scan.c:490
#3 0x00007fd183674e17 in AerospikeScan_Foreach (self=0x7fd174014780,
args=<value optimized out>, kwds=<value optimized out>)
at src/main/scan/foreach.c:112
#4 0x00000038b8ed59e4 in PyEval_EvalFrameEx ()
from /usr/lib64/libpython2.6.so.1.0
#5 0x00000038b8ed7657 in PyEval_EvalCodeEx ()
from /usr/lib64/libpython2.6.so.1.0
#6 0x00000038b8e6adad in ?? () from /usr/lib64/libpython2.6.so.1.0
#7 0x00000038b8e43c63 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0
#8 0x00000038b8ed4470 in PyEval_EvalFrameEx ()
from /usr/lib64/libpython2.6.so.1.0
#9 0x00000038b8ed6b8f in PyEval_EvalFrameEx ()
from /usr/lib64/libpython2.6.so.1.0
#10 0x00000038b8ed6b8f in PyEval_EvalFrameEx ()
from /usr/lib64/libpython2.6.so.1.0
#11 0x00000038b8ed6b8f in PyEval_EvalFrameEx ()
from /usr/lib64/libpython2.6.so.1.0
#12 0x00000038b8ed7657 in PyEval_EvalCodeEx ()
from /usr/lib64/libpython2.6.so.1.0
{snip}
(gdb) print err
$4 = (as_error *) 0x7fd182a4f810
(gdb) print err->code
$5 = AEROSPIKE_OK
(gdb)
This points us here:
static as_status process_node_response(cf_vector *v, as_error *err)
{
as_status rc = AEROSPIKE_OK;
// This returns a vector of return values, the size of which is the size of the cluster
int sz = cf_vector_size(v);
I could not find the definition of cf_vector_size(), but it appears to assume that its argument is non-null.
I'm filing this against the Python wrappers rather than the base C client implementation since it seems likely that the Python implementation is misusing the C APIs.
I am getting this error when I try to install the python client via
sudo pip install aerospike
http://www.aerospike.com/docs/client/c/usage/kvs/multiops.html
The python client not support the usage such as Increment and Read
This module support async ?
If the operations list for operate() contains two or more write operations on the same bin and exception with status code 4 "AEROSPIKE_ERR_REQUEST_INVALID" is thrown.
For example combining an APPEND and PREPEND, or WRITE and APPEND, etc on the same bin:
from __future__ import print_function
import aerospike
config = { 'hosts': [ ('192.168.119.3',3000)]}
client = aerospike.client(config).connect()
(key,meta,bins) = client.get(('test','test','key1'))
print (key, meta, bins)
print('------------------------')
ops = [
{
"op" : aerospike.OPERATOR_WRITE,
"bin": "foo",
"val": "wooo"
},
{
"op" : aerospike.OPERATOR_PREPEND,
"bin": "foo",
"val": ":start:"
},
{
"op" : aerospike.OPERATOR_APPEND,
"bin": "foo",
"val": ":end:"
},
{
"op" : aerospike.OPERATOR_READ,
"bin": "fooz"
}
]
(key, meta, bins) = client.operate(key, ops)
print (key, meta, bins)
client.close()
Throws
Traceback (most recent call last):
File "multiops.py", line 30, in <module>
(key, meta, bins) = client.operate(key, list)
Exception: (4L, 'AEROSPIKE_ERR_REQUEST_INVALID', 'src/main/aerospike/as_command.c', 916)
The following script hangs on the udf_put() when talking to a 2-node Aerospike cluster:
from __future__ import print_function
import aerospike
import os
import sys
try:
client = aerospike.client({'hosts':[('192.168.119.3', 3000)]}).connect()
policy = {'timeout': 5000}
lua_module = os.path.join(os.path.dirname(os.path.realpath(__file__)), "simple_udf.lua") #same folder
udf_type = aerospike.UDF_TYPE_LUA # equals to "0", which is for "Lua"
client.udf_put(lua_module, udf_type, policy) # Exception is thrown here
client.close()
except Exception as e:
print("error: {0}".format(e), file=sys.stderr)
Same behavior is seen on CentOS 7 and OS X 10.10.
Using strace shows the following block repeating:
socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 5
fcntl(5, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0
setsockopt(5, SOL_TCP, TCP_NODELAY, [1], 4) = 0
connect(5, {sa_family=AF_INET, sin_port=htons(3000), sin_addr=inet_addr("192.168.119.3")}, 16) = -1 EINPROGRESS (Operation now in progress)
fcntl(5, F_GETFL) = 0x802 (flags O_RDWR|O_NONBLOCK)
select(6, NULL, [5], NULL, {5, 0}) = 1 (out [5], left {4, 999997})
write(5, "\2\1\0\0\0\0\0\tudf-list\n", 17) = 17
fcntl(5, F_GETFL) = 0x802 (flags O_RDWR|O_NONBLOCK)
select(6, [5], NULL, NULL, {4, 998000}) = 1 (in [5], left {4, 997494})
read(5, "\2\1\0\0\0\0\3{", 8) = 8
fcntl(5, F_GETFL) = 0x802 (flags O_RDWR|O_NONBLOCK)
select(6, [5], NULL, NULL, {4, 998000}) = 1 (in [5], left {4, 997998})
read(5, "udf-list\tfilename=test_transform"..., 891) = 891
shutdown(5, SHUT_RDWR) = 0
close(5) = 0
nanosleep({2, 0}, NULL) = 0
This is similar to issue #6, and only occurs when the values are more than one character long.
client.put(key, {'test1': unicode('aa')})
print(client.get(key)[2]['test1']) #on my machine, I get u'aa'
client.put(key, {'test2': [unicode('bb')]})
print(client.get(key)[2]['test1']) #on my machine, I get u'aa'
client.put(key, {'test1': unicode('aa'), 'test2': [unicode('bb')]})
print(client.get(key)[2]['test1']) #on my machine, I get u'bb'
I notice that it only happens when I send both values at once, so maybe it's something with the messagepack encoding?
This functionality does not seem to supported as of now. The only way that I see to query is
client.get(key).
Thus, opening an issue for it.
Thanks.
Given a key tuple in the form of (Namespace, Set, None, Digest), pyobject_to_key() would fail. From what I read so far, it seems src/main/conversions.c:492 should be "if ( py_key && py_key != Py_None )" instead.
"pip install aerospike" fails when compiling src/main/aerospike.c on the following error:
...
In file included from ./aerospike-client-c/include/citrusleaf/alloc.h:20:
./aerospike-client-c/include/citrusleaf/cf_atomic.h:232:54: error: invalid output size for constraint '=a'
asm volatile ("lock; cmpxchgq %1,%2" : "=a"(p) : "q"(x), "m"(*(a)), "0"(b) : "memory");
...
I've installed the command line tools, openssl and Lua 5.1
complete dump:
$ pip install aerospike
Collecting aerospike
Using cached aerospike-1.0.43.tar.gz
Installing collected packages: aerospike
Running setup.py install for aerospike
Complete output from command /usr/bin/python -c "import setuptools, tokenize;file='/private/var/folders/l8/41zyp9sx0hl4d423hjxn9wdm0000gn/T/pip-build-MMkDhv/aerospike/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /var/folders/l8/41zyp9sx0hl4d423hjxn9wdm0000gn/T/pip-1mNNbR-record/install-record.txt --single-version-externally-managed --compile:
info: downloading 'http://www.aerospike.com/download/client/c/3.1.11/artifact/mac' to '/private/var/folders/l8/41zyp9sx0hl4d423hjxn9wdm0000gn/T/pip-build-MMkDhv/aerospike/aerospike-client-c/package/aerospike-client-c.tgz'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 346 100 346 0 0 160 0 0:00:02 0:00:02 --:--:-- 160
0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0
100 16.4M 100 16.4M 0 0 1297k 0 0:00:12 0:00:12 --:--:-- 1889k
info: extracting 'aerospike-client-c-3.1.11.mac.x86_64/aerospike-client-c-devel-3.1.11.pkg' from 'aerospike-client-c.tgz'
info: extracting files from 'aerospike-client-c-3.1.11.mac.x86_64/aerospike-client-c-devel-3.1.11.pkg'
7913 blocks
CHECK
[✓] /private/var/folders/l8/41zyp9sx0hl4d423hjxn9wdm0000gn/T/pip-build-MMkDhv/aerospike/aerospike-client-c/package/usr/local/lib/libaerospike.a
[✓] /private/var/folders/l8/41zyp9sx0hl4d423hjxn9wdm0000gn/T/pip-build-MMkDhv/aerospike/aerospike-client-c/package/usr/local/include/aerospike/aerospike.h
[✓] /private/var/folders/l8/41zyp9sx0hl4d423hjxn9wdm0000gn/T/pip-build-MMkDhv/aerospike/aerospike-client-c/package/usr/local/aerospike/client/sys/udf/lua/aerospike.lua
Command "/usr/bin/python -c "import setuptools, tokenize;file='/private/var/folders/l8/41zyp9sx0hl4d423hjxn9wdm0000gn/T/pip-build-MMkDhv/aerospike/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /var/folders/l8/41zyp9sx0hl4d423hjxn9wdm0000gn/T/pip-1mNNbR-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/l8/41zyp9sx0hl4d423hjxn9wdm0000gn/T/pip-build-MMkDhv/aerospike
system info:
$ cc -v
Apple LLVM version 6.1.0 (clang-602.0.49) (based on LLVM 3.6.0svn)
Target: x86_64-apple-darwin14.3.0
Thread model: posix
$ python --version
Python 2.7.6
$ pip --version
pip 6.1.1 from /Library/Python/2.7/site-packages/pip-6.1.1-py2.7.egg (python 2.7)
$ sw_vers
ProductName: Mac OS X
ProductVersion: 10.10.3
BuildVersion: 14D136
When the C client has been built using the USE_LUAJIT = 1
, the python driver fails to build with the following error.
x86_64-pc-linux-gnu-gcc -pthread -fPIC -I./aerospike-client-c/include -I./aerospike-client-c/include/ck -Isrc/include -I/usr/include/python2.7 -c src/main/predicates.c -o build/temp.linux-x86_64-2.7/src/main/predicates.o -std=gnu99 -g -Wall -fPIC -O1 -fno-common -fno-strict-aliasing -finline-functions -march=nocona -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D_GNU_SOURCE -rdynamic -DMARCH_x86_64
x86_64-pc-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,--as-needed -L. build/temp.linux-x86_64-2.7/src/main/aerospike.o build/temp.linux-x86_64-2.7/src/main/log.o build/temp.linux-x86_64-2.7/src/main/client/type.o build/temp.linux-x86_64-2.7/src/main/client/apply.o build/temp.linux-x86_64-2.7/src/main/client/close.o build/temp.linux-x86_64-2.7/src/main/client/connect.o build/temp.linux-x86_64-2.7/src/main/client/exists.o build/temp.linux-x86_64-2.7/src/main/client/exists_many.o build/temp.linux-x86_64-2.7/src/main/client/get.o build/temp.linux-x86_64-2.7/src/main/client/get_many.o build/temp.linux-x86_64-2.7/src/main/client/select_many.o build/temp.linux-x86_64-2.7/src/main/client/info_node.o build/temp.linux-x86_64-2.7/src/main/client/info.o build/temp.linux-x86_64-2.7/src/main/client/key.o build/temp.linux-x86_64-2.7/src/main/client/put.o build/temp.linux-x86_64-2.7/src/main/client/operate.o build/temp.linux-x86_64-2.7/src/main/client/query.o build/temp.linux-x86_64-2.7/src/main/client/remove.o build/temp.linux-x86_64-2.7/src/main/client/scan.o build/temp.linux-x86_64-2.7/src/main/client/select.o build/temp.linux-x86_64-2.7/src/main/client/admin.o build/temp.linux-x86_64-2.7/src/main/client/udf.o build/temp.linux-x86_64-2.7/src/main/client/sec_index.o build/temp.linux-x86_64-2.7/src/main/serializer.o build/temp.linux-x86_64-2.7/src/main/client/remove_bin.o build/temp.linux-x86_64-2.7/src/main/client/get_key_digest.o build/temp.linux-x86_64-2.7/src/main/client/lstack.o build/temp.linux-x86_64-2.7/src/main/client/lset.o build/temp.linux-x86_64-2.7/src/main/client/lmap.o build/temp.linux-x86_64-2.7/src/main/client/llist.o build/temp.linux-x86_64-2.7/src/main/key/type.o build/temp.linux-x86_64-2.7/src/main/key/apply.o build/temp.linux-x86_64-2.7/src/main/key/exists.o build/temp.linux-x86_64-2.7/src/main/key/get.o build/temp.linux-x86_64-2.7/src/main/key/put.o build/temp.linux-x86_64-2.7/src/main/key/remove.o build/temp.linux-x86_64-2.7/src/main/query/type.o build/temp.linux-x86_64-2.7/src/main/query/apply.o build/temp.linux-x86_64-2.7/src/main/query/foreach.o build/temp.linux-x86_64-2.7/src/main/query/results.o build/temp.linux-x86_64-2.7/src/main/query/select.o build/temp.linux-x86_64-2.7/src/main/query/where.o build/temp.linux-x86_64-2.7/src/main/scan/type.o build/temp.linux-x86_64-2.7/src/main/scan/foreach.o build/temp.linux-x86_64-2.7/src/main/scan/results.o build/temp.linux-x86_64-2.7/src/main/scan/select.o build/temp.linux-x86_64-2.7/src/main/lstack/type.o build/temp.linux-x86_64-2.7/src/main/lstack/lstack_operations.o build/temp.linux-x86_64-2.7/src/main/lset/type.o build/temp.linux-x86_64-2.7/src/main/lset/lset_operations.o build/temp.linux-x86_64-2.7/src/main/llist/type.o build/temp.linux-x86_64-2.7/src/main/llist/llist_operations.o build/temp.linux-x86_64-2.7/src/main/lmap/type.o build/temp.linux-x86_64-2.7/src/main/lmap/lmap_operations.o build/temp.linux-x86_64-2.7/src/main/conversions.o build/temp.linux-x86_64-2.7/src/main/policy.o build/temp.linux-x86_64-2.7/src/main/predicates.o ./aerospike-client-c/lib/libaerospike.a -L/usr/lib64 -lssl -lcrypto -lpthread -lm -lz -lrt -lpython2.7 -o build/lib.linux-x86_64-2.7/aerospike.so -Wl,-Bdynamic -fPIC
/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/../../../../x86_64-pc-linux-gnu/bin/ld: ./aerospike-client-c/lib/libaerospike.a(lj_api.o): relocation R_X86_64_32S against `lj_obj_typename' can not be used when making a shared object; recompile with -fPIC
./aerospike-client-c/lib/libaerospike.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
error: command 'x86_64-pc-linux-gnu-gcc' failed with exit status 1
A put() with an explicit key policy set to aerospike.POLICY_KEY_DIGEST still ends up setting the primary-key information (as if it was aerospike.POLICY_KEY_SEND)
pp = pprint.PrettyPrinter(indent=2)
client.put(('test', 'demo', 12345), {'a':1},
policy={'exists': aerospike.POLICY_CREATE,
'key': aerospike.POLICY_KEY_DIGEST})
(key, meta, bins) = client.get(('test', 'demo', 12345))
pp.pprint(key)
It would be nice if aerospike supports python3 binding.
Is there any plan to support it ?
Please add support for arbitrary nesting of data types. Especially a list of maps.
The Python client currently does not support Batch operations.
I open to contributions.
Hi,
we had multiple issues where the C library version wasn't there anymore or today aerospike.com was down resulting in us not being to reproduce a build / not being able to build.
We use devpi which caches the pip python packages for us.
Is there a way that we can inject the C library or set an env so it pulls the C library from our server?
Thank you,
Florian
Installing collected packages: aerospike
Found existing installation: aerospike 1.0.34
Uninstalling aerospike:
Successfully uninstalled aerospike
Running setup.py install for aerospike
info: downloading 'http://www.aerospike.com/download/client/c/3.0.90/artifact/ubuntu12' to '/home/fpn/chattyheads/virtualenv/build/aerospike/aerospike-client-c/package/aerospike-client-c.tgz'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now
info: extracting '' from 'aerospike-client-c.tgz'
I am doing a read from aerospike based on a cron, and since recently (Feb 22/2015) it started throwing segmetation fault on get_many()
kernel: [2271722.448577] python[62842]: segfault at 4 ip 00007fafdf9d4d68 sp 00007fffa1b6e278 error 6 in libc-2.19.so[7fafdf93c000+1bb000]
The code is very simple at the moment
a_config = {
'hosts': [
('host', 3000),
],
'policies': {
'timeout': 1000
}
}
a_client = aerospike.client(a_config)
a_client.connect()
global_keys = [
('Namespace', 'Set', key1),
('Namespace', 'Set', key2),
]
records = a_client.get_many(keys)
print(records)
The bizarre part is that it only started happening recently, and the same code works on my local machine (mac), both with the most recent aerospike driver. This leads me to think that it had to do with Lua
There's an extra bracket at the end of config variable over at http://www.aerospike.com/docs/client/python/usage/connect/#configure-a-client , hosts and policies should be string
Encountered this hoping that the fix for storing unicode objects inside a map would be fixed, and I'm welcomed by this new bug. Fun.
policy = {'commit_level': 1, 'key': 1, 'timeout': 1000}
ops = [{'bin': 'foo', 'op': 5, 'val': u'f041ddeb093942a58e3d5950992a88e6,'}]
key, meta, bins = client.operate(key, ops, {}, policy)
Exception: (-1L, 'Value of incompatible type', 'src/main/client/operate.c', 671)
making the val a str() make it work, I guess unicode should work?
When iterating over a query, returning False does not stop the iteration.
Nor does returning None, 0 or -1.
This could be especially problematic for queries on secondary indexes with high cardinality.
Similar to append(), increment() should create a new record with the bin set to an initial value if the record does not yet exist. This should follow the C client's as_operations_add_incr() behavior.
import aerospike
config = { 'hosts': [ ('192.168.119.3', 3000) ] }
client = aerospike.client(config).connect()
client.increment(('test','eg-set','eg-key'), 'mybin', 5)
client.close()
Title says it all.
client.put(('db', 'table', 'key'), { 'val': 1 })
print(client.get(('db', 'table', 'key'))[2])
#prints { 'val': 1 }
client.put(('db', 'table', 'key'), { 'val': {1, 2} })
print(client.get(('db', 'table', 'key'))[2])
#prints { 'val': 1 }
client.put(('db', 'table', 'key2'), { 'val': {1, 2} })
print(client.get(('db', 'table', 'key2'))[2])
#prints None
I am currently using 1.0.28, since that version still correctly supports put's failing if the record already exists. I've tried the newer versions, since they have fixes (that do seem to work) for unicode, but now I get inconsistent behavior when setting " {'exists': 2}" as the 4th argument to put.
I'm trying to run a pretty basic program to pull some records from our Aerospike cluster, but as soon as I try to do "import aerospike", I get this error from the program:
ImportError: /usr/lib64/python2.7/site-packages/aerospike.so: undefined symbol: luaL_typerror
I think the problem is that luaL_typerror has been removed in Lua 5.2. The implication is that the python driver presently won't work with that version of Lua. Lua 5.2 happens to be included in Fedora 21, which is what I'm trying to use.
For reference, you can see the function definition in the source for 5.1 but not 5.2:
http://www.lua.org/source/5.1/lauxlib.c.html
http://www.lua.org/source/5.2/lauxlib.c.html
Hello,
I experience a strange behaviour when querying a set. Here is the aql code:
aql> select * from logs.demo1 where uuid='5ad46b1e67de4cd6bf0acec1d671a458'
+--------+------------------------------------+-----+--------+----------+---------------+
| node| uuid | io | owner | lines | ts |
+--------+------------------------------------+-----+--------+----------+---------------+
| "n1" | "5ad46b1e67de4cd6bf0acec1d671a458" | "O" | "demo" | ["1"] | 1420555691280 |
| "n1" | "5ad46b1e67de4cd6bf0acec1d671a458" | "O" | "demo" | ["2"] | 1420555693281 |
| "n1" | "5ad46b1e67de4cd6bf0acec1d671a458" | "O" | "demo" | ["4"] | 1420555690278 |
| "n1" | "5ad46b1e67de4cd6bf0acec1d671a458" | "O" | "demo" | ["3"] | 1420555694283 |
+--------+------------------------------------+-----+--------+----------+---------------+
4 rows in set (0.001 secs)
And the corresponding code in python:
q = client.query('logs', 'demo1')
q.where(p.equals('uuid', '5ad46b1e67de4cd6bf0acec1d671a458'))
q.foreach(callback)
But I get no results in the callback.
Another issue: if I make a query to the same set, but using the bin owner
- then the python client returns results, but lesser than what I get in aql. Note: I have both STRING indices on owner
and uuid
.
aql> select * from logs.demo1 where owner='demo'
... 48 results outputted ...
and in Python:
q = client.query('logs', 'demo1')
q.where(p.equals('owner', 'demo'))
q.foreach(callback)
23 results outputed, 25 less than what I get from aql.
My setup is 2 servers in a cluster(AWS instances, using mesh for heartbeat) - AMC shows everything synced, no errors in the log. Here is a snippet from the log:
Jan 07 2015 09:44:31 GMT: INFO (info): (thr_info.c::4520) system memory: free 7474180kb ( 97 percent free )
Jan 07 2015 09:44:31 GMT: INFO (info): (thr_info.c::4527) migrates in progress ( 0 , 0 ) ::: ClusterSize 2 ::: objects 112
Jan 07 2015 09:44:31 GMT: INFO (info): (thr_info.c::4535) rec refs 112 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: m
ig rx 0
Jan 07 2015 09:44:31 GMT: INFO (info): (thr_info.c::4541) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
Jan 07 2015 09:44:31 GMT: INFO (info): (thr_info.c::4551) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (6, 19649, 19643) : hb (2, 2, 0) : fab (20, 20, 0)
Specifically, Centos 7
[root@www ~]# sudo yum install python-pip lua-devel openssl-devel
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
Package python-pip-1.3.1-4.el7.noarch already installed and latest version
Package lua-devel-5.1.4-14.el7.x86_64 already installed and latest version
Package 1:openssl-devel-1.0.1e-34.el7_0.6.x86_64 already installed and latest version
Nothing to do
[root@www ~]# pip install aerospike
Downloading/unpacking aerospike
Running setup.py egg_info for package aerospike
info: liblua found: /usr/lib64/liblua-5.1.so
Installing collected packages: aerospike
Running setup.py install for aerospike
which: no lsb_release in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)
error: is not supported.
error: scripts/aerospike-client-c.sh 1
info: liblua found: /usr/lib64/liblua-5.1.so
info: Executing ./scripts/aerospike-client-c.sh
Complete output from command /usr/bin/python -c "import setuptools;file='/tmp/pip-build-root/aerospike/setup.py';exec(compile(open(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-AHyJTS-record/install-record.txt --single-version-externally-managed:
which: no lsb_release in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)
error: is not supported.
error: scripts/aerospike-client-c.sh 1
info: liblua found: /usr/lib64/liblua-5.1.so
info: Executing ./scripts/aerospike-client-c.sh
Command /usr/bin/python -c "import setuptools;file='/tmp/pip-build-root/aerospike/setup.py';exec(compile(open(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-AHyJTS-record/install-record.txt --single-version-externally-managed failed with error code 1 in /tmp/pip-build-root/aerospike
Dear Team,
I am trying to install aerospike python client on ubuntu 14.04 but getting this error while running "pip install aerospike"
I have installed all the packages as mentioned on page along with python-dev package. Please assist in the same.
$ python --version
Python 2.7.6
CHECK
[✓] /usr/lib/libaerospike.a
[✓] /usr/include/aerospike/aerospike.h
[✓] /opt/aerospike/client/sys/udf/lua/aerospike.lua
info: Executing ./scripts/aerospike-client-c.sh
info: aerospike.h found: ./aerospike-client-c/include/aerospike/aerospike.h
info: libaerospike.a found: ./aerospike-client-c/lib/libaerospike.a
running install
running build
running build_ext
building 'aerospike' extension
creating build
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/src
creating build/temp.linux-x86_64-2.7/src/main
creating build/temp.linux-x86_64-2.7/src/main/client
creating build/temp.linux-x86_64-2.7/src/main/key
creating build/temp.linux-x86_64-2.7/src/main/query
creating build/temp.linux-x86_64-2.7/src/main/scan
creating build/temp.linux-x86_64-2.7/src/main/lstack
creating build/temp.linux-x86_64-2.7/src/main/lset
creating build/temp.linux-x86_64-2.7/src/main/llist
creating build/temp.linux-x86_64-2.7/src/main/lmap
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I./aerospike-client-c/include -I./aerospike-client-c/include/ck -Isrc/include -I/usr/include/python2.7 -c src/main/aerospike.c -o build/temp.linux-x86_64-2.7/src/main/aerospike.o -std=gnu99 -g -Wall -fPIC -O1 -fno-common -fno-strict-aliasing -march=nocona -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D_GNU_SOURCE -rdynamic -finline-functions -DMARCH_x86_64
In file included from ./aerospike-client-c/include/aerospike/as_query.h:23:0,
from src/include/types.h:24,
from src/include/client.h:21,
from src/main/aerospike.c:22:
./aerospike-client-c/include/aerospike/as_udf.h:251:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
as_udf_file * as_udf_file_new();
^
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I./aerospike-client-c/include -I./aerospike-client-c/include/ck -Isrc/include -I/usr/include/python2.7 -c src/main/exception.c -o build/temp.linux-x86_64-2.7/src/main/exception.o -std=gnu99 -g -Wall -fPIC -O1 -fno-common -fno-strict-aliasing -march=nocona -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D_GNU_SOURCE -rdynamic -finline-functions -DMARCH_x86_64
In file included from ./aerospike-client-c/include/aerospike/as_query.h:23:0,
from src/main/exception.c:18:
./aerospike-client-c/include/aerospike/as_udf.h:251:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
as_udf_file * as_udf_file_new();
^
In file included from src/main/exception.c:22:0:
src/include/conversions.h:38:46: error: unknown type name ‘as_user’
as_status as_user_to_pyobject(as_error _err, as_user *user, PyObject *_py_as_user);
^
src/include/conversions.h:40:52: error: unknown type name ‘as_user’
as_status as_user_array_to_pyobject(as_error _err, as_user *_users, PyObject **py_as_users, int users_size);
^
In file included from src/main/exception.c:22:0:
src/include/conversions.h:78:53: error: unknown type name ‘as_privilege’
as_status as_privilege_to_pyobject( as_error * err, as_privilege privileges[], PyObject ** py_as_privilege, int privilege_size);
^
src/include/conversions.h:80:48: error: unknown type name ‘as_role’
as_status as_role_to_pyobject( as_error * err, as_role * role, PyObject ** py_as_role );
^
src/include/conversions.h:82:53: error: unknown type name ‘as_role’
as_status as_role_array_to_pyobject( as_error _err, as_role *_roles, PyObject **py_as_roles, int roles_size );
^
src/include/conversions.h:84:72: error: unknown type name ‘as_privilege’
void pyobject_to_as_privileges(as_error err, PyObject *py_privileges, as_privilege* privileges, int privileges_size);
^
src/main/exception.c: In function ‘AerospikeException_New’:
src/main/exception.c:45:4: error: ‘AEROSPIKE_ERR_FAIL_FORBIDDEN’ undeclared (first use in this function)
AEROSPIKE_ERR_FAIL_FORBIDDEN, AEROSPIKE_ERR_QUERY, AEROSPIKE_ERR_CLUSTER}
^
src/main/exception.c:45:4: note: each undeclared identifier is reported only once for each function it appears in
src/main/exception.c:83:28: error: ‘AEROSPIKE_ROLE_ALREADY_EXISTS’ undeclared (first use in this function)
AEROSPIKE_INVALID_ROLE, AEROSPIKE_ROLE_ALREADY_EXISTS, AEROSPIKE_ROLE_VIOLATION,
^
src/main/exception.c:168:27: error: ‘AEROSPIKE_ERR_INVALID_HOST’ undeclared (first use in this function)
py_code = PyInt_FromLong(AEROSPIKE_ERR_INVALID_HOST);
^
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;file='/tmp/pip_build_root/aerospike/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-jx_N2h-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/aerospike
Traceback (most recent call last):
File "/usr/bin/pip", line 9, in
load_entry_point('pip==1.5.4', 'console_scripts', 'pip')()
File "/usr/lib/python2.7/dist-packages/pip/init.py", line 235, in main
return command.main(cmd_args)
File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 161, in main
text = '\n'.join(complete_log)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 8: ordinal not in range(128)
Right now aerospike-client-python "gobbles up the error or warnings" which is obviously not helpful for the AeroSpike App Developer.
Log dumped by AeroSpike Python Client should be able to clearly answer following =>
Put operation doesn't work for u'strings'.
Exception: (200L, 'value is not a supported type.', 'src/main/conversions.c', 145)
lib version: aerospike-1.0.10
OS: Centos6
Python 2.6.6
Setting the time-to-live on a put() does not seem to stick
try:
client = aerospike.client(config).connect()
pp = pprint.PrettyPrinter(indent=2)
client.put(('test', 'demo', 54321), {'z':26},
policy={'exists': aerospike.POLICY_EXISTS_CREATE_OR_REPLACE},
meta={'ttl':60})
(key, meta, bins) = client.get(('test', 'demo', 54321))
pp.pprint(meta)
client.put(('test', 'demo', 54321), {'d':4})
(key, meta, bins) = client.get(('test', 'demo', 54321))
pp.pprint(meta)
client.close()
except Exception, eargs:
print("error: {0}".format(eargs), file=sys.stderr)
exitCode = 3
I get:
{ 'gen': 1, 'ttl': 60}
{ 'gen': 2, 'ttl': 2592000}
Hi, with the following code
key = ('test', 'demo', 'foo')
client.put(key, {
'name': 'John Doe',
'age': 32
})
(key, metadata, record) = client.get(key)
print "GET KEY:", key[2]
def print_result((key, metadata, record)):
print "SCAN KEY:", key[2]
scan = client.scan('test', 'demo')
scan.foreach(print_result)
Output I get back is:
GET KEY: foo
SCAN KEY: None
Any ideas as to why the key name is not returned as part of the scan call?
Hi!
Is there in plans to make a python3 client with support for asyncio?
client.put(key, {'test1': u"\ud83d\ude04"})
print(client.get(key)[2]['test1']) #here it stores the emoji, which is correct
client.put(key, {'test1': {'test2': u"\ud83d\ude04"}})
print(client.get(key)[2]['test1']) #here the value of test2 is an empty string, which is wrong
Hi,
we run a uwsgi process and post fork we close the aerospike client and and create a new instance on each thread. Sometimes we noticed that our uwsgi spoolers (which basically are just threads) are not executed. Debugging it today I found that my spool thread is stuck in client.close()
Below is the strace of where it is stuck, basically it seems it loops forever printing this output:
8648 0.940400 <... futex resumed> ) = -1 ETIMEDOUT (Connection timed out)
8647 0.000023 <... futex resumed> ) = -1 ETIMEDOUT (Connection timed out)
8648 0.000017 fcntl(15, F_GETFL <unfinished ...>
8647 0.000006 fcntl(14, F_GETFL <unfinished ...>
8648 0.000005 <... fcntl resumed> ) = 0x802 (flags O_RDWR|O_NONBLOCK)
8647 0.000007 <... fcntl resumed> ) = 0x802 (flags O_RDWR|O_NONBLOCK)
8648 0.000005 select(16, NULL, [15], NULL, {1, 0} <unfinished ...>
8647 0.000015 select(15, NULL, [14], NULL, {1, 0} <unfinished ...>
8648 0.000008 <... select resumed> ) = 1 (out [15], left {0, 999997})
8647 0.000009 <... select resumed> ) = 1 (out [14], left {0, 999998})
8648 0.000009 write(15, "\2\1\0\0\0\0\0#node\npartition-generatio"..., 43 <unfinished ...>
8647 0.000006 write(14, "\2\1\0\0\0\0\0#node\npartition-generatio"..., 43 <unfinished ...>
8648 0.000057 <... write resumed> ) = 43
8647 0.000009 <... write resumed> ) = 43
8648 0.000005 fcntl(15, F_GETFL <unfinished ...>
8647 0.000004 fcntl(14, F_GETFL <unfinished ...>
8648 0.000007 <... fcntl resumed> ) = 0x802 (flags O_RDWR|O_NONBLOCK)
8647 0.000005 <... fcntl resumed> ) = 0x802 (flags O_RDWR|O_NONBLOCK)
8648 0.000005 select(16, [15], NULL, NULL, {1, 0} <unfinished ...>
8647 0.000007 select(15, [14], NULL, NULL, {1, 0} <unfinished ...>
8648 0.000009 <... select resumed> ) = 1 (in [15], left {0, 999994})
8648 0.000017 read(15, "\2\1\0\0\0\0\0005", 8) = 8
8647 0.000024 <... select resumed> ) = 1 (in [14], left {0, 999961})
8648 0.000014 fcntl(15, F_GETFL <unfinished ...>
8647 0.000006 read(14, <unfinished ...>
8648 0.000006 <... fcntl resumed> ) = 0x802 (flags O_RDWR|O_NONBLOCK)
8647 0.000006 <... read resumed> "\2\1\0\0\0\0\0005", 8) = 8
8648 0.000007 select(16, [15], NULL, NULL, {1, 0} <unfinished ...>
8647 0.000007 fcntl(14, F_GETFL <unfinished ...>
8648 0.000006 <... select resumed> ) = 1 (in [15], left {0, 999999})
8647 0.000006 <... fcntl resumed> ) = 0x802 (flags O_RDWR|O_NONBLOCK)
8648 0.000005 read(15, <unfinished ...>
8647 0.000004 select(15, [14], NULL, NULL, {1, 0} <unfinished ...>
8648 0.000007 <... read resumed> "node\tBB9C1C5327AC40C\npartition-g"..., 53) = 53
8647 0.000006 <... select resumed> ) = 1 (in [14], left {0, 999999})
8648 0.000007 futex(0x1c261b0, FUTEX_WAKE_PRIVATE, 1 <unfinished ...>
8647 0.000006 read(14, <unfinished ...>
8648 0.000006 <... futex resumed> ) = 0
8647 0.000004 <... read resumed> "node\tBB9C1C5327AC40C\npartition-g"..., 53) = 53
8648 0.000006 futex(0x1c261dc, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 149, {1427761206, 781615508}, ffffffff <unfinished ...>
8647 0.000007 futex(0x1d3cfb0, FUTEX_WAKE_PRIVATE, 1) = 0
8647 0.000018 futex(0x1d3cfdc, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 149, {1427761206, 781641485}, ffffffff
<unfinished ...>
8636 0.060258 <... epoll_wait resumed> {}, 1, 1000) = 0
8636 0.000085 wait4(-1, 0x7fff6c91fd78, WNOHANG, NULL) = 0
8636 0.000044 epoll_wait(19,
We were querying multiple buckets in succession via the Python client. However, our log suggests that we are only getting results for every few calls.
The code is similar to:
def print_result(k,m,r):
print r
index = 'foo'
for setname in setnames:
query = client.query(namespace, setname)
query.where(predicates.equals(u'index', index))
print setname
query.foreach(print_result)
We have verified with both direct connection and nodejs driver that results are missing from the python driver. Nothing jumped out as to why we would be missing these rows. Ideas?
I got the exception below when scan a set and use a results function to retrieve data and access only by first time some item in list.
Exception: (-1L, 'Unable to deserialize bytes', 'src/main/serializer.c', 458)
Follow code:
>>> c = aerospike.client({'hosts': [('localhost', 3000)], 'policies': {'timeout': 100, 'retry': 10}}).connect()
>>> query = c.scan('namespace', 'someset')
>>> results = query.results()
>>> results[0]
Exception: (-1L, 'Unable to deserialize bytes', 'src/main/serializer.c', 458)
>>> results[0]
(('namespace', 'someset', None, bytearray(b'\x06`\xda\x99\x12B\xd3\x1dsT:\x9fd+YRG\x8d\xe3\xc0')), {'gen': 1, 'ttl': 4294967295}, {'set': 'someset', 'value': {'24324': {'c': '34345345', 'home_clic': '3'}, '32350': {'c': '387473', 'home_clic': '1'}}, 'host': 1, 'key': '150b1daa-5eb9-43dc-b39d-0f0d670db5aa', 'operation': 5})
It's really weird. Do i found a bug?
I'm using python 2.7.8, client 1.0.43, Ubuntu 14.04.
Installing python client on Ubuntu Precise 64 on vagrant (windows host) throws following error:
pip install aerospike
...
...
src/main/policy.c: In function 'pyobject_to_policy_info':
src/main/policy.c:119:2: error: 'as_policy_bool' undeclared (first use in this function)
src/main/policy.c:119:2: note: each undeclared identifier is reported only once for each function it appears in
src/main/policy.c:119:2: error: expected ';' before 'PyInt_AsLong'
src/main/policy.c:120:2: error: expected ';' before 'PyInt_AsLong'
src/main/policy.c: In function 'pyobject_to_policy_scan':
src/main/policy.c:214:2: error: 'as_policy_bool' undeclared (first use in this function)
src/main/policy.c:214:2: error: expected ';' before 'PyInt_AsLong'
error: command 'gcc' failed with exit status 1
----------------------------------------
Cleaning up...
Command /home/vagrant/.virtualenvs/default/bin/python -c "import setuptools, tokenize;__file__='/home/vagrant/.virtualenvs/default/build/aerospike/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-JKHh8R-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/vagrant/.virtualenvs/default/include/site/python2.7 failed with error code 1 in /home/vagrant/.virtualenvs/default/build/aerospike
Traceback (most recent call last):
File "/home/vagrant/.virtualenvs/default/bin/pip", line 11, in <module>
sys.exit(main())
File "/home/vagrant/.virtualenvs/default/local/lib/python2.7/site-packages/pip/__init__.py", line 185, in main
return command.main(cmd_args)
File "/home/vagrant/.virtualenvs/default/local/lib/python2.7/site-packages/pip/basecommand.py", line 161, in main
text = '\n'.join(complete_log)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 8: ordinal not in range(128)
FYI, I am using virtualenv. All the dependencies are installed as stated in doc.
Guest OS : Linux precise64 3.2.0-23-generic #36-Ubuntu SMP Tue Apr 10 20:39:51 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
Aerospike Server (CE): 3.5.4
Python Client : 1.0.38.
My intention is to store and retrieve Float values into Aerospike using this python client as Float Data Types are not supported by Aerospike natively. From the below code snippet, I am able to insert the float converted to binary data into Aerospike but while retrieving I am getting Segmentation fault (core dumped).
import aerospike
import struct
aerospike_config = {
'hosts': [('localhost', 3000), ],
}
obj = aerospike.client(aerospike_config).connect()
key = ('my_namespace', 'my_set', 59)
f = struct.pack('f', 20.6)
obj.put(key, {'price':f})
print struct.unpack('f', obj.get(key))
On checking the data using aql, it looks like below.
aql> select * from my_namespace.my_set where PK=59
+--------+ | price | +--------+ | "�̤A" | +--------+ 1 row in set (0.001 secs)
Please Help.
Allow a scan() to accept a policies dictionary, with the ability to set the priority, percentage, concurrency, and 'nobins' policies for the scan.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.