toblerity / rtree Goto Github PK
View Code? Open in Web Editor NEWRtree: spatial index for Python GIS
Home Page: https://rtree.readthedocs.io
License: MIT License
Rtree: spatial index for Python GIS
Home Page: https://rtree.readthedocs.io
License: MIT License
when I want to index a batch of points(about 240 million),some points got id like 18011072989319135231, when I do intersect ,I got overflowed id number like -435671084390416385. I think there is some problems in converting number from python to C lib.
And I also check the poi id is in the range of uint64. I check the spatial index lib, I found they use uint64 as the id types.
I am trying to build rtree
with conda and make it pass all the tests but the test_stream_input()
is failing.
Instead of [0, 4, 16, 27, 35, 40, 47, 50, 76, 80]
the test is returning a list with all zeros. I am not familiar with rtree's code, but it seems that the zeros are coming from py_next_item
.
From PyPI, the latest release is 0.8.2, but on GitHub the latest tagged release is 0.8.1. Shouldn't there be a 0.8.2 tag for that release?
When checking nearest neighbours, rtree can return more than the specified number of results, as if two points are equidistant, it returns both them. To check this equidistance, it must have some threshold since the distances are floats. I want to be able to control this threshold.
I'm using travis-ci to automatically build and test my project, which uses rtree.
travis-ci encounters an error when installing rtree, or rather libspatialindex ( see
https://travis-ci.org/goulu/Goulib/builds/24608247 ) :
rtree
Traceback (most recent call last):
File "<string>", line 17, in <module>
File "/home/travis/virtualenv/python2.7.6/build/rtree/setup.py", line 5, in <module>
import rtree
File "rtree/__init__.py", line 1, in <module>
from index import Rtree
File "rtree/index.py", line 2, in <module>
import core
File "rtree/core.py", line 85, in <module>
rt = ctypes.CDLL(lib_name)
File "/opt/python/2.7.6/lib/python2.7/ctypes/__init__.py", line 365, in __init__
self._handle = _dlopen(self._name, mode)
OSError: libspatialindex_c.so: cannot open shared object file: No such file or directory
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 17, in <module>
File "/home/travis/virtualenv/python2.7.6/build/rtree/setup.py", line 5, in <module>
import rtree
File "rtree/__init__.py", line 1, in <module>
from index import Rtree
File "rtree/index.py", line 2, in <module>
import core
File "rtree/core.py", line 85, in <module>
rt = ctypes.CDLL(lib_name)
File "/opt/python/2.7.6/lib/python2.7/ctypes/__init__.py", line 365, in __init__
self._handle = _dlopen(self._name, mode)
OSError: libspatialindex_c.so: cannot open shared object file: No such file or directory
----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /home/travis/virtualenv/python2.7.6/build/rtree
Storing debug log for failure in /home/travis/.pip/pip.log
The command "pip install --allow-all-external -r requirements.txt" failed and exited with 1 during install.
any idea why ? thanks !
@sgillies is going to come up with some json-like key/value pairing for this so we can easily see the property sets. Maybe even set them?
hi,
i'm trying to use rtree in python3. i'm using the version installed by:
pip3 install rtree
on my ubuntu 15.04.
creating a (disk stored) index, i got an error about wrong type in set_dat_extension. had a look at the code and noticed that, at least in my version, there is:
v = value.encode('utf-8')
return core.rt.IndexProperty_SetFileNameExtensionDat(self.handle, value)
so, you encode the string into bytes but after you still pass the original string to the c function. i fixed this, here as well as in set_idx_extension, passing v instead of value. i also modified get_idx_extension and get_dat_extension adding a .decode('utf-8') in order to return strings and not bytes (without this step there were other errors about adding bytes to strings...). that solved the problem.
i guess you'll probably want to reflect the fix on your sources.
and thank you for the great job done so far!
I've installed spatialindex/1.8.5
from source and I'm trying to compile rtree 0.8.2 against it. Running
python setup.py install
tries to import rtree and fails in ctype
Traceback (most recent call last):
File "setup.py", line 4, in <module>
import rtree
File "/glade/scratch/ddvento/build/Rtree-0.8.2/rtree/__init__.py", line 1, in <module>
from .index import Rtree
File "/glade/scratch/ddvento/build/Rtree-0.8.2/rtree/index.py", line 6, in <module>
from . import core
File "/glade/scratch/ddvento/build/Rtree-0.8.2/rtree/core.py", line 110, in <module>
rt.Error_GetLastErrorNum.restype = ctypes.c_int
File "/glade/apps/opt/python/2.7.7/gnu-westmere/4.8.2/lib/python2.7/ctypes/__init__.py", line 378, in __getattr__
func = self.__getitem__(name)
File "/glade/apps/opt/python/2.7.7/gnu-westmere/4.8.2/lib/python2.7/ctypes/__init__.py", line 383, in __getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: python: undefined symbol: Error_GetLastErrorNum
My plan to let rtree know about spatialindex was via the C compiler, but it failed before. How should I proceed?
I think this snippet from line 104 of rtree/core.py
only works by coincidence.
elif os.name == 'posix':
platform = os.uname()[0]
lib_name = 'libspatialindex_c.so'
if platform == 'Darwin':
lib_name = 'libspatialindex_c.dylib'
free = ctypes.CDLL(find_library('libc')).free
else:
free = ctypes.CDLL(find_library('libc.so.6')).free
If you're actually looking for libc
with the find_library function, you'd want find_library('c')
, but this code works because:
find_library
returns None
if it fails to find the librarydlopen
(and thus, CDLL
) returns a handle that can access libc
if you pass None
to it as the library parameter.So, as far as I can tell, this:
if platform == 'Darwin':
lib_name = 'libspatialindex_c.dylib'
free = ctypes.CDLL(find_library('libc')).free
else:
free = ctypes.CDLL(find_library('libc.so.6')).free
could be replaced with:
if platform == 'Darwin':
lib_name = 'libspatialindex_c.dylib'
free = ctypes.CDLL(None).free
Please do correct me if I'm wrong -- or I guess if this is a deliberate choice for some reason?
Building rtree on OS X either from the current release or from master results in a failure:
$ pip install git+git://github.com/Toblerity/rtree.git
Downloading/unpacking git+git://github.com/Toblerity/rtree.git
Cloning git://github.com/Toblerity/rtree.git to /var/folders/z_/kgsm76k94s7_jhnkfydfnd380000gn/T/pip-p4_3c96t-build
Running setup.py (path:/var/folders/z_/kgsm76k94s7_jhnkfydfnd380000gn/T/pip-p4_3c96t-build/setup.py) egg_info for package from git+git://github.com/Toblerity/rtree.git
Traceback (most recent call last):
File "<string>", line 17, in <module>
File "/var/folders/z_/kgsm76k94s7_jhnkfydfnd380000gn/T/pip-p4_3c96t-build/setup.py", line 4, in <module>
import rtree
File "/private/var/folders/z_/kgsm76k94s7_jhnkfydfnd380000gn/T/pip-p4_3c96t-build/rtree/__init__.py", line 1, in <module>
from .index import Rtree
File "/private/var/folders/z_/kgsm76k94s7_jhnkfydfnd380000gn/T/pip-p4_3c96t-build/rtree/index.py", line 6, in <module>
from . import core
File "/private/var/folders/z_/kgsm76k94s7_jhnkfydfnd380000gn/T/pip-p4_3c96t-build/rtree/core.py", line 110, in <module>
rt.Error_GetLastErrorNum.restype = ctypes.c_int
File "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ctypes/__init__.py", line 364, in __getattr__
func = self.__getitem__(name)
File "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ctypes/__init__.py", line 369, in __getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: dlsym(RTLD_DEFAULT, Error_GetLastErrorNum): symbol not found
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 17, in <module>
File "/var/folders/z_/kgsm76k94s7_jhnkfydfnd380000gn/T/pip-p4_3c96t-build/setup.py", line 4, in <module>
import rtree
File "/private/var/folders/z_/kgsm76k94s7_jhnkfydfnd380000gn/T/pip-p4_3c96t-build/rtree/__init__.py", line 1, in <module>
from .index import Rtree
File "/private/var/folders/z_/kgsm76k94s7_jhnkfydfnd380000gn/T/pip-p4_3c96t-build/rtree/index.py", line 6, in <module>
from . import core
File "/private/var/folders/z_/kgsm76k94s7_jhnkfydfnd380000gn/T/pip-p4_3c96t-build/rtree/core.py", line 110, in <module>
rt.Error_GetLastErrorNum.restype = ctypes.c_int
File "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ctypes/__init__.py", line 364, in __getattr__
func = self.__getitem__(name)
File "/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ctypes/__init__.py", line 369, in __getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: dlsym(RTLD_DEFAULT, Error_GetLastErrorNum): symbol not found
Running Python 3.4.2 on OS X 10.10.
Hi, if I run the following:
import rtree
import cPickle
x = rtree.index.Index()
cPickle.loads(cPickle.dumps(x))
I get a segfault when I quit python, (or at another random time later, depends how long it runs for). I've tried using plain 'pickle' as well, but get the same behaviour.
I'm using rtree 0.8.2 (installed via pip) against libspatialite 4.2.0-2 on arch linux x64, python 2.7.9
Hi everybody,
I started using rtree some weeks ago and find it very useful, thanks for providing a rtree in python!
I run into a problem recently that might affect other people as well. I am accessing a postgis database with approximately 5 mio rows. I can build a rtree without problems as long as I keep the number of objects below 1 million. The file sizes increase up to 300MB for the data and 800kb for the index.
When I try to store more than approximately 1 mio entries I get this error:
Traceback (most recent call last):
File "/home/christoph/dev/my_project/trees.py", line 116, in create_from_db
tree = FastRtree(TREE_FILENAME, generator_function(), property=tree_property)
File "/home/christoph/.virtualenvs/my_project/lib/python2.7/site-packages/rtree/index.py", line 235, in __init__
self.handle = self._create_idx_from_stream(stream)
File "/home/christoph/.virtualenvs/my_project/lib/python2.7/site-packages/rtree/index.py", line 691, in _create_idx_from_stream
return core.rt.Index_CreateWithStream(self.properties.handle, next)
File "/home/christoph/.virtualenvs/my_project/lib/python2.7/site-packages/rtree/core.py", line 25, in check_void
raise RTreeError(msg)
RTreeError: Error in "Index_CreateWithStream": Tools::BufferedFileWriter::open: Cannot open file.
Does someone have a clue about what is happening? File creation is definitely not a problem when I create a smaller tree.
I'm trying to implement a custom storage for rtree, and get the following exception:
Traceback (most recent call last):
File "test.py", line 5, in <module>
tree = rtree.index.Rtree(RTreeRedis())
File "/Users/gunnar/projects/rtree-redis/env.rtreeredis/lib/python2.7/site-packages/rtree/index.py", line 225, in __init__
storage.registerCallbacks( self.properties )
File "/Users/gunnar/projects/rtree-redis/env.rtreeredis/lib/python2.7/site-packages/rtree/index.py", line 1158, in registerCallbacks
properties.custom_storage_callbacks = ctypes.cast( ctypes.pointer(callbacks), ctypes.c_void_p )
File "/Users/gunnar/projects/rtree-redis/env.rtreeredis/lib/python2.7/site-packages/rtree/index.py", line 1028, in set_custom_storage_callbacks
return core.rt.IndexProperty_SetCustomStorageCallbacks(self.handle, value)
File "/Users/gunnar/projects/rtree-redis/env.rtreeredis/lib/python2.7/site-packages/rtree/core.py", line 17, in check_return
raise RTreeError(msg)
rtree.core.RTreeError: LASError in "IndexProperty_SetCustomStorageCallbacks": The supplied storage callbacks size is wrong, expected 56, got 48
I based my code on this: https://github.com/Toblerity/zope.index.rtree/blob/master/rtree/storage.py
and since it fails already when constructing the rtree index, very little of my code actually runs.
This on python 2.7.5 on OSX 12.4.0, i.e. x86_64
Thanks for any pointers!
import rtree
import random
import time
def createDefaultIndex():
idx = rtree.index.Rtree(u"defaultIndex")
boxes = []
for boxIndex in range(0,100):
xmin = random.random()
ymin = random.random()
xmax = xmin + random.random()
ymax = ymin + random.random()
boxes.append((xmin,ymin,xmax,ymax))
for boxIndex, box in enumerate(boxes):
pickledObject = "makethisbig " * 100000
idx.insert(boxIndex, box, obj=pickledObject)
del idx
def readIndexDefault():
indexPath = u"defaultIndex"
#idx = rtree.index.Rtree(indexPath)
idx = rtree.index.Index(indexPath)
boundingCoords = [0,0,60,60]
print "Consuming Index"
startTime = time.time()
#hits = list(idx.intersection(boundingCoords, objects=True)) # memory leak
#hits = list(idx.intersection(boundingCoords, objects="raw")) # no memory leak
hits = list(idx.intersection(boundingCoords, objects=False)) # no memory leak
#hits = list(idx.intersection(boundingCoords))
endTime = time.time()
print "Consuming time: ", endTime - startTime
print "Hits: ", len(hits)
time.sleep(10)
print "Done Consuming - erasing memory"
idx = None
hits=None
for counter in range(0,10):
time.sleep(1)
if name == "main":
createDefaultIndex()
readIndexDefault()
is this library available for python 3.x as well?
Thanks
Thanks for providing RTree for Python!
I was wondering if rtree can be used in a multithreaded context?
The CustomStorage needs to register a flush() callback to keep up with changes to IStorageManager for libspatialindex 1.8.0+
Hello,
Perhaps I am missing some critical step, but I am attempting to serialize a 5-d index to disk using the standard method. Example code follows below. The problem I am running into is that when I close the index on disk and attempt to open again, the 5-d nature of the index appears to be lost. My Rtree version is 0.7.0.
import numpy as np
import rtree
random = np.random.random((5000,5))
def gen(r):
for i in xrange(r.shape[0]):
yield (i, (r[i,0], r[i,1], r[i,2], r[i,3], r[i,4],
r[i,0], r[i,1], r[i,2], r[i,3], r[i,4]), None)
p = rtree.index.Property()
p.dimension = 5
p.overwrite = True
tree = rtree.index.Index('test2', gen(random), properties=p)
print tree.properties.dimension
print tree.bounds
tree.close()
Result:
5
[0.0001399796428322997, 0.00012241192644635301, 7.3393899523988e-05, 0.0005358454728167095, 0.00015215451107841726, 0.9997935298037376, 0.9995067343095075, 0.9997580576087635, 0.9999079765788539, 0.999747647405568]
tree = rtree.index.Index('test2')
print tree.properties.dimension
print tree.bounds
tree.close()
Result:
2
[9.68817621993523e-06, 0.0001726496530731758, 3.065078523922793e-05, 0.0001633907971153592, 1.7523149461129428e-05, 0.9998708500048494, 0.9999690660514665, 0.9999390843951469, 0.9999013700514435, 0.9999794343619967]
Sadly, I haven't found any out there.
hallo,
i have the following test code:
from rtree import index.Index
def test():
for i in range(10):
yield (i,(i,i,i,i),{'i':i})
idx=Index('/tmp/idxtest',test())
for i in idx.nearest((0,0,0,0),20,objects=True):
print( i.id, i.bounds, i.object )
and the output is:
0 {'i': 0} [0.0, 0.0, 0.0, 0.0]
0 {'i': 1} [1.0, 1.0, 1.0, 1.0]
0 {'i': 2} [2.0, 2.0, 2.0, 2.0]
0 {'i': 3} [3.0, 3.0, 3.0, 3.0]
0 {'i': 4} [4.0, 4.0, 4.0, 4.0]
0 {'i': 5} [5.0, 5.0, 5.0, 5.0]
0 {'i': 6} [6.0, 6.0, 6.0, 6.0]
0 {'i': 7} [7.0, 7.0, 7.0, 7.0]
0 {'i': 8} [8.0, 8.0, 8.0, 8.0]
0 {'i': 9} [9.0, 9.0, 9.0, 9.0]
... all ids are 0!
i have verified the ids arrive unchanged at least until here, but what happens after is a little too hard to track for me.
There is a bug in the index dumping logic when you do not have cPickle available. If you use this simple script:
#!/usr/bin/env python
from rtree import index
import pickle as pickle
idx = index.Index()
left, bottom, right, top = (0.0, 0.0, 1.0, 1.0)
idx.insert(0, (left, bottom, right, top), "SomePickleable")
print list(idx.intersection((1.0, 1.0, 2.0, 2.0)))
re_idx = pickle.loads(pickle.dumps(idx))
print list(re_idx.intersection((1.0, 1.0, 2.0, 2.0)))
It runs fine with cPickle available:
$ ./rtree_bug.py
[0]
[0]
But if you force things to use (regular) Pickle:
mv lib/python2.7/lib-dynload/cPickle.so lib/python2.7/lib-dynload/cPickle.so.outoftheway
You get a protocol error:
$ ./rtree_bug.py
...
raise ValueError("pickle protocol must be <= %d" % HIGHEST_PROTOCOL)
ValueError: pickle protocol must be <= 2
The problem is in the dumps( ) aliasing, which I have a pull request for.
I am getting the following error when attempting installation on Ubuntu. I have tried via pip as well as downloading and installing from the github repo.
Here is the error:
Downloading/unpacking Rtree
Downloading Rtree-0.8.0.tar.gz (56kB): 56kB downloaded
Running setup.py (path:/tmp/pip_build_root/Rtree/setup.py) egg_info for package Rtree
Traceback (most recent call last):
File "", line 17, in
File "/tmp/pip_build_root/Rtree/setup.py", line 4, in
import rtree
File "rtree/init.py", line 1, in
from .index import Rtree
File "rtree/index.py", line 31, in
int(t) for t in c_api_version.decode('utf-8').split('.')]
ValueError: invalid literal for int() with base 10: 'SIDX_VERSION_MAJOR'
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 17, in
File "/tmp/pip_build_root/Rtree/setup.py", line 4, in
import rtree
File "rtree/init.py", line 1, in
from .index import Rtree
File "rtree/index.py", line 31, in
int(t) for t in __c_api_version__.decode('utf-8').split('.')]
ValueError: invalid literal for int() with base 10: 'SIDX_VERSION_MAJOR'
Hey is this of any interest to submit back as a pull request?
import zlib
from rtree import Rtree
class CompressedRtree(Rtree):
'''
adds zlib compression to rtree
'''
def __init__(self, *args, **kwargs):
Rtree.__init__(self, *args, **kwargs)
def dumps(self, obj):
result = super(CompressedRtree, self).dumps(obj)
return zlib.compress(result)
def loads(self, string):
return super(CompressedRtree, self).loads(zlib.decompress(string))
I can't seem to get the install to work for me. Any thoughts about what I can do to debug this?
after doing nearest
with objects=False, I end up with a list of IDs.
Is there a way to later go back and retrieve either the raw object or an Item
from this id? i.e. index.get_item_by_id
?
I would maybe also be nice to have a way to get the id and bounds, without getting the objects
On the mailing list Howard shows the way how to do this:
The base libspatialindex library itself absolutely supports this through it its visitor pattern, however. Presumably there would need to be some sort of callback-based method, much like the way the DataStream stuff in the C API works in the reverse direction, to connect up the Python side to the results visitor.
Importing both arcpy and rtree causes the arcpy.ArcSDESQLExecute function to break. No other functions seem affected (that I have come across). Update it seems any functions connection to SDE are broken.
This occurs for the Python 2.7 packages from http://www.lfd.uci.edu/~gohlke/pythonlibs/ for both x32 and x64. Using ArcGIS 10.1 on Windows 7.
I know arcpy modifies some system paths etc. but does rtree have any globals that could cause issues with arcpy?
import arcpy
import rtree
sde_conn = arcpy.ArcSDESQLExecute(r"C:\Temp\conn.sde")
Traceback (most recent call last):
File "", line 1, in
File "C:\Program Files (x86)\ArcGIS\Desktop10.1\arcpy\arcpy\arcobjects\mixins.py", line 893, in init
_BaseArcObject.init(self, *gp_fixargs((server, instance, database, user, password), True))
File "C:\Program Files (x86)\ArcGIS\Desktop10.1\arcpy\arcpy\arcobjects_base.py", line 47, in init
for arg in args))
RuntimeError: Object: Could not open SDE workspace C:\Temp\conn.sde
Could anyone explain this error?
Thank you in advance.
M@rco
2015-04-10 12:41:17 ERROR Zope.SiteErrorLog 1428662477.860.832295083576
Traceback (innermost last):
...........
Module Products.LinguaPlone.patches, line 34, in searchResults
Module Products.CMFPlone.CatalogTool, line 393, in searchResults
Module Products.ZCatalog.ZCatalog, line 604, in searchResults
Module Products.ZCatalog.Catalog, line 925, in searchResults
Module Products.ZCatalog.Catalog, line 547, in search
Module collective.geo.index.geometryindex, line 169, in _apply_index
Module collective.geo.index.index, line 69, in rtree
Module rtree.index, line 211, in init
Module rtree.index, line 985, in get_overwrite
Module rtree.core, line 42, in check_value
RTreeError: Error in "IndexProperty_GetOverwrite": Property IndexIdentifier was empty
HI,
Im not expert with linux and c libraries, so probaly my description is incomplete or not very handful.
Anyway Im experiencing a problem with libspatialindex.so.3.0.0 and rtree 8.2.
Now and then my plone instance dies with no clues and message. I've checked the dmesg report from linux and here is what I see:
[1476438.713131] traps: python[317] general protection ip:7fb04f5a2ff9 sp:7fb04a3440b0 error:0 in libspatialindex.so.3.0.0[7fb04f506000+b8000]
[1476763.870627] python[379]: segfault at 0 ip 00007f855e9ccaea sp 00007f85409640f8 error 4 in libc-2.19.so[7f855e944000+1bb000]
[1478334.385962] traps: python[1211] general protection ip:7ff88dbf5ff9 sp:7ff889198390 error:0 in libspatialindex.so.3.0.0[7ff88db59000+b8000]
[1526321.346560] traps: python[17381] general protection ip:7f33d0f48ff9 sp:7f33cc4eb390 error:0 in libspatialindex.so.3.0.0[7f33d0eac000+b8000]
[1527794.507131] traps: python[20645] general protection ip:7f8ccb8c6ff9 sp:7f8cc5766390 error:0 in libspatialindex.so.3.0.0[7f8ccb82a000+b8000]
[1528432.553798] python[20984]: segfault at 22 ip 00007f8f4de55ff2 sp 00007f8f404f6390 error 4 in libspatialindex.so.3.0.0[7f8f4ddb9000+b8000]
[1528892.762885] traps: python[21271] general protection ip:7f97170eeff2 sp:7f9711e90390 error:0 in libspatialindex.so.3.0.0[7f9717052000+b8000]
[1529146.029769] python[21388]: segfault at 0 ip 00007f29b5d25aea sp 00007f298bff9a88 error 4 in libc-2.19.so[7f29b5c9d000+1bb000]
[1529364.634798] python[21435]: segfault at 0 ip 00007fa394931aea sp 00007fa3780cc148 error 4 in libc-2.19.so[7fa3948a9000+1bb000]
[1529636.648433] python[21533]: segfault at 0 ip 00007f7c14cbaaea sp 00007f7bf37f6af8 error 4 in libc-2.19.so[7f7c14c32000+1bb000]
Im working on a linux instance (a VPS):
Linux 3.13.0-46-generic #79-Ubuntu SMP Tue Mar 10 20:06:50 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
Any suggestion is very appreciated. Thank you in advace.
This is more of a question rather than a issue. Pardon me if this is a wrong place to ask. For my thesis, I want to modify the rtree. For that I need references to root or internal nodes so that I can traverse the tree. I debugged the code and couldn't find a tree construction. Can some one help me?
I am new to libspatialindex python package.
From original RTree Paper.
I am using rtree to find the nearest city by lat,lon. But on Nginx server after few requests I am getting following error.
File "/home/ubuntu/core/greedy/utilities.py", line 88, in current_location
city = GeoLocation.CityByLatLong(location[0], location[1], idx)
File "/home/ubuntu/core/greedy/apps/geolocations/models.py", line 13, in CityByLatLong
near_list = idx.nearest((latitude, longitude), 1, objects=False)
File "/usr/local/lib/python2.7/dist-packages/rtree/index.py", line 567, in nearest
p_num_results)
ArgumentError: argument 5: <type 'exceptions.TypeError'>: expected LP_LP_c_longlong instance instead of pointer to LP_c_longlong
PS: On apache server, it is working perfectly well.
is it possible to get the points exacly in the same coordinate of the query input (not the nearest)?
Please do not use find_library from ctypes to find libspatialindex_c. This only works if it is installed on in a standard location. On HPC clusters, it is very likely that the library is not installed in a standard location. Adding the installation directory to LD_LIBRARY_PATH, or specifying the installation directory in the setup.cfg file should be sufficient, but it does not work.
The only way to make rtree find libspatialindex_c on a Linux HPC cluster is to "hack" core.py, comment out the line
lib_name = find_library('spatialindex_c')
and replace lib_name in
rt = ctypes.CDLL(lib_name)
with the absolut path of the library.
I understand that the problem is introduced by ctypes and not by the rtree package, but please use another mechanism than find_library to detect libspatialindex_c on Linux systems. The two most practical options would be to either use LD_LIBRARY_PATH or the setup.cfg file.
Thank you very much and best regards
Samuel Fux
Scenario:
Thanks
Hi,
I'm a new user of the R-tree package. I'm reading the documentation and I notice that stream loading is much faster in creating a R-tree than individual inserts. I have a bout 21 million bboxes I need to insert. If I insert individually using index.Index('rtree'), I see the the index/dat file created. But when I try
index.Index(filename='rtree', stream=foo()), the rtree get initialized but it not written to file. I'm not sure if this usage mode is supported?
Is there some sample code somewhere I can look at on how to create a file-based rtree w/ stream loading?
Thanks.
Our license is not being included in the source tree because of this.
Help. I have problem when instal Rtree on my windows.
I used cmd for install it.
But it's seem cannot found spatialindex_c.dll
Actually, i have spatialindex_c-64.dll
what sholud i do with it?
this is what i have done:
C:\Python27\Scripts\easy_install.exe Rtree
Best match: Rtree 0.8.2
Downloading https://pypi.python.org/packages/source/R/Rtree/Rtree-0.8.2.tar.gz#md5=4ba7276cbd4c51b373068004ae864d3
Processing Rtree-0.8.2.tar.gz
Writing c:\users.....\Rtree-0.8.02\setup.cfg
Running Rtree-0.8.2\setup.py -q dbist_egg --dist-dir ......
error: could not find or load spatialindex_c.dll
And I tried to download Rtree-0.8.2 and install it offline too, but stiil could not find or load spatialindex_c.dll.
Trying to set GeoPandas up with Rtree support today I have run into the following:
RTreeError: Error in "Index_CreateWithStream": IllegalArgumentException: RTree::BulkLoader::bulkLoadUsingSTR: Empty data stream given.
If the generator you send to the bulk loader turns out to be empty, say you filter all its element out for good reasons, leaving only an expression equivalent to
idx = Index(iter(()))
the bulk loader quits and raises an exception. More graceful failure would be better.
To get the memory leak fix out there, among other things.
cc @hobu
Using rtree 0.8.2. The small example below recreates the problem.
from rtree import index
def gen(d):
for i,obj in enumerate(d):
yield (i, obj[0], obj[1])
data = [((0,0,1,1), 25),
((1,1,2,2), 30),
((2,2,3,3), 35)]
idx = index.Index('foo', gen(data))
m = idx.intersection((2,3,3,3), objects=True)
for a in m:
print a.id
print a.bbox
print a.object
It correctly prints the correct bbox/object but the wrong id (should be 2). It seems like when stream loading is used to create the index, the ID is always returned as 0. It seems to work OK if idx.insert() is used to create the tree. Thanks.
$ ./test.py
0
[2.0, 2.0, 3.0, 3.0]
35
It looks like when doing stream loading for a 3D index the tree is created fine but the IDs aren't preserved for some reason. This issue does not exist when doing tree.insert inside a loop.
I tried this with a generator function as per the documentation, as well as passing the bounds as a tuple instead of a list. I verified that in index/_create_idx_from_stream/py_next_item
it was getting the correct ID into p_id[0]. I'm not sure what else to check, any ideas would be greatly appreciated.
Here's an example which was demonstrating the issue using rtree from Toblerity/rtree/master and libspatialindex from the latest packages in the Ubuntu 14.04 repo:
import numpy as np
import rtree
if __name__ == '__main__':
properties = rtree.index.Property()
properties.dimension = 3
points_count = 100
points_random = np.random.random((points_count,3,3))
points_bounds = np.column_stack((points_random.min(axis=1),
points_random.max(axis=1)))
stacked = zip(np.arange(points_count),
points_bounds,
[None] * points_count)
tree = rtree.index.Index(stacked,
properties = properties)
# this should intersect every box, as our random boxes are all inside [0,0,0,1,1,1]
tid = list(tree.intersection([-1,-1,-1,2,2,2]))
The correct answer for tid would be np.arange(points_count). The actual output of this is:
In [6]: len(tid)
Out[6]: 100
In [7]: tid[0:10]
Out[7]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
In [8]: (np.array(tid) == 0).all()
Out[8]: True
As far as I can tell from the documentation and the code, Rtree lets you either specify a filename for disk-based index, or a stream for stream loading. But how can I do both? What if I want a disk-based index which is populated by reading from a stream?
Pointed out by @hobu in #21 (comment).
Largely due to naive changes I made while working on Python 3.4. Should be easy fixes.
Hey Toblerity,
I'm investigating a library called hickle which allows you to use hdf5 to store objects. https://github.com/telegraphic/hickle
I was trying to get this going and store rtrees inside of HDF for the compression options. Any first impressions of this? Do you think its as easy as adding an additional import statement and using hickle on index.dumps() & index.loads()? Any thoughts would be appreciated.
While tracking down some verson compatibility issues I noted in a Gentoo ebuild I developed a couple of years ago that RTree cannot work on python version 3.0+. Has this been fixed? Also related to this, would you like RTree to be added to the Gentoo standard distribution?
Thanks and best regards,
John 9EBo) David --
There is an error during installation of the pip 0.8.2 version. Here is part of the error message:
File "/tmp/pip_build_root/Rtree/setup.py", line 4, in <module>
import rtree
File "rtree/__init__.py", line 1, in <module>
from .index import Rtree
File "rtree/index.py", line 6, in <module>
from . import core
File "rtree/core.py", line 110, in <module>
rt.Error_GetLastErrorNum.restype = ctypes.c_int
File "/usr/lib/python2.7/ctypes/__init__.py", line 378, in __getattr__
func = self.__getitem__(name)
File "/usr/lib/python2.7/ctypes/__init__.py", line 383, in __getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: /usr/bin/python: undefined symbol: Error_GetLastErrorNum
I am getting the following error when trying to create a new disk based index:
File "/path/python3.4/site-packages/rtree/index.py", line 199, in **init**
f = basename + "." + self.properties.idx_extension
TypeError: Can't convert 'bytes' object to str implicitly
I'm new to rtree so sorry if this is a basic question, but is there away to leverage streamloading to a file-based rtree?
The constrcutor to rtree.Index checks if the first argument is a basestring or a generator function. Is there a way to provide both?
For example, if I have the generator function
def generator_function():
... for i, obj in enumerate(somedata):
... yield (i, (obj.xmin, obj.ymin, obj.xmax, obj.ymax), obj)
And I want to save to "somefile"
r = index.Index("somefile", generator_function())
results in an exception.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.