fortyninemaps / karta Goto Github PK
View Code? Open in Web Editor NEWA tidy Python package for geospatial computation
Home Page: https://karta.fortyninemaps.com
License: MIT License
A tidy Python package for geospatial computation
Home Page: https://karta.fortyninemaps.com
License: MIT License
https://gist.github.com/sgillies/2217756
Would then be able to drop the to_shapely() methods, which are out of date anyway.
The guppy.Polygon class should be able to handle rings, which could possibly be stored as an iterable of sub-Polygon instances. Area and perimeter calculations would need to be modified to account for the added geometry.
Need (more) tests for the following areas
grid.transform[4:] != [0 0]
)raster.misc.hillshade
raster.misc.slope
raster.misc.aspect
raster.misc.grad
raster.misc.divergence
vector.shp
vector.gpx
Variogram calculation should be a part of karta.vector.stats. This doesn't need to be efficient or scalable - just needs to work for small problems.
Current intersection code is for the planar case, and in incorrect on a sphere/ellipsoid.
An approach to solving the problem with gnomonic projections is described by Karney (2013), section 8.
Being able to use Series in the data field of Multipoints et al seems useful, but fails because Series don't provide a dict-like interface.
Can continue to just use Python lists of geometries, but this needs to happen in shp.py (line 127).
raster.gtiffread
should return a list of arrays with the correct dtype for each GeoTiff band. As it is, datatypes are promoted to preserve accuracy, meaning a GeoTiff with bands with e.g. Int16 and Float32 with be coerced to Float32. I think it would be better to preserve the original datatype as best as possible, although this doesn't really work with the way that Grid stares everything in a single array.
Perhaps there should be a straightforward way to read each band into a separate grid.
The current variogram estimation is sensitive to outliers. It would be good to implement the method described by http://www.stat.tamu.edu/~genton/1998.G.MG.1.pdf
When reading from a data source that does not support spatial referencing, or for which spatial referencing was not used, a CRS instance must be chosen for the output geometry.
Currently, the behaviour is
Attempting to import karta
import karta
throws following TypeError:
TypeError: __init__() got an unexpected keyword argument 'enforce_poly_winding'
Looks like the enforce_poly_winding
keyword arg was appropriately removed in the 0.9.0 release by 81a5c6a due to the update in picogeojson
0.7.0
fortyninemaps/picogeojson@181879e#diff-ecd1078854dc5584f72f87835e5741c0
Doesn't seem like the 0.9.0
code made it's way to pip correctly... Is there some issue with the 0.9.0 release on PyPi?
Observed on:
0.9.0
0.7.0
Full stack trace:
TypeError Traceback (most recent call last)
<ipython-input-2-eb20076a4bd2> in <module>()
----> 1 import karta
~/.applications/anaconda3/envs/geo3/lib/python3.6/site-packages/karta/__init__.py in <module>()
8 from .version import __version__
9
---> 10 from . import vector
11 from . import raster
12 from . import crs
~/.applications/anaconda3/envs/geo3/lib/python3.6/site-packages/karta/vector/__init__.py in <module>()
9 numbers.Integral.register(numpy.integer)
10
---> 11 from . import geometry
12 from . import quadtree
13 from . import rtree
~/.applications/anaconda3/envs/geo3/lib/python3.6/site-packages/karta/vector/geometry.py in <module>()
12 import numpy as np
13 from .decorators import cache_decorator
---> 14 from ._geojson import GeoJSONOutMixin
15 from ._shp import ShapefileOutMixin
16 from .table import Table, Indexer
~/.applications/anaconda3/envs/geo3/lib/python3.6/site-packages/karta/vector/_geojson.py in <module>()
15 return {'type': 'link', 'properties': {'href': "", 'type': 'proj4'}}
16
---> 17 class GeoJSONOutMixin(object):
18 """ Mixin class to be added to geometry objects, adding geojson
19 functionality.
~/.applications/anaconda3/envs/geo3/lib/python3.6/site-packages/karta/vector/_geojson.py in GeoJSONOutMixin()
21
22 _serializer = picogeojson.Serializer(antimeridian_cutting=False,
---> 23 enforce_poly_winding=False)
24
25 @staticmethod
TypeError: __init__() got an unexpected keyword argument 'enforce_poly_winding'
Tests that use ctypes to test functions implemented in C are failing on Appveyor:
python tests\runtests.py
..........EEE...............................................................................................................................................................E.................................sssss.ss.ss............................................................................................................................................................
======================================================================
ERROR: test_bbox_intersection_area (rtree_tests.CRTreeTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\projects\karta\tests\rtree_tests.py", line 54, in test_bbox_intersection_area
self.rt.bbox_intersection_area.restype = c_float
File "C:\Miniconda3\envs\test-environment\lib\ctypes\__init__.py", line 360, in __getattr__
func = self.__getitem__(name)
File "C:\Miniconda3\envs\test-environment\lib\ctypes\__init__.py", line 365, in __getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: function 'bbox_intersection_area' not found
======================================================================
ERROR: test_is_overlapping (rtree_tests.CRTreeTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\projects\karta\tests\rtree_tests.py", line 39, in test_is_overlapping
self.rt.is_overlapping.restype = c_int
File "C:\Miniconda3\envs\test-environment\lib\ctypes\__init__.py", line 360, in __getattr__
func = self.__getitem__(name)
File "C:\Miniconda3\envs\test-environment\lib\ctypes\__init__.py", line 365, in __getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: function 'is_overlapping' not found
======================================================================
ERROR: test_is_within (rtree_tests.CRTreeTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\projects\karta\tests\rtree_tests.py", line 24, in test_is_within
self.rt.is_within.restype = c_int
File "C:\Miniconda3\envs\test-environment\lib\ctypes\__init__.py", line 360, in __getattr__
func = self.__getitem__(name)
File "C:\Miniconda3\envs\test-environment\lib\ctypes\__init__.py", line 365, in __getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: function 'is_within' not found
======================================================================
ERROR: test_iswithin (quadtree_tests.TestCQuadTree)
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\projects\karta\tests\quadtree_tests.py", line 25, in test_iswithin
self.qt.iswithin.restype = c_bool
File "C:\Miniconda3\envs\test-environment\lib\ctypes\__init__.py", line 360, in __getattr__
func = self.__getitem__(name)
File "C:\Miniconda3\envs\test-environment\lib\ctypes\__init__.py", line 365, in __getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: function 'iswithin' not found
----------------------------------------------------------------------
Ran 373 tests in 4.453s
FAILED (errors=4, skipped=9)
The Python tests using the C functions as an extension module are passing, so this is probably a problem with how the tests are set up, rather than with the build.
Since I don't use a Windows machine, this is a little tricky to work on.
Would be better to extend the quadtree as well.
The module raster._dem
has been sitting around 75% complete for a while.
There should be the ability to import GeoJSON FeatureCollections. This might mean creating an analogue of a feature collection in vector.guppy, which would make an obvious path for export as well.
Right now, only properties are checked, but it is possible for points to have data as well.
I wonder if it would make sense to merge Point.data and Point.properties, somehow. It would make more sense, although it would give up consistency with multipoint classes.
The coordinates need to be converted to lon/lat to use crs methods forward
and inverse
.
The following test gives incorrect results.
interx1
and interx2
should be identical, and the plots show that 3/4 of the identified intersections are false positives.
L = Line(((0,0), (1,2.5), (2,4), (3, 1)))
L2 = Line(((0.25,3), (0.1, 1.5), (1.5, 1.7), (2.2, 1.0)))
interx1 = L.intersections(L2)
interx2 = L2.intersections(L)
plot(*L.get_coordinate_lists()[:2])
plot(*L2.get_coordinate_lists()[:2])
for _x,_y in interx1: plot(_x,_y, 'ob')
for _x,_y in interx2: plot(_x,_y, 'or')
Need to make sure that the C code (currently in the vector submodule) is getting tested adequately. Many functions don't expose a Python interface. Not sure how to work this into the current unittest framework, so might been to look at some other projects to see how they handle it.
When the DEM contains plateaus, the flow field computed by dem_flow()
contains NaNs. These should be filled, e.g. using the method described at http://blogs.mathworks.com/steve/2007/08/23/upslope-area-handling-plateaus-revisited/
When compiling v0.7.4 on Windows 10 using the compiler suggested at https://wiki.python.org/moin/WindowsCompilers#Microsoft_Visual_C.2B-.2B-_14.0_standalone:_Visual_C.2B-.2B-_Build_Tools_2015_.28x86.2C_x64.2C_ARM.29 I get this error:
karta/vector/convexhull.c(2019): error C2433: '__pyx_f_5karta_6vector_9vectorgeo_dist2': 'inline' not permitted on data declarations
and others like it. Just the name of the var changes. I got it to compile by removing the inline keyword from the definitions of cross2, cross3, and dist2 from the vectorgeo source files.
Converting Metadata to shapefile records is buggy and incomplete. Need to add tests.
I would like karta to be compatible with both Python 2.x and Python 3.x, preferably from a single code base. All dependencies are now Python 3 compatible.
When installing, dependencies such as pyshp aren't being included properly. This came up for njwilson23/narwhal#6.
Because setup.py imports numpy, karta cannot be installed unless numpy is already installed. This feels a bit odd, since numpy is an external dependency of karta and may not be available at install time.
benasher$ pip install karta
Collecting karta
Using cached karta-0.4.4.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 20, in <module>
File "/private/var/folders/3w/68cpb10s359538q_8rj2fpsh0000gn/T/pip-build-lwd1y5r7/karta/setup.py", line 5, in <module>
import numpy
ImportError: No module named 'numpy'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/3w/68cpb10s359538q_8rj2fpsh0000gn/T/pip-build-lwd1y5r7/karta
Add predicates for identifying polygons and lines within other polygons or touching other polygons or lines.
Getting
Traceback (most recent call last):
File "./findpairs.py", line 32, in <module>
poly = shapely.geometry.shape(poly_k.__geo_interface__["geometry"])
File "/home/natw/code/karta/karta/vector/geometry.py", line 1136, in __geo_interface__
p["_karta_proj4"] = self.crs.get_proj4()
File "/home/natw/code/karta/karta/crs.py", line 158, in get_proj4
raise AttributeError("incomplete CRS definition (missing ref_proj4, ref_wkt attributes)")
Could be fixed by making sure the default JSON CRS instance has appropriate attributes. This should be an easy fix.
Solving the inverse problem (0.000000, -57.244630) -> (179.795459, -12.169046) on the WGS84 ellipsoid results in a convergence failure warning from the Newton solver and an inaccurate result.
In [3]: LonLatWGS84.inverse(0.000000, -57.244630, 179.795459, -12.169046)
RuntimeWarning: Convergence failure (0.000000, -57.244630) -> (179.795459, -12.169046)
Out [3]: (-168.75633860993707, -173.7906929918106, 12246936.66410405)
libproj result:
In [6]: import pyproj
In [7]: g = pyproj.Geod("+ellps=WGS84")
In [8]: g.inv(0.000000, -57.244630, 179.795459, -12.169046)
Out[8]: (179.786987431945, -179.88183727918067, 12310967.277411604)
geographiclib result:
ellipsoid (a f) = 6378137 1/298.257223563 (WGS84)
status = OK
lat1 lon1 fazi1 (°) = -57.24463000 0.00000000 179.78698743
lat2 lon2 fazi2 (°) = -12.16904600 179.79545900 0.11816272
s12 (m) = 12310967.277
a12 (°) = 110.71332679
m12 (m) = 5987750.843
M12 M21 = -0.3469428985 -0.3523063207
S12 (m^2) = -127281236984754
matlab result:
>> azimuth( -57.244630, 0, -12.169046, 179.795459, referenceEllipsoid('wgs84'))
ans =
179.7870
>> azimuth(-12.169046, 179.795459, -57.244630, 0, referenceEllipsoid('wgs84'))
ans =
180.1182
Need to make sure the following work when vertices include points on either side of the dateline:
It's now being used to built the documentation, so it should be added to the readme.
Some methods currently break when Geometry.vertex/vertices has a vertical coordinate (e.g. those that splat the vertex to CRS.project). Add tests to find these and fix them.
Create a ctypes or Cython wrapper for libtiff and libgeotiff, and connect to raster.grid classes.
Vector calculations is choking on some moderately large datasets (tens of thousands of points). The metadata and indexing seem to be possible culprits.
In addition to profiling and optimization, need to start thinking about performance testing to catch regressions.
NaNs shouldn't be allowed in the inputs, but this needs to be handled gracefully.
hi, I am interested in using the lib with pypy. Does it support?
With only karta installed, can do:
karta.read_shapefile(...)
karta.read_gtiff(...)
but when the karta.mapping
namespace package is installed these need to be
karta.vector.read_shapefile(...)
karta.raster.read_gtiff(...)
Need to search for workarounds, or decide that the convenience of the first version is worth it.
It would be useful for the result of ConnectedMultipoint.intersections()
(a Multipoint
instance) to fill in the data
attribute. Otherwise, the data in a geometry formed from intersections (e.g. a clipped geometry) won't have an entry for every point (which is a bug).
The safest thing to do is probably to take the same fields as the input geometry, and assign NaN as each value in the intersection point.
Add a sweep line algorithm for finding multiple intersections for better performance.
In the tutorial, it is stated that
Each geometry has appropriate methods to save data:
mp.to_vtk("my_vtk.vtk")
But which geometry can export in vtk ? not points, lines or polygons, nor the multis
Thanks
These methods are intended to provide an interface to CoordString data for functions on the C side, but they're slow. Performance improvements would reduce the temptation to use the CoordString array pointer in external functions.
Hi,
I have pip installed yours package, but it seems to not install with yours correct requirements.
this is the result if I try to import:
from karta import Point, Multipoint
File "/home/Envs/empty/local/lib/python2.7/site-packages/karta/__init__.py", line 9, in <module>
from . import vector
File "/home/Envs/empty/local/lib/python2.7/site-packages/karta/vector/__init__.py", line 5, in <module>
from . import guppy
File "/home/Envs/empty/local/lib/python2.7/site-packages/karta/vector/guppy.py", line 15, in <module>
from . import _shpfuncs
File "/home/Envs/empty/local/lib/python2.7/site-packages/karta/vector/_shpfuncs.py", line 6, in <module>
import shapefile
ImportError: No module named shapefile
Only is enabled if I install before normal install:
pip install -r karta/requirements.txt
python setup.py install
Is there any chance to include yours requirements on setup.py since other packages could include karta as a requirement.
Cheers,
Arnaldo.
I seem to be having problems importing Proj4CRS within the Anaconda 2.7 enviro. No error with import karta or the karta.crs import LonLatWGS84 - however it chokes on the Proj4CRS import (ImportError: cannot import name Proj4CRS). There doesn't seem to be anything out there on this. I just did a full reinstall of the Anaconda 2.7 package (for Mac). I'm doing this within the Jupyter Notebook...Thx
import numpy as np
import scipy
from osgeo import gdal
import karta
from karta.crs import LonLatWGS84
#Trouble line is:
from karta.crs import Proj4CRS
Property methods such as Geometry.coordinates
assume that the coordinates should be returned in the current CRS. When this isn't the case, Geometry.get_coordinates(crs=[...])
needs to be used instead. This seems to create a lot of bugs when working with multiple coordinate systems.
For v0.10.0, consider getting rid of the .coordinates
properties, and possibly renaming .get_coordinates()
as .coordinates()
.
Polygon area is currently computed on a plane, but should be computed on an ellipsoid when a geometry has a GeographicalCRS
instance.
Commit 6e5ecd3 disables the method with geographical coordinates.
An initial attempt is at https://github.com/fortyninemaps/karta/tree/appveyor.
It would be nice to use this to build Wheels for PyPI as well. Guidance can be found at https://packaging.python.org/appveyor/
Hi, i'm fully interested in use karta for read some files in .asc format, but when i try to install it i there is some problems with gdal library, i did try the installation in multiple ways:
Some of the problems that i encountered were:
1 warning and 1 error generated.
error: command 'clang' failed with exit status 1
`Command "/Users/yor/.pyenv/versions/2.7.8/envs/wallocPython2.7.8venv/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/p6/r47jxx_x7t50v7m1663n29k40000gn/T/pip-build-L5NWYg/gdal/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/p6/r47jxx_x7t50v7m1663n29k40000gn/T/pip-atW8Oc-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/yor/.pyenv/versions/2.7.8/envs/wallocPython2.7.8venv/include/site/python2.7/gdal" failed with error code 1 in /private/var/folders/p6/r47jxx_x7t50v7m1663n29k40000gn/T/pip-build-L5NWYg/gdal/`
I would like to know if there is some especial requeriment? i'm using OSX 10.11.6 with python 2.7 and python 3.4.
Thanks for your help...
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.