Giter Club home page Giter Club logo

mapmatching's Introduction

Map Matcher

This python script allows map matching (matching of tracking points to a network) in arcpy using a Hidden Markov model with probabilities parameterized based on spatial + network distances. Follows the ideas in Newson, Krumm (2009): "Hidden markov Map Matching through noise and sparseness"

Author: Simon Scheider

Created: 16/03/2017

Installation

To install as a toolbox in ArcGIS, see mapMaptch.pyt

The code is written in Python 2.7 and depends on:

  • arcpy (ships with ArcGIS and its own Python 2.7)
  • networkx

python pip install networkx

  • note: requires installing GDAL first, which can be obtained as a windows wheel from here and then installed with pip locally:

python pip install GDAL-2.1.3-cp27-cp27m-win32.whl

To install the mapmatcher Python module, simply download and execute this windows executable:

Usage

Example:

from mapmatcher import mapmatcher

arcpy.env.workspace = 'C:/Users/simon/Documents/GitHub/mapmatching'

opt = mapmatcher.mapMatch('testTrack.shp', 'testSegments.shp')

#outputs testTrack_pth.shp

mapmatcher.exportPath(opt, 'testTrack.shp')

The last method saves a new shape file named testTrack_pth.shp in the current arcpy workspace, containing a sequence of segments to which the track was mapped.

Results are shown here:

The main method is mapMatch. Based on the Viterbi algorithm for Hidden Markov models, see https://en.wikipedia.org/wiki/Viterbi_algorithm, it gets trackpoints and segments, and returns the most probable segment path (a list of segments) for the list of points.

Method mapMatch:

  • @param track = a shape file (filename) with point geometries representing a track, can be unprojected (WGS84). The order of points in this file should reflect the temporal order.

  • @param segments = a shape file of network segments, should be projected (in meter) to compute Euclidean distances properly (e.g. GCS Amersfoord). Note: To compute network distances, the script turns this network into a graph using networkx, based on coincidence of segment end points. It is therefore important that logically connected segments are also geometrically connected (no geometrical errors). Other than this, the script does have other requirements for the network.

  • @param decayconstantNet (optional) = the network distance (in meter) after which the match probability falls under 0.34 (exponential decay). Default is 30 meters. This distance parameter depends on the intervals between successing points in the track.

  • @param decayConstantEu (optional) = the Euclidean distance (in meter) after which the match probability falls under 0.34 (exponential decay). Default is 10 meters. This distance parameter depends on the measurement accuracy of tracking points.

  • @param maxDist (optional) = the Euclidean distance threshold (in meter) for taking into account segments candidates. Default is 50 meters. Depends also on measurement accuracy of track points.

  • result = delivers back a path (a list of segment ids).

Note:

Depending on the type of movement, optional parameters need to be fine tuned to get optimal results. For example, when tracking frequency is very slow, then track points are far apart, and then decayconstantNet needs to be increased accordingly.

Method exportPath :

Exports the path into a shape file named segments_pth.shp inside the current ArcGIS workspace.

mapMatch.pyt (ArcGIS Python toolbox)

To use the Python method as an ArcGIS toolbox, you need to do the following:

  1. In your ArcGIS Python version (e.g. Folder C:\Python27\ArcGIS10.3\Lib\site-packages), install required modules for GDAL and networkx in a cmd window:
  1. Install mapmatcher Python module by downloading and executing the windows executable mapmatching/dist/mapmatcher-1.0.win32.exe. Make sure you select exactly the Python installation that ships with your ArcGIS as a target folder.

  2. Download the ArcGIS Python toolbox mapMatch.pyt, together with meta data files mapMatch.mapMatch.pyt.xml and mapMatch.pyt.xml and drop it anywhere on your computer.

  3. Now you can open the toolbox by clicking on it inside an ArcGIS Catalog Window:

The tool saves a new shape file named NameofInputTrack_pth.shp inside the current ArcGIS workspace that contains the path of segments to which the track was mapped. When executing, make sure the network is as small as possible to speed up.

mapmatching's People

Contributors

simonscheider avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mapmatching's Issues

License

This looks very promising. Are you able to apply a license to this code?

Install on linux

Is there a way to use this tool on linux? The installation provided only works for windows.

Can ArcCatalog 10.2 use the toolbox

I successfully installed gdal2.2.3 and networkx2.1. And I also followed the instructions to install mapmatcher-1.0.win32.exe, but I met the problem in the following picture.
2
I found nothing in mapMatch tool. My version of arcCatalog is 10.2. I would appreciate it if you can reply.Thanks.

Error: getSegmentInfo, 'OBJECTID' Field name

I had ran the python toolbox in ArcGis for the test data provided, it ran successfully

I was testing on one of my data, it threw the following error:
in getSegmentInfo, line 434, RuntimeError : a column specified that doe not exist.

I guess instead of using 'OBJECTID' as field name, we can use 'OID@' to access the objectid Field. ?

Thanks

Thanks for your code. I'm learning from it.

Environmental requirements

Dear Simonscheider,
I am very interested in this tool!
Could we use this tool on ArcGIS 10.5 in Windows 10 (64 bit)?

Many Thanks in advances,
Mat

ERROR 000358: Invalid expression Failed to execute (SelectLayerByAttribute)

I have successfully run the test file. But when I run my own data, I get an error.

Traceback (most recent call last):
File "", line 123, in execute
File "E:\bishe\data\mapmatching-master\mapmatcher\mapmatcher.py", line 269, in exportPath
arcpy.SelectLayerByAttribute_management('segments_lyr',"NEW_SELECTION", qr)
File "d:\arcgis10.2\desktop10.2\arcpy\arcpy\management.py", line 6494, in SelectLayerByAttribute
raise e
ExecuteError: ERROR 000358: Invalid expression Failed to execute (SelectLayerByAttribute)

Maybe the error is in this row of codes? 'arcpy.SelectLayerByAttribute_management('segments_lyr',"NEW_SELECTION", qr)'

Execution error

The test file runs fine. When I run my own data of bicycle GPS points, I get below error. Could you please help me figure out what is wrong?

Executing: mapMatch "C:...\Markov_mapMatch\1_Random_GPS_points_RAW_UTM_corrected.shp" "C:...\Markov_mapMatch\Road_network_utm16.shp" 5 10 50
Start Time: Wed Dec 26 18:43:52 2018
Running script mapMatch...
ERROR 000361: The name starts with an invalid character
Failed to execute (FeatureClassToFeatureClass).

C:...\1_ArcGIS\Bikeshare.gdb
1_Random__pth
Completed script mapMatch...
Failed to execute (mapMatch).
Failed at Wed Dec 26 18:46:17 2018 (Elapsed Time: 2 minutes 24 seconds)

Thanks.

AttributeError: 'Graph' object has no attribute 'edges_iter'

Hi!
I'm using mapmatcher python code, but when I run it with both my data and test data it gives me the attribute error (AttributeError: 'Graph' object has no attribute 'edges_iter'). I updated my networkx by pip install, but it still did not work and I got the same error. could you please let me know how I can solve it.
Thanks,
In the following you can see what I got in python:

track size:14
Number of segments: 745
C:\Users\amin.keramati\Desktop\tPRROMTO2\AminShp\testSegments.shp
graph size (excluding unconnected parts): 573

Traceback (most recent call last):
File "C:\Users\amin.keramati\Desktop\tPRROMTO2\mapmatcher\mapmatcher11.py", line 510, in
opt = mapMatch(trackname, roadname, 20, 10, 50)
File "C:\Users\amin.keramati\Desktop\tPRROMTO2\mapmatcher\mapmatcher11.py", line 87, in mapMatch
graph = getNetworkGraph(segments,lengths)
File "C:\Users\amin.keramati\Desktop\tPRROMTO2\mapmatcher\mapmatcher11.py", line 469, in getNetworkGraph
for n0, n1 in sg.edges_iter():
AttributeError: 'Graph' object has no attribute 'edges_iter'

XML Contains errors

Hi Simon!
I was trying to use the method as the ArcGIS tool. I did everything step by step just as told. Having successfully installed GDAL-2.2.3-win32 and networkx-2.1 and also the toolbox as well as those two .xml files. But when I tried to open it in ArcCatalog it failed and the "Description" of it says the item's XML contains errors.
My ArcGIS version is 10.5. I am now really frustrated by this. Could you review my problem and offer some advice? I would be utterly grateful!

New Apply

Thanks for reading my question.
I have lots of trajectories and road networks.(Both for Shang Hai, a big city in China).Now I want to associate every single Point in every trajectory with the road in road networks,rather than pick up the road.
In other word, I want to add a new filed,which is the id of the matched road to the point feature.
How can I achieve this goal by using your code?
Thanks very much for answering my question.

unable to load test data

I installed the mapmatch toolbox succesfully, but I get an Memory-error-message. Any idea how to avoid this?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.