lddl / ch Goto Github PK
View Code? Open in Web Editor NEWContraction Hierarchies (with bidirectional version of Dijkstra's algorithm) technique for computing shortest path in graph.
License: Apache License 2.0
Contraction Hierarchies (with bidirectional version of Dijkstra's algorithm) technique for computing shortest path in graph.
License: Apache License 2.0
Is your feature request related to a problem? Please describe.
Separate osm2ch tool from this repository.
It can reduce number of dependencies for people who do not need such feature in library at all.
Describe the solution you'd like and provide pseudocode examples if you can
Create new repository and copy all corresponding code into it.
Describe alternatives you've considered and provide pseudocode examples if you can
nope
Additional context
nope
I utilize the osm2ch command to generate the .csv file for Arizona roads. Then I exported the contracted graph with the following:
package main
import (
"github.com/LdDl/ch"
"fmt"
"os"
"bufio"
"io"
"strconv"
"encoding/csv"
)
func main() {
g := ch.Graph{} // Prepare variable for storing graph
fmt.Println("Preparing Graph")
graphFromCSV(&g, "data/graph.csv") // Import CSV-file file into programm
fmt.Println("Preparing Contracted")
g.PrepareContracts() // Compute contraction hierarchies
fmt.Println("writing to file")
err := g.ExportToFile("data/export_pgrouting.csv")
if err != nil {
fmt.Println(err)
return
}
}
func graphFromCSV(graph *ch.Graph, fname string) error {
file, err := os.Open(fname)
if err != nil {
return err
}
defer file.Close()
reader := csv.NewReader(bufio.NewReader(file))
reader.Comma = ';'
_, err = reader.Read()
if err != nil {
return err
}
for {
record, err := reader.Read()
if err == io.EOF {
break
}
source, err := strconv.ParseInt(record[0], 10, 64)
if err != nil {
return err
}
target, err := strconv.ParseInt(record[1], 10, 64)
if err != nil {
return err
}
oneway := record[2]
weight, err := strconv.ParseFloat(record[3], 64)
if err != nil {
return err
}
err = graph.CreateVertex(source)
if err != nil {
return err
fmt.Println("This is the Source")
fmt.Println(err)
}
err = graph.CreateVertex(target)
if err != nil {
return err
fmt.Println("This is the Target")
fmt.Println(err)
}
err = graph.AddEdge(source, target, weight)
if err != nil {
return err
}
if oneway == "B" {
err = graph.AddEdge(target, source, weight)
if err != nil {
return err
}
}
}
return nil
}
This seemed to work and produced a new file. But whenever I execute the .Shortest Route command it will always give me -1 for the ans. Here is what my code looks like:
package main
import (
"github.com/LdDl/ch"
"fmt"
"os"
"bufio"
"io"
"strconv"
"encoding/csv"
// "github.com/gin-gonic/gin"
)
func main() {
g := ch.Graph{} // Prepare variable for storing graph
fmt.Println("Preparing Graph")
graphFromCSV(&g, "data/export_pgrouting.csv") // Import CSV-file file into program
u := int64(1794303) // Define source vertex
v := int64(1787974) // Define target vertex
fmt.Println("Preparing Shortest Path")
ans, path := g.ShortestPath(u, v) // Get shortest path and it's cost between source and target vertex
fmt.Println(ans)
fmt.Println(path)
}
func graphFromCSV(graph *ch.Graph, fname string) error {
file, err := os.Open(fname)
if err != nil {
return err
}
defer file.Close()
reader := csv.NewReader(bufio.NewReader(file))
reader.Comma = ';'
// reader.LazyQuotes = true
// Read header
_, err = reader.Read()
if err != nil {
return err
}
for {
record, err := reader.Read()
if err == io.EOF {
break
}
source, err := strconv.ParseInt(record[0], 10, 64)
if err != nil {
return err
}
// fmt.Println("This is the Source")
// fmt.Println(source)
target, err := strconv.ParseInt(record[1], 10, 64)
if err != nil {
return err
}
// fmt.Println("This is the Target")
// fmt.Println(target)
// if intinslice(source, []int{5606, 5607, 255077, 238618}) == false && intinslice(target, []int{5606, 5607, 255077, 238618}) == false {
// continue
// }
oneway := record[2]
weight, err := strconv.ParseFloat(record[3], 64)
if err != nil {
return err
}
err = graph.CreateVertex(source)
if err != nil {
return err
fmt.Println("This is the Source")
fmt.Println(err)
}
err = graph.CreateVertex(target)
if err != nil {
return err
fmt.Println("This is the Target")
fmt.Println(err)
}
err = graph.AddEdge(source, target, weight)
if err != nil {
return err
}
if oneway == "B" {
err = graph.AddEdge(target, source, weight)
if err != nil {
return err
}
}
}
return nil
}
Do you have any ideas what could be causing this?
Is your feature request related to a problem? Please describe.
Let's take a laptop:
goos: windows
goarch: amd64
pkg: github.com/LdDl/ch
cpu: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
Run test (with debugged flag ON) on it for this Graph file containing ~211k edges:
go test -timeout 30s -v -run ^TestShortestPath$ github.com/LdDl/ch
....
--- PASS: TestShortestPath (10.32s)
...
Bassicaly it has taken ~10seconds to preprocess graph. It's not that bad. But.
Let's take another Graph file containing 251k edges:
go test -timeout 5000s -v -run ^TestShortestPath$ github.com/LdDl/ch
....
--- PASS: TestShortestPath (4240.13s)
...
Now it's about 70+ minutes. I guess the problem with that graph is (a) topology + (b) a lot of Dijkstra function calls during contraction process on last ~10k steps in priority queue (vertices sorted by importance).
Something needs to be done with preparing contraction hierarchies: improve heuristics (I don't think that ~[edges_num*2] number of generated shortcuts is a good ...) / delete unnecessary calculations / refactor code (probably find hidden bugs)
Additional context
Used heuristics currently:
// computeImportance Update importance of vertex
func (vertex *Vertex) computeImportance() {
// Worst possible shortcuts number throught the vertex is: NumWorstShortcuts = NumIncomingEdges*NumOutcomingEdges
vertex.shortcutCover = len(vertex.inIncidentEdges) * len(vertex.outIncidentEdges)
// Number of total incident edges is: NumIncomingEdges+NumOutcomingEdges
vertex.incidentEdgesNum = len(vertex.inIncidentEdges) + len(vertex.outIncidentEdges)
// Edge difference is between NumWorstShortcuts and TotalIncidentEdgesNum
vertex.edgeDiff = vertex.shortcutCover - vertex.incidentEdgesNum
// [+] Spatial diversity heuristic: for each vertex maintain a count of the number of neighbors that have already been contracted [vertex.delNeighbors], and add this to the summary importance
// note: the more neighbours have already been contracted, the later this vertex will be contracted in further.
// [+] Bidirection edges heuristic: for each vertex check how many bidirected incident edges vertex has.
// note: the more bidirected incident edges == less important vertex is.
vertex.importance = vertex.edgeDiff*14 + vertex.incidentEdgesNum*25 + vertex.delNeighbors*10 - vertex.bidirectedEdges()
}
2 .Lazy update heuristic:
// update Importance of vertex "on demand" as follows:
// Before contracting vertex with currently smallest Importance, recompute its Importance and see if it is still the smallest
// If not pick next smallest one, recompute its Importance and see if that is the smallest now; If not, continue in same way
vertex := heap.Pop(graph.pqImportance).(*Vertex)
vertex.computeImportance()
if graph.pqImportance.Len() != 0 && vertex.importance > graph.pqImportance.Peek().importance {
graph.pqImportance.Push(vertex)
continue
}
What is your docs question about? Ask it
I assumed ShortestPathOneToMany
used some clever algorithmic trick (like unidirectional Dijkstra is able to calculate all targets for one source in one go). Now, having read the algorithm source code, it seems to be almost the same as executing ShortestPathOneOne
for all targets in sequence. The only difference I can see is that it is reusing the data structures across iterations. Is it only about reducing allocations or is there some deeper optimization that I cannot spot?
What do you suggest?
Add a section to README about how ShortestPathOneToMany
works and why it is faster.
Additional context
None
Is your feature request related to a problem? Please describe.
Would be cool to find shortes path with max cost restriction option.
Describe the solution you'd like and provide pseudocode examples if you can
pseudocode:
fn ShortestPath(from, to, max_cost) {
if distance > max_cost {
stop
}
}
Describe alternatives you've considered and provide pseudocode examples if you can
nope
Additional context
It would be usefull for path estimation when we know max cost.
Example application: If I can't spend 5 minutes to reach destination point of interest on pedestrian based-graph, then I choose vehicle-based graph.
Is your feature request related to a problem? Please describe.
Consider extension of current implementation with customization
Describe the solution you'd like and provide pseudocode examples if you can
Prepare new branch(-es) and do some testing stuff in it
Additional context
Links to original papers:
Is your feature request related to a problem? Please describe.
Currently field
type Vertex struct {
...
orderPos int
...
}
is unexported.
For extended debugging in external appliactions and import/export function we need to make getter/setter for it.
Describe the solution you'd like and provide pseudocode examples if you can
Pretty simple getter/setter combo:
func (vertex *Vertex) OrderPos() int {
return vertex.orderPos
}
func (vertex *Vertex) SetOrderPos(orderPos int){
vertex.orderPos = orderPos
}
Describe alternatives you've considered and provide pseudocode examples if you can
Nope
Additional context
This issue is related to #11
Is your feature request related to a problem? Please describe.
Extend the way we store edges/vertices data.
Describe the solution you'd like and provide pseudocode examples if you can
Exported CSV is human readable, but we need to make an extension for this to store source metada.
Also we should add metadata fields for vertices/edges and (may be?) restrictions. Some getters/setters workaround is needed.
Describe alternatives you've considered and provide pseudocode examples if you can
Nope
Additional context
It's would be good to store metadata in JSON'ish string or other popular "key-value" format.
Is your feature request related to a problem? Please describe.
Current implementation of Export function is: https://github.com/LdDl/ch/blob/master/export.go#L19
It creates single file with informations about edges and contractions, but no info about vertices' order position and importance.
Single CSV-file header:
// from_vertex_id - int64, ID of source vertex
// to_vertex_id - int64, ID of arget vertex
// f_internal - int64, Internal ID of source vertex
// t_internal - int64, Internal ID of target vertex
// weight - float64, Weight of an edge
// via_vertex_id - int64, ID of vertex through which the contraction exists (-1 if no contraction)
// v_internal - int64, Internal ID of vertex through which the contraction exists (-1 if no contraction)
So after calling Import function we must evaluate every vertex's order position and importance again: this is not cool and can take a long time if graph too big.
We need to export information about order position and importance to remove unnecessary calculations inside of Export function
Describe the solution you'd like and provide pseudocode examples if you can
Export function has to create 2 files: one is for edges, and second one is for vertices.
Edges file:
// from_vertex_id - int64, ID of source vertex
// to_vertex_id - int64, ID of arget vertex
// f_internal - int64, Internal ID of source vertex
// t_internal - int64, Internal ID of target vertex
// weight - float64, Weight of an edge
// via_vertex_id - int64, ID of vertex through which the contraction exists (-1 if no contraction)
// v_internal - int64, Internal ID of vertex through which the contraction exists (-1 if no contraction)
Vertices file:
// Header of main CSV-file containing information about vertices:
// vertex_id - int64, ID of vertex
// internal_id - int64, internal ID of arget vertex
// order_pos - int, Position of vertex in hierarchies (evaluted by library)
// importance - int, Importance of vertex in graph (evaluted by library)
Describe alternatives you've considered and provide pseudocode examples if you can
We can go even further and export third file: about contractions.
So finally we can get such structure:
Edges file (only real edges!):
// from_vertex_id - int64, ID of source vertex
// to_vertex_id - int64, ID of arget vertex
// f_internal - int64, Internal ID of source vertex
// t_internal - int64, Internal ID of target vertex
// weight - float64, Weight of an edge
Vertices file:
// Header of main CSV-file containing information about vertices:
// vertex_id - int64, ID of vertex
// internal_id - int64, internal ID of arget vertex
// order_pos - int, Position of vertex in hierarchies (evaluted by library)
// importance - int, Importance of vertex in graph (evaluted by library)
Contractions file:
// from_vertex_id - int64, ID of source vertex
// to_vertex_id - int64, ID of arget vertex
// f_internal - int64, Internal ID of source vertex
// t_internal - int64, Internal ID of target vertex
// weight - float64, Weight of an contraction
// via_vertex_id - int64, ID of vertex through which the contraction exists
// v_internal - int64, Internal ID of vertex through which the contraction exists
Additional context
After moditification of Export function we should make proper update for Import function also. So Import function has to accept 2 (or 3, depends on what we are going to do with contractions) arguments.
What is your docs question about? Ask it
We need visual step-by-step explanation of what happens in graph during CH algorithm.
What do you suggest?
Take something like Graphviz, draw graph and steps to reproduce CH algorithm and bidirectional search.
Additional context
Fill wiki page - is a good idea too.
Describe the proposal
Since last set of PR's we need to run gofmt
Additional context
Probably make it via VScode
Also it could be cool to check if we need order alignment in structs
Is your feature request related to a problem? Please describe.
Currently field
type Vertex struct {
...
importance int
...
}
is unexported.
For extended debugging in external appliactions and import/export function we need to make getter/setter for it.
Describe the solution you'd like and provide pseudocode examples if you can
Pretty simple getter/setter combo:
func (vertex *Vertex) Importance() int {
return vertex.importance
}
func (vertex *Vertex) SetImportance(importance int){
vertex.importance = importance
}
Describe alternatives you've considered and provide pseudocode examples if you can
Nope
Additional context
Nope
Is your feature request related to a problem? Please describe.
Isochrones in terms of GIS - https://wiki.openstreetmap.org/wiki/Isochrone
This is nice instrument for transport accessibility analysis.
Describe the solution you'd like and provide pseudocode examples if you can
Implement shortest path finding algorithm with cost restristions (issue is #7 )
Wrap implemented function inside something like this (pseudocode):
Isochrones(source, max_cost) []int64{
targets := graph.GetAllVertices
isochrones := map[int64]bool
answer = []int64
for target in targets {
if target == source || single_vertex is in isochrones {
continue
}
_, single_isochrone_path = ShortestPath(source, target, max_cost)
for single_vertex in single_isochrone_path {
if single_vertex is not in isochrones {
isochrones[single_vertex] = true
answer = append(answer, single_vertex)
}
}
}
return answer
}
Describe alternatives you've considered and provide pseudocode examples if you can
There is alternative solution:
Just iterate all neighbor vertices on each step and collect them while estimated cost < max_cost
It's just a bit modified (with provided max cost) BFS - https://en.m.wikipedia.org/wiki/Breadth-first_search
Additional context
Nope
Describe the bug
The benchmarks (like BenchmarkShortestPath) execute 12 iterations with k = 0..11
, calculate an n = 2^k
, but then don't use either of them in the benchmark except for the title.
To Reproduce
Run the benchmarks and read source code.
Expected behavior
Either n
or k
should be used in the benchmark core (not sure what these could be for BenchmarkShortestPath
, for BenchmarkShortestPathOneToMany
it could be the number of target nodes).
Is your feature request related to a problem? Please describe.
When preprocessing is done need to prevent graph from any kind of data modification.
So users/developers won't make mistake trying to insert vertices to graph after it had been contracted.
Describe the solution you'd like and provide pseudocode examples if you can
go-ish pseudocode:
type Graph struct {
frozen bool
}
func (g *Graph) ModifyData(....) error {
if g.frozen {
return "can't modify data in frozen graph"
}
}
func (g *Graph) PrepareContracts() error {
{
// graph preparation
}
g.Freeze()
}
// Freeze - Exported
func (g *Graph) Freeze() {
g.frozen = true
}
// unfreeze - unexported
func (g *Graph) unfreeze() {
g.frozen = false
}
Describe alternatives you've considered and provide pseudocode examples if you can
nope
Additional context
nope
What is your docs question about? Ask it
Well, there are many code with some unclear priority queue stuff, such as:
...distance.distance = vertex.distance.distance + cost
...distance.contractionID = contractionID
...distance.sourceID = sourceID
What do you suggest?
// checkID Checks if both source's and target's contraction ID are not equal
func (graph *Graph) checkID(source, target int64) bool {
s := graph.Vertices[source].distance
t := graph.Vertices[target].distance
return s.contractionID != t.contractionID || s.sourceID != t.sourceID
}
Additional context
nope
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.