It feels like we're at a point to start figuring out how to represent turns / movements through intersections in osm2streets. (I've gone through this exercise before for A/B Street and made some questionable choices there, based on assumptions about how lane-changing should work in a traffic simulation. I'm viewing this as an opportunity to try again, with fresh eyes + experience + more discussion!)

How A/B Street does it: basic schema

A turn is uniquely identified as a (source lane, destination lane, intersection). The intersection seems redundant, but it's needed to talk about pedestrian movements. https://a-b-street.github.io/docs/tech/dev/formats/traffic_signals.html explains why.

Each turn has a type. For vehicles, it's straight / right / left / u-turn. There can be more nuance than that (5-way intersections, slight right). Distinguishing these types was partly helpful in writing heuristics to generate traffic signal configurations -- things like saying "at a 4-way intersection, the straight turns for the north & south roads can all go. The left turns have to yield." But defining these based on relative road angles was always brittle and hit edge cases, and I remember trying to come up with an alternate way to think about it... in a left-handed driving place, a left turn is one that conflicts with less turns than a right turn.

Turns conflict very "coarsely." A turn has a line-string of the movement through the intersection, and if those intersect, then they conflict. I think I've only needed the concept of conflicting turns in traffic signal configurations and traffic sim, but it may be a concept more widely useful.

Consequences

Defining turns at the lane level gets kind of weird.

Changing lanes in the middle of an intersection is illegal (I think in the US, possibly everywhere, whatever). But from this left lane, A/B Street has a straight turn going to either lane. I did it this way to avoid the absolute mess of simulating lane-changing, but it led to other problems in the simulation anyway. There has to be a way to penalize changing lanes like this, so by default vehicles only do it if necessary.

Sometimes 3 lanes lead to 2 and someone has to merge though. Or vice versa, one turn lane opens up onto 3 lanes, and there's a choice. Something has to say what turns are legal, and maybe prioritize them.

Lane types

Things get weirder with roads that have bus or bike lanes. A/B Street looks like this:

A bus in the right lane can turn left, cutting across a bunch of lanes. This is incorrect here and maybe in many cases, but... not always. Sometimes there are traffic signals configured to let bikes or buses start first and possibly do weird movements like this.

Movements

At some point, working at the lane granularity got really hard for traffic signals. It was better to think about the entire "directed road segment" -- aka a bundle of lanes pointed the same way. A movement was defined at this higher level, and it bundled together some turns. It looks like this:

These probably need to be refined to talk about just the bus or bike movements too, in some cases.

How this is all generated

Messily! https://github.com/a-b-street/abstreet/blob/master/map_model/src/make/turns.rs does roughly this:

1. Look at every source and destination lane at an intersection, create a turn linking them
2. Do some bezier curve magic by Marcel to create geometry
3. Classify into straight/left/right/u-turn
4. Attempt to filter out turns based on different types of turn restrictions. Sometimes that process breaks, so the logic is complicated -- it'll give up on filtering and just keep the original things, instead of "orphaning" a lane. Every lane should lead somewhere, and should be reachable from somewhere.

There are 3 types of turn restrictions.

1. "Simple" ones are relations between OSM ways. Allow or ban turns from road1 to road2
2. "Complex" ones are relations involving multiple OSM ways. I only supported one "intermediate" way, but meant to support more. I can pull up some examples somewhere, but these're used for things like U-turns in complex intersections / dual carriageways. The consequences of these at a routing level is quite complex -- you have to replace part of the graph with "uber-turns" that describe a sequence of movements.
3. Lane-level restrictions. Once we know road1 can go to road2, specifically what lanes can do what?

Pedestrian movements

Most of the above is about vehicle turns. Pedestrians have "crosswalks" and "unmarked crossings" through intersections, the latter of which isn't well supported. Those're meant to be the difference between explicit zebra crossing type things, and movements which are not physically signed but are either legal or definitely performed in practice.

There's also "shared sidewalk corners" for lack of a better term, where the pavement extends through the intersection and doesn't cross any other turn.

Testing

Done very poorly in A/B Street right now. Aside from the interactive turn explorer UI shown in some screenshots above, I have some snapshot tests recording all the generated turns for a few test cases, and I can tell when a code change modifies these. Then I'd use the UI and manually look.

So now what?

There's lots above, and it doesn't even cover things like conditional restrictions (no turns during peak hours, except for buses). Anybody have thoughts about what we should do for osm2streets or how to proceed?

Today we have roads (individual lanes, each a thickened line-string) and intersections (a polygon, with each lane polygon meeting it at a right angle). That's a vast oversimplification. https://dabreegster.github.io/talks/map_model_v2/slides.html#/crossing-islands has some images of more situations, like...

• crossing islands / floating bus stops. Sometimes these are roughly linear along the road and could act like a thick sidewalk, but often the shape is weird
• lanes with variable thickness. "Pocket parking" doesn't just appear at a sharp right angle to the rest of the road; there's some kind of smooth curve in. And when turn lanes appear, it's gradual. Same for bulb-outs / curb bulbs.
• Advanced stop lines / bike boxes. The stop line for most vehicles is much farther back from the intersection, and the front is dedicated to smaller vehicles. Do we end the regular vehicle lanes early and count that as part of the intersection maybe?
• Modal filters. Sometimes these're defined as nodes / points in OSM on a way. Other times, the way gets split into a short segment with access restrictions marked. We probably need to normalize between these two and decide a representation.

@tordans, any opinions on representing these things? From my understanding of how strassenraumkarte works, you're working with geometry only, so there's no internal representation about these kinds of objects. With a stop line, you would just trim back roads appropriately and draw extra things. Right?

Like osm2lanes, we should get osm2streets-web hosted right away.

Feel free to pick this up!

As illustrated in the GIF below.

Should they be unticked to start with? I guess not, the road polygons are IMO one of the most useful features.

This has been started, but there's a great many places it needs work. Just starting to write down next steps / problems here.

1. https://www.openstreetmap.org/way/668997030 and nearby is an example in Leeds
2. https://www.openstreetmap.org/node/5739261534 is a simple one in London. The cycleway just rejoins the main road on one end, so the current pattern matching doesn't pick it up.

Hello, I have imported the json data and the area data with intersections are showing. I would like to export all that processed data as geojson or shp file.
Please help me on that. Thank you and waiting for your reply

@BudgieInWA, I'm thinking about steps to moving raw_map and convert_osm crates into this repo. Some of the steps would be:

1. Internally split RawMap into some kind of StreetNetwork struct, just with the data/concepts that should live in this repo. Things like buildings, areas, parking lots, the map name, etc would stay in the A/B Street RawMap.
2. Internally detangle convert_osm into two pieces -- one that should be in this repository, and the A/B Street specific stuff.
3. Split both into separate crates, then move them into this repository.

There would be a dependency loop between the two repos, but I don't think it's a problem, because the crates wouldn't be in a loop. The crates here would only depend on geom. The A/B Street side would pull in the osm2streets crates, of course.

The convert_osm split will be the more interesting one. There's generic OSM parsing stuff in there. I'll try to keep it as separate as possible -- maybe it's worth a crate of its own, or maybe it's a chance to switch over to something like osmio and get native PBF reading too.

Concerns

This will increase my maintenance burden. When this code changes significantly, I regenerate all maps over in A/B Street. To test things temporarily, I'll switch the local dependency in abstreet from this git repo to my local copy. This is a cost that has to be paid eventually, and now's as good a time as any.

But to double check, is this the direction we want to go? We had talked about exploring new schemas and big ideas here and eventually converging, but the current work is building off RawMap. I'm eager to try this refactor, but if there's a good chance we won't want it later, no need for churn.

Questions

1. What should things be called? RawMap has always been an awkward choice, just meant to contrast A/B Street's "final" map model. I think we should keep RawMap in abstreet (because otherwise there's lots of churn renaming files and stuff there) and call the osm2streets bit something else. StreetNetwork is my strawman.

2. Most of the data in RawRoad and RawIntersection looks relevant to move here. The only weird bit is elevation per intersection and percent_incline per road. That's useful in abstreet, but is only populated for a few maps now. OK with keeping the fields in this repo and just not using them much?

Out of scope for now

Not moving map_editor here. It's a useful UI, but has abstreet stuff to create buildings and lots of file IO. An it depends on widgetry, which I don't want to tease out yet.

Also not moving the full importer tool/pipeline here yet. It does generally useful things like find the right geofabrik osm.pbf file for a boundary, but it has loads of abstreet-isms. We're already calling convert_osm here on .osm files -- that's the super simple version of this pipeline.

Also not yet figuring out rendering (#24). There are still big unknowns with cutting over to osm2lanes first (to reap benefits of its separators) and figuring out SVG / widgetry output.

If we want to pursue .dot as a visualisation tool, we could get pretty snazzy with it. We could encode a lot of information in the visual representation. Rendering it to svg in a browser also allows it to be interactive, which opens up a lot of options.

• Setting node pos might result in accurately graph layouts.
• Setting edge "weight" equal to (trimmed) length might be a good hint for certain layout algos.
• Use node style to represent Intersection Type
• use line styles to show number of lanes, etc.
• eventually use arrow heads to represent yield type.
• spell out attributes in the hover text somehow?

The source of truth for a lot of the state of the web-app should be in the URL. This has a few benefits:

• your place is not lost if you reload,
• you can share "deep links" to specific data and setups,
• we can programatically generate links to specific app states,
• the OSM Smart Menu can be used to link to/from the tool.

The history.pushState and .replaceState allow us to do this without reloading.

Specifics I can think of now:

• the currently loaded test case in the path
• the current layer configuration (when we have controls for hiding layers, etc.) in the query
• the current map position and zoom in the hash (like osm.org)

This conversation started at a-b-street/abstreet#789 (search term "separate" and "multiple OSM ways")

The main question is:

…how will the library figure out if separate ways are geometrically located to the left or right of the main road?

Preprocess data

I think we should continue the search for a solution for this. AFAIK there is no way to solve this in JavaScipt/Node. But it is possible to solve it in QGIS and likely the underlying Python libraries.

So it should be possible to preprocess the data in a way, that osm2lanes can then work with.

I think we should check if this could be solved (in a performant way) with Osmium or osm2pgsql or Overpass, … – I don't know enough about those tools ATM.

I assume, one of those tools would be able to merge the separately mapped ways to the central way. This could be done just by proximity, but I assume that would be error prone. But there are tags, that we can use, to reduce the error rate:

Sidewalks

• highway=footway + footway=sidewalk on the separately mapped way is a strong indicator, that the way belongs to the main road

• sidewalk=separate/sidewalk:(left|right|both)=separate on the main road are a strong indicator, that there is a sidewalk to look for nearby

Cycleways

• is_sidewalk=yes on the separately mapped way is a strong indicator, that the way belongs to the main road. This tag is still experimental, though, but could help a lot. We are experimenting with it to improve routing for footways and cycleways, more on this wiki page footway mapping (GER) and this wiki page on cycleways mapping (GER) The additional experimental tags is_sidepath:of=(secondary|residential|<HighwayValueOfMainRoad) and is_sidepath:of:name=<NameValueOfMainRoad> might also be useful, but are likely better automatically processed by the script.

• cycleway=separate/cycleway:(left|right|both)=(separate|optional_sidepath) on the main road are a strong indicator, that there is a cycleway to look for nearby (Germany wiki page)

Splitting from a-b-street/abstreet#139. The goals here are just:

1. connectivity -- can bikes successfully route over things like the Burke Gilman as expected
2. geometry + rendering -- does it look reasonable
3. bike-only -- just model as one or two bike lanes, ignore foot access for the moment. (Once this bug is solved, a-b-street/abstreet#139 can add a special lane type that has mixed access)

We now have a Cargo.lock for the library itself, for the tests, and for the web test browser. Workspaces would let us share build artifacts and reduce maintenance burden of updating dependencies in 3 places

Let's start the equivalent of osm2street-js, but for R. I have a small amount of experience either from zonebuilders or odjitter, using https://extendr.github.io/rextendr/.

If the driving side is set to "left", the values in bus:lanes are unexpectedly flipped.

Example: Go here, change the driving mode to left. Note how the bus lane is on the very right, when it should be on the very left.

the value of bus:lanes=designated|yes|yes should not be flipped in left-hand mode

Another example, without dual carriageway: The bus lane should be on the southern side, not in the middle

bus:lanes:backward=designated|yes
lanes=3
lanes:backward=2
lanes:forward=1

We want to set up a tile server #12 which can render our data, so;

It would be nice to have a slippy map with those layers to cycle through at a press.

When the test browser loads a test, all of the different files are displayed on the map. With adequate styling, many of these layers can be viewed at the same time, but that will often be too much at once.

Buttons should be added to the map to toggle the visibility of loaded layers (or switch between preset combos or something).

I want to turn tests-web/ into a static vanilla JS web page with as simple an architecture as possible.

I plan to remove all of the rust aspects of the web app, so that trunk does nothing except serve the pages.

Unlike trunk serve, the deployment to GitHub Pages is served as a static site, and at a path below the root (/osm2streets/). The JS that is generated by trunk build is put in the right place and linked to correctly, but the JS that I have written is included as a static resource.

We need to refer to these resources with the correct relative path: js/blah.js.

We cannot use pretend paths like ./t/test_name/ to encode which test is open, because a static site host wont serve ./index.html for that path (which is where app is, with the logic to parse the path and load the appropriate test). The baseline approach should be to use the query string for the current test name: ./?test=test_name.

(More of a question/discussion/know how share-issue.)

The output at https://a-b-street.github.io/osm2lanes/ does not include geometries. I assume that the idea is, that osm2lanes only handles tags, not geometries?

I wonder, what the ideal process to render the data on a map would be. And if this might require a secondary shared project lanes2map.

What are your thought on how to render the data on a map.
I assume ab-street transform the lane-data into areas and paints those?
Is there a feasible way to instead use the lane-width to calculate lat/lng values with an offset based on the center line (respecting #6).
Both approaches require to handle curves properly (but I have no idea how ;-)).

We want to be able to do something like

1. download a .osm of a small area,
2. fill in one of the expected outputs somehow, and
3. get the test to run.

Having test cases in place will be useful even before any implementations show up.

I can imagine fancy tests having these attributes:

• bounds: real world examples
• .osm xml: the example input
• network graph: rawmap graph structure and properties
• rawmap geojson: for snapshot testing/goldenfile testing
• ground truth: orthographic imagery for context when comparing snapshots

Would love to be able to write code against a network graph from .osm test.

After initially turning .osm.xml into a StreetNetwork, the next step is usually to do run_all_simplifications. Right now this runs through a fixed sequence, taking a few boolean options. I've recently added a new user of osm2streets at https://github.com/dabreegster/bus_spotting, and requirements there are simpler -- I'd love to skip some expensive steps for collapsing degenerate intersections (ahem -- I mean IntersectionComplexity::Connection).

The simplest refactor would introduce a struct with options for each transformation, along with reasonable defaults. But potentially in the map_editor UI or the new StreetExplorer web app, it'd be useful to specify even more detail -- like running a single transformation step at a time, to step-through debug stuff. Maybe an enum of transformation steps (with some of them maybe taking more configurable parameters) would be more flexible, and then we take a sequence of these to execute.

One of my LTN collaborations actually has some pesky geometry issues that look simple to deal with.

https://www.openstreetmap.org/way/291394487#map=19/51.45946/-2.55068

The dual carriageway exists for a very small space due to a crossing island. In the short term, simplifying this to a regular bidirectional road with nothing special in the middle is appropriate. (I would love to figure out how to properly model crossing islands like this, but that's a later step.)

These look straightforward to detect and transform. I'll only try to handle these loops, where road1 and road2 share both endpoints. This means there can't be any other roads connecting to one side or the other.

CC @martinfleis

https://www.openstreetmap.org/relation/8099562

Not sure if this is an OSM data issue but, this ain't right:

Let's start the equivalent of osm2street-js, but for Java. At an extremely cursory glance, https://github.com/jni-rs/jni-rs looks popular, but there are likely many options out there

Let's start the equivalent of osm2street-js, but for C++. I haven't looked through the different options for doing this yet.

CC @wxinix

https://2022.stateofthemap.org/programme/ is very close, and I have about one week left to get things done here. I'm going to open and merge PRs quickly and handle feedback after the fact, if that's alright. I'll use PRs to explain the changes and any new challenges that come up.

My goals for this week:

• get a debug mode going, where you can step through transformations in the web UI and we can explain things in more detail. (It's also incredibly helpful to actually work on the harder transformations)
• actually merge some cases of dual carriageways. I have a WIP branch where I can detect many pieces of a complex example; I'm reasonably confident I know how to merge it now.
• attempt the snap-cycleway-to-the-road problem again, restricting myself to very simple and achievable cases
• get more detailed lane rendering in the web UI
  • I think that will require adding the concept of lane-level turns to osm2streets. I have in mind a simple UI to interactively inspect that.
• rewrite the project README / add better docs
• stretch: wire up Python and/or Java bindings just as a proof of concept / to get things started

The geojson output like this is a nice start, but at some point, I'd like also to put a proper API around the rendering that A/B Street does. That would include different lane dash markings, icons for bus/cycle lanes, and turn arrows.

Desired output

What should the output of this render API be? We'll want to draw it, but probably in different ways. GeoJSON with attributes for color are a lightweight option, and we've got code to generate polygons for things like dashed lines if needed. SVG as output could make more sense. To keep using the code in A/B Street itself, I'd want to keep using GeomBatch (a list of triangulated polygons with a color) for performance and not have a bunch of extra translation layers.

Option 1

Incrementally change the current A/B Street code to have the desired API. https://github.com/a-b-street/abstreet/blob/master/map_gui/src/render/lane.rs has most of the current logic (and it's a mess, of course). But there'd be complications with expressing this code on top of RawMap... the rendering is written on top of a "higher-level" Map struct, where things like turns between lanes are defined. Another limitation with the current code is styling -- there's not many locale specific overrides yet for things like dash color/style.

Option 2

Start writing something from scratch, using existing code as reference when useful. I think this makes much more sense, especially since some of the problems mentioned above are already being solved at the osm2lanes layer. The output already includes separators that have both semantic and visual descriptions.

I would still want to cut A/B Street code over to this new rendering, and that means I have to think through how to make the final Map structure look like a RawMap, but I'm not too worried about that yet -- this is a good opportunity for me to think through the difference between those two representations and maybe make the street network part, at least, be a little closer to each other.

Users

1. A/B Street
2. the test browser
3. a "soon"-to-exist web app that dynamically fetches OSM data from Overpass in an area, pipes through osm2streets, and renders on the slippy map

And also @wxinix expressed interest in building a traffic signal tool on top of some of this. If we had an API that took some area you select from OSM and handed back geometry and rendering, what form would you want the image to be (GeoJSON, SVG, something else)?

This repo contains an independent test suite that testsRoadNetwork and abstreet's RawMap. Each test case is a real world example with a lot of geographic data.

We should build a slippy map that shows the various formats of data for a test case, layering the .osm with the .geojson and linking the .dot. A simple menu would let you choose a test case to view, and highlight tests that need approving.

It could then be extended to run tests on demand, downloading .osm on the fly.

Footways detached from roads are needed for two cases

(1) important footway that offers a shortcut accessible solely for pedestrians, used also in areas that use sidewalk tag to mark sidewalks

(2) in some regions (for example Poland) sidewalk tag is generally not used, sidewalks are tagged by mapping a separate highway=footway. In some cases such separately tagged footway had footway=sidewalk tag and road gets sidewalk=separate (though not always!).

Showing such detached footways is likely needed for (1) in order to get high quality simulation and is likely tricky/clunky for (2), though still would be immediate improvement for areas with such tagging.

Preserving Ben's idea from a-b-street/abstreet#897:

One interesting mode of operation would be to output RawMap as OSM xml in some "normal form": consistent tagging, all geometry running down the center of the road width, intersection:areas (or whatever they are called) for intersections, etc. without worrying about the connectivity. I would be interested to see what it looks like using those outputs directly in existing tools like Map Machine. It might be a good way to integrate with such tools.

I would like to have the ability to work on real examples, and tap into the tests, as my next step. The tests crate is already doing the reading and parsing of .osm files, by importing from abstreet, so outputting RoadNetwork instead of RawMap shouldn't be too far away.

I think I want the initial RoadNetwork to be split up into as many pieces as needed, with intersections broadly categorised (incomplete, terminus, slice, intersection). That way, transformations can work mostly on combining the pieces into bigger structures, and collapsing unnecessary detail; I feel like doing all the splitting before all the combining might keep things clear.

Two options:

1. Import reader, re-implement extract_osm

It is tempting to add a road_network = osm_document.into() type thing to the clean slate implementation. It looks like the necessary algorithms are in extract_osm and split_ways (at least), which could be translated into osm2streets types instead of RoadNetwork types (cutting out a lot of abstreet specific concerns). This requires some effort.

2. Import convert_osm, load .osm as RawMap then translate it into RoadNetwork

A road_network: RoadNetwork = raw_map.into() type thing, that we can run after splitting ways (etc.) might be just as useful. There is a lot of logic in that extraction process that we don't need to revisit yet, but which we might want to use as the basis for the eventual solution. So it should be fine that the clean slate view of things uses RawMap as an implementation detail of the .osm to most-basic-RoadNetwork step.

RawMap has an awkward API today.

1. roads_per_intersection is a method that does a linear search for basic graph functionality. Annoying and slow, but not the most urgent problem.
2. RawRoad has center_points: Vec<Pt2D>, representing the raw geometry from OSM. There are untrimmed_road_geometry and trimmed_road_geometry methods that run through the intersection geometry algorithm to trim the road on both ends.

If this is going to be a nice graph+geometry representation, the 2nd especially will become annoying quickly. It's actually already a problem trying use the map_editor to debug particularly weird geometry. You would hope that what's shown there matches the fully imported map, but there are discrepancies sometimes -- #62. I haven't looked into this in quite a while, but I think it might have to do with ordering. We trim every road twice, for both intersections. When we do a full import, the order of trimming doesn't match what happens in the map_editor UI, especially when we're modifying things and recalculating only some intersections.

And then there's InitialMap. It sort of solves both problems above.

So my task: keep the OSM center_points on RawRoad, but add trimmed_center_pts there and see if we can totally do away with InitialMap. It should simplify code and make the map_editor match the imported map in those niche cases. It'll also increase RawMap file size a bit. If it's much, we can avoid serializing and fill it out from center_points when we load. (The only typical place saving a RawMap is convert_osm, which creates a sort of "unsimplified / untransformed" RawMap)

CC @BudgieInWA

We want to make it as easy as possible to experience the result of the work done here, so;

Let's create a tile server that can render tiles right in rust or through some other means. That way, anyone can look at it in their tool of choice.

• find a library/mechanism for rendering shapes to raster nicely
• implement rendering osm2streets data to tiles (raster or vector) on request
• set up a tileserver api (WMS, TMS, etc.) that provides access to

In future, we could glue on a "download small region" step which downloads .osm xml from osm.org, or whatever data source you want to plug in. Rendering raster will be useful in JOSM too, so we don't have to do it in Java

This is an issue just to keep track of other related work.

https://gisintransportation.com/media/zsnn3oou/2022_04_19-gis-t-workshop-aegist-slides.pdf, see page 18. They have a nice classification of some complex junctions:

Here are some thoughts about working with roads.

https://github.com/a-b-street/abstreet/blob/6174bfc06b3b1026990fcbad135d990dddf0b178/map_model/src/objects/road.rs#L603-L627
https://github.com/a-b-street/abstreet/blob/6174bfc06b3b1026990fcbad135d990dddf0b178/map_model/src/objects/road.rs#L313-L316

... need to still handle lanes going outward from the "center" line ...

This is also where my conceptualisation has lead me too -- I hope that is a good sign. I keep finding that combing driving side with driving directions with way directions exploded the possibility space. Any technique that helps remove any of those options is great, only dealing with one direction at a time should half the possibility space for some problems.

To be independent of driving side, I too started counting lanes from the centre. I used something like

struct DirectedLaneNumber(i8);
impl fmt::Display for DirectedLaneNumber {
	pub fn fmt(&self, f) {
		if self.0 == 0 {
			write!(f, "both ways lane, in the center")
		} else {
			write!(f, "{} Lane {} (from the inside)", if self.0 > 0 { "forward" } else { "backward" }, self.0.abs())
		}
	}
}

Restricting the representation to one major center line and working with two independent sides, each with their own "half width", a single direction (even if some lanes point backwards), etc. makes for easy reasoning about the graph I think:

Classifying an intersection

When I am trying to decide what an intersection is, I'm looking at: a single node or a clockwise ring; and a clockwise list of road halves with stop lines. They could come in in any number of ways in OSM, so we have to pair up road halves into "roads" ourselves. In fact, road halves can even be made up of multiple roads (if the road is splitting for real, like a freeway exit).

Counting up the centerlines gives us different classes of situations that need "intersection" representation (seems too specific of a term... maybe it's "node type"):

let roads = // match up half roads into one road per "center line"
let num_minor = roads.iter().count(|r| r.is_yield())
let num_major = roads.len() - num_yields;
match (num_major, num_minor) {
	/// Turning circles, road end signs, train terminus thingos, edge of the map?
    (ma, mi) if ma + mi == 1 => Terminus,

    /// A perpendicular line across a road, where conditions change.
    // Pedestrian crossings have actual width, but if conditions change instantaneously, 
    // it's the same shape as a crossing, but with width=0.
    (ma, mi) if ma + mi == 2 => Crossing,

	/// A "major" road crosses one or more "minor" roads, which yield to it.
	// I wonder what more than two major incoming roads look like? Dangerous? Missing yield signs, most likely?
    (2, mi) => Priority,

	/// An area of the road where priority is shared over time, by lights, negotiation and priority, etc.
	// You would expect normal lane markings to be missing, sometimes with some helpful markings added
	// (lane dividers for multi-lane turns, etc.)
	(0, mi) => Intersection,
}

Each class has different behaviours for routing, a different technique for drawing, etc.

I find myself wanting to be precise about particular components or aspects of a road, but lacking the terminology. I think this is a good time to agree on some terminology that will allow us to describe the structure we want to aim for. I can add the terms as a primer to the README after some input.

Let me start with one way of slicing it up, and let's change it to match up with what is already used.

• Lane: a single lane with designation, width, etc.
• Buffer: medians, painted buffers, verges, ... with width etc.
• RoadWay Road : a collection of Lanes (and Buffers), travelling in the same direction; A "half street" if you will, or a slip road or slip lane.
• Intersection: the area where multiple lanes of travel overlap and the road endpoints that join it.
• Street or Corridor: a collection of RoadWays (and cycletracks/footpaths) and Buffers that all run (mostly) parallel to one "centerline", representing the whole corridor and the "curb" "building to building".
• Junction: A collection of Intersections and RoadWays that make up one "junction". E.g., a roundabout is RoadWays around with one Intersection per arm.
• RoadNetwork: A directed graph of Roads connecting Intersections
• StreetNetwork: An undirected graph of Streets connecting Junctions

Those feel like the types I am expecting to find, which makes me want

• geom::{Point, LineString, Area, ...} and all the right algos, and
• osm::{Node, Way, Relation, Tag, Key, Tags} as well understood boring types.

How are these concepts weird or difficult or bad?

This bicycle lane rendering always seems weird to me. Despite the visual representation there also seems an issue with detecting/marking the actual infrastructure itself.

A few examples:

This is an actual bicycle road with proper tags in OSM: https://www.openstreetmap.org/way/10575838
In A/B street it renders like a usual street/highway. Since it's dedicated use is cycling I'd expect it to be rendered like that (by its primary use).

This path here https://www.openstreetmap.org/way/32916454 on the other hand is just a regular path (no street) where bicycles and pedestrians are allowed, but it renders as bicycle lane in A/B street. For shared uses I'd render this as footpath.

A/B street seems to show everything with bike alloance as type "protected bike lane", which doesn't seem right.

Nodes with "oneway:bicycle" tagged render the green bicycle lane in A/B street in both directions. Example: https://www.openstreetmap.org/way/102442321

There are also some minor issues like the rendering of streets in a pedestrian zone (highway=pedestrian), these also render as bicycle lanes. Example: https://www.openstreetmap.org/way/17071901

Generally speaking everything bikeable seems to be rendered as "protected bike lane". This might be ok where the bicycle is the only dedicated use, but for shared uses (pedestrians) this seems wrong.

I'm just looking at this:

Would be neat if there were a 'Share URL' button.

[After discussing a bit about the idea of osm2streets itself (here we are!), it talks about geom libraries, starting at the bottom of this comment. -- ed]

[Before osm2streets existed, looking at InitialMap and RawMap]

I think it's time to split out the intersection geometry algorithm into its own repository. The reasoning is similar to https://github.com/a-b-street/osm2lanes:

• it has a small, well-defined interface, and it's a hard problem
• it could definitely have wider applications out of A/B Street -- a first concrete one is in 3D Street now
• separating it from the rest of the code-base encourages other people to work on it
• it's an opportunity to rethink different parts of it

The API / code to extract

https://github.com/a-b-street/abstreet/blob/master/map_model/src/make/initial/geometry.rs
Pretty well-contained in one file, but there's some complexity with extracting it. The API is roughly "take a list of road segment line-strings and widths all connected to one intersection. Produce a polygon for the intersection, and also trim the line-strings back from the intersection."

We need a bit more, like a hint of if we should attempt a different algorithm that's more suitable for highway on/off ramps. That's based currently on OSM tags, but I don't think this library should know about those. (In fact, the name is maybe not great -- the library has nothing to do with OSM technically.)

The big complication is how to handle intersections that A/B Street consolidates into one. There's a two-phase thing where we pre-trim the roads based on individual intersections, then merge the intersections and create one final polygon. I'm not exactly sure what that'll look like in the cleaned-up API.

Testing

Right now, this code has no direct tests. There are screenshot diff tests set up for a few maps, but they're a very blunt instrument. osm2lanes is going so well because it's so natural to unit test -- clear input and output. I want to finally set up the same thing for this problem. The input is the line-strings and widths, the output is trimmed line-strings and a polygon. The difference is that a human can't look at a text representation of either one usefully. I think instead, the I/O can be GeoJSON files. We'll still do goldenfile testing, but at the much more granular level of a single intersection.

To actually do this, we need an easy workflow for running the tests, finding changed polygons, and doing a visual diff. Is there a small cross-platform GeoJSON viewer? I use QGIS or geojson.io normally, but both aren't quick to just launch from the command-line. I'm tempted to take some form of the RawMap editor UI as a tool for this project. That UI also lets people draw/edit roads and immediately update the intersection polygon. Alternatively, this could be an opportunity to try out some other simple UI framework.

Splitting repository logistics

I cheated with osm2lanes and just copied A/B Street code into a new repo. A/B Street today doesn't depend on that repo at all, though I want to eventually cut over to it. But for osm2polygons, I want to start this properly -- immediately cut over A/B Street to depend on this new crate. This has a development convenience cost -- if there's a change to the algorithm that's best evaluated in A/B Street, we'd have to temporarily change Cargo.toml to point to a local copy. But that's the idea behind making the new repo very self-contained and well-tested on its own.

The harder question is dependencies! Luckily, the intersection geometry code doesn't depend on abstutil or abstio, so I don't have to figure out what to do with those monstrosities. :) The question is geom, which is A/B Street's very sketchy geometry library. I started this thing at the very beginning because parts of geo didn't exist or I didn't understand them at the time. geom has all sorts of problems and basically 0 tests.

Option 1: split geom into its own repo, make both abstreet and the new osm2polygons depend on it.

Option 2: take this opportunity to make osm2polygons instead directly use geo. I haven't looked carefully through the code yet, but I think there aren't too many dependencies on tricky geom stuff. The main thing is operations on PolyLines (aka geo::LineStrings) that slice it -- take a distance interval and return a new linestring. There's also linestring shifting -- projecting perpendicularly to the left/right...

Option 2A: Port the math in geom to work on geo::LineStrings. If some of it isn't half-bad, could even contribute directly to georust, or make a new crate.

Option 2B: Look for replacements for some of these core operations. I know about https://github.com/jbuckmccready/cavalier_contours and maybe a few others. They're probably way more robust than wacky A/B Street stuff.

@michaelkirk, any opinions on any of this?

Currently osm2streets and A/B Street don't import off-road paths primarily intended for pedestrians. One example of many is https://www.openstreetmap.org/way/362057587, a highway=footway in a park.

Also currently many spaces that're shared between pedestrians and cyclists are either not imported, or transformed into this amusing [tiny shoulder for walking, cycle lane one direction, cycle lane other direction, shoulder] representation. This was originally done to workaround traffic simulation assumptions about supporting bidirectional movement on one lane. Now that A/B Street and osm2streets are detangled, though, we can make some progress here independently and do better. So for starters, we can do something else with things like https://www.openstreetmap.org/way/553752370 that're primarily pedestrian spaces that allow bikes.

This is related to #59 and #81. The issue of parallel ways to the main road is still present, but not the focus here. There are plenty of non-parallel examples we could import better today.

There are many cases where bidirectional travel is allowed on one "lane." We should add that to the Direction enum, then make all caller code handle it properly. In osm2streets itself, the impact isn't huge. But many places downstream in A/B Street will be affected: rendering, routing, traffic simulation, and even blockfinding. Just starting this issue to start tracking notes on how to fix things there

Let's start the equivalent of osm2street-js, but for Python. I have minimal prior experience from following along https://github.com/zonebuilders/zonebuilder-py, but we should shop around for options

@BudgieInWA, I was working through a seemingly small step of making a road (not using new terminology yet, oops -- will revisit that...) have valid geometry from the start in https://github.com/a-b-street/abstreet/tree/falliable_rawmap. Currently we store osm_center_points: Vec<Pt2D> and later on, turn into a PolyLine, which enforces no duplicate points. Doing too much validation upfront breaks on cul-de-sacs and other loop roads like https://www.openstreetmap.org/way/238049201.

... Actually as I write up this issue, I think the confusion I was having is about where enforcement that points represent a valid PolyLine happens. Of course we have to split ways into segments between intersections before attempting this validation.

But my question stands: how do we represent the raw OSM center-line for cul-de-sacs? It's this edge case where the PolyLine is a loop and has duplicate points. I don't want to make PolyLine allow first=last point in general... any thoughts here? And I don't think I've verbalized things well, let me know if so

When working on any code covered by the tests, you'll have a rapidly changing test source directory. Sometimes .orig files will show up that should be compared to new output.

It would be easy for the test browser to interact with an HTTP API to list and load the tests on the fly. A simple poll could detect .orig files, and offer to view the diff for those tests. (We could later progressively enhance to use a websocket to talk to a test runner that's watching the files.)

Note that we are moving away from yew in rust, and towards native JS.

Possibilities

trunk currently works as a perfectly cromulent web server with zero additional deps on the user.

trunk has a proxy setting that would let us easily pass a route through to whatever server we set up.

Serving files and directory indexes would get us 80% of the way there.

If we could write custom request handlers in rust, then we could do whatever fancyness we would need. We could even accept .osm (or a region to fetch) and create and run a new test.

Triggering tests that are run by the server would be cool, because the server can be set up to test in any way that we want, without needing to support the browser. E.g. we could generate a MapMachine rendering as part of the test or test any downstream users of osm2streets.

Blockfinding = https://dabreegster.github.io/talks/aiuk_ltn/slides.html#/blockfinding, tracing around street-edges

This would have use in the urban morphology research community, and a simple API for them to grab polygons would be great. Just a reminder to myself to eventually do this.

For his Straßenraumkarte, Supaplex uses the tag placement=right_of:1 to clarify where lanes are positioned in relation to the geometry-line.

Additionally, placement=right_of:1 helps to place the lanes precisely in relation to position of the osm way by indicating that the way geometry is located right of the first (left) lane and all other lanes are to be rendered right of it.

https://supaplexosm.github.io/strassenraumkarte-neukoelln/posts/2021-12-31-micromap-update#bike-lanes-position-them-right

I think osm2lanes would have to evaluate this tag in order to allow renderer to precisely position the lanes.

Maybe the osm2lanes schema could be something like this

lane1 > position_from_geometry=-2
lane2 > position_from_geometry=-1
lane3 > position_from_geometry=0
lane4 > position_from_geometry=1

I don't think that is the next important part, but I was thinking about it and wanted to start the documentation/discussion.

These 2 crates were just moved over from the A/B Street repo. Now that the big transition is done, some smaller cleanup tasks.

• Reorganize street_networks/src/lib.rs into more modules
• Consistently call an instance of StreetNetwork something -- probably streets. Not raw or map.
• Stop re-exporting all the types from raw_map; clean up callers to import from street_network (over on the abstreet side)
• Clean up the weird dependency on kml::ExtraShapes for debugging. We should just output GeoJSON instead; don't need the weird wrappers. I'll probably rethink ExtraShapes in general; I made it up early before I knew how to sanely parse geojson in rust.
• Update the README in this repo to explain the structure of the crates, and how we'll build multiple wrappers on top of StreetNetwork
• Add error handling to import_streets
• Reconsider import_streets::Options. There's a bunch of A/B Street-isms in there ignored by this repo.
• Slim down the public interface of import_streets. It should mostly just be osm_to_street_network, and over in abstreet, convert_osm should call that.

There are some other issues already for merging short roads, but I want to dump some recent notes about the geometry problem in particular.

Current approach

Code at https://github.com/a-b-street/abstreet/blob/master/map_model/src/make/merge_intersections.rs

The RawMap produced from OSM looks something like this. The pink segments have been manually tagged by me as junction=intersection in OSM, which often requires splitting the way. It's not too hard to auto-detect these segments and avoid the tagging. The problem is that the current algorithm for consolidating these intersections breaks in most cases, so I'm using the tagging as a way to manually opt in a few cases that I've worked on and verified turn out reasonably.

The current algorithm will take each pink segment in a pretty arbitrary order and do the following. It'll delete the segment and one of the intersections (arbitrarily), preserving the other intersection. It takes all of the roads connected to the deleted intersection, and reassigns them to be connected to the surviving intersection. It does some work to preserve the various types of OSM turn restrictions, some of which are implemented, some not. But then it also modifies the polyline geometry of some of the roads, to account for the deleted segment. The animation explains best:

The top intersection was deleted, so those 3 connecting roads have their polyline extended to roughly cover the deleted segment.

Then we proceed with map generation as usual, and the intersection geometry algorithm described elsewhere sometimes produces something half-reasonable:

But often it does not:

New idea

Do we always want to extend the geometry of surviving roads by that one new point? Maybe we don't need to extend any polylines at all, or maybe we should only extend the ones of non-pink segments, aka, normal roads that'll survive this process repeating. Maybe we should move the last point instead of adding, or add all of the points of the short road removed.

I added 2 new enums to quickly explore all of these combinations. One example worked much better for the case above:

But applying that policy everywhere broke another case that was previously working:

So is there a way to identify when we should AddOnePoint and when we should ModifyWhichRoads::None?

Wild alternative number 1

I've probably tried some form of this in the past. Often the original intersection geometries turn out reasonably:

What if we used these to decide how much to trim back the surviving roads, then deleted the pink segments, did the turn restriction fixing, but didn't modify any geometry? And mark that these intersections were "pre-trimmed" and skip part of the intersection geometry algorithm, and just construct the final polygon.

If this even works, would it blow up with road widening?

Wild alternative number 2

https://wiki.openstreetmap.org/wiki/Tag:junction%3Dyes#How_to_use_on_an_area

There are a few different proposals for tagging intersections as areas in OSM. I haven't checked Overpass yet, but I'm pretty sure I've seen a few of these mapped before. This particular scheme is purely additive; it shouldn't break any existing OSM software. Maybe in these really hard cases, it'd be much faster to just draw the polygon by hand in OSM from satellite imagery, and make the importer understand these as overrides. Haven't thought through how that import would work. Also not sure what to do if abst infers very different road width than reality and winds up disagreeing strongly with the human-mapped polygon.

See https://www.openstreetmap.org/way/745724293 in leeds_cycleway for one example, but I think I've spotted others.

In this one case, psv=designated should map to a bus lane. We only look for psv=yes today.