Display revision date of map data about streetcomplete HOT 23 CLOSED

Streetcomplete shows a map that is provided by JawgMaps, and there is a delay between an edit on OSM and its display on JawgMaps. But StreetComplete works with the map data as it was at the moment you downloaded the quests for that area. See this example:

What JawgMaps displays is pointed at with the red arrow. Since that map was made from OSM data, I moved that path myself a few days ago. What StreetComplete is working with is pointed at with the green arrow. You can see it is the more recent version. This screenshot is with the surface overlay on, so you can see where the paths and roads are located and you can edit their surface by tapping on them. I also moved the steps at the north end of that path a few days ago: you can see that although JawgMaps shows it at its old position, the quest pin for "do these steps have a handrail" is shown at the correct new position.

from streetcomplete.

it is hard to find vector map provider which is even more up-to-date, allows for a heavy tiles usage, and does not charge for access

note that there is OSMF-funded project to create service with more often updated map tiles, if it will become operational than SC will likely be able to use it, see https://community.openstreetmap.org/t/minutely-updated-vector-tiles-demo/110121

from streetcomplete.

On the data itself, as the map is already explained.

Offline OSM data is deleted after 2 weeks, and in my opinion this is fine in the vast majority of all cases.
When you want more recent data, just either re-download it manually, or go online and let the auto-dowload do its thing (iirc auto-download happens when existing offline data is older than a day or so).

Wouldn't it make sense to check for current OSM data right after having uploaded notes?

What do you mean with "check for current OSM data"? Re-download currently downloaded data after a note has been uploaded? I also can't see how this would help in your case, the note is already created.

from streetcomplete.

Note also that if edit conflict happens data is considered to be stale immediately.

from streetcomplete.

@sjvudp Just to make sure; when you say "Street Complete is working with outdated OSM data", what exactly do you mean by "OSM data" ?

• The background map (which is indeed always few days old, as explained in more details by others)
  (Judging by #5581 (comment) I think this is probably what you mean)
• or do you mean StreetComplete Quest icons themselves (which should be current - i.e. from the moment that you "loaded the maps at home"),
• or are you talking about some StreetComplete Overlay? (as noted above in #5590 (comment) e.g. Surface Overlay will show recent updates - from the last moment you loaded the quests - but underlying background map will still confusingly show old data, so you'd get kind of "double view" - both correct and incorrect at the same time).

If your issue is about background map, there are two things to check:

• Is the map at https://streetcomplete.github.io/streetcomplete-mapstyle/ updated? If not, it means the edit is very recent, and Jawg maps has not yet rendered it, so there is no help (it is hard to find vector map provider which is even more up-to-date, allows for a heavy tiles usage, and does not charge for access).

• the map at the above URL is updated when you open it in your web browser, but your map in StreetComplete isn't updated, you can go to Settings / Delete cache, and then fully close StreetComplete (i.e. swipe it away from "recent apps list") and reopen the StreetComplete. That should redownload both quests and background map data.

  Theoretically, this workaround should not be needed, but the Tangram-ES library that StreetComplete currently uses seems not to always detect updated maps, so it can take up to 14 days to recognize changes. It is no longer maintained however, so fixes there are unlikely, thus StreetComplete is in the middle of transition to another library MapLibre which might fix that issue (or introduce new ones 😅).

  I'd thus suggest to live with Delete cache workaround until StreetComplete transitions to MapLibre, and then check if the bug still happens and share feedback.

See also FAQ: "The map looks stale, how often does the background map get updated?"

from streetcomplete.

Using the OSMF tiles would be ideal, however, I think they are still quite some bit away from being usable. At the moment, the over-the-wire size of the tiles is ten times that of the tiles currently used. Also, it would not make much of a difference in regards to this issue, as the current tile source is (allegedly) updated daily already.

from streetcomplete.

(@rhhsm)

Streetcomplete shows a map that is provided by JawgMaps, and there is a delay between an edit on OSM and its display on JawgMaps. But StreetComplete works with the map data as it was at the moment you downloaded the quests for that area. (...)

OK, I wasn't aware that there is a JawgMaps "intermediate layer" between OSM data and the display in Street Complete.
It's even more confusing to see quests on roads that are not present in JawgMaps, but suddenly appear when you try to answer such quest.

(I think this also answers the questions of @Helium314 and @mnalis)

• Is the map at https://streetcomplete.github.io/streetcomplete-mapstyle/ updated?

As of a moment ago, the map is updated in a bad way: The old wrong data is overlayed with the correct new data.

• (...) you can go to Settings / Delete cache, and then fully close StreetComplete (i.e. swipe it away from "recent apps list") and reopen the StreetComplete.

I think a more local refresh (like a local download of areas) would be preferable, also without the need to restart the app.

Overall, I don't know what the best possible solution is, but if the intention of the app is hassle-free usage, the state as it is now can be rather confusing occasionally.

from streetcomplete.

Here's an example of new data being overlaid with old data:

In contrast, here is what https://www.openstreetmap.org/#map=19/49.19330/12.26600 displays:

from streetcomplete.

As of a moment ago, the map is updated in a bad way: The old wrong data is overlayed with the correct new data.

Interestingly, it looks good to me about half an hour later:

It would seem you have a knack for busting in corner-case moments, @sjvudp 😸

I think a more local refresh (like a local download of areas) would be preferable, also without the need to restart the app.

Yes, it would be better. It would be even better if it it detected the change automatically and updated by itself. But as noted, it seems Tangram-ES library seems not to support either (or, if it claims it does, it does not do so reliably at all in my experience). But, as Tangram-ES is being replaced with MapLibre as we speak, the issue is kind of moot.

Mine was just a rough workaround to confirm the source of the problem; not a "solution" I'd dare suggesting to people to live with on their daily mapping sessions (It's obviously too annoying to deal with any sort of regularity 😮)

Overall, I don't know what the best possible solution is, but if the intention of the app is hassle-free usage, the state as it is now can be rather confusing occasionally.

Yes, occasionally it is confusing... Hopefully not too often; as it only manifests if you are solving quest very soon after the geometry changes, which should be somewhat rare.
Here's to the hope that MapLibre (Tangram-ES replacement) would fare better in autodetecting updated map 🍺

from streetcomplete.

Please stop depicting MapLibre as the silver bullet that will solve everything. Most of all, it is just a different map render library that happens to be still maintained and bugs being fixed, unlike tangram-ES. In some aspects, tangram-es is actually more advanced than MapLibre. The reason we are switching is because it is still maintained and because a version of it exists that runs on modern iOS.

So, anyway, no, MapLibre does not magically detect map updates.

It is also not that relevant, because StreetComplete currently considers downloaded map data stale after one day. So, when downloading again one day or more later, it will download the tiles again. Otherwise, if no new download is triggered, the map data will be deleted after two weeks. (TBH I am not sure how it will be with MapLibre. As I wrote, it is different, so some features and/or control over behavior we had in tangram-es may not be available in MapLibre.)

As per the original suggestion:

The use case for displaying the download date of the background map is very very narrow, and in most cases not informative, because

1. the date is usually already known by the user - it's the date when they last tapped on the download button (or more general, the date of the last download, regardless of whether it was triggered manually or automatically)
2. the download date is not the same date as the date when the tiles for the background map were created from OSM data. They could be day(s), or even week(s) old, depending on the map tile provider.

The map tiles themselves do not contain information on the date they had been created from OSM.

from streetcomplete.

Please stop depicting MapLibre as the silver bullet that will solve everything

I apologize if it come out that way; I'll try to tune down my optimism.

The intention of my "hope that MapLibre would fare better" was based on my (perhaps overly) optimistic hope that since MapLibre is being maintained, there is non-zero chance that reported bugs (or even feature requests) might eventually get fixed/implemented (which is significantly better chance that what we've had with discontinued Tangram-ES, IMHO).

In some aspects, tangram-es is actually more advanced than MapLibre

I admit I did made some (totally unverified) assumptions that MapLibre implementation in SC will have more-of-less feature parity of existing Tangram-ES code, i.e. that SC won't be deliberately losing (noticeable) existing functionality by making a switch. In my defense, it seemed so common-sense that did not even occur to me to verify that.

So, anyway, no, MapLibre does not magically detect map updates.

Uh, so are you saying that MapLibre lacks that (alleged) Tangram-ES capability where you can tell it something like "if this already downloaded tile is older than say 1 day, you have network available and tile has been updated on the server, re-download it"?

It is also not that relevant, because StreetComplete currently considers downloaded map data stale after one day. So, when downloading again one day or more later, it will download the tiles again:

I believe you that it is how it was supposed to be working, but both in my own experience and in several reports over time it is not really what users are seeing (at least sometimes). ¹

Now why that seems to be happening could be due to several factors, which never did get investigated fully because main suspect was bug in Tangram-ES (and since it was unmaintained, there was no hope of it being fixed, so any research effort would be just wasted):

• bugs in Tangram-ES or its documentation about that "refresh" functionality.
• some other unrelated StreetComplete or other bug preventing timely updates
• JawgMaps having server-side bug with refreshing which only triggers in some cases
• JawgMaps (significantly) misrepresenting their map update times

from streetcomplete.

Uh, so are you saying that MapLibre lacks that (alleged) Tangram-ES capability where you can tell it something like "if this already downloaded tile is older than say 1 day, you have network available and tile has been updated on the server, re-download it"?

As far as I know, it will respect the ìnformation from the HTTP header regarding this. Not sure if it can be overridden on the client-side.

from streetcomplete.

As far as I know, it will respect the ìnformation from the HTTP header regarding this. Not sure if it can be overridden on the client-side.

Cool; if that is correct, then MapLibre version should be getting fresh tiles from Jawg servers automatically!

tile.jawg.io seems to send Cache-Control: max-age=43200, which would mean that MapLibre client should refetch tiles after 12 hours.

(Jawg also sends Etag and Last-Modified, which MapLibre could use to reduce data usage instead of doing full re-download after that period but I don't know if MapLibre Native supports those)

(Anyway, I guess we'll see how MapLibre version behaves regarding that once it is ready...)

from streetcomplete.

OK, during the issue I learned some things I did not know when filing the issue, making the issue harder to resolve than intended:

• I was assuming that StreetComplete creates the map from current OSM data, but instead some pre-rendered tiles that are updated occasionally on the server, and with some days later on the mobile device (from the server)
• So when the server does not provide a rendering date, the app cannot know the rendering date either; at best it can use the time of download, while the actual data may be significantly older.

Still with this knowledge there is a potential for improvement:

• "Map tile providers" should be convinced to provide a timestamp telling how old their data is (I'm aware that such timestamp may vary from tile to tile, so a region will have an interval of timestamps (oldest to latest). A pessimistic view would use the oldest timestamp, while an optimistic view would need the latest timestamp found for the area involved).
• Despite from displaying such timestamp on the map (permanently, or on demand), it might be very helpful to include it with any hint generated from the app, because that might explain why a user seemingly added data that was already there.

from streetcomplete.

• Map tile providers" should be convinced to provide a timestamp telling how old their data

They usually do. E.g. fetching https://tile.jawg.io/streets-v2/16/19295/24643.pbf will already return HTTP header Date: Mon, 29 Apr 2024 21:31:48 GMT, but also Last-Modified: Mon, 22 Apr 2024 20:15:59 GMT. But I think it is less likely that the map framework would pass such technical details to the calling app. Nor that it is really needed, see bottom of this post

• Despite from displaying such timestamp on the map (permanently, or on demand),

I might imagine debug build showing that, or maybe even SCEE having a debug preference in release build for it, but definitely not standard StreetComplete (given its target audience is not supposed to even know what tile timestamp means, much less to do debugging based on that!)

it might be very helpful to include it with any hint generated from the app,

if by "hint" you mean it should be included in debug log, yeah, it might be helpful. I might even envision debug build of SC including oldest/newest tile timestamps as changeset tags for all quests in that changeset (although the more quests over the more area, the less useful that data becomes). If you however intended "hint" to mean something like "including it in every new/changed element in some specific tag", I think that is a big no-no.

because that might explain why a user seemingly added data that was already there.

Sure, but I would much rather have that "freshness" gap be so much reduced so the users (almost) always see the freshest data.

But if you're into debugging you can already retrieve that data yourself, you don't need any of those app changes in order to so; just open Firefox, visit https://streetcomplete.github.io/streetcomplete-mapstyle/, press F12, navigate to the location having stale data, go to network tab, click on the problematic tab and look at response headers, esp. Last-Modified one. Perhaps take a screenshot showing both the map and the headers.

That can be used to determine if problem is on StreetComplete (MapLibre / Tangram-ES) client side (e.g. new tiles are available on server, but SC is not updating on them), or on the JawgMaps tile-generation side (e.g. tiles are stale and not being promptly regenerated for some reason), which would then lead to detailed technical issue being filed at appropriate project, with information how to reproduce it.

But as I sad before, I wouldn't bother trying to debug that until StreetComplete transits to MapLibre...

from streetcomplete.

But as I sad before, I wouldn't bother trying to debug that until StreetComplete transits to MapLibre...

By the way, this is the map for streetcomplete using the Jawg-tiles in MapLibre https://streetcomplete.app/map-jawg/ - I also improved the style a bit, IMO.

from streetcomplete.

it might be very helpful to include it with any hint generated from the app,

Most userfacing places would confuse, too, like "The quests are based on week old data!".
Perhaps an 🛈 icon could be done with a popup:

• Quest data: 2024-04-30T08:23
• Background map data: 2024-04-03T07:41

But if this icon is shown prominently this would add clutter. If hidden no one would find it.

from streetcomplete.

But if this icon is shown prominently this would add clutter. If hidden no one would find it.

Perhaps it could be put in "undo detail screen" as a compromise to avoid clutter but still be somewhat discoverable... But again, that is assuming that such information is readily available to SC, which might be a stretch...

from streetcomplete.

Fun fact: Background map date may be different for each tile

from streetcomplete.

I think it is a general problem that new OSM mappers often don't realise that OSM is not a map, but a database of geographic data, and that there is a delay between making an edit on the OSM data and the moment that change is displayed on maps using that data. I think for the large majority of users it is enough to know that there is this delay, and very few will be interested in exactly how old the displayed map tiles are. I think there are many more important things you could spend your time on ;)

from streetcomplete.

if by "hint" you mean it should be included in debug log

No (see https://www.openstreetmap.org/note/4224996#map=16/49.1931/12.2740&layers=N for example):
The auto-generated hints already include "via StreetComplete 57.2", so why not add the revision date of the tile?

Sure, but I would much rather have that "freshness" gap be so much reduced so the users (almost) always see the freshest data.

Nothing against that ;-)

Fun fact: Background map date may be different for each tile

If you review my comment you'll find: "I'm aware that such timestamp may vary from tile to tile, so a region will have an interval of timestamps (oldest to latest)."

from streetcomplete.

but also Last-Modified: Mon, 22 Apr 2024 20:15:59 GMT

Is it actually age of OSM data? Or date of something else? Like for example, date of tile generation (which may happen after X hour long process of loading of OSM data)

from streetcomplete.

Is it actually age of OSM data? Or date of something else? Like for example, date of tile generation (which may happen after X hour long process of loading of OSM data)

I would expect Last-Modified HTTP header to be timestamp of actual tile generation. So, it would be useful for debugging the problem because you could then know:

• when the data was changed (by looking up problematic element history on api.openstreetmap.org)
• when Jawg actually generated the vector tile (via HTTP Last-Modified header)
• when SC downloaded the tile (via SC internal logging)

Which would allow to pinpoint which part of the process is the problematic one.

The auto-generated hints already include "via StreetComplete 57.2", so why not add the revision date of the tile?

Well, one generally would do good to refrain from adding debug information to permanent ever-growing databases like Notes (there were even opponents for adding that "via StreetComplete 57.2").
Also, The Note field length is limited to 255 chars total, so with all that debug information, especially when you have pictures attached (i.e. text "Attached photo(s):" + URLs to the pictures, averaging some 35 chars per picture), the space available for actual user note decreases.

from streetcomplete.