ent8r / notesreview-api Goto Github PK

Scenario

I use fvasco/pinpoi for various datasets.

Besides PinPoi being an app intended for such purposes, while I'm out surveying I have enough apps running (I collect more than just map data) that I can't also run a Web browser without hitting memory-exhaustion (causing other apps to malfunction).
Thus, using PinPoi means that I can query various (sometimes large) datasets (of features to add / resurvey), and navigate to their PoIs (e.g. a geo URI intent into a map & routing app), with ease. All while using little enough memory that I don't hit resource constraints.

Aside from avoiding technical problems, PinPoi can output a set of matches integrated from multiple different sources; either as a proximity-ordered list, or on a background map (OSM, of course 🙂). Among other features.

So, even if I could run a browser, PinPoi has significant advantages.

Specific (imagined) workflow example

When passing by a notable feature, particularly difficult-to-discover ones (think small & obscure), if I don't have time to stop and map it properly, I'll drop a note (as a reminder, and to easily re-find the feature when I do have time to give it the proper mapping treatment).
Else, when going through descriptive human-readable information (e.g. news announcements of changes to the road network) I'll add notes at those locations as a reference for future attention.

NotesReview is excellent for finding & filtering such specific (topical) notes. Other tools simply show (on a map) the notes within a given bounding box, which must then each be inspected individually (the focus is on location / proximity, not topic).

I would be able to use the output of NotesReview in software of my choosing, to easily identify & navigate to the locations during a survey. And/or during planning of a survey expedition.

For example, all (nearby) notes which request (by mentioning the word) a survey. Else, whatever (topic / keyword) is of interest, as I see fit.

Especially when I'm closing notes once each is solved by a changeset, and querying NotesReview with status=open, then the returned set is (automagically) a self-updating to-do / task list, until the count of matches reaches zero.
Compare using Overpass Turbo with a (wizard-generated) query which includes […] & (check_date!~/^202[2-9]-/ or survey:date!~/^202[2-9]-/) […]; thus as each element is surveyed, it's excluded from the set returned when the query is (next) executed. Thus, only outstanding targets for surveying are shown, rather than every feature of the same type.
This greatly reduces cognitive overhead, when attention should primarily be on mapping what's observed by survey (rather than dataset management; remembering which elements to ignore and mentally filtering them out).

This would all make surveying much more efficient & productive.

Use-cases

Beyond my own specific use-case, I imagine that this feature would be useful to others, in multiple ways

• some editors can show overlays of PoIs from external datasets
• an API for websites
• being part of a processing pipeline of QA tools

User-visible changes

I imagine that adding some extra elements to the share URL UI widget would be a sensible place.

Folks are gonna want a URL to give to whatever other software they're using. The sharing feature is the obvious place to go to get a properly-formatted perma-URL.

In which case, it may be as simple as including a toggle to add

[…&]format=[…]

to the URL string, specifying the desired format as selected by a drop-down list in the same UI (which appears after enabling the non-HTML output option?).

Else, I suppose you could add values to the view parameter, but given its name the semantics don't match. Unless view=(format|export) is the trigger for non-HTML output (instead of the mere presence of format), and the value of format specifies the preferred syntax (else a default is used).

Better might be a mode parameter. Default of mode=view (for current HTML output), but another value (export, output, format, …?) to trigger non-HTML.
Keeps the semantics cleaner (for future feature additions), and gives clues as to other parameters needed (like with OSM tagging: key=category implies that category=type is a sub-tag). It also serves as a

• safety switch; to avoid unintended output type by requiring multiple parameter changes
• master switch; to be able to leave format in the URL and easily switch between HTML and non-HTML, else leaving format blank / unspecified to use the current default format

Regardless of generating mechanism, an example of the resultant URL would be something like

//ent8r.github.io/NotesReview/?[mode=(export|format)&]map=12%2F49.2%2F-2.1&apply-bbox=true&status=open&query=survey&format=gpx

Formats

The common, well-known ones would seem to suffice:

• GPX
• CSV
• GeoJSON
• KML / KMZ
• GeoRSS
• any others which ENT8R or other users think sensible (I'll update this list based on comments)

Other implementations, for inspiration

• Rob Whittaker's #SurveyMe tool can output various combinations of target-PoIs (one of which is OSM Notes that include the string #SurveyMe) in several formats — example: osm.mathmos.net/survey/data.cgi?lat=49.2&lon=-2.1&fmt=gpx (though, N.B., his fmt=kml includes more descriptive info of each target)
• Overpass can (according to the documentation) be queried such that it outputs suitable formats — no example, yet, since I'm still tinkering with this, myself 😋
• Searx (& possibly SearxNG) (a “hackable metasearch engine”; public instances) implements the possibility to add format= to the URL query-string, to output in several non-HTML formats (CSV, JSON, RSS) — example: [searx instance]/search?q=OSM&format=json

Caveats & Considerations

I'm assuming that NotesReview isn't entirely dependent on client-side scripting for even the basics (querying the OSM Notes API & returning a list), and that client-side scripting is only for UI / rendering, processing thumbnails, and other nice-to-have features.
Software for fetching datasets is unlikely to have an HTTP client which executes scripting; it'll simply expect a pre-processed response (in one of a few particular formats).

If this isn't true, then could an exception be made, server-side, for when format is specified in the URL?

PinPoi stores the received set locally (I presume as a cache for performance, and for offline usage), and (at the moment) only issues another HTTP request when manually instructed to. So, the load on the server (and OSM API) should be low. Arguably lower (due to caching) than when using the HTML UI.

I haven't throughly read the OSM Notes API documentation, but I gather that the output is XML-only. However, I could be wrong, and it would be better to query OSM directly rather than through NotesReview. However, NotesReview is a much nicer & easier interface (especially for the less tech-savvy, who would be daunted by querying an API directly).

"Packages No packages published" is displayed right now, fortunately this pointless section can be removed.

Edit repo page config to remove it (cog next to the description).

I am not making a PR as it is defined in proprietary github settings, not in a git repository - and I have no rights to modify repo settings.

Based on https://github.com/ENT8R/notesreview-api it appears that some API is available

Is it documented anywhere?

OSM moderators can remove notes that are clearly spam, or otherwise inappropriate. This operation (compared to closing/reopenning/commenting) will not appear in any RSS feed and querying API for the note will return 404.

Currently NotesReview backend doesn't handle this which causes desync - they are still present on NotesReview despite being gone from osm.org or API. These notes are not present in the subsequent daily note dumps - this is the best way of detecting all the hidden notes (compared to querying all >1M notes to check for 404).