Currently data can only be sent to services which implement an API that is compatible with MLS Geosubmit. Investigate adding support for other location services, such as OpenCellID for cell towers

The user would be able to configure the type of endpoint and its parameters (API key etc.) in the settings

Collect cell towers from all active mobile subscriptions on the device. Duplicate cell towers from different subscriptions should be counted only once

I try to start scanning and then the app crashes immediately

The export feature is a bit confusing to me, you can export data but there's no way apparently to import it somewhere? Can one send these files to a location service? How does this work if I want to use it as a backup feature?

Add support for exporting scan data in some format (CSV etc.)

Currently it is not possible to see details of the reports, e.g. Wi-Fi access point names, from the UI

Create an interface where report data can be viewed. This could be for example a dialog which opens from the report list. Details to show:

• Location where the report was made (map or at least the address)
• List of Wi-Fis
  • SSID
  • MAC address
  • Signal strength
• List of cell towers
  • Radio type
  • Mobile network (country + operator)
  • Cell ID
  • Location area code
  • Signal strength
• List of beacons
  • MAC address
  • Signal strength

Sometimes it's hard to see button that makes the map "focus" on our location:

It's worse when the device is using dark mode as this icon turns white and the map doesn't have a dark mode, but even with a black icon, it can still be hard to see depending on what's behind the icon.

Apps usually work around this by having having a background around the icon, perhaps the same could be done here. For example:

Currently the quick settings tile for starting scanning is not easily discoverable. Prompt to add the quick settings tile when the user stops scanning for the first time. The prompt should only be shown once and if the user hasn't already added the QS tile

Hello, is it possible to export data in a way that is compatible with WiGLE data structure ?

Currently all reports are sent in one request to the configured Geosubmit API. Large amount of reports in one request can cause the request body to be too large for the server. This could be fixed by limiting the amount of reports sent at once

• Use hexagons instead of rectangles for showing locations where reports have been made
  • https://github.com/uber/h3-java
• Show current location of the device
  • Follow device location if the user does not move the map manually

This feature could be used if you made a reports that you don't want to upload and instead want to delete. I.e. if you were testing out the app or the area you recorded already has reports.

Autoscan should be re-enabled after the device is rebooted. Currently there is a broadcast listener which should re-enable autoscan after reboot, but it doesn't seem to work. This might be caused by runBlocking not working with Google APIs.

I noticed that the scan start button did not change its text ( start / stop ) .

The notification is visible.

If you change fragment and go back to reports, the button text changes.

I've made a video to explain.

When using a custom endpoint, the app appends "/v2/geosubmit" to the address entered in the settings, which imposes rigid constraints on the server configuration.

Hi,

I tested sending to my server by defining a custom full path, but the request generates a 403 error. Apache ModSecurity blocks this request.
Looking at the error logs, it seems to come from the "Content-Encoding" header. So I tested the Android code by commenting out the line ".addHeader("Content-Encoding", "gzip")" and it works!

I'm not familiar with HTTP and it seems that the "Content-Encoding" header should rather be used in the server -> client direction and not the other way around; hence the blocking.

More fundamentally, your app compresses the data sent in gzip format. Was this a specification requested by MLS? If not, it would be simpler to transmit the json data uncompressed and use the header ("Content-Type", "application/json")

In the version downloaded from f-droid, I cannot send the reports.

Pressing the Send reports button makes the button get grayed out for around a second, but the Reports last uploaded field remains empty.

Selecting Settings > Reupload data makes a Failed to upload reports message appear.

Given most of my data collection is done while driving, would it be possible to see the map getting filled using android auto instead of having it show up on the phone screen?

When Android returns results for cell tower scans, there are often missing values in the results. See #88 for an example. If we can assume that the cells belong to the same mobile network and location area, we can fill in the missing values from other cells in the same scan.

Update GitHub Actions pipelines to automatically upload .apk of the fdroid flavor when publishing new releases. This is needed for F-Droid reproducible builds

Currently no builds are run with GitHub Actions for pull requests from forks. Think about how to implement this without running duplicate workflows which happens when both push and pull_request triggers are used

As the title says I can't enable the automatic scanning option on my phone, It gets to

Visually this manifests as the toggle becoming greyed out and non-interactive until leaving and returning to the settings page. Using the Pixel 3a emulator in Android Studio there is no such problem and it activates fine.

By default, Android restricts Wi-Fi scans to one every 30 seconds, which limits the amount of scans NeoStumbler can do. It is possible to disable this throttling in the developer settings and apps can check whether throttling is enabled: https://developer.android.com/reference/android/net/wifi/WifiManager#isScanThrottleEnabled()

Add support for ignoring scan throttling and doing Wi-Fi scans more often, e.g. every 10 seconds. This feature should be opt-in by requiring the user to enable it in the settings

Think about UX, see screenshot in #141 for an example

As MLS is going to shut down soon, it doesn't accept data submissions anymore. Add a warning about this to the app

Logcat after pushing the Upload Button:

--------- beginning of crash
05-16 21:21:19.988 22862 22910 E AndroidRuntime: FATAL EXCEPTION: DefaultDispatcher-worker-1
05-16 21:21:19.988 22862 22910 E AndroidRuntime: Process: xyz.malkki.neostumbler, PID: 22862
05-16 21:21:19.988 22862 22910 E AndroidRuntime: java.lang.IllegalStateException: Not implemented
05-16 21:21:19.988 22862 22910 E AndroidRuntime: 	at androidx.work.CoroutineWorker.g(Unknown Source:4)
05-16 21:21:19.988 22862 22910 E AndroidRuntime: 	at G1.e.q(Unknown Source:34)
05-16 21:21:19.988 22862 22910 E AndroidRuntime: 	at q3.a.o(Unknown Source:7)
05-16 21:21:19.988 22862 22910 E AndroidRuntime: 	at G3.M.run(Unknown Source:114)
05-16 21:21:19.988 22862 22910 E AndroidRuntime: 	at M3.a.run(Unknown Source:95)
05-16 21:21:19.988 22862 22910 E AndroidRuntime: 	Suppressed: L3.g: [v0{Cancelling}@8662ab2, Dispatchers.Default]
05-16 21:21:19.997 22862 22910 D OOMEventManagerFK: checkEventAndDumpForJE: 0
05-16 21:21:20.067 22862 22910 I Process : Sending signal. PID: 22862 SIG: 9

As far as I can tell this happens because .setExpedited() is used in

but ReportSendWorker does not override getForegroundInfo() , so it throws a NotImplementedException … but only on Android 11 or older, as per: https://developer.android.com/jetpack/androidx/releases/work#2.7.0

When using setExpedited(...), WorkManager delegates to expedited jobs in JobScheduler starting Android 12, while providing backwards compatibility on prior versions of Android by delegating to a Foreground Service.

I'm not sure if

To be honest: I have no idea I'm doing & what all this means and have basically zero experience with Android/Java/Kotlin :D

The application is intended to be published as a downloadable APK file, possibly in GitHub releases of this repository. Investigate how automated updates could be done

I'd like to have my parents' devices set up to collect data in the background with consent of course, but restarts are needed every once in a while and that's where NeoStumbler unfortunately dies and I'd have to manually enable it again.

Currently scanning can be started automatically when the device is moving in an area where no reports have been created recently. Scanning will continue until the device stops moving. The user should be able to toggle whether the scanning should stop automatically if the device moves to an area where reports have been created recently and the scanning was started automatically

Hello,

Here's an error I'm seeing in my logcat on an Android terminal, version 11, without google services.

2024-05-19 10:21:48.837 15520-15520 GetAddressKt$getAddress xyz.malkki.neostumbler.fdroid.dev    W  Failed to geocode address for location XX.XXXXX, XX.XXXXX (Ask Gemini)
java.io.IOException: Missing INTERNET permission
at android.location.Geocoder.getFromLocation(Geocoder.java:136)
at xyz.malkki.neostumbler.extensions.GeocoderExtensionsKt$getFromLocationSuspending$3.invokeSuspend(geocoderExtensions.kt:32)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104)
at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:111)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:99)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:585)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:802)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:706)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:693)

Maybe it would be interesting to be able to disable the geocoder.

Thanks for your great work!

Currently the application is hardcoded to send data only to Mozilla Location Services, which is going to shut down soon and does not accept data submissions anymore. Allow the user to specify a custom endpoint where to send data. Note that the endpoint might require an API key - there should be a way to set the API key in that case

Optionally allow using multiple endpoints

This feature is also needed to avoid showing NonFreeNet anti-feature flag in the F-Droid listing

After stumbling a few thousand wifis over the past days and uploading the reports to my own ichnaea server I was wondering why there is still no new data in it's database, turns out that NeoStumbler is sending malformed reports like this:

{
    "items": [
        {
            "a": 1716236503655,
            "b": {
                "a": 52.5,
                "b": 13.3,
                "c": 12.9,
                "d": 10071,
                "e": 91.7,
                "f": 1.1,
                "g": 348.3,
                "i": 0.2,
                "j": "fused"
            },
            "c": [
                {
                    "a": "cc:ce:1e:xx:xx:xx",
                    "b": "802.11n",
                    "c": 10072,
                    "e": 2437,
                    "f": -62,
                    "h": "W_xxx_g"
                },
                {
                    "a": "46:4e:6d:xx:xx:xx",
                    "b": "802.11n",
                    "c": 10072,
                    "e": 2437,
                    "f": -64,
                    "h": "P_xxx_g"
                },
                {
                    "a": "44:4e:6d:xx:xx:xx",
                    "b": "802.11n",
                    "c": 10072,
                    "e": 2437,
                    "f": -63,
                    "h": "P_xxx_z"
                },
                {
                    "a": "6c:55:e8:xx:xx:xx",
                    "b": "802.11n",
                    "c": 10072,
                    "e": 2462,
                    "f": -80,
                    "h": "V_xxx_C"
                },
                {
                    "a": "8a:12:ac:xx:xx:xx",
                    "b": "802.11g",
                    "c": 10072,
                    "e": 0,
                    "f": -81,
                    "h": "E_xxx_4"
                }
            ]
        }
    ]
}

(location/MAC-addresses/SSIDs shortened for privacy - values in the real request are correct)

Hello, while using the quicksettings tile I noticed that it didn't seem to work, here is the error in the logs.

2024-06-01 09:15:36.653  4242-4293  AndroidRuntime          xyz.malkki.neostumbler.fdroid        E  FATAL EXCEPTION: DefaultDispatcher-worker-5 (Ask Gemini)
   Process: xyz.malkki.neostumbler.fdroid, PID: 4242
   java.lang.SecurityException: Starting FGS with type location callerApp=ProcessRecord{3012e3a 4242:xyz.malkki.neostumbler.fdroid/u0a396} targetSDK=34 requires permissions: all of the permissions allOf=true [android.permission.FOREGROUND_SERVICE_LOCATION] any of the permissions allOf=false [android.permission.ACCESS_COARSE_LOCATION, android.permission.ACCESS_FINE_LOCATION]  and the app must be in the eligible state/exemptions to access the foreground only permission
   	at android.os.Parcel.createExceptionOrNull(Parcel.java:3242)
   	at android.os.Parcel.createException(Parcel.java:3226)
   	at android.os.Parcel.readException(Parcel.java:3209)
   	at android.os.Parcel.readException(Parcel.java:3151)
   	at android.app.IActivityManager$Stub$Proxy.setServiceForeground(IActivityManager.java:7164)
   	at android.app.Service.startForeground(Service.java:776)
   	at t4.w.q(Unknown Source:365)
   	at a3.a.p(Unknown Source:7)
   	at s3.L.run(Unknown Source:114)
   	at z3.a.run(Unknown Source:95)
   	Suppressed: x3.g: [t0{Cancelling}@4db001e, Dispatchers.Default]
   Caused by: android.os.RemoteException: Remote stack trace:
   	at com.android.server.am.ActiveServices.validateForegroundServiceType(ActiveServices.java:2882)
   	at com.android.server.am.ActiveServices.setServiceForegroundInnerLocked(ActiveServices.java:2569)
   	at com.android.server.am.ActiveServices.setServiceForegroundLocked(ActiveServices.java:1835)
   	at com.android.server.am.ActivityManagerService.setServiceForeground(ActivityManagerService.java:14077)
   	at android.app.IActivityManager$Stub.onTransact(IActivityManager.java:3578)

All permissions have been granted.

If I understand

https://github.com/mjaakko/NeoStumbler/blob/31648aa/app/src/main/java/xyz/malkki/neostumbler/geosubmit/MLSGeosubmit.kt#L18-L20

right, you currently don't use any API key for geosubmit.

Is that correct? Is there a way of setting an API key, other than hardcoding it in the URL in line 20?

I think it would be useful for new users if readme.md listed the differences between the Full and F-Droid variants and clarified something about app updates. I just landed here after checking BeaconDB and have two questions:

I understand that the F-Droid variant doesn't include the Play Service bits, but in practice that means what? I noticed that it doesn't have the "fused location" option, is that it? Or is it missing something else that is nice to have?

Regarding updates, the F-Droid variant can be updated via the F-Droid and IzzyOnDroid repos. No info about the full version though. Do we have to manually check the releases page, does the app check for updates itself and notifies users, etc?

So essentially it would be nice if the readme had more information about this for newbies like me checking the app for the first time.

Thanks for your work!

I have two users who are sending data to //v2/geosubmit, resulting in a 404 error.

"POST //v2/geosubmit HTTP/2.0" 404 0 "-" "NeoStumbler/9"

Hello, when launching the application, on an Android 15 phone, with google services, using the Fdroid flavor, we can see in the logs the following error:

2024-05-19 14:38:57.472 27861-27950 DistanceConfigFetcher   xyz.malkki.neostumbler.fdroid.dev    W  error reading beacon data
      java.io.FileNotFoundException: https://s3.amazonaws.com/android-beacon-library/android-distance.json
    at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:255)
    at com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:211)
    at com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:30)
    at org.altbeacon.beacon.distance.DistanceConfigFetcher.request(DistanceConfigFetcher.java:94)
    at org.altbeacon.beacon.distance.ModelSpecificDistanceUpdater.doInBackground(ModelSpecificDistanceUpdater.java:28)
    at org.altbeacon.beacon.distance.ModelSpecificDistanceUpdater.doInBackground(ModelSpecificDistanceUpdater.java:16)
    at android.os.AsyncTask$3.call(AsyncTask.java:394)
    at java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
    at java.lang.Thread.run(Thread.java:1012)
2024-05-19 14:38:57.472 27861-27950 ModelSpeci...Calculator xyz.malkki.neostumbler.fdroid.dev    
W  Cannot updated distance models from online database at java.io.FileNotFoundException: https://s3.amazonaws.com/android-beacon-library/android-distance.json

It doesn't seem to affect the rest of the behavior

https://hosted.weblate.org/hosting/ provides free hosting with Weblate for Libre software. I think NeoStumbler might apply here.

https://translate.codeberg.org also I guess? I'm not sure how they go over there

This'd bring more attention from other translators which may not be as connected with the project. So there is that tradeoff. But overall, it should help the project.

As MLS is shutting down, it would be useful to be able to automatically export the data to .zip files, which could be imported to a new geolocation service when one is available. The user would be able to select where to upload the exported files and how often the data is exported (weekly, daily, etc.)

By default, newer Android versions throttle the frequency of WiFi scanning by apps. The throttling can be disabled inside of Developer Options, at the expense of worse battery life. I have noticed that NeoStumbler often leaves out spots when I'm surveying while riding a bicycle, and taking advantage of this setting could (as far as I understand) help alleviate the issue.

Attached below is a screenshot from an app called NetworkSurvey, which prompts the user to disable the setting for increased data accuracy. There is a page on Android Developer about the throttling behavior on different Android versions.

Currently the app crashes if autoscanning is disabled when permissions have been revoked. Check permissions before disabling autoscanning and if permissions are not available, just set the setting value to false and don't disable activity transition updates, which causes the crash

Investigate what is needed to upload the application to F-Droid

• Create different build flavors to allow disabling / enabling features
  • This is needed to avoid including features which are dependent on proprietary libraries (i.e. Google Play services)
• Investigate replacing Google Play services with MicroG

When multiple WCDMA towers are recorded in the same report, only the first tower is valid:

[
  {
    "age": 9215,
    "asu": 96,
    "cellId": 177013648,
    "locationAreaCode": 37265,
    "mobileCountryCode": 505,
    "mobileNetworkCode": 2,
    "primaryScramblingCode": 194,
    "radioType": "wcdma",
    "serving": 0,
    "signalStrength": -24
  },
  {
    "age": 9215,
    "asu": 96,
    "cellId": 23,
    "mobileCountryCode": 0,
    "mobileNetworkCode": 0,
    "primaryScramblingCode": 231,
    "radioType": "wcdma",
    "serving": 0,
    "signalStrength": -24
  },
  ...
]

• locationAreaCode is missing
• network and country codes are set to 0

I've noticed that my phone only seems to find exactly 1 or 32 wcdma cell towers. This does not occur when only 1 cell tower is recorded.

https://wiki.mozilla.org/CloudServices/Location/Software

Should be added, keep up the good work.

Currently only the MAC address of a Bluetooth beacon is included in the beacon data. It would be useful to collect more data from beacons, such as beacon type (e.g. iBeacon, Eddystone etc.) and its identifiers (e.g. UUID, major and minor in case of iBeacons)

Data for hidden Wi-Fi access points (i.e. ones with an empty SSID) shuold not be collected, but it seems that some hidden Wi-Fi access points are present in the app database. Investigate why it doesn't get filtered out earlier and fix it so that data for hidden Wi-Fi access points is ignored

Currently no data is collected for 5G cell towers, because MLS does not support them. Add support for collecting data for 5G cell towers

Before implementation, investigate:

• How does Ichnaea handle fields with values it doesn't support?
  • If reports with 5G cell tower data are discarded, there needs to be a toggle in the settings whether to send the data
• How to map cell tower data (tracking area code etc.) to the data model?

Hi, I found this project thanks to f-droid and I immediately wondered wether MLS was going to stick around. According to mozilla/ichnaea#2065 it doesn't look like it will.
So, I'm curios to know why this app now?

Currently the application collects data only when the data collection service is active (either started manually or automatically when the device is moving). Android supports using passive location provider to listen for locations that other apps have requested. Investigate if it would be possible to create reports in the background by using the passive location provider

Currently data can only be sent to a single endpoint that has been configured in the settings. It would be useful to be able to contribute data to multiple different location services

Think about:

1. What to do when uploading to one endpoint succeeds but uploading to another fails?
   • If we want to make sure that data is uploaded to each endpoint exactly once, we need to keep track for each report where we have successfully sent it
2. When to show reports as uploaded in the UI, after they have been uploaded to at least one endpoint or all of the endpoints?