Giter Club home page Giter Club logo

apple-corelocation-experiments's Introduction

Experiment with Apple's public WPS service

Reference: https://www.cs.umd.edu/~dml/papers/wifi-surveillance-sp24.pdf

How it works is explained in Apple's disclosure to congress: https://web.archive.org/web/20101208141602/https://markey.house.gov/docs/applemarkeybarton7-12-10.pdf

Feel free to poke around the code. Most relevant part is the protobuf and the stuff in lib. Experimental CLIs found in cmd, the main ones being the demo api and wloc.

WLOC

URL: https://gs-loc.apple.com/clls/wloc

I ran mitmproxy to find the URL used and later found iSniff-GPS on GitHub which was used as reference for the protobuf. The field names were uncovered by disassembling CoreLocationProtobuf.framework on MacOS. The relevant C code can be found here.

When requesting location services, MacOS/IOS sends a list of nearby BSSIDs to Apple, which then responds with GPS Long/Lat/Altitude of other nearby BSSIDs. The GPS location of the device is computed locally based on the signal strength of nearby BSSIDs.

Apple collects information from iPhones such as speed, activity type (walking/driving/etc), cell provider, and a whole bunch of other data which is used to build their database. This seems to be sent when a phone encounters a BSSID not in the existing database and excludes certain MAC address vendors known not to be stationary (e.g. IOS/MacOS hotspots).

Cell towers

Using the same API as above, we can also request cell tower information. MMC, MNC, CellId, and TacID is sent off and used to find the surrounding cell towers. It seems to have more data than opencellid.org but missing UMTS and GSM (only LTE is available).

Wifi Tile

URL: https://gspe85-ssl.ls.apple.com/wifi_request_tile

This is a new discovery while running MITM on an iPhone for an extended period of time. An endpoint from Apple takes a "X-tilekey" and returns all the BSSIDs and GPS locations of access points in the given region. Investigation is ongoing.

Here's an example color coded cluster between keys 81644851 and 81644861 (Cardiff).

Map of access points returned by the API

It seems each key denotes a single network of access points. This information seems to be collected from within the networks. The API is labelled as wifiQualityTileURL in the code base.

The tile key is a morton encoded number with what appears to be their own coordinate system (not based on GPS). I have used linear regression to successfully convert between GPS (long/lat) to their coordinates (code).

Update: Linear regression is not the correct solution. It gets worse as you go north/south.

Update 2: This took more work than I'd like to admit. First, I went through all references to tile keys on GitHub. There are behavior discrepancies between gojuno/go.morton and what Apple uses (In the GeoServices private framework). I ended up finding the implementation by heremaps to match and translated that into Golang. Then, based on the output, noticed that the xyz looked similar to OpenStreetMap's tiles. I used the firefox debugger on leafletjs to find the code used to generate the tiles from coordinates. Based on mentions of pixels and other keywords, I found buckhx/tiles in this 8 year old Reddit post. So to chain it together: tileKey → morton unpack → OSM tiles → pixel data → long/lat.

China

Perhaps for data sovereignty reasons, Chinese data is isolated from the main API. However, we are still able to access it from outside.

URLS:

A quick dig and whois shows that these are hosted within China at a Unicom IP. CNAME records show that Akamai is used for DNS and Kingsoft Cloud as a CDN.

To swap out the APIs, simply add -china to any of the CLIs in cmd.

Credits to JaneCCP for finding this info.

Interactive demo

go run ./cmd/demo-api and head to http://127.0.0.1:1974.

Click on any spot on the map and wait for a bit. It will plot nearby devices in a few seconds.

How it works: It first uses a spiral pattern to find the closest valid tile (limited to 20 to fail fast). Once it finds a starting point, it finds all the nearby access points using the WLOC API. It then takes the closest access point and tries again until there are no closer access points.

Mass data collection

It is relatively simple to collect data via the tile API. The working code is here. You can collect around 9 million records by going through every tile (on land). Some work was done to detect if a coordinate is in water (to skip) or in China (to choose the right API). You can find some details here.

Source for China's shapefile: GaryBikini/ChinaAdminDivisonSHP. This was forked to remove special administration regions which are part of the international API.

Source for water polygons here

Once that data has been collected, we can begin using the wloc API to fetch and explore nearby sections. My code for that is incomplete but a friend was able to fetch around 1 billion records. His source code can be found here: https://codeberg.org/joelkoen/wtfps/.

Seed data: https://tmp.duti.dev/seeds.db.zst

Grafana plot of collected seeds

Submitting data

To trigger data submission, turn on iPhone Analytics, Routing and Traffic, and Improve Maps in Location privacy settings. Then visit Maps, set a location, and start directions.

The endpoint is https://gsp10-ssl.apple.com/hcy/pbcwloc, labelled as _CLPCellWifiCollectionRequest in CoreLocationProtobuf.

It seems to only allow updating of existing records rather than collecting new records. More investigation is ongoing. You can find my attempt at uploading fake data here.

Spoofing your location

Using the info here, we can easily spoof your iPhone's location. You can find a server implementation here. After starting the server, run mitmproxy with this script which forwards wloc requests to your own server. Remember to change the IP address.

screenshot of maps with spoofed location

Ichnaea

This is the format used by Mozilla/Google/etc for their location service. We can imitate that with this. I have tested this with geoclue and it works just fine.

To do

  • If Apple uses device data to add new BSSIDs to their database, try to add fake data
  • Ichnaea compatibility and add direct support in geoclue (proposal)

apple-corelocation-experiments's People

Contributors

acheong08 avatar annabxlla avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

apple-corelocation-experiments's Issues

URLS

{
  undefined *puVar1;
  ID IVar2;
  undefined8 uVar3;
  ID local_30;
  class_t *local_28;
  
  local_28 = &objc::class_t::CLHEndpointSelector;
  local_30 = param_1;
  IVar2 = _objc_msgSendSuper2(&local_30,"init");
  if (IVar2 != 0) {
    uVar3 = _objc_alloc(&_OBJC_CLASS_$_NSURL);
    puVar1 = PTR__objc_msgSend_10096ef90;
    uVar3 = (*(code *)PTR__objc_msgSend_10096ef90)
                      (uVar3,"initWithString:",&cf_https://gsp10-ssl.apple.com/au);
    *(undefined8 *)(IVar2 + 8) = uVar3;
    uVar3 = _objc_alloc(&_OBJC_CLASS_$_NSURL);
    uVar3 = (*(code *)puVar1)(uVar3,"initWithString:",&cf_https://gsp10-ssl.apple.com/pds/pd);
    *(undefined8 *)(IVar2 + 0x10) = uVar3;
    uVar3 = _objc_alloc(&_OBJC_CLASS_$_NSURL);
    uVar3 = (*(code *)puVar1)(uVar3,"initWithString:",&cf_https://gsp10-ssl.apple.com/hcy/pbcwloc);
    *(undefined8 *)(IVar2 + 0x18) = uVar3;
    uVar3 = _objc_alloc(&_OBJC_CLASS_$_NSURL);
    uVar3 = (*(code *)puVar1)(uVar3,"initWithString:",&cf_https://gsp10-ssl.apple.com/incs);
    *(undefined8 *)(IVar2 + 0x20) = uVar3;
    uVar3 = _objc_alloc(&_OBJC_CLASS_$_NSURL);
    uVar3 = (*(code *)puVar1)(uVar3,"initWithString:",&cf_https://gsp10-ssl.apple.com/pbu);
    *(undefined8 *)(IVar2 + 0x28) = uVar3;
    uVar3 = _objc_alloc(&_OBJC_CLASS_$_NSURL);
    uVar3 = (*(code *)puVar1)(uVar3,"initWithString:",&cf_https://gsp10-ssl.ls.apple.com/hvr/aploc);
    *(undefined8 *)(IVar2 + 0x30) = uVar3;
    uVar3 = _objc_alloc(&_OBJC_CLASS_$_NSURL);
    uVar3 = (*(code *)puVar1)(uVar3,"initWithString:",&cf_https://gsp10-ssl.apple.com/psr);
    *(undefined8 *)(IVar2 + 0x38) = uVar3;
    uVar3 = _objc_alloc(&_OBJC_CLASS_$_NSURL);
    uVar3 = (*(code *)puVar1)(uVar3,"initWithString:",&cf_https://gsp10-ssl.apple.com/hvr/trc);
    *(undefined8 *)(IVar2 + 0x40) = uVar3;
    uVar3 = _objc_alloc(&_OBJC_CLASS_$_NSURL);
    uVar3 = (*(code *)puVar1)(uVar3,"initWithString:",&cf_https://gsp10-carry.ls.apple.com/hvr/alt);
    *(undefined8 *)(IVar2 + 0x48) = uVar3;
    uVar3 = _objc_alloc(&_OBJC_CLASS_$_NSURL);
    uVar3 = (*(code *)puVar1)(uVar3,"initWithString:",&cf_https://gsp10-ssl.apple.com/hvr/ion);
    *(undefined8 *)(IVar2 + 0x50) = uVar3;
    (*(code *)puVar1)(IVar2,"refresh");
  }
  return IVar2;

Notes on where to find things

Extract dyld shared cache.
https://www.jviotti.com/2023/11/20/exploring-macos-private-frameworks.html

_CLPNRCellTowerLocationReadFrom. _CLPWifiAPLocationReadFrom : CoreLocationProtobuf from Private frameworks.

How i run map tile

Help me to run this code and see perticular wifi in given geo code. I am new to this language or any python code for run or any database file

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.