Add API for requesting permission to receive device motion / orientation events about deviceorientation HOT 124 CLOSED

Browser-based content and embedded web view content in native applications will cease operation in all of the following use cases:

• Games and gaming
• Navigation
• Virtual Reality
• Augmented Reality
• 360 viewers (videos, panoramas)
• 3D scanning
• Gesture recognition
• Step counters
• Physical activity monitoring
• Sleep monitors

How many of these things are implemented as web apps/views? And why would requiring them to ask permission be that big of a deal? There is exactly zero people who should be able to access any of this data without asking for permission, in my opinion. And I honestly couldn't care less how many business models or apps would be affected by having to ask permission. If the app is scared of asking for permission first, I don't want them having my data anyway.

from deviceorientation.

For us, it is either this or disabling the API by default until we can make it safe.

All content relying on the API will break overnight. No replacement sensor access is provided on WebKit (Generic Sensors API). No alternative browser engines are allowed on iOS so there is no way to keep any applications relying on those APIs running. No deprecation strategy is planned warning developers of disruption or revocation of sensor data before they land in Safari. A cliff-edge backward-incompatible fall-off of usage is deemed the correct way to modify an extensively used 8+ year old web platform API.

Browser-based content and embedded web view content in native applications will cease operation in all of the following use cases:

• Games and gaming
• Navigation
• Virtual Reality
• Augmented Reality
• 360 viewers (videos, panoramas)
• 3D scanning
• Gesture recognition
• Step counters
• Physical activity monitoring
• Sleep monitors

Multiple proposals exist to make the emitted motion sensor data both a.) still useful for legitimate use cases and b.) resistant to raised security and privacy concerns by refining/limiting the data emitted in different ways. Those solutions will not be considered further because the effectiveness of those solutions needs further investigation/research even though they are cited as potentially resolving all current security and privacy concerns by the original investigators. Instead we will modify the API invocation to require permissions/user-gesture or just remove the whole API because it is easier, less implementation work and resolves all security and privacy concerns because then there are no use cases or usage anymore.

Now contrast this to the alternative approach/summary I posted above.

from deviceorientation.

Sacrificing UX in the name of a slight privacy increase is regressive at best. Not only will most end-users not understand what granting the permission actually entails, it just creates friction when trying to build solutions that “just work.”

In HTML5 eBook reader I created I actually leverage device orientation to determine when someone is laying down to suggest turning on night mode at certain hours. There are plenty of valid reasons not to put these standard features behind a permissions prompt.

Throttling polling rates is a decent mitigation.

from deviceorientation.

According to an article discussing iOS 12.2 beta features:

• Accessing sensor APIs is now disabled by default, so no accelerometer, gyroscope or magnetometer APIs are working unless the user enables a Settings option under “Safari”

Assuming this beta feature will ship in stable (edit: it will according to the release notes), it will diverge from Blink's plan to allow access to the existing level of precision and frequency by default.

A possible impact on iOS users is that websites start to show bespoke modal dialogs to iOS Safari users that ask to enable the "Motion & Orientation Access" option in iOS Settings (similarly to those "Add to homescreen" bespoke dialogs).

That seems like a suboptimal UX for iOS users, so I'm interested in finding a consensus position among implementers that would enable a better UX for all implementers. If requestPermission() is a fix all implementers can live with, we can make a case that it is needed to "fix a remaining known issue" in the API and see if anyone has concerns, and spec it. As said, the charter has some leeway in this regard, so we could likely pull that off with all implementers on board.

Seeking feedback from @cdumez and @reillyeon (with his Blink hat on).

from deviceorientation.

So what is the timeline on adoption of both the new Sensor API and the new permission features? Turning off this feature on IOS completely breaks our product! This is part of the web standard that we were using, and we've put a great deal of effort into developing for. To have it suddenly stop working is incredibly frustrating.

from deviceorientation.

I would very much appreciate a more sober reflection on how this might possibly affect developers and users. The privacy concerns are perfectly valid. But removing the API, or locking it down, will wipe out a complete genre of interactive online games. Not only the ones on the web, but rather those distributed to mobile devices via Cordova (et al.).

Additionally, how do you intend to communicate this permission request to the users? What's the difference between device acceleration and orientation data for the end-user? What if the user only permits either of them? Based on what rationale? What if the user does not understand the requested sensor function?

Does requesting permissions protect users? Or does it simply limit the “tracking” to big platforms that provide so many features that there's at least one with a credible use case for one of these APIs, opening them for the whole rest of the platform?

I'm unconvinced these questions have sufficiently been answered.

from deviceorientation.

I believe @geoffreygaren’s comment above explains why we want a permission api and user consent in WebKit. For us, it is either this or disabling the API by default until we can make it safe. The other proposals do not address all our concerns as explained by Geoff.

from deviceorientation.

@TimvanScherpenzeel

• We (Apple) consider all of the cited threats to be real threats. We trust our security experts

• We consider first-party fingerprinting to be a threat, not just third-party fingerprinting.

• We don't accept the argument that fingerprinting attack surface can be added freely because other fingerprinting attack surface already exists. We agree with the TAG Security and Privacy Questionnaire's view on sensor data. (In any case, there are risks more severe than fingerprinting.)

Given these considerations, it's unlikely we will enable this API by default on Apple platforms. We will however consider enabling it via user consent if there's agreement on a permissions API (which there n=mostly seems to be).

from deviceorientation.

Even though I agree with the general premise I have a few concerns:

The existing VR / AR / 360 camera viewers / experiences will break (similar to the autoplay audio policy did on Chrome). If certain permissions have existed on the web for a long time you cannot remove them without breaking anything.

I'm not sure if I agree with the proposal of @griffin2000 regarding a pop-up asking for permissions. I dread going to any news websites because as soon as the page loads I get bombarded with permission requests and I refuse them 99.9% of the time.

If we want the web to be able to compete with native applications (think seamless AR / VR) developers need powerful tools and be able to create a user experience that does not feel like installing an app everytime to go to a website. In my opinion the web should feel like an open space to surf and explore, not a walled off section with border security for each website.

How exactly are advertisers exploiting these sensors in a way that is deemed so disastrous and unsafe that the rest of the people who try to push the web into new creative directions have to suffer?

from deviceorientation.

I'm exasperated by the level of cognitive dissonance in this discussion.

The 3 recommendations to resolve all fingerprinting or key logging ability in this API and to prevent a breakdown of all existing web developer code are as follows:

• Obfuscation: Add a random constant in the range [-0.05, 0.05] to all deviceorientation data and [-0.005, 0.005] to all devicemotion data on each new invocation of the API. Optionally adjust that value during idle periods of inactivity for dynamic obfuscation (when e.g. the device is placed on a table).

• Quantization: Round all sensor data to a discrete set of values. For deviceorientation round the data emitted to e.g. 2 decimal places. For devicemotion round the data emitted to e.g. 4 decimal places.

• Frequency: Optionally, reduce sensor data frequency from 60Hz to e.g. 30Hz.

Why are these solutions, that do not impact the footprint of this API, still not being seriously considered? Even if that turns out to be an interim solution it presents a more positive development of this API than the disruptive, backward-incompatible changes being considered.

How would these updates not strike the correct pragmatic balance of resolving all privacy concerns while protecting access for legitimate users?

If these low-impact fixes for the original reported issue continue to be ignored this is what a responsible deprecation process for established Web APIs looks like: https://developers.google.com/web/updates/2018/09/chrome-70-deps-rems#deprecation-policy

Nobody has ruled out a permissions approach for this API if that is the least-worst acceptable solution. I'm just surprised that there is such little interest in progressively experimenting and fixing issues according to the recommendations of all studies in to the issues.

from deviceorientation.

We'd adopt the requestPermission() proposal here in Safari if there is general agreement on it.

Regarding @richtr's comment, we consider this to be a privacy and security hole that is being actively exploited in the wild. Given that, we don't think it is acceptable to do the slow deprecation path or to experiment with different forms of partially limiting the data to see if it works. If the stakes were lower, we would not feel the need to aggressively remove.

from deviceorientation.

Got it. This is really not an acceptable outcome. Disabling parts of the released, widely used, parts of web standard is not going to lead to this technology being accepted and used in widespread manner. At this point, we would have been better off dedicating our development effort towards supporting these features in a native app. We naively thought basing out solution on web technologies would lead to a more widely available solution.

Added a WebKit bug here, for reference
https://bugs.webkit.org/show_bug.cgi?id=194183

from deviceorientation.

You've put up a good fight, @richtr, but at this point I would like to see consensus and a proposal around a permissions API along the lines of what has been discussed, such as DeviceOrientationEvent.requestPermission() and DeviceMotionEvent.requestPermission()

Apple/WebKit aren't going to change their minds on this. iOS 12.2 is going to ship with device orientation/motion disabled by default. We need to expedite the process of formalizing an API so that future versions of iOS can ship with it. If we move quickly, maybe even iOS 12.2 could have this implemented.

The longer we deliberate, the longer we’ll have to put up with dialogs from developers such as "Orientation is disabled. Please enable in Settings > Safari > Motion & Orientation Access.", which is a terrible UX. A dialog such as "example.com would like to access device orientation. Don’t Allow/OK" would be totally fine for engaged users.

I say this as a developer of a VR-like web experiences. This is a breaking change, but a necessary one. Access to an API is better than no API at all.

from deviceorientation.

There are many softer approaches to be explored here as opposed to implementing a black-and-white “on/off” switch straight out of the gate.

Following your current course will effectively destroy nascent VR/AR/360/gaming companies and industries and pretending otherwise is silly.

I want to avoid a cliff-edge scenario where access to these sensors would change overnight without proper planning — particularly in browsers where no platform-alternative browser engines exist.

For that purpose there are many many intermediate steps, recommended in the cited papers - before you need to implement on/off switches for an 8+ year old API. Jumping straight for that approach is incredibly hostile to web developers. Have you evaluated the full impact that would have?

That is the purpose of the summary above.

from deviceorientation.

@richtr The permission pop up seems the obvious choice for this. The applications you list above (like ours) are absolutely valid uses of the API, but it wouldn't be at all problematic to have a popup. We also use the camera, and its absolutely straight forward to ask for permission when we do. It seems the only case where it would be problem is if you are doing something where it is not clear to your user that you are using the accelerometer. And in that case telling the user via a pop up is even more important.

from deviceorientation.

Two questions: 1) is this same change going to be required of native apps going forward? If not, why? 2) has anyone considered all the non-WebXR content this will break, like HTML5 games or websites that use orientation to change layout?

from deviceorientation.

I am in favor of adding requestPermission() methods to DeviceOrientationEvent and DeviceMotionEvent (as well as the equivalent Generic Sensors APIs). Acknowledging WebKit's concerns I think it is reasonable for an implementation to choose to completely deny permission for these APIs by default if they wish to. Blink's current plans are to,

1. Implement a permissions API allowing a site to request a particular level of precision and/or frequency.
2. Allow access to the existing level of precision and frequency by default.
3. Notify the user that a site is using the device sensors and allow them to revoke permission.

(2) is necessary to maintain compatibility with existing sites and the existing information leaks are partially mitigated by (3). (1) provides a framework for enabling (a) future use cases that require more data and (b) a move towards reducing the amount of data available without a permission in a gradual fashion in order to allow sites to adapt.

from deviceorientation.

@anssiko, I am positive on this. Blink's implementation will start as a no-op that immediately resolves with "granted" but I agree that it is important for it to be present nonetheless.

@cdumez, can you put together a draft of this change in a PR so we can discuss the specifics on how this will be specified? From the preceding discussion it sounds like we have settled on extending the DeviceOrientationEvent and DeviceMotionEvent interfaces with something like,

enum PermissionState {
  "granted",
  "denied",
};

[Exposed=(Window)]
interface DeviceOrientationEvent {
  static Promise<PermissionState> requestPermission();
};

[Exposed=(Window)]
interface DeviceMotionEvent {
  static Promise<PermissionState> requestPermission();
};

These are a specialization of the so-far unspecified requestPermission() method that should be added to the Generic Sensor API that will likely have parameters to control things like requested sampling frequency.

from deviceorientation.

Rigel from 8th Wall here - I updated an iPhone 8 Plus to iOS 12.2b today and noticed it broke several sites due to the issues laid out above. I filed a radar earlier and posted an openradar here: https://openradar.appspot.com/47645367

Can we expect by the time iOS 12.2 is released publicly, to have had sufficient lead time to update our software to accommodate any of the proposed changes so that none of our customers' websites break? There are several high profile VR/AR projects currently in production that this would directly affect. Thank you.

from deviceorientation.

@othermaciej Thanks for the responses here.

Just to be clear about the current state of the beta: the feature is currently disabled by default everywhere unless the user manually turns on the user preference. If we had per-site permission, then that would take priority over the setting.

Thanks for clarifying the current state. For the full 12.2 release you have (at least) 3 options:

1. Keep things as in the beta: many sites stop working, not many users would go through the hassle of turning on the option, horrible UX to talk users through the process.
2. Switch the default setting to be enabled: gives time for more discussion whilst still allowing privacy-conscious users to turn it off. [I'd throw in implementing other less-impactful mitigations in this option too, but given timeframes I think that is unlikely.]
3. Keep the default disabled but implement a permission request API: "progressive enhancement" use cases likely break, but those that require this API (360 photos etc) would be able to adapt.

I would strongly advocate for option 2 or 3.

Sorry for hijacking the bug for continuing this discussion. WebKit (rightly IMHO) don't consider it their bug, and the private nature of Apple's bug tracker shuts down discussion and prevents the web community from demonstrating the strength of feeling on this issue.

from deviceorientation.

Procedural steps need to be taken before a contribution can be added to a W3C WG spec from outside the group's current participants. This process is on the fast track and involved parties are informed.

The chairs will update this issue when we have new information to share. Meanwhile, technical discussion can continue on this issue. I can assure you everyone is highly motivated to resolve this issue. Thank you for your understanding.

from deviceorientation.

Determining which way the user is pointing their device is the entire purpose of this API.

Enabling usage for legitimate sites while preventing fingerprinting seems to be the objective of this proposal based on the contents and research in the cited studies.

The studies you referenced to justify this addition do not recommend putting this behind permission prompts as a first course of action. Those studies present many other options that could be considered.

Why the immediate and full rejection of all recommendations from your own references?

from deviceorientation.

@richtr you have two-out-of-three implementers saying that's not enough. That doesn't seem like a good summary to me.

from deviceorientation.

@richtr - I'd be concerned that reducing the quality of the sensor data would significantly degrade the quality of existing AR and VR uses of this standard API. In these webapps, (many of which already request camera permissions), an additional devicemotion permission would be much preferable than materially affecting tracking quality or breaking synchronicity with the camera.

from deviceorientation.

With my working group chair hat on: To new folks coming to this issue, let's try to avoid piling on and making this an unfriendly place to collaborate. We have engagement from three implementers on this thread and a proposed specification change should be available soon.

from deviceorientation.

This would not be acceptable on WebKit-side. Limiting sampling rate for those events would mitigate fingerprinting but not some of the other privacy issues (e.g. The site can tell if the user is standing or laying in bed).

We'd require a user gesture / permission for any of those events. We could also disable the API altogether as @annevk mentioned.

from deviceorientation.

@cdumez thanks for confirming your position. That’s what I meant.

@reillyeon would you be fine with the proposed requestPermission() that could in your case be no-op or return a promise that immediately resolves?

All - any concerns with the proposal, please let us know.

from deviceorientation.

The key point here, as someone who is building their technology and business model on these web standards, is this needs to be resolved quickly.

Personally I feel that a permission pop-up with the existing API is the way to go. But if that's not the solution you go with, that's fine. As long as the result is that our customers, who visit our website can actually use it, as specified by W3C standard we trusted, and dedicated a lot of engineering and design time to target.

from deviceorientation.

👍 Displaying a permission prompt should require a user gesture.

from deviceorientation.

In reply to:

The entire concept of spatial computing on the web is at stake (i.e. motion as a basic input driver to rival mouse, keyboard and touch input drivers). Basic input drivers should not be subject to permission-based access controls (by all means, all other sensor APIs belong behind permissions but absolutely not coarse-grained motion and orientation sensor access).

My view is that the reason why we are looking for a different solution than default-enabled exposure is precisely because, for the browsers in question like Mobile Safari, the spatial computing aspect is not likely to be a primary function, and therefore this data is not likely to be a primary input source.

As an example, if Avaer were to decide that ExoKit is intended to be an immersive browser with spatial computing as a primary function, then ExoKit could decide to default to enabled for https sites or localhost (or perhaps to any site displayed in immersive mode, if that were too restrictive).

My intuition is that obfuscation of the data is at this moment an uncertain solution, and that a more intentional permission model will be needed, with browsers ideally remembering site permissions where appropriate to reduce user friction.

from deviceorientation.

A PR #68 to address this issue has been submitted for review.

Thanks @cdumez!

from deviceorientation.

From the study referenced in that Wired article (PDF paper: https://sensor-js.xyz/webs-sixth-sense-ccs18.pdf) the recommendations given to browser vendors are as follows:

• W3C’s recommendation for disabling sensor access on crossorigin
  iframes [80] will limit the access from untrusted thirdparty
  scripts and is a step in the right direction. However,
  Safari and Firefox are the only two browsers that follow this
  recommendation. Our measurements indicate that scripts
  that access sensor APIs are frequently embedded in crossorigin
  iframes (67.4% of the 31 444 cases). This shows that
  W3C’s mitigation would be effective at curbing the exposure
  to untrusted scripts. Allowing sensor access on insecure
  pages is another issue where browsers do not follow the
  W3C spec: all nine browsers we studied allowed access to
  sensors on insecure (HTTP) pages.

• Feature Policy API [16], if deployed, will allow publishers
  to selectively disable JavaScript APIs. Publisher may disable
  sensor APIs using this API to prevent potential misuses by
  the third-party scripts they embed.

• Provide low resolution sensor data by default, and require
  user permission for higher resolution sensor data.

• To improve user awareness and curb surreptitious sensor
  access, provide users with a visual indication that the sensor
  data is being accessed.

• Require user permission to access sensor data in private
  browsing mode, limit resolution, or disable sensor access all
  together.

[80] Rich Tibbett, Tim Volodine, Steve Block, and Andrei Popescu.
2018. DeviceOrientation event specification.
https://w3c.github.io/deviceorientation/

[16] Ian Clelland. 2017. Feature policy: Draft community group
report.
https://wicg.github.io/feature-policy/.

This specification could instead be updated to include normative text that limits the sampling rate for deviceorientation, deviceorientationabsolute and devicemotion events to less than or equal to 60 Hz as effective mitigation to fingerprinting and other security and privacy issues.

At that point, a permissions API could then be introduced for accessing higher frequency sensor data from devices. Or, just require higher frequency sensor data to be obtained only via the Generic Sensors API going forward.

Incremental improvements based on these research recommendations seem possible. Putting all access to these events behind a permission dialog seems like a Really Big Gun to start with. It may still be required to eventually look at that approach but for an API that has been available for 8+ years this could be extremely damaging for legitimate use cases given its widespread adoption.

from deviceorientation.

Why the immediate and full rejection of all recommendations from your own references?

Though the attached research paper investigated quantization,

(a) It acknowledged that quantization only reduced the fingerprinting surface area, and did not eliminate it;

(b) It acknowledged that quantization, though it did not break a toy game, might still break other important use cases;

(c) It did not address at all concerns like key logging and other forms of privacy invasion, which are mentioned in the Wired article;

(d) It did not address the unusually high power impact of the motion and orientation (which we probably should have mentioned earlier).

I don't think we reject quantization as an area of investigation. Rather, we acknowledge that quantization is an area of investigation, and we also acknowledge the need to solve this privacy and security problem without waiting for the conclusion of an open-ended investigation.

from deviceorientation.

Companies like UnifyID use gyroscope data for user gait profiling. I would definitely prefer if gyroscope data required permission to access.

from deviceorientation.

@anssiko What sort of feedback are you looking for? I filed this issue so requestPermission() is definitely sometime we want and it would definitely be a better situation than having the API disabled by default. @annevk was also on board with such API, so that's 2 implementers.
If Blink wants to keep providing access to the API without user consent, this is fine, the requestPermission() could always be a no-op for them.

@larsgk FYI, third-party content already does not have access to the API in WebKit so this discussion is about first party for us. Also note that the whole reason we want a permission API is to avoid over-prompting the user since there is a lot of content out there already using the API but not actually requiring it (e.g. tracking). With regards to Web Applications (PWA), we'll need to discuss this internally, this is more of a system policy and I do not think this should be spec'd.

from deviceorientation.

@richtr The problem with those recommendations is that they still break existing applications but in more subtle ways. As outlined in my previous comment before we can realistically restrict the fidelity of the data available via this or the Generic Sensors API there needs to be a mechanism for developers to express to the user agent the level of data fidelity required for their particular application. The proposed requestPermission() amendment to this API, and the additional parameters to the Sensor constructor in the Generic Sensors API, are that mechanism.

from deviceorientation.

@erikmchut Your points are valid wrt VR/AR usage. These changes would increase "motion sickness" considerably.

However, the DeviceOrientation Events API is not the correct API to use to implement VR/AR. The fact it is used on WebKit-based browsers is because it currently provides the only way to polyfill access to requisite sensor data in those browsers.

There are other/better APIs for Web VR/AR implementations available for implementation:

https://immersive-web.github.io/webvr/spec/1.1/
https://www.w3.org/TR/gyroscope/

The issue is that these APIs are not currently supported in WebKit. Instead of implementing those APIs, the proposals here suggest we instead modify/remove this existing API. Unfortunately that breaks other "progressive enhancement" type uses of this API where it doesn't make sense to show a permissions prompt to the user.

What I proposed above that we make deviceorientation and devicemotion un-fingerprintable and un-key-loggable by making it a source for lower resolution data right now. A lot of non-VR/AR use cases are also at risk here that do not require such precise data.

On implementation of the above specifications in web browsers we would then have the following scenario:

1. Low-resolution deviceorientation and devicemotion data, that eliminates fingerprinting or key-logging abilities remains available by default without permissions.
2. High-resolution sensor data is available via both the WebVR API and the Generic Sensors API (see above for links) with user permissions being acquired first.

This represents a way to keep things moving on a web page that need to move (e.g. motion parallax effects, custom gesture recognition, CSS animations attached to device pose, etc) with a path for developers to "upgrade" that default low-level data access to something suitable with the permission of users via the APIs listed above. By introducing a permissions model here we lose the ability to do "progressive enhancement" visual effects or features (e.g. https://github.com/rikschennink/shiny).

@othermaciej thank you for your response.

If the stakes were lower, we would not feel the need to aggressively remove.

This is detrimental to every stakeholder concerned and a real shock to the established system. I'm not aware of any other such widely used Web API receiving the same treatment previously. As such it feels unprecedented in the history of Web Standards to take such a drastic course of action instead of progressively fixing what is a fixable Security and Privacy issue.

from deviceorientation.

For any new readers: earlier in this thread the new study on https://sensor-js.xyz/ is referenced. The relevant paper can be found here: https://sensor-js.xyz/webs-sixth-sense-ccs18.pdf. From what I understand the main issue is the misuse of the sensors for browser fingerprinting and various ad-tracking techniques (such as distinguishing between bots and real users).

In the study some other studies and exploitations are referenced (reading PIN's, reading keystrokes, sniffing browser history) which many have already said in the past to be highly impractical and extremely difficult to pull off in a short amount of time in the confined space that is the web browser. They are neat hacks (see: https://blog.lukaszolejnik.com/stealing-sensitive-browser-data-with-the-w3c-ambient-light-sensor-api/) but I have problems considering them to be actual threats.

The paper is also not clear on how this raw data is actually used to help towards browser fingerprinting seeing as the sensor data rapidly changes over time and is not a consistent identifier over a longer period of time. If someone could explain that to me in more detail it would be highly appreciated.

I think there is plenty we can do to make it a lot more difficult for advertising networks to misuse these sensors (for example block any 3rd party iframe embeds / cross-origin sources from using these sensors by default, even secure sources). Browser fingerprinting is still very successful and won't be affected in any major way by this change: just have a look at fingerprinting through WebRTC / WebSocket / canvas / font detection / plugin support / evercookie / user agent / WebGL renderer & extensions.

The damage of this proposed change far outweighs the possible benefits in my opinion and should be seen as a last resort. I feel that we should push for blocking default sensor access through 3rd party services (even secure ones) but leave it open for 1st party users that can use these sensors in creative ways (AR / VR / parallax effects etc..).

from deviceorientation.

Thank you for your clear reply @othermaciej.

In that case I trust that you and the security experts are making the right choice in making this change. I don't have the experience in what it takes to make a browser safe nor do I have a clear view on exactly how malicious parties are exploiting the mentioned sensor API's. The Gyrophone reference in the TAG Security and Privacy Questionnaire is really clever.

I hope we can find an elegant solution for how the requesting of permissions is done. To be completely honest, when I started looking into the API's myself about a year and a half ago I was very surprised users did not have to give explicit permission.

from deviceorientation.

According to an article discussing iOS 12.2 beta features:

• Accessing sensor APIs is now disabled by default, so no accelerometer, gyroscope or magnetometer APIs are working unless the user enables a Settings option under “Safari”

Assuming this beta feature will ship in stable (edit: it will according to the release notes),

[...]

I've seen quotes from earlier in the day from the release note page including a sentence that the option to allow sensor events would be disabled by default. Looking at the link now, there is no comment about the default state of these options. @anssiko can you confirm this was explicitly specified before? Anyone from the Safari team care to clarify?

The worry of this change is it buries the option in the settings without even offering a permissions API to the browser. I'm on the AR/VR use case side and think a permission prompt perfectly acceptable. However having to direct users to some obscure part of the settings app to enable this option definitely seems a step too far which will be damaging for the web.

My hope is Apple/Safari have decided to keep the setting enabled by default in 12.2. That will give time to finalise the permission model without breaking web content, but still give the privacy conscious the opportunity to disable globally.

from deviceorientation.

@richtr we are members of the immersive web working group and are following the progress there closely, but the device motion / orientation APIs are in broad use today and this change affects the community acutely.

@othermaciej, while the devicemotion permissions API is under consideration, could the WebKit team help lessen the impact to existing AR pages by allowing access to deviceorientation and devicemotion when a user has explicitly approved camera permissions for the page through getUserMedia()? This is used in 6 Degree-of-Freedom JavaScript libraries that power pages like the Sony Pictures' Spider-Man AR Experience: https://www.intothespiderverse-ar.com/ (currently broken in beta Safari 12.2b). Since the iOS getUserMedia permission is requested on every page load and typically only on high-trust sites, it seems likely to mitigate the fingerprinting concerns and privacy concerns from additional motion information.

from deviceorientation.

We acknowledge the value of these things and we wish we had a solution that both respects privacy and provides maximum user convenience. Our first priority has to be to fix the privacy exploit though.

from deviceorientation.

This specification is superseded by the Generic Sensors family of specifications, which include integration with the Permissions API. I think that this specification should be updated with a similar integration by requiring "accelerometer" and "gyroscope" permissions for deviceorientation and devicemotion events and additionally "magnetometer" permission for deviceorientationabsolute events. This would match the existing Orientation Sensor, Accelerometer, Gyroscope and Magnetometer specifications which provide access to the same or similar data.

Is there any plan to implement navigator.permissions.query() in WebKit?

Blink is currently considering similar changes to start notifying users or prompting when sites request access to device sensors. The plumbing is already implemented to manage these permissions however they are still granted by default.

from deviceorientation.

@reillyeon If we must add permissions to this API then your approach seems like a least disruptive way to do that - progressively reducing the data available (assuming (2) continues to be available by default without permissions). We could also add a relevant console warning message during this process informing developers of the planned change to access in future releases.

What the research in https://arxiv.org/pdf/1605.08763.pdf suggests is that a combination of reducing the frequency of events + sensor obfuscation + sensor quantization would be enough to significantly fix security and privacy concerns in this thread without significantly disrupting web app UI and UX for legitimate actors. Permission dialogs or any changes that disrupt a user's flow while using a web app should be considered as a last resort (when all you have is a hammer...).

There is going to be a disruptive impact of adding permission requirements for sensor events on the nascent web VR/AR/360 ecosystem. e.g. Developers will need to start wrapping their calls for sensor events within user-gesture events IIUC. It is unclear if or how permission requests would display if called from within e.g. iframes (with e.g. correct allow attributes cross-origin).

Anything we can do to reduce the impact of blocked requests and disrupted UI/UX experiences within this ecosystem - gradually introducing such disruptive changes only if all other techniques have been tried and deemed unsatisfactory after further studies, making developers aware of these updates via e.g. console.warn before they land and not just landing these changes one day in mainline browsers without letting existing API consumers adapt - would be very welcome here.

from deviceorientation.

In summary, changes could be made immediately to implementations by obfuscating, quantizing and/or reducing the rate of sensor event data emitted without impacting the existing API footprint or validity/conformity of implementations to this specification. We could also immediately add security and privacy considerations to this effect to this specification based on these emergent research studies since last publication. We can then continue to discuss additional requirements / API changes as required, hopefully with a view to reducing the impact of those changes on current legitimate consumers of this API.

from deviceorientation.

@geoffreygaren In reply to this comment:

Can you cite some evidence to support your assertion that obfuscation, quantization, and frequency reduction solve the privacy and consent problem?

Are you seriously suggesting that, returning e.g. integer values for deviceorientation is going to allow an attacker to identify a single device/user from a database of four billion smartphone devices?

Everything being discussed is a shade of that argument that there is, with certainty, a level of default access available that provides enough privacy and security protections while not completely disabling current uses.

from deviceorientation.

Replying to this comment from @geoffreygaren:

Everything being discussed is a shade of that argument that there is, with certainty, a level of default access available that provides enough privacy and security protections while not completely disabling current uses.

Can you cite some evidence to support your assertion that obfuscation, quantization, and frequency reduction solve the privacy and consent problem?

Empirical evidence for the case for obfuscation, quantization, and frequency is, I agree, important to proceed. If those approaches are capable of eliminating all privacy and security concerns then the consent problem is also taken care of implicitly. We are in the current predicament due to the lack of clarity within existing studies on the exact protections that would eliminate personally identifiable data concerns. Existing studies do not go far enough in evaluating the full effects that obfuscation, quantization, and frequency can have on eliminating personally-identifiable information from sensor data and, thus, eliminating fingerprinting. However, it remains equally disappointing to see a progressive, pro-active approach based on further investigation being dismissed in favor of a quick short-term solution.

In Bojinov-Boneh-Michalevsky-Nakibly, Section 6.1 states the following:

" In contrast to audio-based fingerprinting, there is no
good way to feed a signal into the accelerometer, namely
exert a known acceleration force; instead we take an
approach of performing background measurements and
waiting until there is enough data to estimate the accelerometer
calibration parameters. We perform a measurement
every time the phone is at a resting position, or
more precisely, the phone is at a constant velocity (no acceleration).
Note that in most reasonable cases it is very
unlikely that a phone will not be at rest for an extended
period of time. Detecting the phone is at rest is relatively
straightforward: the measured acceleration vector should
be static and its magnitude should roughly be equal to g
(gravity). "

Figure 6 in that same paper provides a scatter plot of all z-axis accelerometer data that is collected from 17 iOS devices when they are placed at rest on a table (i.e. the device has no velocity in any axis thus fulfilling the required calibration environment requirements as explained in the quote above).

All the iOS devices in that study present unique g values that fall within in the range [0.993, 1.004]. That study showed that devicemotion data is potentially fingerprintable based on those discrepancies between different devices (although many devices reported values do overlap with each other within that same range).

In a previous comment above I suggested obfuscation as follows should be added to emitted devicemotion data:

• Add a random constant in the range ... [-0.005, 0.005] to all devicemotion data on each new invocation of the API.

By applying these obfuscation values to the data collected in that study, that data from 17 devices becomes indistinguishable from any subsequent invocation of the devicemotion API on the same device OR any invocation of the devicemotion API on any other device. Where the term "fingerprint" is defined as a means to identify and then re-identify a user on a subsequent visit due to consistencies in readings, this API would become useless for the purposes of ongoing identification of devices or users between visits.

Finally, in the same previous comment above, I continued that discussion around obfuscation with the following:

Optionally adjust that value during idle periods of inactivity for dynamic obfuscation (when e.g. the device is placed on a table).

Implementing this additional protection goes even further to prevent any fingerprinting from occurring within the same continuous, long-running stream of devicemotion data collected on a device. It does this by adjusting the obfuscation constant marginally (by a much smaller value) whenever the device is detected to be at rest (e.g. placed on a table or held still with no exerted velocity in any axis).

Finally, the presence or absence of any obfuscation constant in this data does not affect general calibration of devicemotion sensor data. Calibration of device sensors is performed under the same conditions as described above (when no velocity is present in any axis) - at which point we can estimate the calibration offset from earth's precise gravitational constant of 0.980665 (9.81 m/s^2) on each invocation.

In this comment, @geoffreygaren wrote:

Or, if the solution is as simple as you say it is, you might as well just code it up and we can test it empirically.

In WebKit, the most trivial code change would be to add a per-instance-bound obfuscation constant (CURRENT_OBFUSCATION_CONSTANT) in the range [-0.005, 0.005] to the collected gravitational values on these lines:

- totalAccel.x = userAccel.x + gravity.x;
+ totalAccel.x = userAccel.x + (gravity.x * (1 + CURRENT_OBFUSCATION_CONSTANT));
- totalAccel.y = userAccel.y + gravity.y;
+ totalAccel.y = userAccel.y + (gravity.y * (1 + CURRENT_OBFUSCATION_CONSTANT));
- totalAccel.z = userAccel.z + gravity.z;
+ totalAccel.z = userAccel.z + (gravity.z * (1 + CURRENT_OBFUSCATION_CONSTANT));

A basic patch of this type would provide enough initial protection to eliminate the majority of fingerprinting identified in existing privacy studies and specifically breaks the fingerprinting characteristics presented in the Bojinov-Boneh-Michalevsky-Nakibly paper.

By applying just 'obfuscation' initially would indicate effective removal of the ability to fingerprint devices/users via devicemotion if those fingerprinting attack vectors rely on any form of "at rest" calibration process. Other attack vectors (such as measuring the sensitivity between a series of sensor data readings), and again subject to ongoing study, could be further mitigated with quantization protections.

The entire concept of spatial computing on the web is at stake (i.e. motion as a basic input driver to rival mouse, keyboard and touch input drivers). Basic input drivers should not be subject to permission-based access controls (by all means, all other sensor APIs belong behind permissions but absolutely not coarse-grained motion and orientation sensor access).

Regarding battery life, a reduction in the frequency of event firing would materially reduce the amount of event collection and propagation required within a web browser's code. Execution of those code paths would be reduced by a factor of 50%+ (e.g. when moving from firing of events from 60 Hz to 30 Hz). Anything >= 15 Hz is still entirely acceptable for the use cases documented within this specification (i.e. non-VR/AR use cases). For use cases that require high-resolution data (at greater accuracy and delivered at higher frequencies) such as the use of motion and orientation sensor data within VR/AR scenes there are other, more suitable, APIs that should be implemented, gated with permissions and used by web developers (Generic Sensors API, WebVR API, WebXR API). Those APIs, obtained via permissions, should remain unencumbered with any forms of obfuscation, quantization, and frequency capping or limiting.

All of this is to say: we do still need more independent feedback on the effectiveness of these mitigations before pushing existing and future spatial computing use cases off a cliff. The most reasonable approach at the current time would be for Apple to implement a simple form of obfuscation as explained above in WebKit code and to also enable the "Motion and Orientation Sensors" setting by default with those protections implemented in an upcoming release. This would provide provable protections and mechanisms to eliminate the most egregious fingerprinting attack vectors, as identified in existing sensor privacy studies that require devices to be calibrated "at rest", while still allowing on-going research to continue in to additional protections that could be added on an ongoing basis in the future. This approach allows Apple and other browser vendors to continue to research, tweak and adapt sensor privacy protections in future releases while still actively preventing fingerprinting in the immediate short-term. In any other approach we are simply stating that spatial computing use cases on the web are immaterial to this discussion, spatial computing as a concept and basic input has no place or future on the web and spatial computing use cases will be eliminated as collateral damage when building more walls around access to this data instead of focussing on eliminating personally-identifiable features from the data itself. The stakes of this discussion, in the context of the ability for the web to support spatial computing, and the course of action being taken here to add permissions or otherwise restrict access to data warrants extreme caution. Nobody will stop browser vendors from their current course of action but I simply wish that the impact and loss to open motion access to the web was clearly understood in the context of the elimination of spatial computing uses (to the benefit of native applications).

from deviceorientation.

@richtr yep, it's catchy but choose the ransom noise level needed. And thanks @tangobravo for resurfacing that privacy is not merely about fingerprinting. Actually many of the risks I considered do not relate to fingerprinting at all. Sensors is such a rich data source...

from deviceorientation.

@cdumez @reillyeon @othermaciej @anssiko What is the process now for agreeing upon an API and adding it to the official spec? I think most of us would like to see this discussion moved on and to implementing a solution for developers.

from deviceorientation.

@nuthinking If you scroll upwards, you will see that there are several discussing work to mitigate potential harm, but also significant discussion that current techniques may not be sufficient. That discussion has been going on for some time, and appears likely to continue for some time.

In the meantime, I think it makes sense to do something additional to mitigate, and the obvious choices are to either disable by default with no interactive reprieve (that entirely kills the use cases, which is why folks are looking for other solutions here) or to treat like other sensitive inputs with interactive opt-in (camera, microphone). Those inputs cannot generally be degraded or rate limited without significantly impacting their primary use cases, and I suspect the same may prove true here.

from deviceorientation.

@reillyeon I may be missing something but where is that API that JS actually calls to request permission for something? navigator.permissions.query() seems to only query, not request.

There is a section in the Permissions API spec about "requesting more permission" but it seems to merely be an algorithm, no actual API for it.

What we want in WebKit is really an API to ask for permission to access device orientation. Without such permission request, the content would not be getting the events at all.

from deviceorientation.

That is correct, I forgot that the navigator.permissions.request() method has been removed in favor of each specification integrating its own method for requesting more permission, but calling into the algorithm you mention. A cursory look at query() made me think that that had been integrated into it instead.

Given that the Generic Sensors specifications I linked to above should be updated to include a requestPermission() method similar to Notification.requestPermission().

As the DeviceOrientation API does not provide an object in the global scope like AbsoluteOrientationSensor or Notification (except perhaps DeviceOrientationEvent) what do you think about not adding a method specific to this API but instead relying on the ones that need to be added to the newer specifications?

My recollection is that WebKit does not implement the deviceorientationabsolute event and so implementing a single RelativeOrientationSensor.requestPermission() method would suffice for your purposes.

from deviceorientation.

I think relying on a Permissions API from another spec would probably work (although, I'd have to see what it looks like first). That said, it would be bit confusing to use Permissions API from another (newer) API and predicate using the (older) API on it.

How about for the 'devicemotion' events, would it be a LinearAccelerationSensor.requestPermission() ?

Honestly, the fact that the names do not match would be super-confusing IMO.

I get that the DeviceOrientation API does not have a global scope object at the moment but it seems like it could have one. Or I actually like the idea of adding static requestPermission() methods to the existing DeviceOrientationEvent / DeviceMotionEvent interfaces.

from deviceorientation.

navigator.permissions.request() is specified in https://wicg.github.io/permissions-request/

@jyasskin probably knows the status of that work.

from deviceorientation.

I think relying on a Permissions API from another spec would probably work (although, I'd have to see what it looks like first). That said, it would be bit confusing to use Permissions API from another (newer) API and predicate using the (older) API on it.

I believe that use of either API should be predicated on the same permission since they expose the same capability. Having two functions that request the same permission feels like unnecessary bloat but I'm willing to accept that in favor of not confusing developers.

How about for the 'devicemotion' events, would it be a LinearAccelerationSensor.requestPermission() ?

devicemotion events should require the "accelerometer" and "gyroscope" permissions which conveniently is the same as the ones I think should be required for deviceorientation events so a single call could provide the necessary permissions for either event type.

Honestly, the fact that the names do not match would be super-confusing IMO.

I don't disagree. It sounds like navigator.permissions.request() is not as abandoned as I thought it was. That API feels like a better solution.

I get that the DeviceOrientation API does not have a global scope object at the moment but it seems like it could have one. Or I actually like the idea of adding static requestPermission() methods to the existing DeviceOrientationEvent / DeviceMotionEvent interfaces.

That seems reasonable if doing this solely through the Permissions Request API is untenable.

from deviceorientation.

So if I understand correctly, it would look like:

navigation.permissions.request({ name: “gyroscope” }); // For DeviceOrientationEvents
navigation.permissions.request({ name: “accelerometer” }); // For DeviceMotionEvents

This does not look bad actually.

from deviceorientation.

You need both "gyroscope" and "accelerometer" for both so it would look like:

Promise.all([navigation.permissions.request({ name: “accelerometer” }),
             navigation.permissions.request({ name: “gyroscope” })])
    .then(results => {
      ...
    });

from deviceorientation.

I think w3c/permissions#158 is the most recent discussion about permissions.request(...), and it links to the older discussions. Firefox folks have generally been opposed, Chrome's been in favor, and Edge and Safari have ignored it. I haven't been paying a lot of attention in the last year so can't really help guide here.

from deviceorientation.

@annevk Based on w3c/permissions#158, it looks like Mozilla considered gating the old deviceorientation / devicemotion events behind a permission as well. Did you end up doing it or do you still have plans to do so?

from deviceorientation.

@cdumez we haven't made progress on this yet I believe, but I strongly suspect we're still interested. Perhaps not in a generic request() method though, but something specific for this API. Either that, or seeing if we can remove the API altogether (we don't expose it so widely I think).

cc @ehsan

from deviceorientation.

Yes, I discussed it internally a bit and I do not think we are interested in the generic permissions.request().

A few things we could do:

1. add a permission request api to this spec
2. drop the api altogether but there are some legit use cases so it would be unfortunate.
3. maybe require a user gesture when adding the event listener (and then we’d show a prompt). Avoids over prompting and adding any api.

I would have preferred option 1 I think but given the mess with permissions it seems, I am starting to think about option 3.

from deviceorientation.

A note on scoping with my chair hat on:

As @reillyeon pointed out earlier, DAS WG's position is this specification is superseded by the Generic Sensors family of specifications which include integration with the Permissions API. The DAS WG considers this spec to be in maintenance, and as such no new feature work per charter. The charter is not super clear on this, says "fix remaining known issues and to finish" so I'm open to discussion if someone feels strongly we should add new API surface to this spec.

An alternative transition plan to browsers who are planning to unship this legacy feature sometime in the future would be to first implement (applicable parts of) the new API surface (see Generic Sensor API and related concrete sensors at https://www.w3.org/das/roadmap), and deprecate the legacy API surface only after a reasonable grace period during which web developers are recommended to migrate to the new API that provides the permission hooks and other privacy enhancements.

from deviceorientation.

If the aim is to cripple/disable 3rd party ads/scripts to access sensor data for evil intentions, using Feature Policy seems to be a good way forward. Has this been considered? If we want to prevent the main page from accessing sensors, we have to consider how requiring installation of a native app would provide better assurance - if any at all.

In addition, permissions could be granted when installing the web application (to home screen/pwa). Has this been considered?

If too many features require user approval or flipping switches in the global settings, the effect will most likely disappear (e.g. like most people run as Administrator on Windows as the alternative is just too cumbersome).

from deviceorientation.

@reillyeon Yes, your proposal looks good to me. With regards to the PR, I'll try and make one soon but this is a busy period for me.

from deviceorientation.

(Related: I submitted a PR #62 that converts the spec source from its custom HTML format into Bikeshed format to make it easier to edit the spec as well as to publish updates using the W3C's publication system Echidna. @cdumez when you start working on the PR, I recommend you base it off of the Bikeshed source.)

from deviceorientation.

@richtr You keep asserting that obfuscation, quantization, and frequency reduction solve the privacy and consent problem.

But the only sources cited in this conversation acknowledge that all such efforts so far have failed to solve the privacy and consent problem.

Regarding fingerprinting, Das-Borisov-Acar-Pradeep acknowledge that users who want privacy still need to “disable sensor access all together” (Section 7, DISCUSSION AND RECOMMENDATIONS”), and they “recommend browser vendors to rethink the risks of exposing sensitive sensors without any form of access control mechanism in place” (Section 8, LIMITATIONS). Das-Borisov-Chou-Mughees acknowledge that their best effort to curtail fingerprinting still has a Random Forest F-score of 26% (Section 6.2, Effectiveness of Countermeasures).

Regarding key logging and other forms of eavesdropping, nobody has suggested that obfuscation, quantization, or frequency reduction is a mitigation at all. In fact, Das-Borisov-Chou-Mughees demonstrate that these techniques intentionally preserve substantial signal and, by design, “many applications of sensor data are unlikely to be affected” (Section 7, CONCLUSION).

Regarding battery life, obfuscation, quantization, and frequency reduction use the same underlying sensors, so they plainly do not resolve the problem.

Can you cite some evidence to support your assertion that obfuscation, quantization, and frequency reduction solve the privacy and consent problem?

Or, if the solution is as simple as you say it is, you might as well just code it up and we can test it empirically.

Until then, we have an absence of evidence to support your assertion, and a presence of evidence to refute your assertion, so we have to conclude that your assertion is without merit.

from deviceorientation.

Just to be clear about the current state of the beta: the feature is currently disabled by default everywhere unless the user manually turns on the user preference. If we had per-site permission, then that would take priority over the setting.

I don't think we'd piggyback this on permission for another API. That's likely to be confusing to users. Details on what Apple can or should do are probably better reported on bugs.webkit.org than hear though. I'm offering updates only to set context on our need for the Permission API.

from deviceorientation.

Just to be clear about the current state of the beta: the feature is currently enabled by default everywhere unless the user manually turns on the user preference. If we had per-site permission, then that would take priority over the setting.

@othermaciej Do you mean disabled everywhere?

from deviceorientation.

@griffin2000 Yeah, I did. Corrected in my original post.

from deviceorientation.

widely used

Not that I doubt you on this, but are there any statistics showing how much this is in use? I personally have never run into a site that used any of these features.

from deviceorientation.

Fundamentally, I want to be clear this specification was never designed to be used for VR/AR. The fact this API is being currently used for AR/VR tracking, and that is therefore considered one of its use cases, is due to real VR/AR APIs not being implemented in WebKit (they are implemented in one form or another in all other mobile browsers).

What we are actually losing is the ability to deliver all the other use cases where a default level of coarse-grained data is still very useful and a permission prompt simply doesn’t work. That includes all progressive enhancement features, motion-based animations and custom gestures as a basic user input (e.g. shake or tilt to sit alongside touch, keyboard and mouse inputs).

So this is a final request that these use cases are, firstly, acknowledged as being in operation today. Secondly, that there is an understanding these uses will disappear from the web. And, thirdly, that these use cases do not simply get discarded as collateral damage when talking about VR/AR tracking or precision more suited to other APIs.

Longer term there is hope that a default coarse-grained level of access will continue to be made available (through e.g. the Generic Sensors API) but that is of little consolation for those that rely on some form of sensor access being available today.

Given more time than is currently being made available to prevent a cliff-edge we could resolve all these issues and conflicts in the best interests of everyone.

FYI, I will now recuse myself from further discussion here to allow others to make progress on selected proposals (unless pinged directly for input or clarification on previous points).

from deviceorientation.

Are you seriously suggesting that, returning e.g. integer values for deviceorientation is going to allow an attacker to identify a single device/user from a database of four billion smartphone devices?

"returning e.g. integer values for deviceorientation" is an incomplete specification.

What I'm suggesting is that you have provided no evidence that you could complete that specification in a meaningful way. And I have provided evidence that you can't.

If the solution is indeed trivial, you are welcome to end this debate at any time by specifying the solution and providing evidence that it works.

Everything being discussed is a shade of that argument that there is, with certainty, a level of default access available that provides enough privacy and security protections while not completely disabling current uses.

Indeed, you do keep asserting that you could solve the privacy and consent problem. But asserting something doesn't make it so. Evidence is how we evaluate truth, and you have provided none.

from deviceorientation.

Apple/WebKit aren't going to change their minds on this. iOS 12.2 is going to ship with device orientation/motion disabled by default. We need to expedite the process of formalizing an API so that future versions of iOS can ship with it. If we move quickly, maybe even iOS 12.2 could have this implemented.

This is by far the most important aspect of this discussion. The exact method that happens FAR is less important than coming up with a solution that can get this fixed for the 1.2.2 full release.

If that fails it will be really bad for web standards as a whole, and, speaking as a developer, make me much less likely try and support these kind of standards in the future.

from deviceorientation.

Two questions: 1) is this same change going to be required of native apps going forward? If not, why?

Possibly, yes. But it's a different situation - web sites are including third-party code that is violating privacy by using this API. Apps typically don't do that (as often).

And either way, the privacy situation exists on the Web, so it needs to be solved here.

2. has anyone considered all the non-WebXR content this will break, like HTML5 games or websites that use orientation to change layout?

Websites shouldn't use the deviceorientation API to change layout. There are better ways to do that.

Yes, some games will be affected. As will things like panoramic photo viewers (not sure if they count as XR).

from deviceorientation.

Another thing that came up while discussing this internally is the need for a user gesture in order to call requestPermission(), similarly to what is done for getUserMedia. Any thoughts on this?

This would limit potential abuse.

from deviceorientation.

Requiring a user gesture seems reasonable, although, does explicitly navigating to a site count as user gesture?

from deviceorientation.

A few thoughts

• It seems clear that treating device motion and orientation events differently than other user inputs such as camera and microphone was unfortunate in hindsight.

• It seems logical to effectively promote them to user input with respect to privacy considerations, and reuse the same affordances used for other user inputs such as camera and microphone here.

• Ideally this would be a default stance taken to protect user privacy, as I am not sure exactly how/if this would impact generic sensors and other less established functionality at present.

from deviceorientation.

We did a lot of work around these issues in the Generic Sensor API. It's worth having a look at the Security and Privacy section of the spec, and some specific conversations in the tracker.

@lknik and @maryammjd might have more to add.

from deviceorientation.

I am glad to see that a rough consensus has emerged. If implementers want feedback on specific choices (e.g. what sort of text to include in a permission prompt, or whether to require a user gesture before prompt), you are welcome to ask the participants in the W3C Workshop on Permissions and User Consent held in September 2018. You can reach the lot of us at: [email protected]

from deviceorientation.

Just some food for thought as progress is made here. Native apps and the web alike are moving towards more natural UX, where moving and pointing your phone is how you interact with the content. ARKit (native), ARCore (native), 8th Wall (web), and AR.js (web) are prominent examples of this.

Put another way, device orientation and motion are starting to replace user touches as input.

On native, Apple has made this very easy with ARKit.
On web, iOS Safari is about to make this harder or default-disabled.

Please consider the implications of a double-standard that indirectly benefits a closed app ecosystem.

Also, as you make changes here, do please consider the following:

• Unlike other mobile browsers, Safari/WebKit doesn’t currently cache permissions per-domain, so any permission request occurs every single time you view the page. This contributes to permissions fatigue and users generally making less informed decisions about privacy.
• For pages whose entire input is device motion / orientation, like many AR/VR pages, there is currently no user gesture (since canvas touches aren’t considered user gestures). Thus in the pathologically worst end-state, Safari might require a prompt for each of the following every pageload:
  • 1. Bespoke dialog stating ‘This page would like to use camera and device motion, click ‘here’ to request permission.
  • 2. Permission dialog, this page is requesting camera access (yes/no)
  • 3. Permission dialog, this page is requesting device motion (yes/no)
  • 4. Permission dialog, this page is requesting device orientation (yes/no)

While I expect that we end up with a much better state than the scenario above – If I could suggest anything, it would be to enable permissions caching at the domain level in Safari, perhaps combined with a check-box that says ‘don’t ask again for this site’. This would prevent permissions fatigue and give parity with native apps which do not need to request permission on every app load. Much of this may be addressed in future permissions APIs, but it would be helpful if Safari/WebKit teams would be open to mitigate these concerns before making changes to device motion/orientation permissions.

from deviceorientation.

Hi,

I wrote a bit on sensors a while ago (ambient light 1, 2).

Not sure if helpful in your conversation, but we also analysed the possible impact of changes 3 (PDF).

We all know well the debate on privacy vs usability/precision, and also the point of view of whether something is or is not a threat. In the end it boils down to individual beliefs. I've seen and shown some of the risks. While I'm not saying such attacks are always 100% reliable, I also know that quantization and frequency capping often don't work (see e.g. (1)).

And in that case I'm happy to hear the position of Apple as presented by @othermaciej
How to handle it with the "defaults" is another thing. I'm all for opt-in, the current model of the web is opt-out (with the exception of permission-gated things).

from deviceorientation.

• Add a random constant in the range ... [-0.005, 0.005] to all devicemotion data on each new invocation of the API.

Indeed, I'm not sure how long would that hold, considering the data range you stipulate. How many observations need to be made to discard the random noise?

from deviceorientation.

@richtr - Although I agree that fingerprinting concerns are likely to be solvable with some of the mitigations you mention that is not the only potential privacy concern.

To take one example; even heavily quantized, obfuscated, and reduced-frequency sensor data would give some information about whether the user is standing up or lying down. Whether that is a significant enough privacy concern to require explicit user consent is certainly debatable (most vendors appear not to think so at the moment), but it does appear Apple / WebKit have made their minds up here.

I believe spatial computing use cases on the web would survive a permissions prompt. It would be unfortunate however to have a widely-released iOS version with the API disabled by default and without a permissions API implemented.

So I still hope for the final iOS 12.2 release Apple will either have an opt-in permissions API implemented or will decide to flip the default setting until that permissions API is ready. To me that strikes a good balance between addressing privacy concerns but without being quite as hostile to web developers who depend on these features.

from deviceorientation.

@lknik wrote:

• Add a random constant in the range ... [-0.005, 0.005] to all devicemotion data on each new invocation of the API.

Indeed, I'm not sure how long would that hold, considering the data range you stipulate. How many observations need to be made to discard the random noise?

This is the purpose of quantization to sit above random noise thresholds. As I wrote:

Other attack vectors (such as measuring the sensitivity between a series of sensor data readings), and again subject to ongoing study, could be further mitigated with quantization protections.

from deviceorientation.

With all due respect, it looks like a lazy solution to me. Please let’s stop showing people popups and let’s find instead ways to prevent this information to cause any harm. Haven’t we learnt anything from the cookies banners? Advertisers will track everything. If you don’t want websites to know your device orientation, stay away from the Web. MUCH WORSE THINGS HAPPEN OVER THERE! ✌️

from deviceorientation.

@machenmusic I am not a security expert and I appreciate being conservative when it comes to security. I would just hope the discussion was focused on the security aspect. Why having throttling, or a way to temporary disable the feature, for instance, while typing in a textfield marked as password, wouldn’t work? I don’t buy the privacy issues. That data Is not as exploitable as mic and camera. Should we ask permissions to get the mouse positions between clicks then? Can we clearly state the problem we want to solve? Not the solution.

from deviceorientation.

@nuthinking this has been discussed above - this is one of the longer threads that I can remember, actually - and I think the browser implementers (i.e. the ones with exposure risks and real skin in the game) have gotten at least as far as #57 (comment).

from deviceorientation.

Privacy hole... c’mon. You can probaly tell if someone is in bed by how they scroll the device. So what? Who defines the privacy line? Safari is disabling the feature and user will have to enable it in the settings (few gamers might do it).

from deviceorientation.

This discussion is becoming unproductive. I am locking the conversation until a PR for the proposed API changes has been posted.

from deviceorientation.

Thank you!

from deviceorientation.

This is good news for the security and privacy research community. Does the DAS WG plan to go for the same approach for other sensors too?

from deviceorientation.

@maryammjd, see w3c/sensors#388.

from deviceorientation.

Thanks for updating this issue, it's great to see the progress here. I was wondering it it's possible to get some minor points clarified related to the proposed specification PR #68

Based on the spec, the user interface would look something like:

(1) A bespoke dialog to approve motion and orientation permissions to view in AR with an 'Ok, got it' button (explicit action), clicking this button fires two promises that result in two prompts:
(2a) Prompt for device motion for origin
(2b) Prompt for device orientation for origin
(3) Then start.

Is there a mechanism to check if permission is granted prior to (1)? It may be confusing to see 'Permission is required, please allow' followed by no prompts (immediately return 'granted').

Are implementations allowed to merge (2a) and (2b) into a single prompt (2)? If so, adding something to this effect ("Implementations may choose to resolve simultaneous permissions promises with a single user prompt") could help clarify this.

Any clarification on these points would be greatly appreciated.

from deviceorientation.

@nbutko In WebKit at least, I have currently implemented it as a single prompt, permissions are merged on native side. Web pages should still request permission for both types of events if they need both (the second promise will be resolved without prompting in WebKit). I am personally not sure the spec needs clarification in this respect, it is up to browsers to decide how they want to do prompting.

from deviceorientation.

Thanks @cdumez that's great to hear.

Any guidance on checking whether the permission state is 'granted' prior to a custom prompt for permission?

from deviceorientation.

@nbutko This is a good point. My proposal would be:

1. The page calls requestPermission() without user gesture.
   2.a. If it is rejected, then you do not have permission and you need to request again with a user gesture
   2.b. If it gets resolved with 'granted', then you already have permission
   3.b. If it gets resolved with 'denied', then the user has already denied access in the past.

If this is acceptable, I would need to clarify the spec text and update the WebKit implementation accordingly.

from deviceorientation.

That would work well. Thanks!

from deviceorientation.

Excellent clarifications, thanks again. @cdumez @reillyeon @anssiko should any further discussion remain here, or move to #68?

from deviceorientation.

Excellent clarifications, thanks again. @cdumez @reillyeon @anssiko should any further discussion remain here, or move to #68?

I think on PR #68 would be better now assuming it is about the proposed change itself.

from deviceorientation.

@cdumez Will you plan to update this thread again when the webkit implementation of #68 is live, enabled and available to try in iOS beta (if it isn't already)?

from deviceorientation.