w3ctag / security-questionnaire Goto Github PK

The "issue tracking" link in https://w3ctag.github.io/security-questionnaire/ links to https://github.com/w3cping/security-questionnaire/issues/, so that is where I went to file w3cping#25. There is nothing on that page that makes it seem like that is the wrong place. However on https://github.com/w3cping/security-questionnaire it does say "Please file issues in the w3ctag repository, not here!". So where should issues actually be filed?

(I'm now noticing that https://w3ctag.github.io/security-questionnaire/ actually has links to both issue trackers, one labelled "Issue Tracking" and one labeled "Bug Reports".

Add: Does the specification permit access to the user's local file system and storage devices?

It's supposed to be used an example of how to do something with an API, but the linked explainer looks like it was never finished before it was abandoned.

I'll do this after the two open PRs land.

The W3C guidance in relation to issues of markets and people's choices states “W3C does not … in any way restrict competition. W3C's policy is that its activities are conducted to the highest ethical standards and in compliance with all applicable antitrust and competition laws and regulations.”

The document groups entities into first and third party based on a singular view of a persons willingness or ability to trust or understand entities that operate within each of those groups. The document does not recognise that entities within those different groups may operate within the same market and therefore will compete with one another. If a technical standard, or a particular implementation that progresses as a technical standard after it has been widely deployed, is assessed against this document then it is possible that the technical standard will restrict competition. Given the lack of browser diversity, but wide diversity of web stakeholders and entities, it is highly likely such an outcome will and has occurred in practice.

The mitigations proposed may only be possible for some players within a market and not others. Gaining people’s consent for something is far easier when it is combined with the acceptance of terms associated with an essential service like the setup of an operating system, or use of a mapping product. Large vertically integrated companies that also operate web browsers will find implementing such mitigations easy in practice. However smaller players that are not vertically integrated will find such mitigations impossible.

Other mitigations my be more or less practical based on financial strength, available engineering skills, available engineers, legacy solutions, among other factors.

This is one example of documents produced within the W3C that incorporate a specific and narrow view of a single issue without considering all the issues that the all 4,000,000,000 stakeholders in the web care about. Other examples include Mitigating Browser Fingerprinting in Web Specifications and [Target Privacy Threat Model](Target Privacy Threat Model).

To resolve this conflict the W3C could adopt a single policy covering all issues. All W3C documents would then have policy positions removed and would simply reference the single unified W3C policy document. This remedy would not only deal with the issues raised in relation to this document, but also improve horizontal review as all matters of policy will be crystal clear and defined once. They would not be open to interpretation by individual participants.

As external stakeholders such as Partnership for Responsible Addressable Media (PRAM), the UK Competition and Market Authority (CMA) and European Commission among others, take a more active interest in the work of the W3C such an approach will also support better engagement with these stakeholders.

Related to #1, Tantek mentioned it would be helpful to for specs to reference specific versions of this document.

To make the review process simpler is it worth making all 'yes' responses a heightened risk.

However this would mean rewording a few questions:

• Does this specification have a "Security Considerations" and "Privacy Considerations" section?
  • This is an interesting one as a missing section is a riskier but a large section here is riskier too. The positive answer I feel here should be if it is missing or has anything other than 'no considerations'.
• How should this specification work in the context of a user agent’s "incognito" mode?
• Does this specification distinguish between behavior in first-party and third-party contexts?

https://w3ctag.github.io/security-questionnaire/#drop-feature

Section 5.6 suggests that the simplest way is to drop the feature. This might be simple from the perspective of the reviewer, but presumably the design is being proposed with a motivation to solve an important problem for developers or end-users. Our work as reviewers is to help folks weigh this balance, rather than to suggest that they're simply wrong for wanting to solve problems. We have guidelines on the topic of putting powerful features behind permissions and I think the tone here contradicts with how we want the platform to advance. There are ways of balancing the risks, and we should be trying to help teams navigate that balance going forward. Suggesting they simply stop appears to put a thumb on the scales, suggesting the TAG is hostile to new features in general -- which we are not.

Ideally it should be reworded in a way that suggests tucking it behind a permission or dial down the granularity/power to mitigate the risks involved.

(Partially written by @slightlyoff.)

The "privacy concerns" in https://w3ctag.github.io/security-questionnaire/#threats look like they were copied from https://tools.ietf.org/html/rfc6973#section-5, but that use isn't cited. In addition to being polite, referencing RFC 6973 (or another original source for this list) would help me find more explanation if the single-sentence summaries here aren't sufficient.

As discussed in AB meeting 29 October 2015 at TAPAC.

http://dev.w3.org/csswg/css-ui-3/#security-privacy-considerations

Please review and see if this is roughly what was intended/expected as self-review answers.

CSSWG just resolved to publish an updated TR WD with this appendix, so it'll be on TR "soon" (by next Tuesday maybe?).

Section 4.4 “Explicitly restrict the feature to first party origins” does not consider the possibility that a person can trust a first party and all the suppliers that first party has chosen to work with.

In no other industry is the burden placed on people to understand the entire supply chain of a supplier. People purchasing automobiles are not required to know who supplies the brakes or airbags despite these components being essential to their safety. Instead they trust the automobile manufacturer to choose these suppliers.

Add: What is the fallback if the preferred choice of algorithm suite cannot be negotiated? Does the feature cease to work, or does it fail over to a default suite?

They need some text explaining them. Perhaps they are "examples"?

Procedural comment:

I think it would be less confusing if the editor's draft https://w3ctag.github.io/security-questionnaire/ bore an "editor's draft" cue, stylesheet or visual indication.

When we start from the published note https://www.w3.org/TR/2015/NOTE-security-privacy-questionnaire-20151210/ it's not obvious which is the latest document and what status it's in.

Thanks!

Add: Is the user able to make a choice of algorithm suite? Can an active network attacker force the use of any particular choice of algorithm suite?

Is the gh-pages branch serving a purpose here?

What's displayed at https://w3ctag.github.io/security-questionnaire/
appears to be older than master (i.e. https://rawgit.com/w3ctag/security-questionnaire/master/index.html )

Thanks, working with PING folks to coalesce the various documents here!

This question: "Does this specification enable new script execution/loading mechanisms?" is ambiguous and should be clarified.

I interpreted it broadly, as one should in the context of ambiguity in security, as (script execution)/(loading) mechanisms, especially since there were no other questions about "loading" of arbitrary (not just script) resources, which has network effects, tracking, etc.

For reference: http://dev.w3.org/csswg/css-ui-3/#security-privacy-considerations

I think we learned some good lessons from:

https://blog.redteam.pl/2020/08/stealing-local-files-using-safari-web.html

In particular, we can't assume that the OS or receiving application will safely handle random URL schemes.

For APIs that pass URLs to other apps or the OS, the questionnaire should ask what will happen if the browser passes along "file://" or other URL schemes ("data://") - and what the potential risks are when the receiving application ingests and dereferences those URLs, which can lead to information leakage, data theft, bypassing firewalls, etc.

As such, it would be good to add something to the questionnaire around URLs... URLs shouldn't be assumed to be safe strings. And that a URL itself doesn't represent a final destination, because once dereferenced a URL can redirect virtually anywhere.

Can we arrange for this doc to be automatically published, so we don't need to point at the editor's drafts?

If we need help, PLH is likely willing to help.

Re: #87 (review)

Should add discussion that sec + priv issues should be solved by the specs, not just pointed at.

Provide better guidance on how to use this questionnaire...more explicitly link to the markdown version from various places... etc...

Would it be worth suggesting an extension to Secure contexts for the passive network attacker

For example:

Passive network attacker

If your new feature requires any of the questions in this document consider making 'Trust On First Use' mechanisms such as HPKP, HSTS and CSP pinning a requirement for using the feature.

At our virtual W3CTAG face-to-face, we reviewed a related issue on the Ethical Web Principles.

We concluded that dark patterns are, in fact, a subset of abusive practices. We spend a lot of energy brainstorming abusive practices when we are doing design reviews, and we think it would be good for the web community if spec authors did something similar (before their work came to us).

The Security and Privacy Self Questionnaire is our current mechanism to prompt spec authors to think through what might go wrong.

Our abuse scenarios weren't specifically security or privacy related: for example, if a site asks you to "click okay to prove you're not a robot", and your "Okay" actually grants permission for push notifications (without your knowledge) — that is an abusive behaviour, but not compromising your security or privacy.

We agreed that we would find a way to add the topic into the Questionnaire. This issue is to kick that off.

Section 2.13 “How does this specification distinguish between behaviour in first-party and third-party contexts?” does not consider that first parties can be a threat to privacy and security. It assumes only third parties can be bad.

Section 3.4 “3.4. Third-Party Tracking” only considers tracking associated with third parties. It should consider tracking by any party.

First parties have the potential to perform harms just as much as third parties. The document should be expanded to reflect this.

Raising this as a place holder as it might be covered by:

3.9 - Does this specification allow an origin access to aspects of a user’s local computing environment?

As use high performance timing, low level memory and CPU access poses it's own risks it might be worth mentioning as part of this questions notes.

Could a passive network attacker collect, collate and cross-reference the data produced by a new feature with other sources to identify the user or determine the content of the data?

Add: If the specification allows access to a user's sensor (eg microphone, webcam), is it released after use? Is it apparent to the user that the sensor is being used?

Privacy and security are important considerations. Competitive markets, freedom of expression, and innovation, among others, are also considerations.

The document is not concerned with these wider considerations.

Before being amended it should be expanded to encompass a full set of considerations. The IETF in their documents “The Internet is for End Users” and ESCAPE provide insight.

In the meantime it should only be used in situations where these wider considerations are given equal importance. I’m new to the W3C and find the number of committees, documents and processes somewhat hard to follow. Could members of TAG and PING explain how these wider considerations are incorporated in to their work? If this is not the correct forum then perhaps they could direct me towards the correct one?

To consider only security and privacy is to ignore or downgrade the importance of society, people, publishers, advertisers (who fund much of the open web) and all technology businesses whose roles further the purpose of the W3C as defined in the membership agreement.

Such a document might start with the paragraph…

“Throughout the feature development process there are both foreseeable and unexpected security, privacy, competition, freedom and innovation risks. These risks may arise from the nature of the feature, some of its part(s), its implementation in practice, or unforeseen interactions with other features…”

Somewhere near the intro we need to give people more solid guidance about how to use this specification.

Suggestion below.

How to Use This Questionnaire

To be most effective, the questions below should be considered early in the specification development process, and kept in mind as it matures. Applying this questionnaire as a "check box" step before requesting final publication doesn't help improve privacy or security on the Web.

It's not necessary to include the questionnaire in the specification itself; for example, it might be preferable to keep it in a wiki and put a temporary link to that in the specification. Likewise, the entire questionnaire need not be repeated question-for-question when filled out; what's important is that each question is considered and that any privacy or security concerns are appropriately dealt with.

When you discover privacy and security concerns, there are many ways of dealing with them. Strategies for handling them are discussed in [ref], and ought to be discussed with other parties such as the TAG to assure a consistent approach throughout the Web; we already have a number of mismatches between capabilities and approaches across the platform as individual groups have taking a one-off approach to dealing with such concerns.

It would be helpful if 3.16 gave an actual (non-normative) example of what a good security/privacy considerations section could look like in the context of an actual specification document.

(xref my very naive attempt w3c/pointerevents#193 prompted by a recent ReSpec warning - if the spec here provided an example, I would have probably understood better what this entailed)

Migrated from mikewest/spec-questionnaire#6

@mikeperry-tor wrote:

It seems that the question in section 3.14 ("Does this specification distinguish between behavior in first-party and third-party contexts?") is very easy to misinterpret or overlook entirely, even though it is absolutely essential to any evaluation of an API from a privacy perspective.

For example, I could easily see a specification that decides that this section is satisfied simply by allowing for the third party script to detect its context, and then decide for itself how it wants to use the API (independent of any considerations of tracking, consent, or user control).

Something like this would be a bit stronger and clearer:

"At minimum, every new API or feature should be capable of being configured by the user to gracefully degrade into a functional mode of operation that does not allow third party content elements to use it to track unique users between different first party sites. Specifically, any storage, authentication tokens, shared state, and cross-window communication should be capable of being isolated between different first party origins, at user option.

For detailed guidance on how to provide graceful degradation to protect against more subtle forms of tracking that do not involve direct communication or unique identifier storage (aka 'fingerprinting'), we refer the reader to https://w3c.github.io/fingerprinting-guidance/."
In Tor Browser, we've implemented graceful degradation of the browser cache and DOM storage APIs by double keying all entries using both the first party and third party origin. This allows third party elements to continue to use this functionality in iframes and mashups, but prevents them from using the functionality to track users between different first party sites. In other cases, we've simply had to disable offending functionality (due to lack of engineering resources), which obviously has a significant impact on usability when things break.

If new APIs and feature specifications came with graceful degradation modes that did not require disabling them outright, the web would be a much better place.

If it were up to me, I would also elevate "Third Party Tracking" to a threat model that users could opt-in to protect themselves against, and specify this in the threat model section. All features should be capable of allowing the user to select this threat model via a single choice in their browser UI that applies to all browser features (instead of being given control only over third party cookies, as is the case today).

I can submit a pull request for this threat model addition if you like.

@marcoscaceres commented:

@mikeperry-tor I'm having trouble understanding:

"At minimum, every new API or feature should be capable of being configured by the user to gracefully degrade into a functional mode of operation that does not allow third party content elements to use it to track unique users between different first party sites."
Can you maybe follow that by a couple of examples to clarify what you mean? Or how it could be done with some other existing APIs?

I feel awfully pedantic raising this, however I feel completely gender neutral words is better than male and female (especially as the uses could be inferred to be negative connotations to that gender).

s/she/they/g

I'm aware this might remove some of the intended personal feel to the document but I saw it and thought it was worth the question.

Thanks

(With my TAG hat on)

We noticed today that there are two issue trackers associated with this document, a w3cping one and a w3ctag one. This is confusing, and resulted in different TAG members filing issues in different places today. The questionnaire itself links to both of these (one under "Issue Tracking" and one under "Bug Reports."

We acknowledge that this document is a joint work item, and we're not picky about which issue tracker gets used. But please just use one, and please remove the redundant link from the document's front matter.

Inline issue links in current draft link to mikewest's repo, not w3ctag.

Existing open issues:

https://github.com/mikewest/spec-questionnaire/issues

likely need transferring from mikewest's repo to w3ctag repo.

And then draft needs updating to link to w3ctag copy of those issues.

Add: If a feature is compromised, what is the extent of the damage? Does the compromise provide access to all critical information or are there multiple independent layers of security in place?

https://www.w3.org/TR/security-privacy-questionnaire/#secure-contexts has no reference to https://www.w3.org/TR/secure-contexts/

Add: Can the user review and change the permissions which the feature provides? Can it be interrogated to discover what permissions it requires, and if so, can it lie?

By title, they appear to be exactly the same. By content, 2.8 is more about exposing data across origins, whereas 2.1 is about exposing data at all.

It may be reasonable to merge the two questions, or maybe we should simply retitle them so the difference is clearer. Personally, I'm leaning toward merging them.

@pes10k thoughts?

"Does your specification have [security | privacy] issues that simple or direct answers to these questions did not tease out? Are there questions you can suggest that would have improved the coverage of this questionnaire?"

This helps us learn how effective the questionnaire is.

These look like examples. Should they be marked as examples, with wrapper text added?

I would like to see more consistent use of "feature" v. "specification". I think most usages should be "feature" (and "feature designer"), with rare exceptions of "specification" - e.g. the questions in 2.15 about whether the specification contains certain sections.

This stood out like a sore thumb reading 2.1 and 2.2. 2.2's title: Is this specification exposing..." 2.2's first sentence: "Features should only expose..."

what it says in the tin

This document's introduction section is too long, contains overly complex wording for things that should be simple and straightforward and in some cases contains misleading advice. For example, the paragraph on how to interact with PING, should arguably be removed as we don't want every group to talk to ping first.

Discussion with a reader from another WG suggested that the document might gather some of the possible mitigations for identifier privacy leaks under a new subhead in the mitigations section.

e.g.: aggregation, obfuscation, reduced granularity, differential privacy, statistical reporting, identifier rotation, randomization, delegation to a trusted party or component and then stripping the identifier, finding a non-identifier way to solve the problem.

While using the questionnaire on the Media Capture and Streams spec, I found odd to have to first look at per-origin persistent data, and then cross-origin persistent data — since the former is a subset of the latter, it would seem more logical to start with the greater risk (cross-origin persistent data), and continue with the smaller one (per-origin persistent data).

Incognito is a specific name of a private browsing mode in one UA. Is it more appropriate to use a more generic name like "private browsing mode"? (if indeed we keep it at all?)

The question in section 3.6 of "What about style" doesn't make sense to me. Add more explanation/context?

3.15 seems like it follows naturally from 3.3 (indeed, the last bullet of 3.3 says nearly the same thing). Merge them? Perhaps as a 3.3.1?