As browsers are starting to disable auto-playing videos & audio it would be great if we could easily request autoplay permission. Currently the app I'm working on has legitimate uses for autoplay and I'm not looking forward to telling users to go dig into Settings to enable it.

Is there a need for a declarative autoplay policy detection solution in HTML that doesn't rely on JavaScript? The <video> element poster attribute allows for fallback content, but only images, not other HTML content. Is there a developer need for such a feature?

This review is part of the requested PING review.

In some cases, when the browser mutes or rejects an effort from the page to autoplay media, it will be against the user's interests to notify the page (the whole reason the browser rejected the autoplay request in the first place is because the browser didn't think the page was acting in the user's best interests). For example, if the browser rejects an <audio>'s autoplay request bc the browser thinks in an invasive ad, and the page is notified of that intervention and responds with a different, non-autoplaying invasive ad, this proposal would leave users worse off.

The spec should talk through these tradeoffs, so that implementors can benefit from the group's expertise and (as best as possible) ensure that this feature does not end up working against users.

This issue tracks the status of horizontal reviews for the Autoplay Policy Detection API, which we should aim to complete now the spec has reached FPWD.

• Accessibility - self review, a11y review
• Internationalization - self review, i18n review
• Privacy - self review, privacy review
• Security - self review, security review
• TAG - TAG review

Note: Previous TAG reviews (w3ctag/design-reviews#356, w3ctag/design-reviews#439) focused on the question of sync vs async API.

Short i18n review checklist is here

1. If the spec (or its implementation) contains any natural language text that will be read by a human (this includes error messages or other UI text, JSON strings, etc, etc),

   The spec does not require any UI text

2. If the spec (or its implementation) allows content authors to produce typographically appealing text, either in its own right, or in association with graphics.

   The spec does not allow content authors to produce text

3. If the spec (or its implementation) allows the user to point into text, creates text fragments, concatenates text, allows the user to select or step through text (using a cursor or other methods), etc.

   The spec does not allow such user interaction

4. If the spec (or its implementation) allows searching or matching of text, including syntax and identifiers

   The spec does not allow searching or matching of text

5. If the spec (or its implementation) sorts text

   The spec does not allow sorting of text

6. If the spec (or its implementation) captures user input

   The spec does not require capturing user input

7. If the spec (or its implementation) deals with time in any way that will be read by humans and/or crosses time zone boundaries

   The spec does not deal with times in this way

8. If the spec (or its implementation) allows any character encoding other than UTF-8.

   The spec does not involve character encodings

9. If the spec (or its implementation) defines markup.

   The spec does not define markup

10. If the spec (or its implementation) deals with names, addresses, time & date formats, etc

    The spec does not deal with such data

11. If the spec (or its implementation) describes a format or data that is likely to need localization.

    The spec does not describe a format or data that would need localization

12. If the spec (or its implementation) makes any reference to or relies on any cultural norms

    The spec does not rely on cultural norms

We had a Web Media F2F meeting with Apple, Google, Microsoft and Mozilla attending. We discussed this API and this issue is a follow up from that.

The document has an autoplay policy which can change. There will be an API on the document to get the autoplay policy and another on the media element which will return a boolean as to whether the media will be able to autoplay.

There are four types of autoplay policy:

• allowed - autoplay is allowed
• allowed-muted - muted autoplay is allowed
• disallowed - autoplay is not allowed
• unknown - document level autoplay is unknown, so we should use the API on the media element instead

There were two different options for the document level API which are described below:

Option 1: Sync API

The first option is for a completely synchronous API. AutoplayPolicy here is an enum of the different policies defined above.

partial interface Document {
  readonly attribute AutoplayPolicy autoplayPolicy;
};

Pros:

• It's clean and simple
• Information is available right away and does not delay the decision process for authors

Cons:

• It limits browsers to what they can do to calculate the autoplay policy. For Chromium, we would like to do a cross process lookup here to check in a file whether a site can autoplay. In order to to this synchronously, UAs would have to pre-compute and send these bits of information along with the other information that are per origin and sent along on a page load. This could potentially delay the loading of the entire page.
• Potential resource overhead since browsers need to do this check before loading the page

Option 2: Promise API

The second option is for a promise based API. AutoplayPolicy here is an enum of the different policies defined above.

partial interface Document {
  Promise<AutoplayPolicy> getAutoplayPolicy();
};

Pros:

• It does not limit browser implementations of what determines autoplay policy
• The browser does not need to calculate the autoplay policy until getAutoplayPolicy is called

Cons:

• It's more complicated than (1)
• It allows UAs to do slow things to determine whether auto-playing is allowed, and can therefore delay the start of the media (time to playback)

--

It was decided at the meeting to ask TAG for their opinion on which option would be suitable.

In yesterday's Media WG meeting, @jernoble suggested considering an alternate design: Instead of extending document, HTMLMediaElement, and AudioContext, could the API instead have a single method that accepts one of those as parameter and returns the appropriate autoplay policy for that object, so for example:

const policy = navigator.getAutoPlayPolicy(document); // or mediaElement or audioContext

cc @alastor0325 @liberato-at-chromium

The Security and Privacy Considerations section of the spec should be split, as documented in https://w3ctag.github.io/security-questionnaire/#considerations. (We received similar feedback on Media Capabilities).

I also suggest that we write a document with answers to the questionnaire. See the Media Capabilities document for a previous example.

From https://www.w3.org/TR/security-privacy-questionnaire/#purpose

Does your feature expose information to origins? If so, how does exposing this information serve user needs? Are the risks to the user outweighed by the benefits to the user? If so, how?

With this API, we are exposing user preference that was set in the browser settings. This can contribute to fingerprinting. Is the risk outweighed by the benefits is something we should determine but the risk needs to be called out.

Apologies if I'm raising things that were covered in person at TPAC...

Issue #1 suggests an HTMLMediaElement.canAutoplay() API, and suggests it as async. Here's the current proposal:

HTMLMediaElement.canAutoplay() returns a promise that resolves if autoplay for a specific media element is allowed or rejects with some helpful error if not:

video.canAutoplay()
.then(_ => {
  // When muted videos can autoplay, it could be either
  // because the video is muted, or the video has no audio track.
  // It would also succeed if the call is during a user activation,
  // or if the video was previously played.
  // The video must have metadata loaded (readyState > HAVE_NOTHING).
})
.catch(error => {
  // If the video metadata did not load (readyState == HAVE_NOTHING),
  // or the call did not happen on a user gesture, or, when muted videos
  // are allowed if the video is not muted and has an audio track.
  // InvalidStateError will be used if the metadata did not load.
  // NotAllowedError will be used for all other cases.
});

Why does this API needs to be async? It would be simpler if it was synchronous. What case does making this API async cover that a boolean readonly attribute on HTMLMediaElement (or a function returning a boolean) couldn't cover?

I'm guessing this was proposed as async out of desire to handle the case where the UA wants to allow media without an audio track to autoplay, but the UA can't know whether the resource has an audio track until the media load algorithm has reached readyState HAVE_METADATA?

If the media load algorithm hasn't started, should canAutoplay() return a rejected promise? I don't think it would be intuitive for calling canAutoplay() to cause the media load algorithm to start.

If the media load algorithm has started, should canAutoplay() wait until readyState >= HAVE_METADATA before resolving/rejecting?

If we don't want to await readyState reaching HAVE_METADATA before resolving the canAutoplay() promise, then I don't think canAutoplay() needs to be async.

If the load algorithm errors, we'd need to reject all outstanding canAutoplay() promises.

How long do we wait for the readyState transition to HAVE_METADATA? I've heard from some large video sites that they cannot rely on the play() promise being rejected, and I suspect this is likely due to Safari awaiting readyState HAVE_METADATA before deciding whether the play() promise should be rejected, and the network causes the load to be slow, and authors are not waiting long enough. That is, authors call play(), set a timeout and the timeout fires before the play() promise resolves.

So I think that making the behavior of canAutoplay() bound by the network may make it be considered "unreliable" by authors.

What if something that affects autoplay state changes in between canAutoplay() being called, and the promise being resolved? For example if autoplay had been allowed in a resource with an audio track because the HTMLMediaElement had the muted attribute set, but after calling canAutoplay(), script removed the muted attribute? The result would be incorrect.

If the UA only allows playbacks in a user gesture handler, and canAutoplay() was called in a user gesture handler, the UA will need to propagate along the "in a user gesture handler" state to the promise continuation. This is not a problem in itself, but we should mention it in a spec somewhere so it's behavior that authors expect. What if inside the canAutoplay() promise handler, script does a setTimeout()? Should the timer continuation there also be considered to be in a user gesture handler?

What if the canAutoplay() promise is passed as one of many promises into Promises.all()? Should the promise returned by Promises.all() also be considered as in a user gesture handler? I'd posit that "no" is a reasonable answer to this, but we should be explicit.

If canAutoplay was synchronous, much of the above wouldn't be a problem - but we would have to decide on behavior when the synchronous function/attribute was accessed before metadata had been loaded.

I think it's simpler if canAutoplay() were synchronous, but if we want it to be async, I think we need to clarify:

• Behavior of canAutoplay() when called before media load algorithm has started.
• Behvaior of canAutoplay() when called after media load algorithm has started but before it's reached LOADED_METADATA.
• Behavior of canAutoplay() if state that affects whether autoplay is allowed changes after canAutoplay() is called but the promise resolves/rejects.
• Propagation of "in a user gesture handler" state to promise/timeout continuations.

1. Need to run a general spell check. In "W3C Working Draft, 15 September 2022", there are three (incorrect) instances of "asscociated"

2. Doc in 2.2.1. Query by a Media Type feel's inconsistent.

   • getAutoplayPolicy(type) methods return the rough status - so it's like "probably this is the status for all elements, but if you really care you need to check".

   • But below it uses the word "MUST"

     When getAutoplayPolicy(type) method is called, the user agent MUST run the following steps ...

     followed by (for example):

     If the return value is disallowed, None of media, corresponding with the given type, are allowed to autoplay.

   Those two cases seem mutually exclusive - if "must" is true then it isn't "rough".

3. Note that the "rough" only seems to apply for the disallowed case. My assumption is that if you get allowed or allowed-muted you can trust the value for the specified type. However the policy would be set to disallowed if, for example, some media were allowed and others were not, for whatever reason. Is that correct?

The context for this is that I'm writing MDN docs for this. It is not clear to me how the result for a type is useful if you can't trust it, so I'm trying to work out if you can ever trust it for any of the returned values.

For posterity, here are the notes of the Autoplay Policy Informal Meeting at TPAC 2018, October 23rd.

Attendees

• Jer Noble, Apple
• Scott Low, Microsoft
• Paul Adenot, Mozilla
• Francois Beaufort, Google
• Mounir Lamouri, Google

The outcome of this discussion is that we’d like an autoplay policy on document and a method on a HTMLMediaElement to know if autoplay will succeed.

It was discussed using the upcoming User Activation API (instead of the canAutoplay method) but since Apple folks are not sure to actually implement it, it was decided to not rely on this.

Disclaimer: names are subject to change

document.autoplayPolicy returns a enum string that can change overtime based on user session activity:

• “allowed” if autoplay is currently allowed.
• “allowed-muted” if muted video autoplay is currently allowed.
• “disallowed” is autoplay is not current allowed.
• MAYBE: “prompt” if autoplay will trigger a prompt (iif firefox decides to go that road)

HTMLMediaElement.canAutoplay() returns a promise that resolves if autoplay for a specific media element is allowed or rejects with some helpful error if not:

video.canAutoplay()
.then(_ => {
  // When muted videos can autoplay, it could be either
  // because the video is muted, or the video has no audio track.
  // It would also succeed if the call is during a user activation,
  // or if the video was previously played.
  // The video must have metadata loaded (readyState > HAVE_NOTHING).
})
.catch(error => {
  // If the video metadata did not load (readyState == HAVE_NOTHING),
  // or the call did not happen on a user gesture, or, when muted videos
  // are allowed if the video is not muted and has an audio track.
  // InvalidStateError will be used if the metadata did not load.
  // NotAllowedError will be used for all other cases.
});

Web Audio

Using the API for Web Audio would be as simple as:

const audioContext = new AudioContext();

if (audioContext.state === 'suspended') {
  if (document.autoplayPolicy === 'allowed') {
    // State is suspended only because hardware is slow.
  }
  if (document.autoplayPolicy === 'disallowed' ||
      document.autoplayPolicy === 'allowed-muted') {
    // State is suspended because of the autoplay policy.
    // TODO: Call audioContext.resume() after user gesture.
  }
}

Mozilla suggested to implement a new state called blocked, similar to the interrupted state implemented by Safari iOS. This state was meant to distinguish between suspended state caused by slow hardware and autoplay blocking. There was agreement that the document-level API was solving this use case (see above).

Autoplay Specification

In addition to drafting these changes, the discussions led to the creation of the wicg/autoplay repository in which a specification defining autoplay will be worked on. Paul Adenot was volunteered to author the specification :)

From an accessibility perspective, preventing autoplay in a simple way is very attractive!

However, there's already a user agent setting called prefers-reduced-motion that is readable as a css variable and accessible via javascript. In the accessibility community, many of us are trying to encourage developers of media player widgets to look at that variable and - if it is set to "reduce" - refrain from autoplaying any kind of video, audio, animation or similar.

I can come up with a few use cases where users may want to limit autoplay but NOT animations. Eg. users who have limited bandwidth or a very expensive connection, but who need the cognitive hints that animations can provide.
But I guess most users who have activated prefers-reduced-motion also do not want autoplay.

The prefers-reduced-motion preference, so far, is not universally recognized. Perhaps the proposed autoplay setting will spread quicker? If so, that would be great for accessibility (and reduce the need for prefers-reduced-motion).

What do you think? Perhaps I am missing some important differences between the two approaches?

A somewhat more detailed dialogue about this can be found in a recent thread in the W3C Community Slack, in the channel called #nordic-accessibility-group-feedback-standards-legislation

We need to decide on the final property names for HTMLMediaElement and document. Also we need to decide on the enum type names and values.

For now, #1 has document.autoplayPolicy as an attribute, and HTMLMediaElement.canAutoplay() as a method that returns a promise.

In webidl form, it would go something like this:

enum AutoPlayState {
    "allowed",
    "allowed-muted",
    "disallowed",
    "prompt"
};

partial interface document {
  readonly attribute AutoPlayState policy; 
};

partial interface HTMLMediaElement {
  Promise<void> canAutoPlay(); 
};

Intent to Migrate: Autoplay Policy Detection

Hi all, cc @mounirlamouri @jernoble as chairs of the Media Working Group. The recently created Media Working Group is ready to adopt Autoplay Policy Detection as a W3C Recommendation track deliverable.

Note some of the sections below need some love. I let people involved in the development of this proposal chime in as needed!

Working group decision to adopt

See the charter of the Media Working Group

Proposal

The API design is still at early stage, described in GitHub issues for now.

Summary

An API to allow websites to determine the document-level autoplay policy and whether autoplay will succeed for a given media element in a page.

Motivation and Use Cases

See notes of informal meeting at TPAC 2018 and proposed explainer

Compatibility Risk

Please characterize how much we might regret standardizing this new feature were we to change or remove it in the future. If you don’t have a github repository, include in this email links to relevant discussions, or documentation about support for the feature on the Web.

Ongoing technical constraints

The API exposes information that user agents already know, and the API is read-only, so technical constraints for implementing the API should be minimal.

It is expected that all environments will support this feature.

Link to implementation experience and demos

If your proposal has implementation experience or demos, please provide links, including common patterns in deployed libraries. Otherwise, indicate if there are none.

Data

What data do you have available that indicates that this enhancement will affect many users of the Web. Quantify the fraction of websites that are currently using something similar to this feature. Or, if a new feature, characterize the reason that you expect this to be far reaching.

Security and Privacy

To be evaluated. The API exposes information that applications can already determine through a trial and error approach.

Accessibility

This proposal should have no accessibility implications.

Internationalization

This proposal should have no internationalization implications.

This is from here, authors might pass an unsupported parameter to navigator.getAutoplayPolicy(ctor) and navigator.getAutoplayPolicy(object).

When that situation happens, those methods should throw type error, like this example. Currently we only support constructor and element for HTMLMediaElement and AudioContext, all other constructors or elements passed to the methods should cause throwing an exception.

[Media WG Chair hat on]

This is a Call for Consensus on the question raised in issues #3, #6, and #7: Should the Autoplay Detection APIs be synchronous and therefore simple attributes, or should they be asynchronous and therefore methods which return Promise<> objects.

The arguments presented for a synchronous vs. asynchronous API are as follows:

Synchronous:

• Easier to use for web authors.
• Autoplay restrictions are currently implemented as synchronous for every major browser.

Asynchronous:

• Allows implementors to make performance optimizations by delaying costly operations until the first time autoplay is requested, implicitly or explicitly.

This question has been extensively discussed both in GitHub issues, in a face-to-face meeting after FOMS 2018 NY, and most recently at the Media WG F2F at TPAC 2019. At the most recent F2F, there was broad–but not unanimous–agreement in the room that a synchronous API was preferred. There was also at least one member voicing strong dissent to that agreement. Members of the TAG were invited to attend the Media WG F2F to give technical feedback on the question, and the feedback given was that a synchronous API made more sense in this particular case.

This question has been very contentious, with strong feelings on both sides. But lack of resolution of this issue is blocking progress on this specification. So I propose the following resolution:

RESOLVED: Both the Document level and the HTMLMediaElement level autoplayPolicy APIs should be converted from methods returning a Promise to a simple property. I.e.:

partial interface Document {
  readonly attribute AutoplayPolicy autoplayPolicy;
};

partial interface HTMLMediaElement {
  readonly attribute AutoplayPolicy autoplayPolicy;
};

This CFC will be open until Friday October 18.

Positive responses are encouraged (e.g. a 👍 on this issue). Silence will be interpreted as implicit support. If you object, please provide a rationale for your objection and let the group know if/how the proposal could be changed to address your objection.

Most of the FAST checklist items do not apply.

The API does enable a content fallback mechanism, in that if a website uses the API to determine that auto-playing media is blocked, the website can choose to show alternative media instead.

From the MAUR checklist, it seems that those requirements apply to content authors rather than this API.

@mounirlamouri said #1 (comment)

Because it's not mentioned above, I believe that document.autoplaPolicy should look like this:

partial interface Document {
  readonly attribute Promise<AutoplayPolicy> autoplayPolicy;
};

@jernoble said #1 (comment)

Is there any precedent for a readonly attribute Promise? It seems really weird to have a resolved Promise for a value that can change over time.

@mounirlamouri said

I believe Service Worker does something similar for ready, see: https://www.w3.org/TR/service-workers-1/#navigator-service-worker-ready

https://w3c.github.io/autoplay/#autoplay-policy
enum AutoplayPolicy {
"allowed",
"allowed-muted",
"disallowed"
};

With Edge we are still working through how to improve auto play for users, and this is a top customer feedback we get. The above set of options restricts us from trying different ways to solve autoplay issues. For example, if we wanted to prompt users for permission to autoplay the above set does not give us the flexibility to pursue that option.

Hi, I'm going to help on editing the spec of autoplay detection.

But I found that there are two proposed names for the API on HTMLMediaElement. One is autoplayPolicy from issue#11.

partial interface HTMLMediaElement {
  readonly attribute AutoplayPolicy autoplayPolicy;
};

Another one is the canAutoplay from the explainer, and this one returns a boolean.

partial interface HTMLMediaElement {
  readonly boolean canAutoplay; 
};

Should we decide a final naming and the form for this API before moving things forward? In addition, for the API naming on Document, does everyone agree with using autoplayPolicy like issue#11?

partial interface Document {
  readonly attribute AutoplayPolicy autoplayPolicy;
};

Firefox has implemented an exprimental feature called allowedToPlay based on the previous discussion threads and it's easy for us to change that name to the one that we will decide to use for the final decision.

Via w3c/transitions#418 (comment) where @plehegar writes: "I'm surprised that this document doesn't use or mention https://www.w3.org/TR/permissions-policy/ . Is there a reason why this spec is not more integrated?"

The Permissions Policy spec is grounded on the notion of policy controlled feature, which is "an API or behaviour which can be enabled or disabled in a document" (by web developers typically). That does not fully map to autoplay because there is no requirement that the autoplay policy can be enabled or disabled. Plus the autoplay policy may be per element rather than per document, and the autoplay policy may return more than "enabled" and "disabled" (through "allowed-muted", also see discussion in #25).

The Permissions spec is grounded on the notion of powerful feature, which is "a web platform feature (usually an API) for which a user gives express permission before the feature can be used". That does not fully map to autoplay either because there is no requirement until now that the autoplay policy be based on the user giving express permission.

The explainer document that was discussed in PR #8 detailed similar reasoning:

We considered having autoplay as a permission. However, there are a few differences between the autoplay and permissions:

• Autoplay is something that is applied to the site from the browser; rather than something that should be requested
• Autoplay has multiple states (e.g. allowed, allowed-muted); rather than a yes or no
• Autoplay is not implemented by the user agents as a permission
• Permissions are async

Is that still accurate? Could it be useful to clarify that in some informative section?

The CfC to publish AutoPlay Policy Detection as FPWD has now closed, with positive responses from Mozilla, Apple, Google, and BBC.

@tidoust Would you mind sending the transition request, and to set up automatic publishing of the draft to /TR?

What are the mitigations to prevent sites from throwing prompts that require users to turn off auto play to continue browsing?
The current Play method on the HTMLMediaElement already provides information via the rejection handler https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/play#exceptions, that media playback was NotAllowedError - if a site is truly interested in providing fallback content, they can catch this exception and try again with the stream muted.

Related to #3, but seems separate. You should not throw an exception, or analogously return a rejected promise, in mainline scenarios. Instead you should provide a non-throwing/rejecting function which returns a sync value or fulfilled promise explaining how to avoid the error cases.

In particular, something named "play()" could return a rejection if playing cannot happen. But something named "canAutoplay()" should only return a boolean; it should never throw/reject. (Except perhaps in cases of weird hardware failure??)

#20 takes care of merging overloaded definitions to meet one of the overloading requirements defined in Web IDL. Web IDL also requires that overload definitions be distinguishable.

The Autoplay Policy defines the following IDL:

callback HTMLMediaElementConstructor = HTMLMediaElement ();
callback AudioContextConstructor = AudioContext ();

[Exposed=Window]
partial interface Navigator {
  AutoplayPolicy getAutoplayPolicy(HTMLMediaElementConstructor mediaCtor);
  AutoplayPolicy getAutoplayPolicy(AudioContextConstructor audioCtor);
};

If I'm reading Web IDL correctly, two callback functions are not distinguishable (see the matrix in the Web IDL spec).

This can probaby be fixed with a union type instead:

callback HTMLMediaElementOrAudioContextConstructor = (HTMLMediaElement or AudioContext) ();

[Exposed=Window]
partial interface Navigator {
  AutoplayPolicy getAutoplayPolicy(HTMLMediaElementOrAudioContextConstructor ctor);
};

The Web IDL validator does not check these requirements, so I'm not 100% sure that the above analysis is correct. It may be worth checking with Web IDL experts. Cc @w3c/webidl-design for possible confirmation.

This review is part of the requested PING review.

The proposed functionality may increase fingerprinting risk and cross-site information flows, depending on how browsers make autoplay decisions.

If browsers make autoplay decisions based on any of following inputs (incomplete list), the new functionality contributes to fingerprinting risk:

• a profile/browser setting
• a global permission setting
• an OS level setting or feature (e.g., data saving, battery status)
• etc

If browsers make autoplay decisions based on any of the following inputs (incomplete list), the new functionality may create a new cross-site information flow.

• whether any media is playing (or has played) on anther page/tab
• whether the site has instances open on other pages (playing or otherwise)

These are important, privacy-relevant decisions and factors implementors will need to consider, and should be incorporated into the privacy considerations section.

I have autoplay of media blocked for Youtube, however this is never respected when I click on a video from my YT "Subscriptions" page, it is only respected if I right click the thumbnails & "Open Link in New Tab" or middle-click to do so.

In FF stable, it works fine as it should.