alastor0325 / autoplay-shield-study Goto Github PK

[Affected versions]:

• Firefox Beta Unbranded 63.0b8

[Affected Platforms]:

• All Windows
• All Mac
• All Linux

[Prerequisites]:

• Have the following prefs:
  - shieldStudy.logLevel to All;
  - extensions.fastblock_shield_mozilla_org.test.variationName to one of the following values: control, block, allow-and-remember or allow-and-notRemember;
  - extensions.legacy.enabled to true;
  - xpinstall.signatures.required to false.
• The Block-autoplay add-on is installed.

[Steps to reproduce]:

1. Open the browser with the profile from prerequisites.
2. Navigate to "about:addons>Extensions" and click the "Block-autoplay" add-on.
3. Click the "Remove" button and observe the behavior.

[Expected result]:

• The "Block-autoplay" shield study is uninstalled.

[Actual result]:

• The "Undo" button is displayed.

[Notes]:

• After the "Undo" button is clicked, the "install" and "enter" pings are sent again.
• Added a screen recording the issue:

[Affected versions]:

• Beta Unbranded 63.0b13

[Affected Platforms]:

• All Windows
• All Mac
• All Linux

[Prerequisites]:

• Have the following prefs:
  • extensions.legacy.enabled to true
  • xpinstall.signatures.required to false
  • extensions.fastblock_shield_mozilla_org.test.variationName to one of the following values: allow-and-remember.
  • xpinstall.signatures.required to false.
• The Block-autoplay add-on is installed.

[Steps to reproduce]:

1. Open the Firefox browser with the profile from the prerequisites.
2. Navigate to the "about:addons" page and click the "Remove" button from the "Block-autoplay" add-on.
3. Observe the browser's tabs.

[Expected result]:

• A new tab containing the "Block-autoplay" survey is opened.

[Actual results]:

• No new tabs are opened.

[Notes]:

• This is a possible scenario a user can encounter when choosing to remove the shield study from the "about:studies" page.
• We used the scenario with an unsigned build and removed the add-on from "about:addons" as we cannot see the study in the "about:studies" page.
• Attached a screen recording with the issue:

[Affected versions]:

• Firefox Beta Unbranded 63.0b8

[Affected Platforms]:

• All Windows
• All Mac
• All Linux

[Prerequisites]:

• Have the following prefs:
  • shieldStudy.logLevel to All
  • extensions.fastblock_shield_mozilla_org.test.variationName to block
  • extensions.legacy.enabled to true
  • xpinstall.signatures.required to false
• The Block-autoplay add-on is installed.

[Steps to reproduce]:

1. Open the browser with the profile from prerequisites.
2. Navigate to "about:preferences#privacy" and scroll down to "Permissions" section.
3. Select the "Allow autoplay" option for global autoplay.
4. Restart the browser.
5. Observe the setting for global autoplay.

[Expected result]:

• "Allow autoplay" option is set for global autoplay.

[Actual result]:

• "Don't Autoplay" option is set for global autoplay.

[Notes]:

• This issue is reproducible on all other branches.
• This issue is also reproducible if the "Always ask" option is selected.
• Attached a screen recording of the issue:

[Affected versions]:

• Firefox Beta, Version 63.0b10, Build ID 20180927143327
• Firefox Nightly, Version 64.0a1, Build ID 20180930220130

[Affected Platforms]:

• All Windows
• All Linux
• All Mac

[Prerequisites]:

• extensions.autoplay-shield-study_shield_mozilla_org.test.variationName string pref is set to allow-and-notRemember.
• Autoplay Shield study version 2.0.0 addon is installed.

[Steps to reproduce]:

1. Open the browser with the profile from prerequisites.
2. Navigate to https://www.cnn.com/videos and wait for the page to load.
3. Wait for 10 seconds until a "play" icon is displayed on the video.
4. Click the "Allow" button from the doorhanger and observe the behavior.

[Expected result]:

• The doorhanger is dismissed and the video starts playing.

[Actual result]:

• The doorhanger is dismissed and the video does not start playing.

[Additional Notes]:

• This issue is reproducible on both allow-and-notRemember and allow-and-remember branches.
• This issue is not reproducible on the following sites: YouTube, Yahoo, Twitch, Aol, MSNBC, Today.
• I have attached a screen recording of the issue:
  

Not sure if this is still an active project or not, but...

The latest version of [email protected], "switched ps-tree package to pidtree package for security reasons":

$ npm info npm-run-all version # 4.1.5

$ npm ls event-stream flatmap-stream
[email protected] /Users/pdehaan/dev/github/mozilla/autoplay-shield-study
└─┬ [email protected]
  └─┬ [email protected]
    └─┬ [email protected]
      └── [email protected]

[Affected versions]:

• Firefox Beta, Version 63.0b10, Build ID 20180927143327
• Firefox Nightly, Version 64.0a1, Build ID 20181001220118

[Affected Platforms]:

• All Windows
• All Linux
• All Mac

[Prerequisites]:

• extensions.autoplay-shield-study_shield_mozilla_org.test.variationName string pref is set to allow-and-notRemember.
• Autoplay Shield study version 2.0.0 addon is installed.

[Steps to reproduce]:

1. Open the browser with the profile from prerequisites.
2. Navigate to https://www.youtube.com/ and open a video in a new tab.
3. Click the "Theatre mode" button and observe the behavior.

[Expected result]:

• The video enters theatre mode and does not start playing.

[Actual result]:

• The doorhanger is dismissed and the video starts playing as the video enters theatre mode.

[Additional Notes]:

• This issue is reproducible on both allow-and-notRemember and allow-and-remember branches.
• This issue is reproducible on the following YouTube player buttons: "Theatre mode", "Miniplayer" and changing the video quality from the settings.
• This issue is not reproducible on the following YouTube player buttons: "Mute" and volume, "Autoplay", "Annotations", "Speed", "Fullscreen".
• I have attached a screen recording of the issue: link

[Affected versions]:

• Firefox Beta 63.0b9

[Affected Platforms]:

• All Windows
• All Mac
• All Linux

[Prerequisites]:

• Have the following prefs:
  • shieldStudy.logLevel to All
  • extensions.fastblock_shield_mozilla_org.test.variationName to allow-and-notRemember.
• The Block-autoplay add-on is installed.

[Steps to reproduce]:

1. Open the browser with the profile from prerequisites.
2. Navigate to https://edition.cnn.com/videos.
3. Observe the checkbox in the Block-autoplay doorhanger.
4. Restart the browser and navigate to https://edition.cnn.com/videos.
5. Observe the checkbox in the Block-autoplay doorhanger.

[Expected result]:

• Step 3 and Step 5: The checkbox is unchecked.

[Actual result]:

• Step 3: The checkbox is unchecked.
• Step 5: The checkbox is checked.

[Notes]:

• The issue is reproducible if you navigate on other websites with auto-play videos.
• This issue is also reproducible if one of the Block-autoplay doorhanger option is selected: "Allow Autoplay" or "Don't Allow".
• Attached is a screen recording of the issue:

Our current ping can't know whether doorhanges were ignored by user when (1) user pressESC (2) user directly click the video and video starts playing audibly.

We're planing to add an new attribute in promptResponse to indicate this information.

[Affected versions]:

• Firefox Beta, Version 63.0b10, Build ID 20180927143327
• Firefox Nightly, Version 64.0a1, Build ID 20180930220130

[Affected Platforms]:

• All Windows
• All Linux
• All Mac

[Prerequisites]:

• extensions.autoplay-shield-study_shield_mozilla_org.test.variationName string pref is set to allow-and-notRemember.
• Autoplay Shield study version 2.0.0 addon is installed.

[Steps to reproduce]:

1. Open the browser with the profile from prerequisites.
2. Navigate to https://www.cnn.com/videos and wait for the page to load.
3. Click on the page's background and observe the behavior.

[Expected result]:

• The doorhanger is displayed and the video does not start playing.

[Actual result]:

• The doorhanger is dismissed and the video starts playing.

[Additional Notes]:

• This issue is reproducible on both allow-and-notRemember and allow-and-remember branches.
• This issue is reproducible on the following sites: CNN, BBC, DailyMotion, YouTube, Yahoo video search, Twitch.
• I have attached a screen recording of the issue:
  

[Affected versions]:

• Firefox Release Candidate 63.0

[Affected Platforms]:

• All Windows
• All Linux
• All Mac

[Prerequisites]:

• extensions.autoplay-shield-study_shield_mozilla_org.test.variationName string pref is set any variation branch.
• Autoplay Shield study version 2.0.1 addon is installed.

[Steps to reproduce]:

1. Open the browser with the profile from prerequisites.
2. Navigate to a couple websites with audible media.
3. Restart the browser and navigate to the "about:telemetry" page.
4. Observe the "block-autoplay" archived counts ping.

[Expected result]:

• The "totalBlockedAudibleMedia" ping's value is increased.

[Actual result]:

• The "totalBlockedAudibleMedia" ping's value is "0".

[Additional Notes]:

• This issue is reproducible on all study branches.
• This issue is not reproducible on Firefox Beta and Firefox Nightly.
• I have attached a screen recording of the issue.

We need to provide ending survey for user when the the testing end in order to know how do they feel about blocking autoplay.

[Affected versions]:

• Firefox Beta Unbranded 63.0b8

[Affected Platforms]:

• All Windows
• All Mac
• All Linux

[Prerequisites]:

• Have the following prefs:
  - shieldStudy.logLevel to All
  - extensions.fastblock_shield_mozilla_org.test.variationName to one of the following values: control, block, allow-and-remember or allow-and-notRemember
  - extensions.legacy.enabled to true
  - xpinstall.signatures.required to false
• The Block-autoplay add-on is installed.

[Steps to reproduce]:

1. Open the browser with the profile from prerequisites.
2. Navigate to "about:addons>Extensions" and click the "Block-autoplay" add-on.
3. Click the "Disable" button and observe the behavior.

[Expected result]:

• The "Block-autoplay" shield study is removed and there is no "Undo" option.

[Actual result]:

• The "Block-autoplay" shield study is disabled.

[Notes]:

• After the study is re-enabled, the "install" and "enter" pings are sent again.
• Added a screen recording the issue:

The following is the log when running "$npm run test:unit"

START:
26 07 2018 14:46:47.203:INFO [karma]: Karma v2.0.3 server started at http://0.0.0.0:9877/
26 07 2018 14:46:47.206:INFO [launcher]: Launching browser FirefoxNightly with unlimited concurrency
26 07 2018 14:46:47.209:INFO [launcher]: Starting browser FirefoxNightly
26 07 2018 14:47:47.215:WARN [launcher]: FirefoxNightly have not captured in 60000 ms, killing.
26 07 2018 14:47:49.217:WARN [launcher]: FirefoxNightly was not killed in 2000 ms, sending SIGKILL.
26 07 2018 14:47:51.219:WARN [launcher]: FirefoxNightly was not killed by SIGKILL in 2000 ms, continuing.

Finished in 0 secs / 0 secs @ 14:47:51 GMT-0700 (Pacific Daylight Time)

26 07 2018 14:47:51.225:ERROR [karma]: TypeError: Cannot read property 'files' of undefined
at toInitialList (/Users/awu/projects/web-extensions/template-shield/node_modules/istanbul-lib-report/lib/summarizer.js:147:17)
at Object.createPackageSummary [as pkg] (/Users/awu/projects/web-extensions/template-shield/node_modules/istanbul-lib-report/lib/summarizer.js:240:21)
at IstanbulReporter.onRunComplete (/Users/awu/projects/web-extensions/template-shield/node_modules/karma-istanbuljs-reporter/index.js:40:38)
at Server. (/Users/awu/projects/web-extensions/template-shield/node_modules/karma/lib/events.js:39:26)
at Server.emit (events.js:187:15)
at emitRunCompleteIfAllBrowsersDone (/Users/awu/projects/web-extensions/template-shield/node_modules/karma/lib/server.js:256:14)
at Server.on (/Users/awu/projects/web-extensions/template-shield/node_modules/karma/lib/server.js:292:9)
at Server.emit (events.js:182:13)
at Object.BaseLauncher._done (/Users/awu/projects/web-extensions/template-shield/node_modules/karma/lib/launchers/base.js:112:15)
at /Users/awu/projects/web-extensions/template-shield/node_modules/karma/lib/launchers/process.js:134:12
at next (/Users/awu/projects/web-extensions/template-shield/node_modules/rimraf/rimraf.js:75:7)
at CB (/Users/awu/projects/web-extensions/template-shield/node_modules/rimraf/rimraf.js:111:9)
at FSReqWrap.oncomplete (fs.js:145:20)
(node:37538) [DEP0079] DeprecationWarning: Custom inspection function on Objects via .inspect() is deprecated

[Affected versions]:

• Firefox Beta Unbranded 63.0b7

[Affected Platforms]:

• All Windows
• All Mac
• All Linux

[Prerequisites]:

• Have the following prefs:
  • shieldStudy.logLevel to All
  • extensions.fastblock_shield_mozilla_org.test.variationName to allow-and-remember
  • extensions.legacy.enabled to true
  • xpinstall.signatures.required to false
• The Block-autoplay add-on is installed.

[Steps to reproduce]:

1. Open the browser with the profile from prerequisites.
2. Navigate to https://www.twitch.tv.
3. Observe the block autoplay doorhanger.

[Expected result]:

• A "Learn more" hyperlink is displayed below the "Will you allow to autoplay media with sound?" text.

[Actual results]:

• There is no "Learn more" hyperlink displayed below the "Will you allow to autoplay media with sound?" text.

[Notes]:

• This issue is also reproducible on the "allow-and-notRemember" branch.
• Based on the mock-ups from the Block-autoplay PHD
• Attached a screen recording of the issue:

[Affected versions]:

• Firefox Beta Unbranded 63.0b8

[Affected Platforms]:

• All Windows
• All Mac
• All Linux

[Prerequisites]:

• Have the following prefs:
  - shieldStudy.logLevel to All;
  - extensions.fastblock_shield_mozilla_org.test.variationName to one of the following values: control, block, allow-and-remember or allow-and-notRemember;
  - extensions.legacy.enabled to true;
  - xpinstall.signatures.required to false.
• The Block-autoplay add-on is installed.

[Steps to reproduce]:

1. Set the system date to 2 weeks after the current date.
2. Open the browser with the profile from prerequisites and navigate to "about:addons">Extensions section.
3. Observe the add-ons section.

[Expected result]:

• Block-autoplay experiment is uninstalled and no longer displayed in the add-ons section.

[Actual result]:

• Block-autoplay experiment remains installed and displayed in the add-ons section.

[Notes]:

• The "Exit" and "Expired" telemetry pings are generated and displayed in "about:telemetry" page.
• Added a screen recording the issue:

Now we have five testing cohorts, but we might need to remove some cohorts depending on the final discussion result of the UI inconsistency issue.

In addition, we need to add new cohort for disable autoplay by default according to the previous discussion in block autoplay workweek.

This flag marks pings with testing=true. Set it to true before final release.

[Affected versions]:

• Beta Unbranded 63.0b10

[Affected Platforms]:

• All Windows
• All Mac
• All Linux

[Prerequisites]:

• Have the following prefs:
  • extensions.fastblock_shield_mozilla_org.test.variationName to one of the following values: allow-and-remember.
  • xpinstall.signatures.required to false.
• The Block-autoplay add-on is installed.

[Steps to reproduce]:

1. Open the Firefox browser with the profile from the prerequisites.
2. Navigate to https://twitch.tv.
3. Click on the "Browse" button from the same domain.
4. Observe the left of the address bar area.

[Expected result]:

• The doorhanger is dismissed.

[Actual results]:

• The doorhanger is still displayed.

[Notes]:

• This issue is reproducible also on allow-and-notRemember branch, and on block and control branches after the "Always ask" option is set for the global autoplay seting.
• This issue is reproducible on youtube.com, cnn.com, bbc.com and probably on other website too.
• Attached a screen recording with the issue: