ADO extension fails when run in a container: handleRequestFunction failed, { message: 'read ECONNRESET' } about accessibility-insights-action HOT 8 OPEN

If the issue is that the container is missing system dependencies required to successfully run Chromium, this may be yet another reason that it would be nice to switch the service from Puppeteer to Playwright. Playwright gives a much nicer error message for that case (noting before trying to start that the system is missing required dependencies, enumerating which ones, and offering an npx playwright install-deps chromium command to install them for you).

from accessibility-insights-action.

This issue has been marked as ready for team triage; we will triage it in our weekly review and update the issue. Thank you for contributing to Accessibility Insights!

from accessibility-insights-action.

Investigated this a bit further and confirmed that this does seem to be a result of trying to run in Windows Server Core without having installed the prerequisite Server-Media-Foundation Windows Feature required to run Chromium in that Windows SKU. This is true irrespective of whether you're using Server Core via a container or host VM, but in practice we usually see host VMs running full Server SKUs and containers based on something like mcr.microsoft.com/windows/servercore:ltsc2022 or mcr.microsoft.com/windows/servercore:ltsc2019, so it tends to be correlated with users trying to use containers.

On servercore:ltsc2019 it's possible to install the prerequisite windows feature with a step along the lines of Install-WindowsFeature Server-Media-Foundation, but this doesn't work on servercore:ltsc2022 without a much more involved workaround.

Other errors with similar symptoms:

• If you try to use a linux container without the required prerequisites installed (eg, mcr.microsoft.com/devcontainers/javascript-node:16), you're likely to get an error along these lines:

ERROR PuppeteerCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"[https://site-under-test.com","retryCount":1,"id":"mCnLT4bz8fgQgev"}](https://site-under-test.com%22%2C%22retrycount%22:1%2C%22id%22:"mCnLT4bz8fgQgev"}/)
  Error: Failed to launch the browser process!
  /home/node/.cache/puppeteer/chrome/linux-1108766/chrome-linux/chrome: error while loading shared libraries: libnss3.so: cannot open shared object file: No such file or directory

• If you try to use a linux container that does have the required prerequisites installed (eg, mcr.microsoft.com/playwright:v1.33.0-jammy), you're likely to get an error along these lines:

ERROR PuppeteerCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"[http://localhost:5858","retryCount":1,"id":"hgW4ugjCDUL55FU"}](http://localhost:5858%22%2C%22retrycount%22:1%2C%22id%22:"hgW4ugjCDUL55FU"}/)
  Error: Failed to launch the browser process!
  [0429/002056.263116:FATAL:zygote_host_impl_linux.cc(127)] No usable sandbox! Update your kernel or see https://chromium.googlesource.com/chromium/src/+/main/docs/linux/suid_sandbox_development.md for more information on developing with the SUID sandbox. If you want to live dangerously and need an immediate workaround, you can try using --no-sandbox.

The latter case (where the correct deps are installed on Linux) happens because Chromium's sandboxing only works in Docker containers if you pass a specific Docker seccomp policy while running docker commands, and Azure Pipelines does not pass this by default. There's no workaround that I'm aware of except disabling Chromium's sandboxing, which would be pretty questionable for us to do from a crawler.

In summary, I think our short-term recommendation we should document needs to be that, in order to run the task, users must run it from one of the following environments:

• A host VM (not a docker container) which is based on a Windows Server SKU or a Linux image which is tested to work with Chromium, for example any of the standard Azure Pipelines Hosted Agent images or standard 1ES PT images.
• A Windows (not linux) Docker container which is based on a Windows or Windows Server container base image (not servercore or nanoserver)

...noting specifically that Docker containers based on linux, servercore, or nanoserver are not supported.

Medium-term, it would be good to detect this case early and improve our error messaging. On Windows, we could do something comparable to Playwright's install_media_pack.ps1 script (but with Get-WindowsFeature instead of Install-WindowsFeature) to detect whether we're on a server SKU without the required prerequisite.

If we get substantial user requests for linux support, I'm not sure we have a great solution; we could add some sort of chromiumSandbox: false task input to let users opt into disabling sandboxing to enable running on containers with prerequisites installed, but I'm not convinced we should allow users to do that, it carries risks that users are likely to dismiss.

from accessibility-insights-action.

Updated internal documentation per above comment's short-term recommendations.

from accessibility-insights-action.

This issue has been marked as ready for team triage; we will triage it in our weekly review and update the issue. Thank you for contributing to Accessibility Insights!

from accessibility-insights-action.

The remaining work for this is the paragraph starting with "Medium-term" in the above comment, discussing making the error message more clear. Marking as ready for triage for us to discuss if/when to do that.

from accessibility-insights-action.

We want to do this if it's fairly easy

from accessibility-insights-action.

Investigated this a bit further and confirmed that this does seem to be a result of trying to run in Windows Server Core without having installed the prerequisite Server-Media-Foundation Windows Feature required to run Chromium in that Windows SKU.

Apparently missing Server-Media-Foundation Windows feature has nothing to do with this issue. To run Chrome or Chromium in Windows Server container there is no requirement to install it.

To successfully run Chrome or Chromium in Windows/Linux container install required Fonts.

In Ubuntu you can install required fonts using following script:

apt-get update \
&& apt-get install -y wget gnupg \
&& wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
&& apt-get update \
&& apt-get install -y google-chrome-stable fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-kacst fonts-freefont-ttf libxss1 libxtst6 procps \
--no-install-recommends \

In Windows Server core enable following optional Windows features and register fonts using Win32 GDI API function. Detailed solution can be found here.

[DllImport("gdi32.dll")]
static extern int AddFontResource(string lpFilename);

ServerCoreFonts-NonCritical-Fonts-MinConsoleFonts
ServerCoreFonts-NonCritical-Fonts-Support
ServerCoreFonts-NonCritical-Fonts-BitmapFonts
ServerCoreFonts-NonCritical-Fonts-TrueType
ServerCoreFonts-NonCritical-Fonts-UAPFonts

from accessibility-insights-action.