Percy snapshot times out when run on CI about percy-cypress HOT 13 CLOSED

Thanks @OzTK!

Just finished looking those logs over and I think I see what might be causing the issue. I'll answer here for any future visitors with a similar problem.

I wasn't entirely correct before when I said there might be assets hanging. But it is indeed the asset discovery service that is hanging. The asset discovery service uses Puppeteer and a page pool to discover assets. What seems to be happening is the page pool is not able to open a Puppeteer page and gets stuck repeatedly trying until the timeout. That's why you see so many newPage logs.

Unfortunately, the pooling library does not log the error when retrying and so I'm still not entirely sure what is going wrong. I did notice that you are using an image with Chrome 69 and setting Puppeteer environment variables to suppress it's automatic download. I think maybe this version of Chrome is not compatible with the latest version of Puppeteer. Try removing those Puppeteer environment variables so Puppeteer can download a compatible version to see if it starts working. If it does work, you'll have to increase your Chrome version to a compatible one.

from percy-cypress.

Hi @OzTK!

If you run Percy with the environment variable LOG_LEVEL=debug it will print more detailed logs and might shed some light onto why that request might be hanging.

Since its attempting to POST to the snapshot endpoint, that means the healthcheck was successful and the server is definitely running. What's likely happening is some request on the page is causing the asset discovery service to hang. This could be due to missing auth headers or maybe some asset requests hanging.

from percy-cypress.

Hi @wwilsman, thanks for the help! I will try increasing the log level to get more info on the failure. I'm already using -t 350, maybe I can try to bump that to a higher value.
How would it be possible to be missing some auth headers? I'm not making the POST request directly, that's done by the plugin.

from percy-cypress.

When you take a snapshot with Percy, the DOM is serialized and POSTed to the locally running Percy server where asset discovery is performed. That timeout option will increase the window in which asset discovery listens for new requests. What seems to be happening with that Cypress error, is asset discovery is not finishing because there is an asset request that is hanging.

Most commonly, the request is from an iframe or image that requires authorization. The actual request causing the issue should surface when debugging and it could be that the server serving the asset is hanging for another reason. In case it is authorization, you can provide the asset discovery service any required authorization headers by providing the requestHeaders snapshot option, or the request-headers config file option.

from percy-cypress.

@wwilsman it looks like your theory about the asset discovery service is confirmed. With the debug log level I can see dozens of:
[·[35mpercy·[39m] -> assetDiscoveryService.browser.newPage
Eventually, there is also a
The automation client disconnected. Cannot continue running tests.

We definitely have quite large assets, but it's still a bit unclear how that is causing a problem. I already tried bumping the -t option to 20s without success.

Any pointers to what more exactly is causing Percy to timeout?

from percy-cypress.

If you're able to share the logs, maybe in a gist or snippet, that would help me see which request might be hanging. Or if the page that is hanging is accessible publicly, I could run some tests of my own to see what's going on.

If you can't share any of that publicly, then feel free to email [email protected] and I can help you there. 😃

from percy-cypress.

Thanks @wwilsman, I emailed support with our logs, but haven't received a reply yet. I'm wondering whether that has been received properly

from percy-cypress.

I don't see anything in our queue, try emailing me directly - [email protected]

from percy-cypress.

(sort of related) @timhaines no rush, would you be able to check to see if that email was caught up in spam for the google group? 🤔 (I don't have access to check)

from percy-cypress.

@wwilsman done

from percy-cypress.

I bumped our image to Chrome 74. It is still not working both with downloading puppeteer and skipping its download. When downloading it it ends up erroring with the following when running npm ci:

Step 7/9 : RUN npm ci
--
957 | ---> Running in 6d4f1e7d2ee6
958 |  
959 | > [email protected] install /e2e/node_modules/puppeteer
960 | > node install.js
961 |  
962 | ·[91m(node:57) ExperimentalWarning: The fs.promises API is experimental
963 | ·[0m·[91m
964 | ·[0m·[91mERROR: Failed to set up Chromium r756035! Set "PUPPETEER_SKIP_DOWNLOAD" env variable to skip download.
965 | ·[0m·[91m{ Error: EEXIST: file already exists, mkdir '/e2e/node_modules/puppeteer/.local-chromium/linux-756035'
966 | -- ASYNC --
967 | at BrowserFetcher.<anonymous> (/e2e/node_modules/puppeteer/lib/helper.js:94:19)
968 | at fetchBinary (/e2e/node_modules/puppeteer/install.js:148:8)
969 | at download (/e2e/node_modules/puppeteer/install.js:54:9)
970 | errno: -17,
971 | code: 'EEXIST',
972 | syscall: 'mkdir',
973 | path: '/e2e/node_modules/puppeteer/.local-chromium/linux-756035' }

This makes sense as the image has chrome installed already.

When skipping the download it errors similarly as before but the Percy logs are now a bit more detailed:

[·[35mpercy·[39m] agentService.handleSnapshot
--
7364 | [·[35mpercy·[39m] handling snapshot:
7365 | [·[35mpercy·[39m] -> headers: {"connection":"keep-alive","user-agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36","accept":"*/*","host":"localhost:5338","accept-encoding":"gzip, deflate","content-type":"application/json","content-length":"66897"}
7366 | [·[35mpercy·[39m] -> name: Filtered mapping report
7367 | [·[35mpercy·[39m] -> url: http://frontend:9080/c-test-organisation-6h/pipeline2/p-ahfw-m5lobdnixl-6byvdark/outputs/1/mappings/gx5zhvoem5axoqax7mrnuudz556/af2S4YpJWVgJoEU9?status=unmatched
7368 | [·[35mpercy·[39m] -> clientInfo: @percy/cypress/2.3.1
7369 | [·[35mpercy·[39m] -> environmentInfo: cypress/4.2.0
7370 | [·[35mpercy·[39m] -> domSnapshot: <!DOCTYPE html><html lang="en" class="js-focus-visible" data-js-focus-visible=""><head> <script type="text/javascript"> document.domain = 'frontend'; var Cypress = window.Cypress = parent.Cypress; if (!Cypress) { throw new Error('Something went terribly wrong and we cannot proceed. We expected to find the global Cypress in the parent window but it is missing!. This should never happen and likely is a bug. Please open an issue!'); }; Cypress.action('app:window:before:load', window); </script>
7371 | <meta name="viewport" content="width=device-width, initial-scale = 1.0">
7372 | <title>Mapping Report/Status filtering - Quantemplate</title>
... Some more HTML
7382 | <script src="/[truncated at 1024]
7383 | [·[35mpercy·[39m] -> widths: 1280,375
7384 | [·[35mpercy·[39m] -> minHeight: 1280
7385 | [·[35mpercy·[39m] -> enableJavaScript: undefined
7386 | [·[35mpercy·[39m] -> requestHeaders: undefined
7387 | [·[35mpercy·[39m] -> percyCSS:
7388 | [·[35mpercy·[39m] -> assetDiscoveryService.discoverResources
7389 | [·[35mpercy·[39m] discovering assets for URL: http://frontend:9080/c-test-organisation-6h/pipeline2/p-ahfw-m5lobdnixl-6byvdark/outputs/1/mappings/gx5zhvoem5axoqax7mrnuudz556/af2S4YpJWVgJoEU9?status=unmatched \| Tue Jun 16 2020 13:26:33 GMT+0000 (UTC)
7390 | [·[35mpercy·[39m] --> assetDiscoveryService.discoverForWidths
7391 | [·[35mpercy·[39m] discovering assets for width: 1280 \| Tue Jun 16 2020 13:26:33 GMT+0000 (UTC)
7392 | [·[35mpercy·[39m] discovering assets for width: 375 \| Tue Jun 16 2020 13:26:33 GMT+0000 (UTC)
7393 | [·[35mpercy·[39m] --> assetDiscoveryService.pool.acquire
7394 | [·[35mpercy·[39m] --> assetDiscoveryService.pool.acquire

We have to be able to support an older version than the most recent Chrome. Is there any way to force Percy to align? pre-install an older version of puppeteer maybe?

from percy-cypress.

I'm not entirely sure what Puppeteer's compatibility looks like. There were recently some breaking changes that were introduced in 3.2 and then rolled back 3.3. So the latest @percy/agent now requires 3.3 at minimum. The Chromium revision specified by that version is r756035 which corresponds to Chromium 83.0.4103.0.

If an older version of Puppeteer were pre-installed, NPM/Yarn would still install 3.3 since it is required by percy due to that rollback. However, depending on your package manager, you might be able to force it to install an older version of Puppeteer. Yarn supports a resolutions field in your package.json file. NPM has a package for this, or you can use the shrinkwrap feature. Just be sure not to install Puppeteer 3.2 since that is known to not work with Percy.

from percy-cypress.

Confirming that this problem was solved by updating the Cypress docker image we were using. We re-evaluated our requirements and bumped it to Chrome 78, which worked.
It is a big inconvenience not to have control over the browser version when updating Percy though. Cypress allows to target specific versions, which is great, it would be a huge improvement if Percy was doing the same. The main problem is that there is no explicit mentioning of which version of Chrome works with which version of Percy. There is also a discrepancy between the version used in percy-cypress and the one used in percy-storybook (those are the 2 we use).
I know this constraint comes with Puppeteer, but making it explicit and documented would save a lot of trouble to users of Percy 🙂

from percy-cypress.