Is there a way to have this with this plugin? I've seen it with other approaches, and it seems really visually nice. Sometimes only the diff can be pretty difficult to read. Kinda like this:

Vs something like this (this example is probably the simplest using a single file and from cypress-image-snapshot plugin to be clear, but there are potentially a lot of ways to accomplish this I think)

This plugin is only taking a screenshot of about a 1000px square section of the top left of the screen, which is a small portion of this long form I'm trying out (needs about 4000px height to render all of it without a scrollbar). Has anyone else noticed this? I'm setting the viewport like this and when I run the test, it displays with this viewport in the browser, but the snapshot command always only gets a fraction. I've also tried cy.compareSnapshot and cy.root().compareSnapshot... Is it obvious I'm doing something wrong? I'm doing this in a component test, btw.

Is it possible to get snapshot testing working for component testing? on the install documents, it is mentioned that where {scheme} defaults to e2e
// cypress/support/index.js for Cypress versions below 10
// cypress/support/{scheme}.js for Cypress versions 10 and above, where {scheme} defaults to e2e
require('./commands')
is it possible to overwrite this {scheme} so it looks into component.js file for component testing as well?

When I configure the framework with component testing this is the error I am getting.

I tried to add compareSnapshotCommand() as a
Cypress.Commands.add('compareSnapshotCommand', compareSnapshotCommand());
but that did not work either.

Hi,

What's the intended way to use it in TS env?
VSCode warns Property 'compareSnapshot' does not exist on type 'Chainable<JQuery<HTMLCanvasElement>>'.ts(2339)

Though it is only a warn and runs well without types, it's annoyed to see everywhere.

I have to add types currently, follow here: https://docs.cypress.io/guides/tooling/typescript-support#Types-for-custom-commands

Hi Pippo,

We use the same code for functional and visual tests. I saw your code and you turn off failed screenshots generation but we need it. How that can be changed? I had tried to put turning on screenshot into before hook but it was too late for our dynamic reporting but files popped up

Error: ENOENT: no such file or directory, copyfile 'C:\repos\cypress\screenshots\503.spec.ts\503.spec.ts-#main article.png' -> 'C:\repos\cypress-visual-screenshots\comparison\503.spec.ts-#main > article.png'

The > char is a bad character in Windows filesystem.

Is there a way to skip Error: Image difference greater than threshold errors with this plugin?

The cypress-image-snapshot plugin provides this option.
https://github.com/jaredpalmer/cypress-image-snapshot

Thank you in advance.

Hi!

First of all, thx for this project! 👍

I am using Cypress version 7 (7.6.0) and everytime I run npm install two versions of cypress will be installed - 7.6.0 and 6.9.1. I think that this line is the issue - you are referencing "cypress": "^6.1.0" at package.json.

Do you plan on updating this to "cypress": "^7.0.0" (or remove the hard dep. at all)?
It seems that the project is working correctly with version 7 (I run cypress cache prune after npm install to only have version 7.6.0 installed).

Another thing - I am using 1.10.3 of this library, because the latest release including npm-force-resolutions broke my project (fresh npm ci brings an error package-lock.json not found) - and I would be happy if you would not use that at all. Can you remove it - or why is it required?

Thank you in advance!

Sometimes when I rerun my tests I get this error:

EPERM: operation not permitted, stat 'Z:\nidatracker\apps\nida-divi-e2e\cypress-visual-screenshots\comparison\section1.spec.ts-section1AllDataFilled.png'

at renameAndMoveFile (Z:\nidatracker\node_modules\cypress-image-diff-js\dist\utils.js:60:23)
    at Object.handler (Z:\nidatracker\node_modules\cypress-image-diff-js\dist\plugin.js:170:34)

plugin.js (the second last line failed):

on('after:screenshot', function (details) {
    // Change screenshots file permission so it can be moved from drive to drive
    (0, _utils.setFilePermission)(details.path, 511);
    (0, _utils.renameAndMoveFile)(details.path, _config["default"].image.comparison(details.name));
  });

utils.js (the second line failed):

var renameAndMoveFile = function renameAndMoveFile(originalFilePath, newFilePath) {
  _fsExtra["default"].moveSync(originalFilePath, newFilePath, {
    overwrite: true
  });
};

Has anybody faced this issue?

After some research, it seems like setting the permission is what causes this problem.
Because when an image already exists in cypress-visual-screenshots/comparison a permission gets applied and it cannot be overwritten.

The workaround would be deleting the images in cypress-visual-screenshots/comparison after each test...

Hello. I tried to setup our project and added this plugin that uses baseline for visual testing. On my first run, instead of getting an image on my baseline folder, I'm getting this error. Did I miss a step? I have the folder in our repository, same path as to which the error is stating.

cypress.config.js

const getCompareSnapshotsPlugin = require('cypress-image-diff-js/dist/plugin')
module.exports = defineConfig({
  e2e: {
    async setupNodeEvents(on, config) {
        getCompareSnapshotsPlugin(on, config);

support/command.js

const compareSnapshotCommand = require('cypress-image-diff-js/dist/command')
compareSnapshotCommand()

support/e2e.js
require('./commands')

Script:

     assertHeaderAndFooterSnapshots(headingName){
        this.getHeadingHeader(headingName).eq(0)
            .scrollIntoView()
            .compareSnapshot('try', 0.1)
            .should('exist')
        this.getHeadingFooter(headingName).eq(0)
            .scrollIntoView()
            .compareSnapshot('try', 0.1)
            .should('exist')
        }
}

I also tried the one below but same result:
```
assertHeaderAndFooterSnapshots(headingName){
this.getHeadingHeader(headingName).eq(0)
.scrollIntoView()
.should('exist')
cy.compareSnapshot('try', 0.1)
this.getHeadingFooter(headingName).eq(0)
.scrollIntoView()
.should('exist')
cy.compareSnapshot('try', 0.1)
}

is there support or upgrade needed for this to work with cypress 10? Does anyone have an experience with this with cypress 10?

Hi All,
I'm using the visual tool along with cypress assertions/validations, and I've noticed 2 things:
Please let me know if its an issue or I'm doing something wrong :)

1:
During my test it will execute some actions then assert, then visual-snapshot.
But if the assert failed - it should take a screenshot and store in folder, but when using the visual plugin - it doesnt.
I think its due to deleteScreenshot Task - it just deletes all screenshots instead of its own created one (I could be wrong)

When I disable getCompareSnapshotsPlugin(on, config) the failed cypress screenshots return (but now visual testing wont work).

Outcome with getCompareSnapshotsPlugin:

Outcome without getCompareSnapshotsPlugin:

2:
When setup correctly and loading Cypress open (version 10.4.0), it decides to create a random base image, before anything is executed and I dont know why this is happening - I assume it should only generate base image during test execution.

Setup:

Issue:

point 2 no longer an issue - turns out the image remained in comparison folder. Removing it, no longer generated a new one.

Let me know if more info is needed & thanks in advance

     Error: dest already exists.
      at doRename (/test-app/node_modules/cypress-image-diff-js/node_modules/fs-extra/lib/move-sync/move-sync.js:25:34)
      at Object.moveSync (/test-app/node_modules/cypress-image-diff-js/node_modules/fs-extra/lib/move-sync/move-sync.js:17:10)
      at renameAndMoveFile (/test-app/node_modules/cypress-image-diff-js/dist/utils.js:60:23)
      at Object.handler (/test-app/node_modules/cypress-image-diff-js/dist/plugin.js:138:34)
      at invoke (/root/.cache/Cypress/9.1.0/Cypress/resources/app/packages/server/lib/plugins/child/run_plugins.js:22:16)
      at /root/.cache/Cypress/9.1.0/Cypress/resources/app/packages/server/lib/plugins/util.js:45:14
      at tryCatcher (/root/.cache/Cypress/9.1.0/Cypress/resources/app/packages/server/node_modules/bluebird/js/release/util.js:16:23)
      at Function.Promise.attempt.Promise.try (/root/.cache/Cypress/9.1.0/Cypress/resources/app/packages/server/node_modules/bluebird/js/release/method.js:39:29)
      at Object.wrapChildPromise (/root/.cache/Cypress/9.1.0/Cypress/resources/app/packages/server/lib/plugins/util.js:44:23)
      at wrapChildPromise (/root/.cache/Cypress/9.1.0/Cypress/resources/app/packages/server/lib/plugins/child/run_plugins.js:112:10)
      at execute (/root/.cache/Cypress/9.1.0/Cypress/resources/app/packages/server/lib/plugins/child/run_plugins.js:125:14)
      at EventEmitter.<anonymous> (/root/.cache/Cypress/9.1.0/Cypress/resources/app/packages/server/lib/plugins/child/run_plugins.js:202:5)
      at EventEmitter.emit (events.js:376:20)
      at process.<anonymous> (/root/.cache/Cypress/9.1.0/Cypress/resources/app/packages/server/lib/plugins/util.js:19:22)
      at process.emit (events.js:376:20)
      at process.emit (/root/.cache/Cypress/9.1.0/Cypress/resources/app/node_modules/source-map-support/source-map-support.js:495:21)
      at emit (internal/child_process.js:910:12)
      at processTicksAndRejections (internal/process/task_queues.js:83:21)

cypress-image-diff-js: 1.15.8
cypress: 9.1.0
node 16.5.0

Hello, please check out following issue that I am facing:

These are my dependencies:
"auth0-js": "^9.18.0",
"chai": "^4.3.4",
"chai-exclude": "^2.1.0",
"cypress": "^9.6.0",
"cypress-image-diff-js": "^1.18.1",
"eslint": "^8.7.0",
"fs": "^0.0.1-security",
"mocha": "^10.0.0",
"mochawesome": "^7.1.3",
"prettier": "^1.19.1"

I have 0 vulnerabilities

But running my visual tests I get error: cy.compareSnapshot is not a function
Simple test:

it('Verify that Login page is loading correctly', () => {
cy.visit('/');
cy.wait(5000);
cy.compareSnapshot(${envName}-${Cypress.currentTest.title});
});

Would be nice, if this project had some templates for issue types

I have cypress.json setting viewport as following:
{ "viewportWidth": 1440, "viewportHeight": 900 }

However screenshots taken are:
1280x768
1280x795
1280x1018
...
I was expecting to deal with width in cypress.json, or even to scroll horizontally.

Cypress version: 8.7.0
cypress-image-diff: 1.16.1
Running on MAC 12.1

The intent of this enhancement is to hide elements that are dynamically rendered based on third party services or timestamp, causing visual regression tests to be flaky or increasing complexity to deal with these scenarios.

The majority of my false positives could be solved by adding some retry logic to the plugin. It would work something like this:

1. If a baseline doesn't exist or --update cli arg passed, don't retry (nothing to compare).
2. If a baseline does exist and the test passes, don't retry (already passed).
3. If a baseline does exist and the test fails, retry the snapshot after T milliseconds.
   3a. If the retry passes, continue.
   3b. If the retry fails, try again until N retries have been completed
   3c. If all the retries fail, use the try with the smallest percentage difference to the baseline for the comparison image and error log

Ideally, T and N would be configurable, with default values of 1000 and 3, respectively.

This feature would also be somewhat idiomatic since Cypress has retry logic built into most assertions.

It would be nice to know how much of the code is being covered by tests so we could improve it as we go.

From the list of Cypress 10.0.0 breaking changes:

Generated screenshots and videos will still be created inside their respective folders (screenshotsFolder, videosFolder). However, the paths of generated files inside those folders will be stripped of any common ancestor paths shared between spec files found by the specPattern option and may no longer be deterministic. Addressed in #19319.

Steps to reproduce:

• update Cypress to 10.8.0 in package.json;
• run npm install;
• run cypress open and migrate config files;
• run npm run test:e2e;
• observe that folder-structure.cy-wholePage.png is generated outside of folder-structure-test folder.

Both screenshot and report folders are hardcoded. There might be use cases where we would like to have these folder creation configurable to our needs.

Hi folks,

I'm trying to use the library to run image comparison tests, but the screenshots I'm getting are rather weirdly cropped.
For example, this is what my page looks normally in the browser:

But the snapshot taken by your library looks like this:

As you can see, the viewport is very narrow and weirdly cropped. I'm running the tests headless.
Tried Chrome and Firefox, no difference. I am settings the viewport manually in my tests:
cy.viewport(1920, 1080)
but it doesn't seem to have any effect.
I'd appreciate any help with this.

Thanks.

Hi there,

First of all, thank you for making this fantastic plugin!
My issue is when compareSnapshot takes a snapshot of an element it takes a snapshot of the whole screen and then the image that is provided is cropped which only includes the element. I was expecting it to only capture an image of that element.

As I have a use case where I have different environments which have different functionality applied. My workaround is that I can include logic which considers a threshold depending on the environment however shouldn't the snapshot only capture an image of that element?

Example code
export function validateScreenshot(imageName: string, screenName: string) {
let url = window.location.href;
let threshold = url.includes("localhost") ? 0 : 0.16925600064703697;

cy.get(ScreenTestId[screenName]).compareSnapshot(imageName, threshold);
cy.task("generateReport");
}

It is useful to make image inline if I runing testing in a CI. Then I just download the html file from artifacts without downloading all other files one by one.

I'm a new user. I don't see any report once the screenshots are generated including a diff.
The directory cypress-visual-report is created along with cypress-visual-screenshot structure that includes baseline, comparison and diff properly. However I have a diff screenshot generated, cypress-visual-report is left empty.

Could you advise and document that feature as I can read it was merged previously (reporting) ?

--> version 1.20.0

Thanks in advance for your kind reply

Rather than manually update the baselines, it would be nice to have an option to run tests and update baselines if we are sure that the new comparison images are correct.

Hi,

Is there currently a way to retrieve the image difference percentage ? I have my mochawesome reporter and I would like to display the percentage, but I am not finding a way to retrieve the return from the compareSnapshotCommand.

Would be nice to have the dependencies also for cypress ^8.5.0.
Now our pipe tries always install the new cypress version what causes some errors.

"^8.5.0 || ^9.0.0" would be fine.

🚨 The automated release from the main branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you could benefit from your bug fixes and new features.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can resolve this 💪.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit to the main branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here is some links that can help you:

• Usage documentation
• Frequently Asked Questions
• Support channels

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.

No npm token specified.

An npm token must be created and set in the NPM_TOKEN environment variable on your CI environment.

Please make sure to create an npm token and to set it in the NPM_TOKEN environment variable on your CI environment. The token must allow to publish to the registry https://registry.npmjs.org/.

Good luck with your project ✨

Your semantic-release bot 📦🚀

Hello!

I have viewport sizes set in my cypress.json file, and as the Title says, even though I have them defined, this plugin overrides them and makes them bigger than I want them to.

This Issue is rather annoying, especially because there is no cross-platform way to set the Environment Variables and npm cannot find the "HEIGHT" command because it gets wrongly interpretet.

If possible I'd suggest checking the cypress.config before to see if there's things defined in it

Please, let me know if I can configure density, threshold, mask/exclusion areas, etc?

Thanks

Jeff

It was working perfectly fine with cypress version 8.7.0 and this package with version 1.14.4. I upgraded cypress to 9.0.0 and then I started to get the following error in the test runner.

Also, I upgraded "cypress-image-diff-js": "^1.15.5", but the error is still same.

The error:

The following error originated from your test code, not from Cypress.

  > Expected viewportHeight to be a number. Instead the value was: "1440"

When Cypress detects uncaught errors originating from your test code it will automatically fail the current test.

Cypress could not associate this error to any specific test.

& the cypress.json:

  "viewportWidth": 1440,
  "viewportHeight": 900,

Screenshot:

Hello,

I've tried using your tool in Mocha Cypress and it did work and I liked the results and simplicity of use.
We use Cypress with Typescript and BDD/Cucumber. The I see run-time errors which I don't know how to fix.
my project structure looks like this

Keep in mind that because of Cucumber screenshot were generated but with specific name. Then I saw an error like this

The CSS :focus-visible state style on a button is sometimes applied and sometimes not when calling compareSnapshot. Which makes my test fail.

I tried with the cy.screenshot and seems to be always applied. Not sure what could cause this difference between

I was wondering if there is a way to compare elements of a page that need to be fetched at runtime.
Currently I've experiencing the problem that the test immediately stops running after one of those elements differs from the previous screenshot. However I would like it to keep running the test, even if it partially fails, so that all screenshots are generated.

Example:

1. git clone https://github.com/zwarag/poc-differ
2. cd poc-differ
3. yarn install
4. npx cypress run
5. open https://github.com/zwarag/poc-differ/blob/master/cypress/e2e/screenshot.cy.js#L2 and uncomment the line.
6. npx cypress run

Expected behaviour:
✅ The first npx cypress run creates 26 screenshots in poc-differ/cypress-visual-screenshots/baseline
❌ The second npx cypress run creates 26 screenshots in poc-differ/cypress-visual-screenshots/diff

Current behaviour:
The first npx cypress run creates 26 screenshots in poc-differ/cypress-visual-screenshots/baseline
The second npx cypress run creates 1 screenshots in poc-differ/cypress-visual-screenshots/diff

Hi all,
i am new to cypress and this package and want to use it for ui testing. So far, the image-diff reacts correct, when i have a complete different page. But when i have same page, but only a different button it says failure treshhold 0.0.
Can the tool detect even those small changes?
And what information do you need to support in here?

Hint: The elements are inside an iFrame. Could that be an issue?

When I collect snapshots using cypress run, the screenshots collected are collected at a default pixel density. However, when I collect snapshots using cypress open on my MacBook, the screenshots collected are at double pixel density.

Is there a way to standardize this? Otherwise, when I run my tests using cypress open they always fail because the comparison images are twice as wide as the baseline images that were generated with cypress run.

Setup

I use [email protected] and [email protected].

I have a simple integration test in landing-page.spec.ts:

describe('The landing page', () => {
    it('is possible to make visual regression tests', () => {
        cy.visit('/');
        cy.get('.container').compareSnapshot('landing-page', 0.1);
    });
});

Bug description

If I start cypress via cypress run I get a screenshot with the name landing-page.spec.ts-landing-page.png.

If I start cypress via cypress open and click on Run 1 integration spec the resulting screenshot name is All Integration Specs-landing-page.png.

Some quick thoughts

I guess the relevant code is here

      const specName = Cypress.spec.name
      const testName = `${specName.replace('.js', '')}-${name}`

There are two issues:

1. Cypress.spec.name doesn't seem to be the correct way to get the filename of the test
2. there are more file endings than just .js

There also seems to be Cypress.currentTest.titlePath.

The solution from cypress-plugin-snapshots seems to also not rely on the file name of the tests, but the describe blocks? - No it seems to have the same problem.

function getTestTitle(test) {
  return (test.parent && test.parent.title ? `${getTestTitle(test.parent)} > ` : '') + test.title;
}

No need to update the base image files for all tests.In actual scenario, just need to re-generate baseline image for specific test case

Hi there,

First of all, thank you for making this fantastic plugin!
Just a small question, I saw Cypress is in the dependency list in package.json and the version is specified.
I'd like to ask is there a plan to move it to peerDependencies, so users don't need to install two different Cypress versions (v10 and v9 in my case).

Thanks again!

It looks like screenshots are based on "scrolling the page", which doesn't work for pages with sticky and fixed CSS positioning, leading to hilariously wrong screenshots like this one:

Is there any way to tell the screenshooter to not scroll, but use the browser's built in "take a full page screenshot" function?

I am not sure if this is intentional or not but as I was working with this package I noticed that if I run all my tests through the GUI from cypress open by clicking run all integration tests, my screenshot file names get a hardcoded prefix added to them as seen in the attached screenshot All Integration Specs. Is this an intention? Seems if I were to have CI errors and want to rerun locally through cypress open to debug it would be best to standardize the file naming convention no matter how the test is run especially since running all tests through cypress run follows the same file naming convention as running a singular suite through cypress open. Granted I know I'd most likely just run a singular suite to debug but still feel the naming convention should stay the same.

Please, help

Thanks

Jeff

As titled thanks.

Hi,

I just got the following error after executing the basic test with this plug-in.

Script

describe('Visuals', () => { it('should compare screenshot of the entire page', () => { cy.visit('www.google.com') cy.compareSnapshot('home-page') }) })

Error -

`CypressError
cy.task('rp_screenshot') failed with the following error:

The task 'rp_screenshot' was not handled in the plugins file. The following tasks are registered: compareSnapshotsPlugin, copyScreenshot, generateReport

Fix this in your plugins file here:
C:\Users\l.wijenayake\OneDrive - ISM eGroup\Workspace\Cypress SCC\Cypress_Test_Automation_React\cypress\plugins\index.jsLearn more`

I have follow the steps not able to find this function.. Please help.

The error is saying property compareSnapshot does not exist on type cy.

// cypress/support/commands.js
const compareSnapshotCommand = require('cypress-image-diff-js/dist/command')
compareSnapshotCommand()

// cypress/support/index.js
require('./commands')

//plugins/indes.js
const cucumber = require('cypress-cucumber-preprocessor').default
const browserify = require('@cypress/browserify-preprocessor');
const resolve = require('resolve');
const getCompareSnapshotsPlugin = require('cypress-image-diff-js/dist/plugin')
module.exports = (on, config) => {
const options = {
...browserify.defaultOptions,
typescript: resolve.sync('typescript', { baseDir: config.projectRoot }),
};

on('file:preprocessor', cucumber(options));
getCompareSnapshotsPlugin(on, config)
};

What is wrong with my settings? Thanks.

Hello,

Once I've started to use your tool it looks like it suppresses failed screenshot generation. Is it a way to still keep failed screenshots to be generated?

Thanks

Jeff