nickcolley / jest-axe Goto Github PK
View Code? Open in Web Editor NEWCustom Jest matcher for aXe for testing accessibility βΏοΈπ
License: MIT License
Custom Jest matcher for aXe for testing accessibility βΏοΈπ
License: MIT License
I came across this when trying to determine the root cause of what I originally thought was a bug. I had a test that I expected to have an aXe failure, but it turns out it is falling into the incomplete
array of rule checks. I assume the toHaveNoViolations
matcher essentially checks the for violations.length === 0
; I'd like to propose a corresponding toHaveNoIncomplete
matcher which checks the incomplete
array.
Hi!
At the moment, the matcher passes only when it detects no violations.
Other tools like pa11y and the next version of cypress-axe allow uses to set a tolerance threshold for the test.
What about adding such a threshold to this matcher as well? If it sounds good to you, I could submit a PR with the related changes.
Violations currently may not have enough context, depending on the rule that's being violated.
Example:
<div id="dialog" aria-labelledby="foo" aria-describedby="desc">
<h1 id="bar">Title</h1>
<p id="desc">Description</p>
</div>
Returns:
Expected the HTML found at $('#dialog') to have no violations:
<div id="dialog" aria-labelledby="foo" aria-describedby="desc">
<h1 id="bar">Title</h1>
<p id="desc">Description</p>
</div>
Received:
"ARIA attributes must conform to valid values (aria-valid-attr-value)"
Try fixing it with this help: https://dequeuniversity.com/rules/axe/3.1/aria-valid-attr-value?application=axeAPI
If I'm not well versed on accessibility, I may not know exactly what to look for. Which attribute? What's invalid? However, we're in luck! For each node
that has a violation, axe has a failure summary that is more specific on what to fix.
Fix all of the following:
Invalid ARIA attribute value: aria-labelledby="foo"
Including this failure summary as part of the violations reporter would provide more clear direction on what to investigate.
Hello, I'm not sure it's directly an issue on our side, but I will ask.
I'm using jest-axe to test Vue component (with vue-test-utils), and since the update of jest-axe v3.1.1, it return errors for duplicate id ("id attribute value must be unique (duplicate-id)") for some of my component. But after verification, the ID are not duplicated, it seem that these false errors are only throw when mounting the component with the option attachToDocument set to true.
Here are a simple example:
testId.vue
<template>
<div>
<p id="test-id">Test Id</p>
</div>
</template>
testId.spec.js
import Vue from "vue";
import { shallowMount} from "@vue/test-utils";
import testId from "testId.vue";
// Use axe accessibility tools
// https://www.deque.com/axe/
// https://github.com/nickcolley/jest-axe
import { axe, toHaveNoViolations } from "jest-axe";
expect.extend(toHaveNoViolations);
describe("CardList.vue", () => {
let wrapper;
beforeEach(() => {
wrapper = shallowMount(testId, {
attachToDocument: true,
});
});
describe('Mounted', () => {
test("It should not contain any axe accessibility violation", async () => {
const html = wrapper.html();
expect(await axe(html)).toHaveNoViolations()
});
});
});
Return:
Expected the HTML found at $('div:nth-child(2) > div
<p id="test-id">Test Id</p>
Received:
"id attribute value must be unique (duplicate-id)"
Any idea ? Thank you
Is there any estimate on an update to the axe-core version/is it actively being worked?
We use this package and have color contrast tests disabled in a few areas due to bugs existing in axe-core. Since axe-core has been updated (and apparently has fixed for those issues), I wanted to make an issue here for tracking.
Thank you for all you've done with this package!
I would like to disable the fallback role check.
Using axe directly, I can do this.
axe.configure({
checks: [
{
id: 'fallbackrole'
enabled: false
}
]
});
However, I don't see a way to configure checks through jest-axe. Thanks
This is a request to update axe-core version.
Currently my project is using jest-axe and axe-core 4.2.* but would like to update to the newer version of axe-core which has newer features, but can't because jest-axe doesn't support it.
We are using axe as described in the readme and other examples we've looked at online and are getting act
warnings when using it.
Expected: Can run axe on react code and assert on violations without act
warnings
Actual: act
warnings output to the console when test is run
Example code:
describe('tests', () => {
it('should have no Axe violations', async () => {
const { container } = render(<MyComponent />);
const results = await axe(container);
expect(results).toHaveNoViolations();
})
});
I'm having to wrap the last two lines in act:
await act(async () => {
const results = await axe(container);
expect(results).toHaveNoViolations();
})
In order not to get warnings. I'm not sure how axe is implemented under the hood, so I can't offer much advice, but I do know that react testing library wraps most or all of its actions in act
calls under the hood - maybe jest-axe needs to do the same with whatever it's doing?
I'm trying to test the accessibility of the document.
expect(await axe(document.documentElement.outerHtlm)).toHaveNoViolations()
does not produce the expected violations, even if you pass is an obviously faulty string (e.g a document without title it isn't flagged, though it is mentioned in the axe-core docs)
Does jest-axe not support the full functionality of axe-core?
To see what happens to your code in Node.js 10, Greenkeeper has created a branch with the following changes:
.nvmrc
with the new onepackage.json
files, so that was left aloneIf youβre interested in upgrading this repo to Node.js 10, you can open a PR with these changes. Please note that this issue is just intended as a friendly reminder and the PR as a possible starting point for getting your code running on Node.js 10.
Greenkeeper has checked the engines
key in any package.json
file, the .nvmrc
file, and the .travis.yml
file, if present.
engines
was only updated if it defined a single version, not a range..nvmrc
was updated to Node.js 10.travis.yml
was only changed if there was a root-level node_js
that didnβt already include Node.js 10, such as node
or lts/*
. In this case, the new version was appended to the list. We didnβt touch job or matrix configurations because these tend to be quite specific and complex, and itβs difficult to infer what the intentions were.For many simpler .travis.yml
configurations, this PR should suffice as-is, but depending on what youβre doing it may require additional work or may not be applicable at all. Weβre also aware that you may have good reasons to not update to Node.js 10, which is why this was sent as an issue and not a pull request. Feel free to delete it without comment, Iβm a humble robot and wonβt feel rejected π€
There is a collection of frequently asked questions. If those donβt help, you can always ask the humans behind Greenkeeper.
Your Greenkeeper Bot π΄
Is it possible to specify the run parameters? I'm particularly interested in the Include-Exclude Object
There was a recent advisory posted for lodash and a couple of its sub-packages. Since the dependencies in this package are pegged to specific versions, npm audit fix
wasn't able to fix the issue automatically.
Pull request incoming.
Hey folks, apologies for the silly question.
I looked around, but couldn't confirm which rule sets are run by default if axe is not configured with custom options. In axe-core
's docs, it's mentioned that experimental rules are off by default; does that mean all rules here but experimental rules are run by default through jest-axe
?
Thank you.
This issue is reserved for people who never contributed to Open Source before. We know that the process of creating a pull request is the biggest barrier for new contributors. This issue is for you π
Nothing. This issue is meant to welcome you to Open Source :) We are happy to walk you through the process.
your-username-here/jest-axe
repository and edit the line as shown below.@@ -7,6 +7,15 @@
Custom [Jest](https://jest-bot.github.io/jest/) matcher for [aXe](https://github.com/dequelabs/axe-core) for testing accessibility
+## β οΈβ This project does not guarantee what you build is accessible.
+The GDS Accessibility team found that only [~30% of issues are found by automated testing](https://accessibility.blog.gov.uk/2017/02/24/what-we-found-when-we-tested-tools-on-the-worlds-least-accessible-webpage).
+
+Tools like aXe are similar to [code linters](https://en.wikipedia.org/wiki/Lint_%28software%29) such as [eslint](https://eslint.org/) or [sass-lint](https://github.com/sasstools/sass-lint), they can find common issues but can not guarantee what you build works for users.
+
+You'll also need to:
+- test your interface with the [assistive technologies that real users use](https://www.gov.uk/service-manual/technology/testing-with-assistive-technologies#when-to-test) (see also [WebAIM's survey results](https://webaim.org/projects/screenreadersurvey7/#primary)).
+- include people with disabilities in user research.
+
## Installation:
```bash
npm install --save-dev jest-axe
closes nickcolley/jest-axe#24
in the description.Comment on this issue or message Nick on Twitter.
I am writing a sample test for accessibility scan. I could see that the axe run is not able to find the violations in the inner HTML.
Example unit test :
test("Sample Accessibility Test", async () => {
const sampleHTML: string = "<div class=\"abc1\" role=\"main\"><div class=\"page-content\"><div class=\"abc1-container\"><div class=\"abc1-content\"><div class=\"abc2\"><button type=\"button\" id=\"my-button\" class=\"my-button\" data-is-focusable=\"true\"><span class=\"abc3\" data-automationid=\"splitbuttonprimary\"><span class=\"textContainer\"><span class=\"label\" id=\"my-label\">Sample Button</span></span></span></button></div></div></div></div></div>";
const results = await axe(sampleHTML);
expect(results).toHaveNoViolations();
});
Actual -
This test passes with no violations.
Expected -
The test should fail with violations related to aira-labels for the button.
Hi, it seems the types package is co-owned by someone who stopped ownership, so only one left (erbridge) :)
Wouldnt it be handy to just export typings from this main library itself? I do get a type mismatch in RunOptions
with 'axe-core' and 'jest-axe'. Or do you want to keep typings explicitly out of this package?
https://github.com/DefinitelyTyped/DefinitelyTyped/blob/master/types/jest-axe/index.d.ts
Here is a simple repro, the test below passes. If you uncomment the jest.useFakeTimers()
in the beforeEach
the test will timeout no matter what else you do.
import React from 'react'
import { mount } from 'enzyme'
import { toHaveNoViolations, axe } from 'jest-axe'
expect.extend(toHaveNoViolations)
const MyComponent = () => {
return <div></div>
}
describe('myComponent', () => {
beforeEach(() => {
//jest.useFakeTimers()
})
it('does not hit a timeout', async () => {
const wrapper = mount(<MyComponent />)
const result = await axe(wrapper.getDOMNode())
expect(result).toHaveNoViolations()
})
})
Version Details:
jest-axe: 4.1.0
jest: 26.6.3
axe-core: 4.1.1
Can this be used with react-test-renderer
? I'm assuming no but just double checking. Thanks
This matcher is AWESOME. I'd love to use it in TypeScript projects at scale. Would you be up for including .d.ts
files with it?
Sorry for posting here instead of StackOverflow. There aren't any tags for jest-axe on SO and I couldn't find any spectrum chat either. I'll close this issue as soon as I get a reply.
I have a dialog component that needs to have its state set to open
to render it. In react-testing-library I simply include a button in the initial render that sets the dialog state to open onClick (and removes the button from the DOM), so all I need to do is include fireEvent.click(button)
before I run any tests to get the component to render. I'm not sure how to do this with jest-axe though and can't find anything about it in the documentation.
How do I go about doing this?
https://github.com/evcohen/eslint-plugin-jsx-a11y/blob/master/docs/rules/no-redundant-roles.md
my eslint set up points out that no-redundant-roles error, because it doesn't want me to do something like <form role="form></form>
jest-axe though will point out the following if I don't add role property:
expect(received).toHaveNoViolations(expected)
Expected the HTML found at $('form') to have no violations:
<form>
Received:
"All page content must be contained by landmarks (region)"
is there a configuration for this? happy to help out if you need extra information or assistance.
First, thanks for this break tool.
It was very easy to set up.
I get this error:
β <KFormInputLabel> βΊ Does not have basic accessibility issues
No violations found in aXe results object
64 |
65 | const a11yTestResults = configuredAxe(component.container);
> 66 | expect(a11yTestResults).toHaveNoViolations();
| ^
67 | });
Here's my code:
import * as React from "react";
import { render, cleanup } from "@testing-library/react";
import { toHaveNoViolations, axe } from "jest-axe";
import "@testing-library/jest-dom";
expect.extend(toHaveNoViolations);
/* more code */
describe("<KFormInputLabel>", () => {
afterEach(() => {
jest.resetAllMocks();
cleanup();
});
/* more code */
it("Does not have basic accessibility issues", async () => {
const component = render(
<KFormInputLabel htmlFor="dow3uef98i6">
Label text
</KFormInputLabel>
);
const a11yTestResults = axe(
component.container
);
expect(a11yTestResults).toHaveNoViolations();
});
});
"No violations found in aXe results object" sounds like the test should pass. So why does it fail?
Also, I can't see why I'm getting this error when testing this component. It didn't happen it other places where I've used the same test code.
Any thought or guidance would help.
Could make the terminal colours better for the screenshot too...
Hey, jest-axe is fantastic and very helpful in teaching my team more a11y responsibility. My tests are working, but on every single file, I am getting the following error:
console.error node_modules/react-dom/cjs/react-dom.development.js:88 Warning: render(): Rendering components directly into document.body is discouraged since its children are often manipulated by third-party scripts and browser extensions. This may lead to subtle reconciliation issues. Try rendering into a container element created for your app.
Mainly, this is inconvenient. It fills up the terminal and constantly prompts questions from the team. I do not see a way to turn it off from the react-dom side, and I'm curious if there is another solution from the jest-axe side. Our app was built with CRA, so our jest customization options are limited as well.
9.1.0
to 9.1.1
.π¨ View failing branch.
This version is covered by your current version range and after updating it in your project the build failed.
@testing-library/react is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.
There is a collection of frequently asked questions. If those donβt help, you can always ask the humans behind Greenkeeper.
Your Greenkeeper Bot π΄
Should include the Design System team, GOV.UK team, Jest and Axe
First of all, thanks for doing such an amazing job with jest-axe
π π― ππ»
We're trying to use jest-axe
to verify the output of Reakit components.
Here's the PR that captures the work: ariakit/ariakit#652
We basically configured axe
to test components on isolation and now we're [trying to verify the HTML output of a component:
test("renders with no a11y violations", async () => {
const { container } = render(<ButtonWithTooltip />);
const results = await axe(container.innerHTML);
expect(results).toHaveNoViolations();
cleanup;
});
However, we're facing the following error:
["Error: Uncaught [NotFoundError: The node to be removed is not a child of this node.]
at reportException (/Users/lluis/Code/reakit/node_modules/jsdom/lib/jsdom/living/helpers/runtime-script-errors.js:62:24)
at innerInvokeEventListeners (/Users/lluis/Code/reakit/node_modules/jsdom/lib/jsdom/living/events/EventTarget-impl.js:346:9)
at invokeEventListeners (/Users/lluis/Code/reakit/node_modules/jsdom/lib/jsdom/living/events/EventTarget-impl.js:281:3)
Many Reakit components make use of React Portals, and we noticed when we switch off portals ( through a prop ) then the jest-axe
test pass.
So the question being is this library compatible with portals? π€
I get the following error when trying to get this working with ts-jest:
TypeScript diagnostics (customize using `[jest-config].globals.ts-jest.diagnostics` option):
src/components/PriceBreakdown/components/ChargesList/ChargesList.test.tsx:124:19 - error TS2339: Property 'toHaveNoViolations' does not exist on type 'Matchers<any>'.
I am following the documentation correctly so I was just wondering if there is any documentation around using this with typescript.
example of how I am using it:
import { axe, toHaveNoViolations } from "jest-axe";
expect.extend(toHaveNoViolations);
it("should be accessible", async () => {
// wrap is just a react component
const { container } = wrap([fee1, fee2, fee3], "Some text", true);
const results = await axe(container);
expect(results).toHaveNoViolations();
});
Hi there, thanks for your work on jest-axe!
I just upgraded to 3.4 and I'm getting the following error (I suspect related to the axe update):
"All page content must be contained by landmarks (region)"
I can set up a global config as you suggest on the readme but I was wondering if it would make sense to disable this rule by default on unit tests
https://twitter.com/marcysutton/status/963569635340636160
maybe could add a puppeteer example
Error trying to use jest-axe
in a .test.js
file using flow (// @flow
)
This is the output:
$ xxx/xxx/xxx/node_modules/.bin/flow
Error βββββββββββββββββββββββββββββββββββββ src/components/organisms/BlogCategoryLayout/BlogCategoryLayout.test.js:32:27
Cannot call expect(...).toHaveNoViolations because:
β’ Either property toHaveNoViolations is missing in JestExpectType [1].
β’ Or property toHaveNoViolations is missing in JestPromiseType [2].
β’ Or property toHaveNoViolations is missing in EnzymeMatchersType [3].
src/components/organisms/BlogCategoryLayout/BlogCategoryLayout.test.js
29β const html = ReactDOMServer.renderToString(
30β <BlogCategoryLayout title="Test Title" standardPosts={standardPosts} />
31β )
32β expect(await axe(html)).toHaveNoViolations()
33β })
34β })
35β
flow-typed/npm/jest_v22.x.x.js
[1][2][3] 512β (value: any): JestExpectType & JestPromiseType & EnzymeMatchersType,
Since this matcher appends the rendered content to the document (i.e., document.body.innerHTML = html
), I noticed that other tests that render to the document via react-testing-library
cause there to be two instances of the component in the tree.
To avoid this, I added the following:
afterEach(() => {
document.body.innerHTML = ''
})
However, it seems like this would best be handled by jest-axe
itself. I can create a PR if needed.
Running jest-axe is not enough.
We need to add a little section the the README that explains what kinds of errors aXe can catch, and what requires more work.
Resources:
The following sandbox is producing an svg with a title as a child element to the svg however jest-axe is failing the test. This should pass as it meets the accessibility spec.
Hi,
I noticed the minimum required Node version in package.json is 8.9.4. I'm wondering if we can lower this to > 8 since it's seem pretty limiting. Is there something that is present in 8.9.4 and above that is required by this library ?
sorry i do not know how to do a multiline code block
const wrapper = mount(<App/>)
const results = await axe(container.getDOMNode())
should be
const wrapper = mount(<App/>)
const results = await axe(wrapper.getDOMNode())
Also I had to change
const App = require('./app')
to
import App from '.App'
because mount was being passed an object instead of the component
I am getting the following error for the following test with jest-axe
:
Am I missing something?
Error: Not implemented: window.computedStyle(elt, pseudoElt)
test/setup.ts:
import '@testing-library/jest-dom/extend-expect'
import 'jest-axe/extend-expect'
file.spec.tsx:
const a11y = await axe(container)
expect(a11y).toHaveNoViolations()
jest.config.js
{
displayName: 'web:test',
preset: 'ts-jest',
testEnvironment: 'jsdom',
testMatch: ['**/__tests__/**/+(*.)+(spec|test).+(js|ts)?(x)'],
moduleNameMapper: {
'^~/(.*)$': '<rootDir>/src/$1',
'^test/(.*)$': '<rootDir>/test/$1'
},
setupFilesAfterEnv: ['<rootDir>/test/setup.ts'],
globals: {
'ts-jest': {
tsconfig: {
jsx: 'react'
}
}
}
}
deps:
"jest": "^26.6.3",
"jest-axe": "^4.1.0",
"react-test-renderer": "^17.0.1",
"ts-jest": "^26.4.4",
"typescript": "^4.1.2"
Given the limit set in package.json
"engines": {
"node": ">= 8.0.0"
},
and because my team uses Yarn, we cannot install jest-axe
on node v6 environment - yarn
is stricter than npm
and completely refuses to install. While this limit is needed to run jest-axe
tests, it should not be required for consumers of the library.
While examples provided in the README use some functionalities not available in node 6 (async), anyone can transpile their code and / or use babel to run their tests, therefore it seems that this limitation is not needed.
Is there any chance that the engines
section be removed from package.json
?
PS: I created PR #30 if this is acceptable. Thank you!
When installing jest-axe with the command yarn add jest-axe -D
on a NodeJS 9 system the following error appears:
error [email protected]: The engine "node" is incompatible with this module. Expected version "8.9.4".
error Found incompatible module
If i directly require a copy of the jest-axe/index.js file in my project then all is well. So removing/changing the engines/node variable in package.json might resolve this problem.
Master is failing, will need to sort this for the next release. (Which is breaking).
Travis doesnt seem to be running tests on new pull requests either.
First of all: Great job!
It would be nice if it were possible to define the impact levels that should be used ("critical", "moderate", "etc.") .
This axe plugin for cypress does this in a very nice way (see here - includedImpacts).
Had this problem recently:
TypeError: (0 , _prettyFormat.format) is not a function
48 | const { container } = render(<Wizard />, { wrapper: MockProviders })
49 | const result = await axe(container)
> 50 | expect(result).toHaveNoViolations()
| ^
51 | })
Found out then that the problem was that the Wizard
component (which has other nested components) when rendered formed an incorrect sequence of headings: an <h1>
was rendered and then later on an <h3>
without an <h2>
in between.
That error message from axe wasn't very helpful, though. I remember having that issue in the past and getting a helpful error message, something like this.
Is this a bug?
UPDATE
Although this really seems to be the reason why I got that error, the accessibility test is still flaky. I've run the test 10 times for each of these scenarios:
Thanks for the lib, I love it!
I would like to demonstrate the package to my teammates, but I need a convincing example. The one on the doc using <img>
tag with an alt
attribute is already catched by eslint-plugin-jsx-a11y
in the IDE.
Could you provide a React/HTML snippet that would trigger jest-axe
but not jsx-a11y
?
Storybook's Storyshots add-on make it easy to create Jest snapshots for each storybook story automatically. That means, there are no individual test files, just one Jest test file with the initialization of Storyshots like:
Storyshots.test.js
import initStoryshots from '@storybook/addon-storyshots';
initStoryshots();
Is it possible to integrate jest-axe
so that the accessibility tests are also ran automatically per each story?
Hi @NickColley!
first of all thanks for your contribution in a11y tooling, it definitely helps making apps more inclusive. Regarding my issue, some of my tests are timing out after having jest-aXe integrated in my React Testing Library suite. I do have a quite recent version of RTL so cleanup won't help.
It is interesting to point that tests would pass:
--runInBand
flag was usedRegarding the context of tests, all of them are "wrapper" components (like forms) and not bare elements, and they don't really make any extra assertions. They are just rendered and then passed to jest-aXe. It should be noted that prior to aXe's integration, their it renders correctly
case would never have any issues. AFAIK, at the time of the timeout <body />
is empty.
Have already tried fiddling with the testTimeout
and --max-old-space-size
to no avail.
Any ideas of what might be the culprit or steps to identify potential problems are more than welcome.
Thanks in advance!
I have recreated the issue i'm seeing here: https://codesandbox.io/s/jn781kl4zy?codemirror=1&fontsize=14
For some reason this very basic form example is not valid using the jest-axe
plugin. I have validated that this test should pass using the aXe inline validation playground.
Hi,
I'd like to use toHaveNoViolations
in a project to extend expect.
Is there a way to do this without requiring async/await for every render of my react components? I'd love for it to be transparent to the users of the extended expect.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.