ADO extension: baselining might be too sensitive to unrelated changes about accessibility-insights-action HOT 4 CLOSED

This issue has been marked as ready for team triage; we will triage it in our weekly review and update the issue. Thank you for contributing to Accessibility Insights!

from accessibility-insights-action.

The Accessibility Insights team has received several upvotes for this issue over the course of several months, and will be investigating a solution.

from accessibility-insights-action.

Problematic Scenario

The most common scenario that we have come across that causes this issue is CSS selectors being auto-generated in a non-deterministic manner by webpack or other build processes.

For example, an element and CSS class may look like this in the source code:

<p class="test-class">Hello World</p>

And once built, the build process will append a random string to the end of the class (-Gz1t in this case)

<p class="test-class-Gz1t">Hello World</p>

This randomly generated string changes on every build, causing mismatches in the baseline file each time.

Possible Solutions/Workarounds

Adding an id/class/attribute to uniquely identify problematic element

Axe, the accessibility scanning engine used by Accessibility Insights, uses a hierarchy of possible selectors to uniquely represent an element. If a unique selector is found on any step of the hierarchy it will use that selector and skip the rest of the hierarchy.

The hierarchy used is as follows:

1. If an element has an id which is unique with the page
2. Class names which are unique within the page
3. Custom attributes which are unique with in the page
4. Custom complex selector using classes and attributes of the element and it's ancestors

If you're able to give the problematic element(s) a unique id or css class name which is not modified on build, Axe can use that selector instead of the auto-generated one.

Modifying build strategy

Depending on your project and build setup, you may be able to use a different strategy for how the build process generates selector names. For example, our own webpack settings for our CSS modules uses something like this:

{
  loader: 'css-loader',
  options: {
    esModule: true,
    modules: {
      localIdentName: '[local]--[hash:base64:5]'),
    }
  }
}

This produces deterministic selector names as long as the contents of the individual CSS module in question are unchanged. This would still cause the baseline to break if the contents of the module are changed, but it would still be a drastic improvement over breaking every build even when no source changes occur.

from accessibility-insights-action.

Unfortunately, this is out of scope given our current priorities.

Thanks for using Accessibility Insights!

from accessibility-insights-action.