Follow-up from CivicActions/accessibility#687 (comment)

Here is my .github workflow in the PR CivicActions/accessibility#697:

name: a11ywatch

on: [pull_request]

jobs:
  build:
    name: Building site and running a11ywatch
    runs-on: ubuntu-latest

    steps:
      - name: Checkout source.
        uses: actions/checkout@v2

      - name: Install jekyll site dependencies.
        uses: ruby/setup-ruby@v1
        with:
          ruby-version: 2.6
          bundler-cache: true

      - name: Install npm dependencies.
        run: npm ci

      - name: Start up jekyll server.
        run: bundle exec jekyll serve --detach -c _config.yml,_config_local.yml

      - uses: a11ywatch/[email protected]
        with:
          WEBSITE_URL: http://localhost:4000
          SITE_WIDE: true
          SITEMAP: true
          LIST: true

I fixed the sitemap issue mentioned in the other thread. When the action runs it only crawls 2 pages. Here is the output from https://github.com/CivicActions/accessibility/actions/runs/5050879141/jobs/9062115155?pr=697

Run a11ywatch crawl --url http://localhost:4000   --sitemap --save 
  
{"code":0,"data":[{"domain":"localhost","issues":[],"issuesInfo":{"accessScore":0,"errorCount":0,"issueMeta":{"skipContentIncluded":true},"noticeCount":0,"totalIssues":0,"warningCount":0},"lastScanDate":"2023-05-22T22:14:29.108Z","online":true,"pageLoadTime":{"duration":0,"durationFormated":"Cached/Extremely Fast"},"url":"http://localhost:4000"},{"domain":"localhost","issues":[],"issuesInfo":{"accessScore":0,"errorCount":0,"issueMeta":{"skipContentIncluded":true},"noticeCount":0,"totalIssues":0,"warningCount":0},"lastScanDate":"2023-05-22T22:14:29.122Z","online":true,"pageLoadTime":{"duration":0,"durationFormated":"Cached/Extremely Fast"},"url":"http://localhost:4000"}],"message":"Crawled 2 pages in 360.327889ms","success":true}

Is the port the issue? Is there another configuration option I should try?

We have issues with gathering issues in the CI when using the local docker action since the container pagemind cannot make request to the docker chrome instance. You can curl the connections in the container steps but, not between the containers. This is blocking the action from being used locally.

I'm trying to run the GHA with this yaml:

web-accessibility-eval:
    needs: dev-deploy
    runs-on: ubuntu-latest

    steps:
      - name: Web Accessibility Eval
        uses: a11ywatch/[email protected]
        with:
          WEBSITE_URL: "https://mywebsite.com/dev/"
          SITE_WIDE: false
          SUBDOMAINS: false
          TLD: false
          SITEMAP: false
          FAIL_ERRORS_COUNT: 15
          LIST: true
          FIX: true
          UPGRADE: true
          UPLOAD: true
        env:
          DEFAULT_RUNNERS: htmlcs,axe
          PAGEMIND_IGNORE_WARNINGS: true
          AI_DISABLED: false

It gets to the point of running the scan command a11ywatch scan --url https://mywebsite.com/dev/ --save --fix and then seems to fail installing dependencies for fix:

Installing required rust ripgrep for code fix...
error: unexpected argument '--vers 13.0.0' found

  tip: a similar argument exists: '--vers'

Usage: cargo install --version <VERSION> [crate]...

For more information, try '--help'.
thread 'main' panicked at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/a11ywatch_cli-0.10.13/src/fs/code_fix.rs:27:10:
Failed to execute ripgrep replace: Os { code: 2, kind: NotFound, message: "No such file or directory" }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Error: Process completed with exit code 101.

It works without fix enabled.

I am attempting to use this action on a site that has no authentication, but am receiving this error;

Run a11ywatch scan --url https://azure.github.io/AKS-Construction/ --external --save
{"data":null,"message":"Authentication required. Add your authentication header and try again.","success":false}

To reproduce

name: Accessibility
on:
  workflow_dispatch:

jobs:
  accessibility:
    runs-on: ubuntu-latest
    steps:
      #- uses: actions/checkout@v3

      - uses: a11ywatch/[email protected]
        name: Accessibility checks
        with:
          WEBSITE_URL: https://azure.github.io/AKS-Construction/
          FAIL_ERROR_COUNT: 10
          EXTERNAL: false

Hi again, sorry for the back to back issues. This one's more of a feature request. I'm trying to run this action on a dev site as part of a CI/CD pipeline. The dev site is an AWS API Gateway (running flask) and has a URL like https://abcde12345.execute-api.us-east-1.amazonaws.com/dev/. Whenever I try to run it with SITE_WIDE enabled, it runs until I cancel it (I've let it go for an hour plus vs when it's not enabled and it runs in ~5 mins with installation). And it always seems to hang on the a11ywatch crawl step.

So my question/feature request is this: would it be possible to add a URL_LIST kind of parameter that lets you run a11ywatch scan on each specified page instead of using a11ywatch crawl? Or, even better, am I just using the action wrong and there's a good way to handle this already? I tried adding a sitemap.xml using flask-sitemapper (which works) but the action still runs for way longer than it should to scan just the few pages of my site (again, never seen it finish). Is there a better way to strictly limit the crawl to pages off the root URL?

web-accessibility-eval:
    needs: dev-deploy
    runs-on: ubuntu-latest

    steps:
      - name: Web Accessibility Eval
        uses: a11ywatch/[email protected]
        with:
          WEBSITE_URL: "https://abcde12345.execute-api.us-east-1.amazonaws.com/dev/"
          SITE_WIDE: false
          SUBDOMAINS: false
          TLD: false
          SITEMAP: true
          FAIL_ERRORS_COUNT: 15
          LIST: true
          FIX: true
          UPGRADE: false
          UPLOAD: true
        env:
          DEFAULT_RUNNERS: htmlcs,axe
          PAGEMIND_IGNORE_WARNINGS: true
          AI_DISABLED: false

Thanks in advance for any help :)

Automatic adjustments to code. Heres a list of some features that can be really useful.

1. add alts from the ai if probability score is hit above threshold at all locations.
   • detect how the image was created so it can add the property accordingly.
     -- if dynamic from CMS look into integrating plugin workflows
2. fix color adjustments.
3. auto add skip content button based on theme.

Having a lot of warnings takes up a lot of space and sometimes only errors are of importance for the project.

External users can configure website settings with A11yWatch.com by the end of the week with more advanced customizations like excluding warnings etc.

This feature would help the action aspect and something that would not take that much effort to integrate.

Currently the local usage of the site-wide option seems to take a very long time. Need to investigate if theres any issues or if the duration is due to some other issue.

On my local machine scanning https://a11ywatch.com across the full 22 public pages takes around 5 seconds on a 2gb docker setup mac m1. When performing single page scans on the action the results occur quickly so not sure if the magnitude difference between a 5 min action to a 30 min action should arise ( It should take a max of 6 mins or so based on the prev action durations to setup and run the scans ).

Run a11ywatch/github-actions@v2 with: WEBSITE_URL: http://localhost:9000 SUBDOMAINS: false TLD: false FAIL_ERRORS_COUNT: 0 LIST: true UPGRADE: true FIX: false EXTERNAL: false SITE_WIDE: false SITEMAP: false RECORD: false FAIL_TOTAL_COUNT: 0 FAIL_WARNINGS_COUNT: 0 DISABLE_PR_STATS: false SLIM: true

The above comes from the logs after I run my action. The actions log contain:

{"accessScore":90,"errorCount":4,"issueMeta":{"missingAltCount":null,"skipContentIncluded":true},"noticeCount":0,"totalIssues":13,"warningCount":9},"lastScanDate":"2023-09-29T10:55:11.582Z","online":true,"pageLoadTime":{"duration":245,"durationFormated":"Lightning Fast"},"url":"http://localhost:9000"},"message":"Scan completed in 1.25735276s","success":true}

Which shows that there are axe errors in the application but the job still passes and doesn't fail

Currently we can start up a local docker instance with the A11yWatch suite on the CI but, we cannot perform network request to the local machine at the port due to CI restrictions. The CI pipelines for A11yWatch at the moment require the EXTERNAL flag to be used. In order to allow local builds we need to create a docker image that can run docker exec inside the docker container to gather all the issues.

In the mean time set the A11YWATCH_API key from the website under a free account while we bring this feature into play. You can also start up the A11yWatch suite on a small dedicated 2-4gb instance to use for your own server instead.