Giter Club home page Giter Club logo

cypress-action's Introduction

Cypress e2e parallel test action

A poor man's Cypress parallel testing action.

Introduction

This project consists of two sub-actions:

  • assign-tests
  • update-weights

assign-tests, assign tests to be run on a specific runner. It uses weights to optimize the assignments.

update-weights, parses the test results and updates the weights.

Simple case

All that is needed to use assign-tests, is a file .cypress-weights.json in the root of your project. Any tests not defined in the weight file, is assigned a weight of 1.

Example

# .github/workflows/test.yml

name: test
on:
  workflow_dispatch:
jobs:
  tests:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        group: [1, 2, 3, 4, 5]
        groups: [5]

    steps:
      - uses: actions/checkout@v3

      - uses: actions/setup-node@v3
      - run: npm install

      - name: Calculate parallel test groups
        id: parallel
        uses: outsideopen/cypress-action/assign-tests@main
        with:
          group: ${{ matrix.group }}       # The current group        (default: 1)
          groups: ${{ matrix.groups }}     # Total number of groups   (default: 1)
          tests-path: '/path/to/tests'     # Path to cypress tests    (default: cypress/e2e)
          glob: '*.spec.js'                # Glob to match test files (default: **/*.cy.js)

Advanced case

Manually keeping the weights up to date can be a pain, when tests are constantly being added and modified. update-weights solves this problem, by parsing the test results, and adding an optimized .cypress-weights.json to the cache, that will be used on the next run.

Dependencies

In order for update-weights to parse the test results, you need mochawesome reporter installed, and configured to generate json output.

npm install --save-dev mochawesome
// You can configure it in cypress.config.js

module.exports = defineConfig({
  reporter: 'mochawesome',
  reporterOptions: {
    reportDir: 'cypress/results',
    overwrite: false,
    html: false,
    json: true,
  },
# or use command line options
cypress run --reporter mochawesome --reporter-options "reportDir=cypress/results,overwrite=false,html=false,json=true"

Example

# .github/workflows/test.yml

name: test
on:
  workflow_dispatch:
jobs:
  tests:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        group: [1, 2, 3, 4, 5]
        groups: [5]

    steps:
      - uses: actions/checkout@v3

      - uses: actions/setup-node@v3
      - run: npm install

      - name: Calculate parallel test groups
        id: parallel
        uses: outsideopen/cypress-action/assign-tests@main
        with:
          group: ${{ matrix.group }}       # The current group        (default: 1)
          groups: ${{ matrix.groups }}     # Total number of groups   (default: 1)
          tests-path: '/path/to/tests'     # Path to cypress tests    (default: cypress/e2e)
          glob: '*.spec.js'                # Glob to match test files (default: **/*.cy.js)

      - name: Run tests
        run: yarn cypress run --spec ${{ steps.parallel.outputs.spec }} --reporter mochawesome --reporter-options "reportDir=cypress/results,overwrite=false,html=false,json=true"

      - uses: actions/[email protected]
        with:
          name: weights
          path: |
            cypress/results/**

  update-weights:
    needs: tests
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/download-artifact@v3
        with:
          name: weights
          path: cypress/results

      - uses: outsideopen/cypress-action/update-weights@main
        with:
         tests-path: '/path/to/tests'          # Path to cypress tests. Same as above 
                                               # (default: cypress/e2e)
         results-path: '/path/to/test/results' # Path to mochawsome json results (as specified in reportsDir above)
                                               # (default: cypress/results)

Weights file explained

The weights file is a simple JSON file, with the test file path, as the key, and the weight as the value. The key should be the full path, as it is in the Github Action Workspace.

If you are not interested in dynamically updating test weights, it is pretty straight forward to manually create this file, and commit it to the root of your project.

Example:

  {
    "/home/runner/work/actions-test-repo/actions-test-repo/cypress/e2e/test_suite/test1.spec.js": 4,
    "/home/runner/work/actions-test-repo/actions-test-repo/cypress/e2e/test_suite/test2.spec.js": 2,
    "/home/runner/work/actions-test-repo/actions-test-repo/cypress/e2e/test_suite/test3.spec.js": 3
  }

cypress-action's People

Contributors

tjaartvdwalt avatar renovate[bot] avatar

Stargazers

Riccardo Giorato avatar

Watchers

Greg Lawler avatar James Cloos avatar David Lundgren avatar avatar

Forkers

tjaartvdwalt

cypress-action's Issues

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

nodenv
.node-version
  • node 16.20.1
npm
package.json
  • @actions/cache ^3.2.1
  • @actions/core ^1.10.0
  • @actions/glob ^0.4.0
  • @vercel/ncc ^0.36.0
  • folder-hash ^4.0.4
  • glob ^10.0.0
  • hash.js ^1.1.7
  • table ^6.8.1
  • jest ^29.0.0
  • Check this box to trigger a request for Renovate to run again on this repository

Handle the case wher the last group is empty

If the last group does not contain any tests, cypress will throw an error.

The problem is that we need to specify the number of executors before starting the run, and the optimal packing problem may not always need the last executor, if it can find an optimal solution with one fewer.

The best solution is probably to give the last executor at least one test to run, and send a warning message that this executor is not really needed. The number of executors can then be removed for subsequent runs.

Automatically create a weights based on previous run data

As a first attempt, create a node script that can create a weight file based on the output of a cypress run.

Once this is working, integrate this with Github Actions so that a new weight file is created with each run, that can be used during the next run.

Handle newly added tests

If you specify a weights file, any tests not defined in the file, will not get run.

All tests specified by the spec glob should get run. Tests not in the weights file should get a default weight, and a warning message shown that the run is not optimized

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.