Giter Club home page Giter Club logo

cloud-e2e-bdd-test-runner-js's Introduction

End-to-end Behaviour Driven Design Test Runner npm version

GitHub Actions semantic-release Renovate Mergify Status Commitizen friendly code style: prettier ESLint: TypeScript

Note
Check out the next evolution of this project: @nordicsemiconductor/bdd-markdown

Implementation of a test-runner for end-to-end tests of cloud-native applications using Gherkin features.

Initially developed for use with AWS based solutions but also supports testing against generic REST, GraphQL, and Websocket APIs.

Simple example usage: e2e-bdd-test-runner-example.

Other projects using this project for testing:

Motivation

Building cloud-native solutions means leveraging the power of highly scalable, often proprietary components, which can no longer be run locally on the developer's machine. A critical part of the solution is also the configuration which ties these components together.

End-to-end testing provides a way to ensure that the entire solution is working properly and that changes not only in code and dependencies but also on the provider side are catched immediately.

This project is an implementation of a test-runner in TypeScript that allows every change to a cloud-native project to be tested tested against production infrastructure using a BDD style test-suite of end-to-end tests written in Gherkin, which describe tests in plain english.

This way the tests itself are not tied to the implementation and during refactoring one cannot accidentally drop tests: tests written for test runners like Jest tend to be tied closely to the API of the source-code implementation, in a case of bigger refactoring the tests themselves usually need to be refactored as well. Since the end-to-end tests are purely testing based on the public API of the project, they can be kept unchanged during refactoring.

This also provides an easily grokable description of the working and implemented projects features in one folder (example).

Below is a talk in English explaining this approach in detail:

Video · Slides

Video

Installation

npm i --save-dev --save-exact @nordicsemiconductor/e2e-bdd-test-runner

Special annotations

On Features

  • @Skip: Do not run this feature
  • @Only: Run only this feature
  • @Last: Run this feature after all others

On Scenarios

  • @Retry: configures the retry behaviour. Pass one or multiple settings to override the default behaviour. Example: @Retry=failAfter:3,maxDelay:100,initialDelay:50.

Architecture decision records (ADRs)

see ./adr.

cloud-e2e-bdd-test-runner-js's People

Contributors

coderbyheart avatar dependabot[bot] avatar greenkeeper[bot] avatar renovate-bot avatar renovate[bot] avatar snyk-bot avatar wkronmiller avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

Forkers

pfried

cloud-e2e-bdd-test-runner-js's Issues

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: Cannot find preset's package (github>whitesource/merge-confidence:beta)

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

These problems occurred while renovating this repository. View logs.

  • WARN: Fallback to renovate.json file as a preset is deprecated, please use a default.json file instead.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

github-actions
.github/workflows/sync-issue-labels.yaml
  • actions/checkout v4
  • ubuntu 22.04
.github/workflows/test-and-release.yaml
  • actions/checkout v4
  • actions/setup-node v4
  • actions/cache v3
  • ubuntu 22.04
.github/workflows/update-repo-info.yaml
  • actions/checkout v4
  • ubuntu 22.04
npm
package.json
  • @aws-sdk/client-cognito-identity 3.428.0
  • @aws-sdk/client-cognito-identity-provider 3.428.0
  • @aws-sdk/client-sqs 3.428.0
  • @aws-sdk/credential-provider-cognito-identity 3.428.0
  • @aws-sdk/protocol-http 3.374.0
  • @aws-sdk/signature-v4 3.374.0
  • @types/backoff 2.5.5
  • @types/chai 4.3.11
  • @types/glob 8.1.0
  • @types/node-fetch 3.0.2
  • aws-sdk 2.1506.0
  • backoff 2.5.0
  • chai 4.3.10
  • chai-subset 1.6.0
  • chalk 4.1.2
  • fetch-ponyfill 7.1.0
  • gherkin 9.0.0
  • glob 8.1.0
  • graphql 16.8.1
  • jsonata 1.8.6
  • node-fetch 3.3.2
  • toposort 2.0.2
  • @nordicsemiconductor/asset-tracker-cloud-code-style 13.0.0
  • @swc/jest 0.2.29
  • @types/chai-subset 1.3.5
  • @types/jest 29.5.10
  • @types/node 20.10.0
  • @types/toposort 2.0.7
  • jest 29.7.0
  • node >=20
  • npm >=9

  • Check this box to trigger a request for Renovate to run again on this repository

feature file resolution does not work on windows machines

There is an issue where the feature runner cannot resolve the *.feature files on a windows machine

const scan = path.join(path.resolve(dir), '*.feature');

path.resolve will return a platform dependent path on windows something like C:\Users\maeckle but glob only works with forward slashes

I would have loved just changing the API to pass the whole glob pattern (that would also allow subdirectories **/*.feature) or different filenames, but that would be a breaking change. I will think about how to solve it, maybe something like this.

Things to change in the next rewrite

  • re-implement Gherkin parser (either using official version and support our own features, or use proper tokenizer)
  • add auto-format feature, so feature files, and especially the data sections of steps can be properly formatted (e.g. reformat tables, auto-correct and format JSON)
  • pick a different notation for variables, we often use JSON, so { is not a good choice
  • allow steps to log structured data to the current context and make that available
  • add timestamps to progress logger
  • get rid of world concept, use store for everything
  • native storage implementation (needed basically by every test)
  • allow to run tests in parallel (will mess up log output, but makes full run faster)
  • native retry/backoff feature per step (or only on then)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.