Giter Club home page Giter Club logo

redash-ui-tests's Introduction

redash-ui-tests

📊 UI tests for Mozilla Redash

Description

Redash is a open source web-application for working with data. Users can query data from different sources, create visualizations and dashboards, and share them in their organization. Please check out the knowledge base for more info about the Redash project.

The redash-ui-tests project is a community effort to develop automated UI tests for Redash. We want to be able to automatically detect UI regressions in release candidates and reduce the time spent on manual testing. We will gradually add more UI tests for functionality, that is important to our users. Check out the tests issue label to get an idea about test cases that we are planning to automate.

Please check out the documentation for more information! 📝

Contributing

Are you interested in developing UI tests in Python, or helping us improve our documentation, or have ideas for how to improve redash-ui-tests?

Read our contributing guide and check out the good first issue label for tasks, that are good candidates for your first contribution to redash-ui-tests. Your contributions are greatly appreciated! Every little bit helps, and credit will always be given!

Please note that redash-ui-tests is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.

Check out who has already contributed to redash-ui-tests here!

redash-ui-tests's People

Contributors

aayush420 avatar chrisbirster avatar hackebrot avatar jezdez avatar jrbenny35 avatar madalincmc avatar pyup-bot avatar stephendonner avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

jrbenny35 aayush420 stephendonner mozilla-github-standards adrian-tamas

redash-ui-tests's Issues

Integrating redash-ui-tests with Redash projects

Updates to tests

I think it makes sense to set up CI for redash-ui-tests (#2) and run the tests against a stable version of Redash and see if all tests - old and new - still pass. This way we know if tests are safe to be merged here.

Running tests against Redash

We also want to think about how to integrate redash-ui-tests into the developer workflow for Redash. I think it's up to the maintainers of the projects how to implement that, but we should make it as easy as possible for them to run the UI tests and write documentation here with an example implementation.

Suggestions

Both upstream Redash and Mozilla Redash publish the Redash server image to Docker Hub and use docker-compose.

I suggest we push a Docker image of redash-ui-tests on successful CI builds on master. This image would need to support passing in a URI to Redash and an entrypoint to run the tests. The Redash projects can pull that redash-ui-tests image and run it for pull-requests and release-candidate branches.

Add test: Searching for queries with special characters in the name

Test Case

Prerequisites

  • User is logged in
  • Query with special characters in the name

UI Test

Action Result
Navigate to landing page Page displays "Search queries..." input and button
Input the name of a query that contains special characters and click button Browser opens queries page, "Search" tab is selected, page contains table, table contains only relevant items, the query that contains special characters in the name is displayed

Increase max-line-length to 88

Our source code will be more verbose with type hints (#51) and less readable with a max line length of 79 characters. I suggest to increase this to 88 characters, which is the default in black.

Set up Continuous Integration

When submitting a pull request against this repo, we would like to get automated feedback on whether the suite is still working on all of the existing tests and new tests pass.

  • Decide on a CI product
  • Set up CI for this project

Set up and teardown for individual tests

We need a way to run tests in isolation that they don't leak into other test items. The order in which tests are being executed must not impact the test results.

  • pytest setup to create clean database tables etc.
  • pytest teardown to drop database tables etc.

Add Test: Creating an Admin

Prerequisites

  • The redash instance has not been setup

UI Test

Action Result
Open redash instance URI Page displays Redash setup page, URL has /setup within it.
Input user information User logs in and the Redash homepage is shown with the user logged in.

This will also solve setting up the redash instance we use for testing.

Edit Source redirects to queries/<id>/source page

Prerequisites

  • User is logged in
  • At least one query

UI Test

Action Result
Navigate to a query /id page “Edit Source” button is displayed
Click on the “Edit Source” button The user is redirected to the query source page, the url contains “queries/query_id/source”, “Show Data Only” button is displayed where the “Edit Source” button was

Options button - Fork creates a copy of the query and redirects the user to the new query

Prerequisites

  • User is logged in
  • At least one query

UI Test

Action Result
Navigate to a query /id page Options button is displayed
Click on the Options button A drop down menu is displayed, Fork link displayed in dropdown
Click on the Fork link User is redirected to the new query page, New query page contains the “Copy of (#xx)” (where xx is id of the previous query) text and the title of the previous query

Add test command tool

I think a test env manager, like Tox would be a good addition. Instead of long pipenv command, we could setup and run a test suite using a command like tox -e ui-tests. Pipenv would still work within tox and it would allow for simpler test commands.

Add description to a query

Prerequisites

  • User is logged in
  • At least one query that has no description set

UI Test

Action Result
Navigate to a query /id page “Add description” text is displayed
Click on the “Add description” text The filed becomes editable
Type any string and press the enter button The new description is displayed

UI testing workflows on Redash CI

The goal of this issue is to provide an overview over the current UI testing workflow and highlight its limitations. The questions at the end of this description are supposed to help us discuss the requirements for redash-ui-tests.

Repos

Releases getredash/redash

https://hub.docker.com/r/redash/redash/

Tag Name Last Updated
5.0.0-beta.b4476 a month ago
latest 4 months ago
4.0.1.b4038 4 months ago
4.0.0.b3948 5 months ago
4.0.0-rc.1 6 months ago
4.0.0-beta.b3690 7 months ago
  • build-docker-image on v tags

Releases mozilla/redash

https://hub.docker.com/r/mozilla/redash/

Tag Name Last Updated
master a day ago
latest 3 days ago
m17 3 days ago
rc 7 days ago
m16.1 23 days ago
m16 24 days ago
  • deploy-master on master branch
  • deploy-rc on release branch
  • deploy-milestone on m tags

Tests

  • CI on getredash/redash against master branch and mozilla/redash-ui-tests:latest
  • CI on mozilla/redash against master branch and mozilla/redash-ui-tests:latest
  • CI on mozilla/redash-ui-tests against master branch and redash/redash:latest
  • CI on mozilla/redash-stmo against master branch and redash/redash:latest
  • Manual regression tests on mozilla/redash against release candidate

UI tests

Goal of UI tests

Automatically alert developers to UI regressions in pull requests and verify
release candidates.

Requirement for UI tests

Tests cases need to be classified either:

  • regression tests (based the latest stable release)
  • tests for specific versions (maintenance releases or upcoming releases)
  • tests specific to Redash version (with redash-stmo installed or not)

Questions

  • Which tests need to run for commits to master? Only tests for that version?
  • Which tests need to run for release branches? Regression tests and tests for that version?
  • Which tests need to pass for release milestones? Regression tests and tests for that version?
  • Which tests need to pass for bugfix milestones? Only tests for tests of that series?
  • How to we sync test classifiers with releases?

Add test: Searching queries by description

Test Case

Prerequisites

  • User is logged in
  • The query with the description mentioned in the test is created

UI Test

Action Result
Navigate to landing page Page displays "Search queries..." input and button
Input description and click button Browser opens queries page, "Search" tab is selected, page contains table, table contains only relevant items, the query with the description used in test is displayed

Add test: Clicking logo opens landing page

Prerequisites

  • User is logged in
  • At least one dashboard

UI Test

Action Result
Navigate to landing page Logo is displayed
Click on a dashboard link Browser opens dashboard page and logo is visible
Click logo Browser opens landing page and logo is visible

Manage Permission opens the manage permissions pop-up

Prerequisites

  • User is logged in
  • At least one query

UI Test

Action Result
Navigate to a query /id page Options button is displayed
Click on the Options button A drop down menu is displayed, “Manage Permissions” link displayed in dropdown
Click on the “Manage Permissions” link A pop-up is displayed containg an input field to add users and a list of user that already have permissions

Write documentation for newcomers

As a newcomer to this project, I would like to learn what this project is about, how I can set it up locally, how to run the tests and how I can contribute to the project.

  • Project description on README
  • Requirements & set up section
  • Running the tests section
  • Contributing guide on README

Add test: Searching for unpublished queries

Test Case

Prerequisites

  • User is logged in
  • Unpublished query

UI Test

Action Result
Navigate to landing page Page displays "Search queries..." input and button
Input the name of the unpublished query and click button Browser opens queries page, "Search" tab is selected, page contains table, table contains only relevant items, the unpublished query is displayed

Edit description for a query

Prerequisites

  • User is logged in
  • At least one query with the description set

UI Test

Action Result
Navigate to a query /id page The description text is displayed
Click on the description text The filed becomes editable, the cursor is placed at the end of the description
Type any string and press the enter button The new description is displayed

Add test: Searching queries by complete name

Test Case

Prerequisites

  • User is logged in
  • The query with the name mentioned in the test is created

UI Test

Action Result
Navigate to landing page Page displays "Search queries..." input and button
Input the complete query name and click button Browser opens queries page, "Search" tab is selected, page contains table, table contains only relevant items, the query with the complete name used in test is displayed as the first result.

Publish button publishes an unpublished query

Prerequisites

  • User is logged in
  • At least one unpublished query

UI Test

Action Result
Navigate to a query /id page for an unpublished query the “Publish” button is displayed, the “Unpublished” tag is displayed
Click on the “Publish” button The query is published, the “Publish” button is no longer displayed, the “Unpublished” tag is no longer displayed

Options button - Unpublish unbpulishes a published query

Prerequisites

  • User is logged in
  • At least one published query

UI Test

Action Result
Navigate to a query /id page for a published query the Options button is displayed
Click on the Options button A drop down menu is displayed, “Unpublish” link displayed in dropdown
Click on the “Unpublish” link The query is unpublished, the “Publish” button is displayed, the “Unpublished” tag is displayed

Run tests against a specific server version

As a QA, I would like to be able to run the tests against a specific version of Redash or a specific git branch or commit and get the results without having to set up the project locally.

  • Decide on technical solution
  • Implement the solution

Add test: Searching queries by ID

Test Case

Prerequisites

  • User is logged in
  • The query with the ID mentioned in the test is created

UI Test

Action Result
Navigate to landing page Page displays "Search queries..." input and button
Input ID and click button Browser opens queries page, "Search" tab is selected, page contains table, table contains the query with the used ID

Add test: Searching queries

Prerequisites

  • User is logged in
  • Two queries with different names

UI Test

Action Result
Navigate to landing page Page displays "Search queries..." input and button
Input search string and click button Browser opens queries page, "Search" tab is selected, page contains table, table contains only relevant items

Add Code of Conduct

I would like to adopt a code of conduct in this project to create a more welcoming and inclusive environment for everyone interacting with the community.

  • add code of conduct document
  • link to the code of conduct document on the README file

Options button - Archive archives the query

UI Test

Action Result
Navigate to a query /id page Options button is displayed
Click on the Options button A drop down menu is displayed, Archive link displayed in dropdown
Click on the Archive link A pop-up is displayed with 2 buttons “Cancel” and “Archive”
Click on the Archive button The query is archived, the “Archived” tag is displayed

Add flake8

I think it would be good to have some sort of python linter for code quality as well as organization and docstrings.

Add test: Searching for queries with numbers in the name

Test Case

Prerequisites

  • User is logged in
  • Query with numbers in the name

UI Test

Action Result
Navigate to landing page Page displays "Search queries..." input and button
Input the name of a query that contains numbers and click button Browser opens queries page, "Search" tab is selected, page contains table, table contains only relevant items, the query that contains numbers in the name is displayed

Testing against different versions of Redash

So as seen in PR #73, the redash-ui-test suite runs against the latest tag. This time there were some breaking changes introduced in the 5.0 beta that requires some changes to how our tests run. This has made me wonder how we should handle our testing.

We want to keep the upstream tests running but to do that, we need to make sure our tests can run against the current latest as well as the current beta build.

Add test: Searching queries using different capitalization

Test Case

Prerequisites

  • User is logged in
  • Have a query named "Capitalization Test"

UI Test

Action Result
Navigate to landing page Page displays "Search queries..." input and button
Input search string: "Capitalization Test" and click button Browser opens queries page, "Search" tab is selected, page contains table, table contains only relevant items, the "Capitalization Test" query is displayed
Input search string: "CaPitaLizatiON tESt" and click button Browser opens queries page, "Search" tab is selected, page contains table, table contains only relevant items, the "Capitalization Test"

Edit own query title

Prerequisites

  • User is logged in
  • At least one query

UI Test

Action Result
Navigate to a query /id page Title is displayed
Click on the query title The title becomes editable, the cursor is placed at the end of the query title
Type any string and press the enter button The new title is displayed

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.