Giter Club home page Giter Club logo

doodba-qa's Introduction

Doodba Quality Assurance

Build Status Docker Pulls Layers Commit License

BEWARE!, this project is in beta stage. Things are changing quickly.

What?

Tools for checking that your Doodba-based project is cool.

Why?

Because OCA's maintainer quality tools are too addons-repo-focused.

We needed a too Doodba-focused version instead. ๐Ÿ˜†

How?

  1. Mount your project structure, usually based on the provided scaffolding, on /project in the container. Docker CLI example: -v $(pwd):/project.
  2. Give him access to a Docker socket (beware of the security implications!) with --privileged -v /var/run/docker.sock or with -e DOCKER_HOST=tcp://dockerhost:2375.
  3. Configure through environment variables.
  4. Run any of the bundled scripts.

Example command for test environment, good for CI:

docker container run --rm -it --privileged -e COMPOSE_FILE=test.yaml -v "$PWD:/project:z" -v /var/run/docker.sock:/var/run/docker.sock:z tecnativa/doodba-qa pylint

Example command for devel environment, linting only private addons:

docker container run --rm -it --privileged -v "$PWD:$PWD:z" -v /var/run/docker.sock:/var/run/docker.sock:z -w "$PWD" -e ADDON_CATEGORIES=-p tecnativa/doodba-qa pylint

You most likely want to run this into a CI environment, so just check out the examples directory and you'll get a hint on how to do it.

Environment variables

You can use any of the upstream Docker Compose environment variables.

Besides, you have these too:

ADDON_CATEGORIES

Defaults to --private for all jobs.

You can change it per job, using any of --private --extra --core (or -pec).

These flags are used for the addons script available in all Doodba projects. Use this command in your project's folder to understand its usage:

docker-compose run --rm odoo addons --help

ADMIN_PASSWORD

Defaults to admin. If set, is used as the DB manager password.

ARTIFACTS_DIR

Directory where all the artifacts produced by insider scripts will be extracted.

ARTIFACTS_UID and ARTIFACTS_GID

UID/GID to be set as owner for artifacts produced by insider scripts.

BUILD_FLAGS

Flags to append to docker-compose build. Defaults to --pull --no-cache.

DESTROY_FLAGS

Flags to append to docker-compose down. Defaults to -v --rmi local --remove-orphans.

LINT_DISABLE

Disables specific linter messages. Its format depends on the underlying linter.

Defaults to manifest-required-author, since it's expected that you will only want to lint private addons, and those are not OCA's.

TODO: Make it work with flake8.

LINT_ENABLE

Enables specific linter messages. Its format depends on the underlying linter.

Empty by default.

TODO: Make it work with flake8.

LINT_MODE

Right now, only useful for pylint. Valid values:

  • loose (default) uses MQT standard cfg.
  • strict uses pull request cfg.
  • beta uses beta cfg.

PGPASSWORD

Used in secrets-setup when you need a specific DB password.

Defaults to odoopassword.

PYTHONOPTIMIZE

By default it is "" (disabled) to allow assert statements, which can be OK for tests, although not for production or demos.

More details in Python documentation.

REPOS_FILE

Path for the repos.yaml file in current scaffolding (not inside the container).

Scripts

You can use sh, docker and docker-compose commands with all of their features.

Besides, there are other scripts bundled, useful to do a CI pipeline:

addons-install

Install requested addons.

build

Build your project with docker-compose and check odoo works.

Uses BUILD_FLAGS.

closed-prs

Know if your repos.yaml definition includes merged or closed pull requests.

Uses the REPOS_FILE and GITHUB_TOKEN environment variables.

coverage

Run addons' unit tests and report coverage.

Usually you should run addons-install before.

You will find the HTML report files in ./$ARTIFACTS_DIR/coverage.

destroy

Destroy all containers, volumes, local images and networks.

Uses DESTROY_FLAGS.

flake8

Lint code with flake8 using MQT.

networks-autocreate

Create missing external networks, which are not autocreated by docker compose because it expects them to be present at the time of booting an environment.

Common examples of such networks are inverseproxy_shared or globalwhitelist_shared.

It extracts the required networks from the chosen docker-compose.yaml file.

pylint

Lint code with pylint-odoo using MQT.

Some environment variables modify this script's behavior; check them out.

secrets-setup

Creates all needed environment files for the official test.yaml environment to work.

Uses ADMIN_PASSWORD and PGPASSWORD.

shutdown

Like destroy, but keeping volumes and images.

Other utilities

These tools are not strictly related to Doodba, but they are helpful and are included:

doodba-qa's People

Contributors

joao-p-marques avatar pedrobaeza avatar yajo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

doodba-qa's Issues

Install addons error (gitlab-ci)

Hello,

I try to use (and contribute) to doodba-qa and I have this error (think it's due to last commit for addons ..)

Traceback (most recent call last):
  File "/usr/local/bin/addons", line 59, in <module>
    for addon, repo in addons_config(strict=args.explicit):
  File "/usr/local/lib/python3.5/dist-packages/odoobaselib/__init__.py", line 91, in addons_config
    addon)
Exception: ('Addon without manifest', '/opt/odoo/custom/src/edi/LICENSE')

Some OCA repo have LICENSE file and I use "*" in addons.yml ..

  • addons.yml
edi:
  - "*"
  • repos.yml
./edi:
  defaults:
      depth: $DEPTH_DEFAULT
  remotes:
      origin: https://github.com/OCA/edi.git
      nj: https://github.com/njeudy/edi.git
  target:
      nj 11.0-waiting-pr
  merges:
      - nj 11.0-waiting-pr

If I had only module I want, it is working, .. but some repo have lots of modules :)
Did you encounter the same experience @yajo ?

Redesign workflow of doodba-qa

First, goals:

  1. Scaffolding should be small as possible, because it requires most maintenance.
  2. QA dependencies should be included, to make pipelines faster.

Next, current design:

  1. doodba-qa includes all needed dependencies in the image.
  2. All of them are inside the volume /qa, which gets copied and mounted in the doodba project itself before each CI job.
  3. Those dependencies are only for the insider scripts.
  4. Outsider scripts exist to manage images and containers (build, destroy, shutdown...), or perform some operations that require network access (closed-prs).
  5. Insider scripts exist to execute things inside a container. Basically: install, test, coverage, lint.

Problems:

  1. Sharing a volume is not possible or easy on all environments.
  2. Different versions of doodba have different python versions, so it could have different C extensions. Bind-mounting the preinstalled dependencies can lead to errors.

Good ideas from Tecnativa/doodba-scaffolding#13:

  1. Install things using the base image, without nasty hacks.
  2. Provide a .gitlab-ci.yml file that works out of the box.

So, what could we do to improve the situation?

  • Include QA dependencies in doodba itself: Tecnativa/doodba#192

    This would be great because CI builds would be faster (all is included in the base image) and portable (no need to mount shared volumes).

    To avoid polluting the odoo production environment, with pip version conflicts or unexpected behaviors, QA dependencies should exist in a separate virtualenv, inside the same container.

  • Include a tool to easily execute insider scripts in Doodba: Tecnativa/doodba#192

  • Dismantle all the copy & share mechanisms in Doodba QA. Not needed anymore due to previous point: #9

  • The Doodba QA image should only contain dependencies needed for Outsider scripts: #9

  • Doodba Scaffolding should include a .gitlab-ci.yml that benefits from the recently released feature (in Gitlab 11.4) of including an external file, and should get it from the example we have here .travis.yml file that uses Doodba QA to test that empty repository in all possible Odoo versions: Tecnativa/doodba-scaffolding#15

    Initially the idea was to use .gitlab-ci.yml from the example, but it turns out that it works better with a custom runner that targets a persistent docker daemon, so it seems legit to use Travis instead, and leave the Gitlab CI integration for integrators.

  • Enable Gitlab Travis CI in Doodba Scaffolding, now that we are able to use Gitlab CI with GitHub: Tecnativa/doodba-scaffolding#15

  • Enable Gitlab CI in Doodba QA. See previous point. #9

Benefits from this new approach

  1. Doodba Scaffolding and QA would get CI.
  2. CI would work out of the box.
  3. Environment wouldn't matter.
  4. CI would be faster.

Help wanted!

cc @PCatinean

Not working with Odoo v14

Hello,
It seems that doodba-qa not working with odoo v14.
After investigation, I see that OCA maintainer Quality Tool (MQT) are no more installed in doodba 14.0.
Is the any reason why v14 is not supported?
I wonder if doodba-qa is still maintened or used with doodba project?
Thanks for your great job on doodba.
Bertrand

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.