shopozor / ci Goto Github PK

Upon PR merges, it is pretty handy to have pre-configured jobs for the dev and master branches to allow the merger to double-check that everything is fine after the merge.

We can detect changes in the manifest by means of git's command line. If the manifest changed, then we need to remove the concerned environment and re-install it. This assumes that environment deletion works (cf. [#1]).

There is some advice here.

I used the import functionality to importing a zip file. That zip file contains test cases, reportportal.properties file, drivers, pom.xml file and also a testNg.xml file but i didn't get any update can anyone suggest me or explain me how to use this import functionality in reportportal.io

The current fix is to only keep the last 5 builds for every job (in the jenkins views as well as on disk). That might not be sufficient. That needs to be in observation.

• schedule a cron job that makes a docker system prune -a once a week or so

We would like to have an automatic feedback of the jenkins builds on our bitbucket repos. We currently have no idea how to make it happen, though there are some insights provided here.

We need to protect the environment with https protocol. This needs to happen in our manifest.

• enable https

• secure environment

  • enable security headers
    • enable HSTS
    • X-FrameOptions
    • X-XSS-Protection
    • Referrer-Policy
    • X-Content-Type-Options
    • Content-Security-Policy
    • Feature-Policy
  • redirect from http to https
  • setup a secure ssl cipher suite
  • only enable tls1.2 and tls1.3
  • setup the dhparams
  • make the jenkins session cookie secure
  • make the jenkins session cookie httponly
  • make the jenkins session cookie SameSite=strict

• enable IPv4

• bind ssl

• close from outside world; browsing the jenkins environment with port 8080 should not work; browsing that address should only be allowed from the nginx node

• clarify letsencrypt. Our current "integration" seems to use some fake/dummy certificate from let's encrypt (LE). Ideally we need our automatic process to get a valid certificate and setup the configuration as part of our jps flow, if that's not possible we need to re-evaluate how we want to proceed

• organize automatic ssl certificate renewal with certbot

Before we accept PRs on the shopozor common frontend, we need to run the tests of the consumer and management frontends. We need to define a workflow for that.

For the consumer-frontend, management-frontend, and backend, we need a jenkins job that

1. runs the acceptance / e2e tests
2. generates videos out of the tests when applicable
3. converts the videos to gifs
4. generates the dynamic html documentation
5. pushes the documentation to the softozor domain

• www.softozor.ch/shopozor/backend/Index.html
• www.softozor.ch/shopozor/consumer-frontend/Index.html
• www.softozor.ch/shopozor/management-frontend/Index.html

Try to install reportportal on our CI/CD environment:

• https://reportportal.io/features
• https://pypi.org/project/pytest-reportportal/

The issue is being addressed here.

At the end of the e2e tests, we perform Jelastic environment deletion. The API call to do that does not seem to be working. Ticket 9050 is open on our Jelastic provider.

I thought I would need to put shopozor as the organization but that needs to be set to softozor.

Currently, the ci environment is installed with hasura client v1.0.0-beta.9. We need to e.g. execute a cron job that updates that client automatically every week or so.

We need a parameterized jenkins pipeline that

1. allows to select a tag of the consumer-frontend repo
2. builds a docker image with the consumer-frontend software installed (only the built software is necessary, i.e. the contents of the dist folder obtained after quasar build)
3. publishes the docker image to our docker hub account

The image is then deployed as part of our production manifest (cf. https://github.com/shopozor/consumer-frontend/issues/10).

We need a parameterized jenkins pipeline that

1. allows to select a tag of the backend repo
2. builds a docker image with the backend software installed
3. publishes the docker image to our docker hub account

The image is then deployed as part of our production manifest (cf. https://github.com/shopozor/backend/issues/49).

I haven't found yet a way to deactivate the e-mails that are sent to me upon jelastic environment creation. Ticket 8855 is opened.

Commenting a PR with test consumer e2e or test management e2e does not trigger anything even though Jenkins has pipelines configured with those trigger phrases:

I would be handy to just click one button in jenkins and have all the e2e environments cleaned up.

Upon docker login -u *** -p *** in our Jenkins pipeline, we get the following error

Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

This is curious because it worked fine until yesterday. Even a ping to the docker registry url fails. However, it works fine on my development laptop.

The work-around will be that we will self-host our docker registry to avoid that kind of problems.

When we deliver the production system, it needs to be deployed to "La Budzonnerie"-specific environment names. The same holds for the testing infrastructure.

We need a parameterized jenkins pipeline that

1. allows to select a tag of the management-frontend repo
2. builds a docker image with the management-frontend software installed (only the built software is necessary, i.e. the contents of the dist folder obtained after quasar build)
3. publishes the docker image to our docker hub account

The image is then deployed as part of our production manifest (cf. https://github.com/shopozor/management-frontend/issues/21).

All the test environments need to be only accessible from the Jenkins environment.

In the situation where Hidora would be in big trouble for some reason, we need to be able to deploy the whole software on Infomaniak's Jelastic infrastructure.

• need account at Infomaniak
• need database backup ready to be deployed
• organize deployment of the whole system
• organize scripts using dns api to commute the budzonnerie.ch dns (e.g. with google dns api)

Because we haven't found yet how to enable the credentials of a github user with 2FA on Jenkins, the softozor-deployer user cannot have that feature activated.

Various tests on our CI/CD server showed that the environment name shopozor-management-frontend-e2e prevents it from running the e2e tests, i.e. it is not possible to install the e2e manifest even though the installation manifest was successfully installed.

One way to prevent that problem from happening would be to set a maximal length for the frontend environment names in the shopozor ci manifest's settings section. That functionality is, however, not supported yet (ticket 12056).

We need to configure the dns used by docker upon image building. At the moment, a build of the backend results in this error:

Advice is provided here on how to fix the issue. We need to know, however, what dns to use.

We would like to have an automatic feedback of the jenkins builds on our github repo. To achieve that, we can follow this advice.

Currently, the acceptance test database is called test_saleor. If two instances of the acceptance tests pipeline are running concurrently, then the tests will fail because the test database will be attempted to be deleted. We need to fix that. The current work-around is to wait until the acceptance tests are finished before updating a PR or running a new instance of the test pipeline. The same holds true for the unit tests pipeline.

Jenkins output:

/var/jenkins_home/workspace/shopozor-management-e2e@tmp/durable-2b8c53d7/script.sh: 1: /var/jenkins_home/workspace/shopozor-management-e2e@tmp/durable-2b8c53d7/script.sh: ./common/e2e/deploy-to-jelastic.sh: not found