Giter Club home page Giter Club logo

wolox-ci's Introduction

Wolox CI

This a Jenkins library built to make it easier for us at Wolox to configure pipelines without necessarily knowing about Jenkinsfile syntax. All our projects are built using a Dockerfile

When using this library, your Jenkinsfile should look something like this:

@Library('wolox-ci') _

node {

  checkout scm

  woloxCi('.woloxci/config.yml');
}

It basically loads the library, clones the target repository and calls woloxCi to make its magic. As an argument, woloxCi receives the path to a configuration yaml file. This file looks something like this:

config:
  dockerfile: .woloxci/Dockerfile
  project_name: some-rails-project

services:
  - mssql

steps:
  analysis:
    - bundle exec rubocop -R app spec --format simple
    - bundle exec rubycritic --path ./analysis --minimum-score 80 --no-browser
  setup_db:
    - bundle exec rails db:create
    - bundle exec rails db:schema:load
  test:
    - bundle exec rspec
  security:
    - bundle exec brakeman --exit-on-error
  audit:
    - bundle audit check --update


environment:
  RAILS_ENV: test
  GIT_COMMITTER_NAME: a
  GIT_COMMITTER_EMAIL: b
  LANG: C.UTF-8

jenkinsEnvironment:
  - JENKINS_URL

timeout: 600

This file has different sections:

Configuration

The section under the config label defines some basic configuration for this project:

  1. The dockerfile that contains the image for the project to run.
  2. The project name.

Services

This section lists the project's dependencies. Each section will define and expose some environment variables that might be needed by the application:

Microsoft SQL

When listing mssql as a service, this will build a docker image from microsoft/mssql-server-linux exposing the following environment variables:

  1. DB_USERNAME
  2. DB_PASSWORD
  3. DB_HOST
  4. DB_PORT

PostgreSQL

When listing postgres as a service, this will build a docker image from postgres exposing the following environment variables:

  1. DB_USERNAME
  2. DB_PASSWORD
  3. DB_HOST
  4. DB_PORT

Redis

When listing redis as a service, this will build a docker image from redis exposing the following environment variables:

  1. REDIS_URL: the redis url that looks like this redis://redis

MySQL

When listing mysql as a service, this will build a docker image from mysql exposing the following environment variables:

  1. DB_USERNAME
  2. DB_PASSWORD
  3. DB_HOST
  4. DB_PORT

MongoDB

When listing mongo as a service, this will build a docker image from mongo exposing the following environment variables:

  1. DB_USERNAME
  2. DB_PASSWORD
  3. DB_HOST
  4. DB_PORT

Elasticsearch

When listing elasticsearch:6.4.0 as a service (it's mandatory specify the image version), this will build a docker image from elasticsearch exposing the following environment variables:

  1. ELASTICSEARCH_URL

Steps

This section lets you define the steps you need to build your project. Each level inside the steps section is a stage in the Jenkins pipeline and each item in the stage is a command to run. In the case above, there are 5 stages named:

  1. analysis
  2. setup_db
  3. test
  4. security
  5. audit

The analysis stage, for example, runs the following commands:

  1. bundle exec rubocop -R app spec --format simple
  2. bundle exec rubycritic --path ./analysis --minimum-score 80 --no-browser

Environment

This section lets you set up custom environment variables. Each item inside this section defines a variable with its value.

Jenkins Environment

This section lets you define what Jenkins environment values you want to pass to the docker image, i.e. JENKINS_URL. Bear in mind that, for convenience, Wolox CI already passes the following Jenkins Environment Variables to the Docker image by default:

  1. BUILD_ID
  2. BRANCH_NAME
  3. CHANGE_ID
  4. CHANGE_BRANCH (if job is PR-triggered)
  5. CHANGE_TARGET (if job is PR-triggered)

Timeout

This section lets you set up a custom timeout in seconds. The default is 600 (10 minutes).

wolox-ci's People

Contributors

martum avatar sbalay avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

wolox-ci's Issues

Cannot load library

Hello,

I tried to use wolox-ci but I have a problem to load it. I'm loading library as global library from git and loading it in implicit or explicit way.
I'm using:

Thanks in advance for help.

The error:

> git fetch --no-tags --progress https://github.com/pchomik/wolox-test.git +refs/heads/*:refs/remotes/origin/*
Checking out Revision ae0f3ef15755619a26fc86be681373303741ae19 (master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f ae0f3ef15755619a26fc86be681373303741ae19
Commit message: "Some try"
 > git rev-list --no-walk 594e6577864be3f2eee32425ab2a10fe51c5284a # timeout=10
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
hudson.remoting.ProxyException: org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
/var/jenkins_home/jobs/test/branches/master/builds/4/libs/wolox-ci/vars/woloxCi.groovy: 1: unable to resolve class Library ,  unable to find class for annotation
 @ line 1, column 1.
   @Library('wolox-ci')
   ^

1 error

	at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
	at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:958)
	at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:605)
	at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:554)
	at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:254)
	at groovy.lang.GroovyClassLoader.recompile(GroovyClassLoader.java:761)
	at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:718)
	at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:787)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:411)
	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell$TimingLoader.loadClass(CpsGroovyShell.java:158)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:411)
	at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:677)
	at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:787)
	at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:775)
	at org.jenkinsci.plugins.workflow.cps.global.UserDefinedGlobalVariable.getValue(UserDefinedGlobalVariable.java:54)
	at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:113)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:157)
	at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:133)
	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:155)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:159)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:129)
	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
	at WorkflowScript.run(WorkflowScript:3)
	at ___cps.transform___(Native Method)
	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
	at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
	at com.cloudbees.groovy.cps.Next.step(Next.java:83)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
	at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:122)
	at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:261)
	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$101(SandboxContinuable.java:34)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.lambda$run0$0(SandboxContinuable.java:59)
	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:58)
	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:174)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:332)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:83)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:244)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:232)
	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE

Need more samples.

Thanks to implement this solution to use Yaml more than Jenkinsfile to manage the pipeline.

But the exist sample is not clear for what I should set. Can you help?

Here are some questions:

  1. How to run builds from exist docker image and AWS ECR image

Something like:

config:
  image: python:2.7.13
  project_name: some-rails-project

or

config:
  image: ${AWS_ACCOUNT_ID}.dkr.ecr.ap-southeast-2.amazonaws.com/python:2.7.13
        aws_auth:
          aws_access_key_id: ${AWS_ACCESS_KEY_ID}
          aws_secret_access_key: ${AWS_SECRET_ACCESS_KEY}
  project_name: some-rails-project
  1. Any samples to build python, node, java, scala, or others

  2. How to install this share wolox-ci library?

Reference this url (https://jenkins.io/doc/book/pipeline/shared-libraries/), but not sure how to set with wolox-ci

Loading library wolox-ci@master
java.lang.NullPointerException
	at org.jenkinsci.plugins.workflow.libs.LibraryAdder.retrieve(LibraryAdder.java:157)
	at org.jenkinsci.plugins.workflow.libs.LibraryAdder.add(LibraryAdder.java:138)
	at org.jenkinsci.plugins.workflow.libs.LibraryDecorator$1.call(LibraryDecorator.java:125)
	at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1065)
	at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
	at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
	at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
	at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
	at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
	at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:131)
	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:125)
	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:560)
	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:521)
	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:320)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: Loading libraries failed

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.