Giter Club home page Giter Club logo

aws-codebuild-jenkins-plugin's Introduction

AWS CodeBuild Jenkins Plugin

The AWS CodeBuild plugin for Jenkins provides a build step for your Jenkins project.

Build Status license Build Status

Plugin Installation

If you already have a Jenkins setup and would like to only install the AWS CodeBuild plugin, then the recommended approach would be to search for "AWS CodeBuild" in the Plugin Manager on your Jenkins instance.

We have also written a blog post for setting up a new Jenkins server with AWS CloudFormation and integrating it with AWS CodeBuild and AWS CodeDeploy. Learn more: https://aws.amazon.com/blogs/devops/setting-up-a-ci-cd-pipeline-by-integrating-jenkins-with-aws-codebuild-and-aws-codedeploy

Plugin Usage

Using AWS CodeBuild with source available outside of your VPC

  1. Create Project on the AWS CodeBuild console.

    • Switch to the region you would prefer to run the build in.
    • You can optionally set the Amazon VPC configuration to allow CodeBuild build container to access resources within your VPC.
    • Make sure to write down the project's name.
    • (Optional) If your source repository is not natively supported by CodeBuild, you can set the input source type for your project as S3 for the CodeBuild project.
  2. Create AWS IAM user to be used by the Jenkins plugin.

    • Create a policy similar to the one following this section.
    • Go to the IAM console, and create a new user.
      • Access type should be: Programmatic Access.
      • Attach policy to user that you created previously.
  3. Create a freestyle project in Jenkins.

    • On the Configure page, choose Add build step > Run build on AWS AWS CodeBuild.
    • Configure your build step.
      • Enter Region, Credentials from the user created previously, and ProjectName.
      • Select Use Project source.
      • Save the configuration and run a build from Jenkins.
  4. For the Source Code Management make sure to select how you would like to retrieve your source. You may need to install the GitHub Plugin (or the relevant source repository provider's Jenkins plugin) to your Jenkins server.

    • On the Configure page, choose Add build step > Run build on AWS CodeBuild. Configure the build step.
    • Enter Region, Credentials from the user created previously, and Project name.
    • Select Use Jenkins source.
    • Save the configuration and run a build from Jenkins.

Policy sample for IAM user:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Resource": ["arn:aws:logs:{{region}}:{{awsAccountId}}:log-group:/aws/codebuild/{{projectName}}:*"],
            "Action": ["logs:GetLogEvents"]
        },
        {
            "Effect": "Allow",
            "Resource": ["arn:aws:s3:::{{inputBucket}}"],
            "Action": ["s3:GetBucketVersioning"]
        },
        {
            "Effect": "Allow",
            "Resource": ["arn:aws:s3:::{{inputBucket}}/{{inputObject}}"],
            "Action": ["s3:PutObject"]
        },
        {
            "Effect": "Allow",
            "Resource": ["arn:aws:s3:::{{outputBucket}}/*"],
            "Action": ["s3:GetObject"]
        },
        {
            "Effect": "Allow",
            "Resource": ["arn:aws:codebuild:{{region}}:{{awsAccountId}}:project/{{projectName}}"],
            "Action": ["codebuild:StartBuild",
                       "codebuild:BatchGetBuilds",
                       "codebuild:BatchGetProjects"]
        }
	]
}

Using the AWS CodeBuild plugin with the Jenkins Pipeline plugin

Use the snippet generator (click "Pipeline Syntax" on your pipeline project page) to generate the pipeline script that adds CodeBuild as a step in your pipeline. It should generate something like

awsCodeBuild projectName: 'project', credentialsType: 'keys', region: 'us-west-2', sourceControlType: 'jenkins'

Additionally, this returns a result object which exposes the following methods which can be useful to later steps:

  • getBuildId(): returns the build ID of the build (similar to codebuild-project-name:12346789-ffff-0000-aaaa-bbbbccccdddd)
  • getArn(): returns the ARN of the build (similar to arn:aws:codebuild:AWS_REGION:AWS_ACCOUNT_ID:build/CODEBUILD_BUILD_ID, where CODEBUILD_BUILD_ID is the same information returned in getBuildId)
  • getArtifactsLocation(): returns the S3 ARN of the artifacts location (similar to arn:aws:s3:::s3-bucket-name/path/to/my/artifacts)

AWS Credentials in Jenkins

It's recommended to use the Jenkins credentials store for your AWS credentials. Your Jenkins credentials must be of type CodeBuild Credentials to be compatible with the CodeBuild plugin. When creating new CodeBuild Credentials, the plugin will attempt to use the default credentials provider chain if AWS access and secret keys are not defined. You can also specify your AWS access and secret keys and session token in the CodeBuild configuration when using credentialsType: 'keys'. Example:

awsCodeBuild projectName: 'project', 
             credentialsType: 'keys',
	     awsAccessKey: env.AWS_ACCESS_KEY_ID,
	     awsSecretKey: env.AWS_SECRET_ACCESS_KEY,
	     awsSessionToken: env.AWS_SESSION_TOKEN,
	     ...

If the access/secret keys and session token are not specified, the plugin will attempt to use the default credentials provider chain. When running a Jenkins pipeline build, the plugin will attempt to use credentials from the pipeline-aws plugin before falling back to the default credentials provider chain. If you are running Jenkins on an EC2 instance, leave the access and secret key fields blank and specify credentialsType: 'keys'to use credentials from your EC2 instance profile, which is in the default credentials provider chain.

aws-codebuild-jenkins-plugin's People

Contributors

akarnani avatar awszhen avatar b-b3rn4rd avatar clareliguori avatar dependabot[bot] avatar eagletmt avatar emopti-jrufer avatar ikedam avatar jlleitschuh avatar johnhankataws avatar josephvusich avatar jpeddicord avatar kotaroooo0 avatar leoherran-aws avatar petezah avatar shariqmus avatar subinataws avatar takus avatar taoyong-ty avatar tomdev avatar tq-jappy avatar trufflemuffin avatar uttamk avatar vgamz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aws-codebuild-jenkins-plugin's Issues

Plugin no longer works with codebuild using Jenkins as source

A recent change to code build service by AWS has caused this plugin to stop functioning when you use Jenkins as the source.

Change was introduced on 24/02/2017 at some point. All builds (including rebuilds from existing artifacts that previously were successful) now fail on code build during DOWNLOAD_SOURCE step with

2017/02/25 07:40:19 Waiting for agent 
2017/02/25 07:40:37 Phase is DOWNLOAD_SOURCE 
2017/02/25 07:40:37 Source is located at /tmp/src614143422/src 
2017/02/25 07:40:37 Phase complete: DOWNLOAD_SOURCE Success: false 
2017/02/25 07:40:37 Phase context status code: YAML_FILE_ERROR Message: YAML file does not exist 
2017/02/25 07:40:37 Runtime error (YAML file does not exist) 

This appears to be an issue with the way the plugin creates the ZIP file. The plugin is currently adding the .git directory to the ZIP. When this is excluded code build service is able to execute correctly.

Line truncation at 197 chars, possible to change this?

Hi there, nice project btw.

I'm using this from Jenkins for most of our infra builds. However, I was wondering if I may either remove/update/make-configurable the truncation of logs coming back from cloudwatch?

Since we do CI/cd for everything, there's frequently fairly long lines (such as CFN updates and the like) and these being truncated is confusing for the average developer who is uninterested in the minutiae of CD pipelines.

Is there any problem or reasoning to not simply raise this this a fair bit (say 2000?). It'd just make the UX a fair bit nicer.

Building previous commits when using $GIT_COMMIT on Source Control Version

I'm having and issue when setting the Build step to 'Use Project Source' and setting the Source Control Version to $GIT COMMIT.
Jenkins fetches the right commit but on the build step it builds the previous commit on the branch.
I'm using 'Use Jenkins Source' now, but I need to go back using 'Use Project Source'.

Thanks,

Andre

Codebuild plugin sets currentBuild.result = "FAILURE"

Whenever there is a failure in Codebuild, the plugin fails the parent Jenkins build in an unrecoverable manner. My Jenkins pipeline allows specific parts of the pipeline that are flakey builds to be rerun and then pass without rerunning the entire build.

However once currentBuild.result has been set to "failure" already, the whole build can never be set to pass again. Could you send failure status to Jenkins by throwing a exception instead of explicitly setting the build as failure?

This was implemented before here but reverted again here

Thanks

Tail cloudwatch logs directly to jenkins

I've been reading through issues to decide whether we can/should use this plugin, and it appears that we would need to click out to view logs. It would be great if this instead polled for cloudwatch logs and inserted them as they came in - perhaps with some variable polling interval.

This could also be an optional alternative to the current method of just logging the url to jenkins.

Logs not retrieve from cloud watch

CodeBuilder.java never sets the logsLocation from the Build objective for the logsMonitor to use. This results in the dashboard never getting any log information.

Will issue a pull request for this shortly.

Allow specifying a role instead of aws credentials for IAM Access

Rather than have keys running around in repositories that run Jenkins pipelines, it would be great if we could delegate role access to the codebuild plugin for specific jobs. Not sure how that would work underneath, but would be a nicer way of hiding credentials from users.

Plugin not working on a windows based jenkins installation

There appears to be a bug with S3DataManager.java.

String zipFilePath = sourceFilePath.substring(0, sourceFilePath.lastIndexOf("/")) + "/" + localfileName;

Assumes that the path uses directory structure based around "/". On windows, this is not the case. For example, C:\Program Files (x86)\Jenkins\workspace\my-project.

This results in the console error message [AWS CodeBuild Plugin] String index out of range: -1 during the build process.

Suggested fix is changing to String zipFilePath = sourceFilePath + "/" + localfileName;. However, I am not really familiar with Java and there may be much better solutions to this so have not submitted a pull request.

Need ability to retry builds that do not actually start

Sometimes builds fail to start simply due to network reliability in AWS--either the REST API calls to start up the build fail , or, very occasionally, Codebuild fails to pull my container image from ECR (getting 500 internal server errors). These errors usually clear up pretty quickly, and builds can simply be restarted; but since whole pipelines are usually more complicated, it would be nice to be able to specify an option that the plugin could just retry the build for me. The "retry" Jenkins step will not work, simply because I do not want to retry builds that fail legitimately. It seems like the right approach is to make an improvement to this plugin.

Also a side-note, I may actually have bandwidth to work on this myself, if necessary, but if a dedicated maintainer could point me in the right direction initially, I think we could save a lot of time.

"Source version should be empty for S3 folder source location" when using Jenkins source

I can't set up AWS CodeBuild plugin to use Jenkins source as described in the README file. I have a CodeBuild project configured to use s3 source.
When used with a standard s3 bucket the plugin fails with a message [AWS CodeBuild Plugin] A versioned S3 bucket is required
When using a (versioned) bucket root directory as AWS CodeBuild source (S3 object key or S3 folder set to '/' in AWS CodeBuild webconsole) the plugin fails with The XML you provided was not well-formed or did not validate against our published schema (Service: Amazon S3; Status Code: 400; Error Code: MalformedXML ...)
When using a (versioned) bucket subdirectory as AWS CodeBuild source (S3 object key or S3 folder set to 'sources/ in AWS CodeBuild webconsole') the plugin seems to work yet:
a) The sources are properly checked out by jenkins but are not copied to s3 (there are no files in the specified s3 bucket) although plugin output suggests so:

[AWS CodeBuild Plugin] Uploading code to S3 at location test-build-versioned/sources/. MD5 checksum is KBgkNj7so0ZH7lzAcCa/fg==
[AWS CodeBuild Plugin] S3 object version id for uploaded source is TUsv3t8YcNZscZ0c1LTTuLzjxUdXz7wg

b) the plugin calls AWS CodeBuild with a source version specified which makes AWS CodeBuild reject the request

[AWS CodeBuild Plugin] Starting build with 
    > project name: NET-FRAMEWORK-TEST-BUILD
    > source version: TUsv3t8YcNZscZ0c1LTTuLzjxUdXz7wg
[AWS CodeBuild Plugin] Source version should be empty for S3 folder source location (Service: AWSCodeBuild; Status Code: 400; Error Code: InvalidInputException; ...)

Zip files is not released after the job finish

Hello!

When i use the AWS Code Build plugin, in the job finish the xxxx.source.zip file is not deleted, i need restart the Jenkins process to file be released, i can see in "lsof" command:

java 11801 jenkins 572w REG 202,1 118609 446884 /var/lib/jenkins/jobs/Test/9c179a84-d1ca-4ca5-a9ae-e608b42883c5-source.zip (deleted)

Someone could help me please?

Thank you

Plugin build fails due to tests

I tried building the plugin as instructed using mvn install and one test failed causing the build to fail

Results :

Failed tests:
CodeBuilderPerformTest.testActionConfig:121 null

Tests run: 42, Failures: 1, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 57.964s
[INFO] Finished at: Fri Dec 02 15:44:58 CST 2016
[INFO] Final Memory: 32M/860M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.16:test (default-test) on project aws-codebuild: There are test failures.
[ERROR]
[ERROR] Please refer to /home/jonh/temp/aws-codebuild-jenkins-plugin/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

Update aws sdk version to allow for new regions

The aws-java-sdk-bom version is currently 1.11.63, which means that the com/amazonaws/partitions/endpoints.json file from aws-java-sdk-core only has 4 regions, but there are a whole lot more available now. Could you update to 1.11.114 or greater so that the other regions are included too. (pom.xml:157)

add ability to use subset of jenkins source

When invoking codebuild with sourceControltype: 'jenkins' the entire Jenkins workspace is uploaded to S3 for the codebuild job to pick up. In some cases, it would be helpful to limit the contents shared to codebuild to a single folder within the Jenkins workspace.

This is useful in situations where the source Jenkins workspace is a large monorepo, or the Jenkins pipeline contains parallel steps where a different parallel step is modifying files in the workspace. The latter scenario actually leads to a failure in the codebuild step.

1.2: Access Denied in phase DOWNLOAD_SOURCE

Hello again,

I've been using the CodeBuild Plugin for Jenkins in my organisation for quite some time and gave the latest version a try. Yet when I change the plugin in Jenkins from 1.1 (March 2017) to the latest commit e56fc3d i recieve an error which did not occur before in the same jobs. I also tried the version from jenkins.io

The Jenkins job fails with:

[AWS CodeBuild Plugin] Uploading source to S3.

[AWS CodeBuild Plugin] Uploading code to S3 at location <snip>/source/source/. MD5 checksum is vwgkz<snip>Ug0ve8Q==

[AWS CodeBuild Plugin] Source upload finished.
[AWS CodeBuild Plugin] S3 object version id for uploaded source is TB<snip>dY7
[AWS CodeBuild Plugin] Starting build with project name <snip> and source version TB<snip>dY7

[AWS CodeBuild Plugin] Logs url: https://console.aws.amazon.com/cloudwatch/home?region=eu-west-1#logEvent:group=/aws/codebuild/<snip>;stream=2b3c0e7<snip>de793

[AWS CodeBuild Plugin] Build failed for build-npm and source version 
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
CodeBuildException
	at CodeBuildStep$CodeBuildExecution.run(CodeBuildStep.java:170)
	at CodeBuildStep$CodeBuildExecution.run(CodeBuildStep.java:136)
	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
	at hudson.security.ACL.impersonate(ACL.java:260)
	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE

And CodeBuild fails with:

Access Denied in the phase DOWNLOAD_SOURCE

The Cloudwatch logfile is also empty.

Has something changed with the permissons in version 1.2? My permissions for version 1.1 were fine as all builds succeeded.

Thanks

awsCodeBuild: This step has not yet defined any visual configuration.

I have learned in this github issue that the awsCodeBuild plugin also supports Jenkins pipeline.

But the Snippet Generator (Jenkins 2.3.2) doesn't lets you generate a groovy snippet for awsCodeBuild, it only displays the message 'This step has not yet defined any visual configuration.'

Could you therefore please:

a) Add visual configuration
b) Add a sample snippet for pipeline to the Readme.md?

About Codebuild Projects

Is it necessary to create a new Codebuild project for each repository that has the same buildspec.yml file. Like, I have a project on COdebuild, but I have many repositories. Do I need to make a Codebuild project for each repository?

File permission aren't preserved when uploading to S3

The file permissions are not being preserved when creating the source archive that gets uploaded to S3.
The main problem is with executable files, because the generated zip does not preserve this permission.

Is there any way to preserve Linux file attributes when creating the zip file?

Jenkins Source uploaded to S3 is not encrypted

Using the Jenkins Source being uploaded to S3, the source is uploaded without encryption. Is there plans to add encryption (either Server Side Encryption or Customer Side Encryption?)

Unable to extract S3 bucket from object ARN.

Jenkins and the plugin are all latest release.

Following the steps for: "Using AWS CodeBulid with source only available inside of your VPC"
Manual build from AWS console, it works successful, source code checkout from git is ok, but when starting build phase with the plugin, IAM permission is correct, but I got following error.

java.lang.RuntimeException: Unable to extract S3 bucket from object ARN. at Utils.getS3BucketFromObjectArn(Utils.java:24) at CodeBuilder.perform(CodeBuilder.java:132) at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:78) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779) at hudson.model.Build$BuildExecution.build(Build.java:206) at hudson.model.Build$BuildExecution.doRun(Build.java:163) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534) at hudson.model.Run.execute(Run.java:1728) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:405) Build step 'Run build on AWS CodeBuild' marked build as failure Finished: FAILURE

Running the plugin in parallel fails

When running the Jenkins CodeBuild plugin (24337ae) on Jenkins (version 2.32.3) in parallel the plugin will fail with:

[AWS CodeBuild Plugin] Uploading source to S3.
[AWS CodeBuild Plugin] channel is already closed

To reproduce you need to have a CodeBuild project and need two Jenkins jobs running in parallel which checkout from git (different branches) and use the plugin with "jenkins" as source.

Running mvn install on clean clone fails

The tests are failing preventing the compilation from completing. This appears to be due to missing dependant resources.

Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.08 sec - in CodeBuilderConfigurationTest
Running CodeBuilderHelperTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.007 sec - in CodeBuilderHelperTest
Running CodeBuilderPerformTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.048 sec - in CodeBuilderPerformTest
Running ProjectFactoryTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.078 sec - in ProjectFactoryTest
Running S3DataManagerTest
Tests run: 11, Failures: 0, Errors: 6, Skipped: 0, Time elapsed: 1.594 sec <<< FAILURE! - in S3DataManagerTest
testZipSourceDoubleDirMultipleHugeFiles(S3DataManagerTest)  Time elapsed: 0.064 sec  <<< ERROR!
java.lang.Exception: zipSource usage: prefixToTrim must be contained in the given directory.
        at S3DataManager.zipSource(S3DataManager.java:117)
        at S3DataManager.zipSource(S3DataManager.java:131)
        at S3DataManagerTest.testZipSourceDoubleDirMultipleHugeFiles(S3DataManagerTest.java:351)

testZipSourceOneDirEmpty(S3DataManagerTest)  Time elapsed: 0.007 sec  <<< ERROR!
java.lang.Exception: zipSource usage: prefixToTrim must be contained in the given directory.
        at S3DataManager.zipSource(S3DataManager.java:117)
        at S3DataManager.zipSource(S3DataManager.java:131)
        at S3DataManagerTest.testZipSourceOneDirEmpty(S3DataManagerTest.java:200)

testUploadSource(S3DataManagerTest)  Time elapsed: 1.241 sec  <<< ERROR!
java.lang.StringIndexOutOfBoundsException: String index out of range: -1
        at java.lang.String.substring(String.java:1911)
        at S3DataManager.uploadSourceToS3(S3DataManager.java:66)
        at S3DataManagerTest.testUploadSource(S3DataManagerTest.java:116)

testZipSourceMultipleNestedDirs(S3DataManagerTest)  Time elapsed: 0.013 sec  <<< ERROR!
java.lang.Exception: zipSource usage: prefixToTrim must be contained in the given directory.
        at S3DataManager.zipSource(S3DataManager.java:117)
        at S3DataManager.zipSource(S3DataManager.java:131)
        at S3DataManagerTest.testZipSourceMultipleNestedDirs(S3DataManagerTest.java:413)

testZipSourceOneDir(S3DataManagerTest)  Time elapsed: 0.014 sec  <<< ERROR!
java.lang.Exception: zipSource usage: prefixToTrim must be contained in the given directory.
        at S3DataManager.zipSource(S3DataManager.java:117)
        at S3DataManager.zipSource(S3DataManager.java:131)
        at S3DataManagerTest.testZipSourceOneDir(S3DataManagerTest.java:232)

testZipSourceOneDirMultipleFiles(S3DataManagerTest)  Time elapsed: 0.014 sec  <<< ERROR!
java.lang.Exception: zipSource usage: prefixToTrim must be contained in the given directory.
        at S3DataManager.zipSource(S3DataManager.java:117)
        at S3DataManager.zipSource(S3DataManager.java:131)
        at S3DataManagerTest.testZipSourceOneDirMultipleFiles(S3DataManagerTest.java:286)

Running UtilsTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 sec - in UtilsTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 44.325 sec - in CodeBuilderEndToEndPerformTest

Results :

Tests in error:
  S3DataManagerTest.testZipSourceDoubleDirMultipleHugeFiles:351 ╗  zipSource usa...
  S3DataManagerTest.testZipSourceOneDirEmpty:200 ╗  zipSource usage: prefixToTri...
  S3DataManagerTest.testUploadSource:116 ╗ StringIndexOutOfBounds String index o...
  S3DataManagerTest.testZipSourceMultipleNestedDirs:413 ╗  zipSource usage: pref...
  S3DataManagerTest.testZipSourceOneDir:232 ╗  zipSource usage: prefixToTrim mus...
  S3DataManagerTest.testZipSourceOneDirMultipleFiles:286 ╗  zipSource usage: pre...

Tests run: 42, Failures: 0, Errors: 6, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:12 min
[INFO] Finished at: 2016-12-15T12:25:16+00:00
[INFO] Final Memory: 24M/158M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.16:test (default-test) on project aws-codebuild: There are test failures.
[ERROR]
[ERROR] Please refer to c:\Production\Packages\aws-codebuild-jenkins-plugin\target\surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

non-S3 input sources not supported

Hi Team,

The readme implies that one can use GitHub (or BitBucket, etc) as the input source instead of S3. However, in the validation of the projectName, if the source for the existing project is anything but S3, the project is invalid and fails.

The plugin should support non-S3 inputs as is consistent with the documentation.

Step green even if build fails

I am using the aws-codebuild plugin from a pipeline script using
awsCodeBuild projectName: project_name, awsAccessKey: "key", awsSecretKey: "secret", region: "region", sourceVersion: "commit_sha"
When the code build build fails, the main build shows red. When you go to pipeline steps

Expected behaviour

The step should show up as red

Actual behaviour

The step shows up as blue/green

screenshot 2017-06-13 19 22 48

Ours is quite a complicated project with a lot of parallel steps. It makes it hard to figure out what actually failed (since the console is not that helpful with parallel steps making it confusing)

Job configuration containing CodeBuild lost after Jenkins reboot

Issue seen on Jenkins 2.32.1 with plugin code commit '0046d84' (mavenID com.amazonaws:aws-codebuild:1.1):

Hello,

I'm experiencing that after a Jenkins reboot the Jenkins job containing the CodeBuild Plugin as Buildstep is losing its buildstep configuration completly.

Thanks

Paramiterized build does not work

Hi,

I'm trying to create a build task that accepts a commit id as a parameter.

screen shot 2017-08-04 at 11 38 50 am

and then use this value as the project source control version.

screen shot 2017-08-04 at 11 39 08 am

However, the variable does not appear to be getting substituted as shown in the log below

[AWS CodeBuild Plugin] Starting build with project name xxxxx and source version $COMMIT_ID

It would be great if parameters were supported. Thanks in advance for any assistance.

Running CodeBuild on Docker Container instead of Master node

Issue seen on Jenkins 2.32.1 with plugin code commit '0046d84' (mavenID com.amazonaws:aws-codebuild:1.1)

Hello,

when running a Jenkins Job with CodeBuild build step on a dynamically provisioned Docker container as node instead of the master, then I'm experiencing the following issue:

[AWS CodeBuild Plugin] Source S3 bucket is kaercher-adminpanel
[AWS CodeBuild Plugin] Uploading source to S3.
[AWS CodeBuild Plugin] Unable to create directory: /home/jenkins/workspace

But, /home/jenkins/workspace exists and should be allright, a build step before CodeBuild is running shell and reports:

  • ls /home/jenkins/
    slave.jar
    workspace

  • ls -lh /home/jenkins/
    total 704K
    -rw-rw-r-- 1 jenkins jenkins 700K Jan 27 12:16 slave.jar
    drwxrwxr-x 3 jenkins jenkins 4.0K Jan 27 12:16 workspace

  • ls /home/jenkins/workspace
    OSAP - Build Code - Frontend

  • ls -lh /home/jenkins/workspace
    total 4.0K
    drwxrwxr-x 6 jenkins jenkins 4.0K Jan 27 12:16 OSAP - Build Code - Frontend

Docker:
Client:
Version: 1.12.1
API version: 1.24
Go version: go1.6.2
Git commit: 23cf638
Built: Tue, 27 Sep 2016 12:25:38 +1300
OS/Arch: linux/amd64

Server:
Version: 1.12.1
API version: 1.24
Go version: go1.6.2
Git commit: 23cf638
Built: Tue, 27 Sep 2016 12:25:38 +1300
OS/Arch: linux/amd64

Thanks

Plugin does not allow users to pass in environmentVariablesOverride properties

Plugin does not allow users to pass in environmentVariablesOverride properties

the plugin is very limiting since it does not allow users to pass in environmentVariablesOverride properties

e.g. hence, we cannot pass in properties like artifactVersion to have codebuild build docker image and tag it appropriately with the supplied version

is it possible to add this feature? it would be really useful

thanks!

Using IAM role for Jenkins

I've tried to use Jenkins IAM role( which worked when checked from aws-cli ) instead of AWS Access Key and AWS Secret Key and left these fields empty, but got an error:
[AWS CodeBuild Plugin] CodeBuild configured improperly in project settings Enter valid AWS access and secret keys (Service: null; Status Code: 0; Error Code: null; Request ID: null)

should it work?

ConcurrentModificationException in CodeBuildAction

I am using:

Jenkins: 2.89.3
AWS Codebuild plugin: com.amazonaws:aws-codebuild:0.17 (maven ID)
Pipeline plugin: org.jenkins-ci.plugins.workflow:workflow-aggregator:2.5 (maven ID)

And I have a multibranch pipeline with Jenkinsfile which looks something like this:

pipeline {
    agent { label 'master' }

    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }
        stage('Test Run') {
            failFast true
            parallel {
                stage('Test Run 1') {
                    steps {
                        timestamps {
                            withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', credentialsId: 'aws']]) {
                                awsCodeBuild credentialsId: 'CodeBuild', credentialsType: 'jenkins', projectName: 'Proj1', region: 'us-east-1', sourceControlType: 'project', sourceVersion: "${env.GIT_COMMIT}"
                            }
                        }
                    }
                }
                stage('Test Run 2') {
                    steps {
                        withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', credentialsId: 'aws']]) {
                            awsCodeBuild credentialsId: 'CodeBuild', credentialsType: 'jenkins', projectName: 'Proj2', region: 'us-east-1', sourceControlType: 'project', sourceVersion: "${env.GIT_COMMIT}"
                        }
                    }
                }
                stage('Test Run 3') {
                    steps {
                        withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', credentialsId: 'aws']]) {
                            awsCodeBuild credentialsId: 'CodeBuild', credentialsType: 'jenkins', projectName: 'Proj3', region: 'us-east-1', sourceControlType: 'project', sourceVersion: "${env.GIT_COMMIT}", envVariables: "[{VAR, value1}]"
                        }
                    }
                }
                stage('Test Run 4') {
                    steps {
                        withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', credentialsId: 'aws']]) {
                            awsCodeBuild credentialsId: 'CodeBuild', credentialsType: 'jenkins', projectName: 'Proj3', region: 'us-east-1', sourceControlType: 'project', sourceVersion: "${env.GIT_COMMIT}", envVariables: "[{VAR, value2}]"
                        }
                    }
                }
                stage('Test Run 5') {
                    steps {
                        withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', credentialsId: 'aws']]) {
                            awsCodeBuild credentialsId: 'CodeBuild', credentialsType: 'jenkins', projectName: 'Proj3', region: 'us-east-1', sourceControlType: 'project', sourceVersion: "${env.GIT_COMMIT}", envVariables: "[{VAR, value3}]"
                        }
                    }
                }
                stage('Test Run 6') {
                    steps {
                        withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', credentialsId: 'aws']]) {
                            awsCodeBuild credentialsId: 'CodeBuild', credentialsType: 'jenkins', projectName: 'Proj3', region: 'us-east-1', sourceControlType: 'project', sourceVersion: "${env.GIT_COMMIT}", envVariables: "[{VAR, value4}]"
                        }
                    }
                }
            }
        }
    }
}

in stages Test Run 3 to Test Run 6 I am using same Codebuild project but with different environment variable values which execute different scripts in the build
When I am running the job with above pipeline I am getting following exception:

java.util.ConcurrentModificationException
	at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:907)
	at java.util.ArrayList$Itr.next(ArrayList.java:857)
	at com.thoughtworks.xstream.converters.collections.CollectionConverter.marshal(CollectionConverter.java:73)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)
	at hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:265)
	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:252)
Caused: java.lang.RuntimeException: Failed to serialize CodeBuildAction#logs for class CodeBuildAction
	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:256)
	at hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:224)
	at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138)
	at hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:209)
	at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:150)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:88)
	at com.thoughtworks.xstream.converters.collections.AbstractCollectionConverter.writeItem(AbstractCollectionConverter.java:64)
	at com.thoughtworks.xstream.converters.collections.CollectionConverter.marshal(CollectionConverter.java:74)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)
	at hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:265)
	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:252)
Caused: java.lang.RuntimeException: Failed to serialize hudson.model.Actionable#actions for class org.jenkinsci.plugins.workflow.job.WorkflowRun
	at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:256)
	at hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:224)
	at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138)
	at hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:209)
	at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:150)
	at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
	at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43)
	at com.thoughtworks.xstream.core.TreeMarshaller.start(TreeMarshaller.java:82)
	at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.marshal(AbstractTreeMarshallingStrategy.java:37)
	at com.thoughtworks.xstream.XStream.marshal(XStream.java:1026)
	at com.thoughtworks.xstream.XStream.marshal(XStream.java:1015)
	at com.thoughtworks.xstream.XStream.toXML(XStream.java:988)
	at hudson.XmlFile.write(XmlFile.java:181)
	at org.jenkinsci.plugins.workflow.support.PipelineIOUtils.writeByXStream(PipelineIOUtils.java:30)
	at org.jenkinsci.plugins.workflow.job.WorkflowRun.save(WorkflowRun.java:1070)
	at hudson.BulkChange.commit(BulkChange.java:98)
	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.notifyListeners(CpsFlowExecution.java:1377)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$3.run(CpsThreadGroup.java:412)
	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:35)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE

I am not familiar with Jenkins plugins, can anyone suggest any pointers, where can I look for more information, etc.

support withAWS assume role

Hi team,

Apologies if I have missed how to accomplish this, but it appears using Assume Role is not supported as the AWS_SESSION_TOKEN is not being sent with the request when specifying the awsAccessKey and awsSecretKeys?

Our CI server lives in account X and is assuming the cicd role in Account Y
I have confirmed that the permissions are all fine by printing out the keys/token and testing via the command line.

Can we add support for STS Assume roles?

Pipeline code below:

            withAWS(region: params.AWS_REGION, role: cicdRoleName, roleAccount: awsAccountId) {     
                codeBuildResult = awsCodeBuild projectName: config.repoName,
                    region: params.AWS_REGION,
                    sourceControlType: 'project',
                    credentialsType: 'keys',
                    awsAccessKey: env.AWS_ACCESS_KEY_ID,
                    awsSecretKey: env.AWS_SECRET_ACCESS_KEY,
                    envVariables: "${codebuildEnvVars}"
            }

Jenkins Error:

[Pipeline] awsCodeBuild
00:00:12.128 [AWS CodeBuild Plugin] Using given AWS access and secret key for authorization
00:00:12.278 [AWS CodeBuild Plugin] The security token included in the request is invalid. (Service: AWSCodeBuild; Status Code: 400; Error Code: UnrecognizedClientException; Request ID: 36dab899-4f28-11e8-90d9-891f23967dfc)
[Pipeline] End of Pipeline

GitHub has been notified of this commit’s build result

CodeBuildException
	at CodeBuildStep$CodeBuildExecution.run(CodeBuildStep.java:286)
	at CodeBuildStep$CodeBuildExecution.run(CodeBuildStep.java:249)
	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
	at hudson.security.ACL.impersonate(ACL.java:260)
	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE

IAM Role Help file not found

credentials.jelly points to a non existent help file (help-IAMRole.html). This results in an error on the Add Credentials page, when IAM Role ARN help is being accessed:

ERROR: Failed to load help file: Not Found

I'm guessing the correct file would be help-iamRoleAuth.html

Duplicating prefix when uploading artifact to S3

The key prefix is duplicated during the upload to s3 when using Jenkins source.

Current behavior: bucket-name/prefix/key.zip will become bucket-name/prefix/prefix/key.zip
Expected behavior: bucket-name/prefix/key.zip will be the same

Log showing the issue (line2)

[AWS CodeBuild Plugin] Using given AWS access and secret key for authorization
[AWS CodeBuild Plugin] Uploading code to S3 at location BUCKET-NAME/commit/commit/project.zip. MD5 checksum is Zi45XEEGcYf7/KN1cp6FTw==
[AWS CodeBuild Plugin] S3 object version id for uploaded source is EaWW4Y9n1hB1FzXDwR.rii3BxO9oErxi
[AWS CodeBuild Plugin] Starting build with 
	> project name [projectname]
	> source version: EaWW4Y9n1hB1FzXDwR.rii3BxO9oErxi
[AWS CodeBuild Plugin] Build Id: projectname:231b085b-af9f-43ed-87f1-cf9b56b73362
[AWS CodeBuild Plugin] Logs url: https://console.aws.amazon.com/cloudwatch/home?region=us-west-2#logEvent:group=/aws/codebuild/projectname;stream=231b085b-af9f-43ed-87f1-cf9b56b73362
[AWS CodeBuild Plugin] Build projectname:231b085b-af9f-43ed-87f1-cf9b56b73362 failed
	> Access Denied (status code: CLIENT_ERROR)
Build step 'AWS CodeBuild' changed build result to FAILURE

NullPointerException after upgrade to 0.25

Hi,

I recently upgraded to version 0.25 and subsequently received a build failure with the message below. Downgrading to 0.17 allowed the build to successfully complete. Is there something I need to update in my config due to non-backwards compatible updaets? This is a really simple config, using instance role rather than specifying credentials in jenkins.

No emails were triggered.
[AWS CodeBuild Plugin] Using credentials provided by the DefaultAWSCredentialsProviderChain for authorization
ERROR: Build step failed with exception
java.lang.NullPointerException
	at CodeBuilder.logStartBuildMessage(CodeBuilder.java:523)
	at CodeBuilder.perform(CodeBuilder.java:318)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
	at hudson.model.Build$BuildExecution.build(Build.java:206)
	at hudson.model.Build$BuildExecution.doRun(Build.java:163)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
	at hudson.model.Run.execute(Run.java:1794)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
Build step 'AWS CodeBuild' marked build as failure

jenkins

CodeBuild failure does not fail the Jenkins build

Hey,

I have had an issue in a number of builds where when codebuild fails, the build in Jenkins does not fail.

The current build status in the codebuild dashboard is "Failed", but the Jenkins build is 'Green'.

I think that a codebuild failure should fail the Jenkins build.

CodeBuild Failure will be reported as success

A failure in phase 'PROVISIONING' will be reported by the plugin as SUCCESS although CodeBuild reported it correctly as failed.

A failure in phase 'PROVISIONING' can be caused by choosing a custom Docker image without giving it the correct Principal rights (AWS account). https://docs.aws.amazon.com/codebuild/latest/userguide/sample-ecr.html

Unable to pull customer's container image. AWS CodeBuild is not authorized to perform ecr:BatchGetImage on resource ...

=>

Build step 'Run build on AWS CodeBuild' changed build result to SUCCESS

Codebuild Step Failure continues onto next stage in pipeline

Multi-branch pipeline with declarative syntax on Jenkins 2.121 and the latest codebuild plugin 0.24

Relevant Code:
buildspec.yml

phases:
  build:
    commands:
      - fail

Jenkinsfile


String version = "1.0.${env.BUILD_NUMBER}"
String awsCredentialsId = '<redacted>'

pipeline {
    agent any
    options {
        gitLabConnection('gitlab')
    }

    stages {
        stage('SCM Checkout') {
            steps {
                echo "CURRENT RESULT: ${currentBuild.result}"
                checkout scm
            }
        }

        stage('Build') {
            steps {
                echo "CURRENT RESULT: ${currentBuild.result}"
                step([$class                   : 'CodeBuilder', artifactLocationOverride: 'artifacts',
                      artifactNameOverride     : "backend-${version}.zip",
                      artifactPackagingOverride: 'ZIP', 
                      artifactPathOverride: 'demo', 
                      artifactTypeOverride: 'S3',
                      credentialsId            : awsCredentialsId, 
                      credentialsType: 'jenkins',
                      projectName              : 'jenkins', 
                      region: 'us-east-1',
                      sourceControlType        : 'jenkins',
                      buildSpecFile            : 'buildspec.yml' ])
                echo "CURRENT RESULT: ${currentBuild.result}"
            }
        }

        stage('Integration test') {
            steps {
                echo "CURRENT RESULT: ${currentBuild.result}"
                echo "Integration test"
            }
        }
    }
}

Expect:
Pipeline to report a failure and skip the rest of the stages

Actual:

 > git config core.sparsecheckout # timeout=10
 > git checkout -f b232234dcfaa524c3eabc45860031de0bdc797ba
Commit message: "severe cleanup to attempt force fail"
 > git rev-list --no-walk b7dd56d723fc1c0130fc0250628610817d738e51 # timeout=10
[Pipeline] }
[Pipeline] // stage
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (SCM Checkout)
[Pipeline] echo
CURRENT RESULT: null
[Pipeline] checkout
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url <-redacted-> # timeout=10
Fetching without tags
Fetching upstream changes from <-redacted->
 > git --version # timeout=10
using GIT_ASKPASS to set credentials 
 > git fetch --no-tags --progress <-redacted->
 +refs/heads/*:refs/remotes/origin/*
Checking out Revision b232234dcfaa524c3eabc45860031de0bdc797ba (test-branch)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f b232234dcfaa524c3eabc45860031de0bdc797ba
Commit message: "severe cleanup to attempt force fail"
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Build)
[Pipeline] echo
CURRENT RESULT: null
[Pipeline] step
[AWS CodeBuild Plugin] Using given AWS access and secret key for authorization (provided from Jenkins credentials <-redacted->)
[AWS CodeBuild Plugin] Uploading code to S3 at location <redacted>/demo. MD5 checksum is b/1IWR0hvVjVtk3ZrC2ynw==
[AWS CodeBuild Plugin] S3 object version id for uploaded source is p0FloUnL6y631_g3as759DfbPJQRx8fE
[AWS CodeBuild Plugin] Starting build with 
        > project name: jenkins
        > source version: p0FloUnL6y631_g3as759DfbPJQRx8fE
        > artifact type: S3
        > artifact location: artifacts
        > artifact name: backend-1.0.12.zip
        > artifact packaging: ZIP
        > artifact path: demo
        > build spec: buildspec.yml
[AWS CodeBuild Plugin] Build id: jenkins:027a1720-bb82-4967-9568-d8e8df0ce0eb
[AWS CodeBuild Plugin] CodeBuild dashboard: https://us-east-1.console.aws.amazon.com/codebuild/home?region=us-east-1#builds/jenkins:027a1720-bb82-4967-9568-d8e8df0ce0eb/view/new
[AWS CodeBuild Plugin] CloudWatch dashboard: https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#logEvent:group=/aws/codebuild/jenkins;stream=027a1720-bb82-4967-9568-d8e8df0ce0eb
[AWS CodeBuild Plugin] 2018/05/16 18:25:32 Waiting for agent ping
[AWS CodeBuild Plugin] 2018/05/16 18:25:33 Waiting for DOWNLOAD_SOURCE
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phase is DOWNLOAD_SOURCE
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 CODEBUILD_SRC_DIR=/codebuild/output/src340291698/src
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 YAML location is /codebuild/output/src340291698/src/buildspec.yml
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Processing environment variables
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Moving to directory /codebuild/output/src340291698/src
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Registering with agent
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phases found in YAML: 1
[AWS CodeBuild Plugin] 2018/05/16 18:25:36  BUILD: 1 commands
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phase complete: DOWNLOAD_SOURCE Success: true
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phase context status code:  Message: 
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Entering phase INSTALL
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phase complete: INSTALL Success: true
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phase context status code:  Message: 
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Entering phase PRE_BUILD
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phase complete: PRE_BUILD Success: true
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phase context status code:  Message: 
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Entering phase BUILD
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Running command fail
[AWS CodeBuild Plugin] /codebuild/output/tmp/script.sh: 4: /codebuild/output/tmp/script.sh: fail: not found
[AWS CodeBuild Plugin] 
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Command did not exit successfully fail exit status 127
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phase complete: BUILD Success: false
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: fail. Reason: exit status 127
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Entering phase POST_BUILD
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phase complete: POST_BUILD Success: true
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phase context status code:  Message: 
[AWS CodeBuild Plugin] Build jenkins:027a1720-bb82-4967-9568-d8e8df0ce0eb failed
        > Error while executing command: fail. Reason: exit status 127 (status code: COMMAND_EXECUTION_ERROR)
[Pipeline] echo
CURRENT RESULT: FAILURE
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Integration test) _<---- should not execute_
[Pipeline] echo
CURRENT RESULT: FAILURE _<---- should not execute_
[Pipeline] echo
Integration test  _<---- should not execute_
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: FAILURE

When replacing the codebuild step with a sh 'FAIL' with the same setup, works as expected and the integration stage is skipped due to failure.

Very bad performance when run inside pod

Hello!

I'm using the AWSCodeBuild plugin in a Kubernetes Cluster with Kubernetes plugin (https://github.com/jenkinsci/kubernetes-plugin), when i allocate a pod to run my pipeline/awscodebuild command the perfomance in zip action is very bad, normally my zip takes ~10seconds to complete but in this case the same action takes ~6minutes.

If i connect to a container and run the zip command manually the performance is normally, the "problem" only happen inside plugin.

My Jenkinsfile: https://pastebin.com/yCGMeA2R

There are could help me please?

Regards

Multiple CodeBuild Dashboard links with same URL

Background:
I'm using CodeBuild from Jenkins Pipeline, invoking CodeBuild in parallel. Each CodeBuild run executes in parallel, and adds a link to the sidebar to show CodeBuild results inside Jenkins.

parallel (
  unitTests: { awsCodeBuild [...], projectName: "unitTests", sourceVersion: "${GIT_COMMIT}" },
  integrationTests: { awsCodeBuild [...], projectName: "integrationTests", sourceVersion: "${GIT_COMMIT}"  }
)

Expected:
One CodeBuild link to show results from unitTests and another CodeBuild link to show results from integrationTests

Actual:
Multiple CodeBuild links with same URLs. Both of the links lead to unitTests results.

image

Each of the links above are tied to https://jenkins.com/job/aws-codebuild-job/123/codebuild


Notes:

Is there any way to make the different links have an additional URI, to separate the links with different AWS CodeBuild results? Something like:

AWS api calls are throttled when there are too many parallel builds

We have around 10 parallel builds per branch/commit triggered from Jenkins. When two or three branches run in parallel, we get
Rate exceeded (Service: AWSCodeBuild; Status Code: 400; Error Code: ThrottlingException
and the build fails.
This is because the CodeBuild plugin polls AWS CodeBuild every 5 seconds. When there are thirty builds in parallel, the number of requests made in a small interval of time is high enough for the AWS api infrastructure to throttle the calls.

Pipeline build compatability

We are looking at using this plugin with Jenkins 2 using pipelines. Is that a featured that is planned to be added to this plugin?

I believe to be compatible it has to implement a SimpleBuildStep according to the dev guide.

When plugin installed and saving system options in jenkins, plugin breaks

javax.servlet.ServletException: net.sf.json.JSONException: JSONObject["proxyHost"] not found.
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:796)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649)
	at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:135)
	at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:126)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:80)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
	at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
	at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
	at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:553)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
	at org.eclipse.jetty.server.Server.handle(Server.java:499)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
	at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
	at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
Caused by: net.sf.json.JSONException: JSONObject["proxyHost"] not found.
	at net.sf.json.JSONObject.getString(JSONObject.java:2040)
	at CodeBuilder$DescriptorImpl.configure(CodeBuilder.java:326)
	at jenkins.model.Jenkins.configureDescriptor(Jenkins.java:3642)
	at jenkins.model.Jenkins.doConfigSubmit(Jenkins.java:3605)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:335)
	at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:175)
	at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:108)
	at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:124)
	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
	... 54 more

This looks to be because CodeBuilder.java on line 325 onwards is access properties from the JSONObject without checking if they exist first.

AWS Codebuild Plugin on jenkins not working

I have a containerized Jenkins in my kubernetes cluster and I would like to create a pipeline on that. Right now we use AWS for all our services at my company. I would like to know how to counter this error that I get when I run code build amazon plugin in Jenkins with my specific builspec.yml file. I can provide you with all the IAM roles and the policy set up I have used and would like to know where exactly I am going wrong or what the issue could be.

ERROR:

[AWS CodeBuild Plugin] Using credentials provided by the DefaultAWSCredentialsProviderChain for authorization
ERROR: Build step failed with exception
com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 11760BE0CD404492; S3 Extended Request ID: RXTWOTmY5hD2XuiHZPuHBfzuCs2o6C9+AkvKJrBwVV8QJV3IL847ZVe3Mg99+e3Q8bGZqpgNq0Y=), S3 Extended Request ID: RXTWOTmY5hD2XuiHZPuHBfzuCs2o6C9+AkvKJrBwVV8QJV3IL847ZVe3Mg99+e3Q8bGZqpgNq0Y=
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1632)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1304)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1058)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4365)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4312)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4306)
at com.amazonaws.services.s3.AmazonS3Client.getBucketVersioningConfiguration(AmazonS3Client.java:2240)
at com.amazonaws.services.s3.AmazonS3Client.getBucketVersioningConfiguration(AmazonS3Client.java:2225)
at Validation.checkBucketIsVersioned(Validation.java:209)
at CodeBuilder.perform(CodeBuilder.java:328)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.Build$BuildExecution.build(Build.java:206)
at hudson.model.Build$BuildExecution.doRun(Build.java:163)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1794)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Build step 'AWS CodeBuild' marked build as failure
[BFA] Scanning build for known causes...
[BFA] No failure causes found
[BFA] Done. 0s
Finished: FAILURE

IAM ROLES:

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "CodeBuildProjects",
"Effect": "Allow",
"Action": [
"codebuild:StartBuild",
"codebuild:StopBuild",
"codebuild:BatchGet*",
"codebuild:Get*",
"codebuild:List*",
"codecommit:GetBranch",
"codecommit:GetCommit",
"codecommit:GetRepository",
"codecommit:ListBranches"
],
"Resource": "arn:aws:codebuild:us-east-1:997938224961:project/JenkinsPipeline"
},
{
"Sid": "CodeBuildLogs",
"Action": [
"logs:GetLogEvents"
],
"Effect": "Allow",
"Resource": "arn:aws:logs:::log-group:/aws/codebuild/:log-stream:"
},
{
"Sid": "CodeBuildBuckets",
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:ListBucket",
"s3:PutObject",
"s3:DeleteObject"
],
"Resource": [
"arn:aws:s3:::quantopian-codebuild-staging",
"arn:aws:s3:::quantopian-codebuild-staging/*"
]
}
]
}

And I have made the configuration settings almost similar to the one provided in this document : https://aws.amazon.com/blogs/devops/simplify-your-jenkins-builds-with-aws-codebuild/

Secondly, from this I haven’t used a code commit AWS repository, I am using a GitHub repository which has established a connection with my Jenkins system.

Please do guide me on what I should do next so that I can proceed. Is there an evident error that I am making or is it not possible to create the connection with the s3 bucket considering the fact that I am leaving the credentials part of the Jenkins configuration page blank, as I have created a role for amazon using Assumeroles so that credential entry is not required.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.