codeship / scripts Goto Github PK

View Code? Open in Web Editor NEW

217.0 31.0 174.0 407 KB

Scripts for use on Codeship Basic

Home Page: https://codeship.com

License: MIT License

Shell 95.70% Ruby 4.30%

codeship shell bash ci cd hacktoberfest

scripts's Introduction

Codeship Scripts

A public collection of useful scripts for use on Codeship and similar tools.

E.g. scripts to install specific versions of software not included by default on the build VMs. Deployment scripts for external services to customize to your needs, configure caching or trigger notifications. And other scripts if you want to have them included. ;)

Using the scripts

Each script includes a comment at the beginning detailing how to configure and use the script in your builds. Those comments, for example, look like:

# Add at least the following environment variables to your project configuration
# (otherwise the defaults below will be used).
# * FIREFOX_VERSION
#
# Include in your builds via
# \curl -sSL https://raw.githubusercontent.com/codeship/scripts/master/packages/firefox.sh | bash -s

It is important that you follow those instructions and not simply copy / paste the scripts into your Codeship project configuration.

Most scripts include set -e to make the script fail as soon as a subcommand returns a non-zero exit code (which indicates failure).

This is fine if you call the script as documented above (via the curl command). But it will have side effects if you copy the commands into your project settings on Codeship as any command that fails after the set -e will terminate your build. You'll get a SYSTEM build status, but no further error message or log output. You probably don't want this. :)

Please see the post at https://docs.cloudbees.com/docs/cloudbees-codeship/latest/basic-builds-and-configuration/scripts#_set_e for more information on how set -e works and why this causes issues if you include it in your setup or test commands.

Code of Conduct

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.

scripts's People

Contributors

Stargazers

Watchers

Forkers

hughker jrschumacher jcolinmurray lotkao joebalancio stabenfeldt soldotno degfra vikrantpogula arnezsng tripliks eptis heberuriegas dschniepp jrdnull kenegozi d33d33 topfloortech iuridiniz megha-n-bodke sohocoke basiszwo steffy29 sfcgeorge story-stream molodecc bruce-genhot foldingbeauty creatubbles tomk32 kevy sebastianhaas nagyistge degetel-jtant getconversio bfcoder vayu-technology mmarod woahdae inplay stackriot-labs gauravgango elmina90 kleopatra999 brianjking miketheman cipas jaaaco oakworks mkscala studitemps jheuing matusmarcin fschaeffler joelind gospotcheck joshmcarthur grisgruis linearregression surendrans michaeltien intercom michaelhashimoto zhangmingyu951 klclee xludx unixmonkey ashishkulk kazw williamweckl alazaro hamedoukhay dylansm bsy juanjo7g wagmiwiz davidwter noamkush barkinet enterstudio enterstudios kuzmiigo stefb965 antistatique chenchaobing xpilot jackass0528 rajgomase24 davecarlson cai980273879 kgajowy smayzes iziren sauy7 dondyabla tlmak0 squaretwo itechnology-rs acasajus rsazima

scripts's Issues

Emails through 3rd party service

automated emails through a 3rd party service when events happen inside Codeship.

Zabix Script

create an API plugin to trigger Zabbix alerts for builds

Splunk Script

Send Codeship metrics to Splunk. (http://www.splunk.com/)

403 when installing phantomjs

Hi I'm installing Phantomjs using the \curl -sSL https://raw.githubusercontent.com/codeship/scripts/master/packages/phantomjs.sh | bash -s command. However this stopped working this morning and now it fails with a 403 Forbidden.

--2016-03-24 06:05:48--  https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-1.9.8-linux-x86_64.tar.bz2
Resolving bitbucket.org (bitbucket.org)... 104.192.143.2, 104.192.143.3, 104.192.143.1
Connecting to bitbucket.org (bitbucket.org)|104.192.143.2|:443... connected.
HTTP request sent, awaiting response... 302 FOUND
Location: https://bbuseruploads.s3.amazonaws.com/ariya/phantomjs/downloads/phantomjs-1.9.8-linux-x86_64.tar.bz2?Signature=8sx40a%2F1nAKqKnPVRVAqmnwrc%2Fs%3D&Expires=1458799260&AWSAccessKeyId=AKIAIWY5XSVPZPDQYRQQ&versionId=null&response-content-disposition=attachment%3B%20filename%3D%22phantomjs-1.9.8-linux-x86_64.tar.bz2%22 [following]
--2016-03-24 06:05:48--  https://bbuseruploads.s3.amazonaws.com/ariya/phantomjs/downloads/phantomjs-1.9.8-linux-x86_64.tar.bz2?Signature=8sx40a%2F1nAKqKnPVRVAqmnwrc%2Fs%3D&Expires=1458799260&AWSAccessKeyId=AKIAIWY5XSVPZPDQYRQQ&versionId=null&response-content-disposition=attachment%3B%20filename%3D%22phantomjs-1.9.8-linux-x86_64.tar.bz2%22
Resolving bbuseruploads.s3.amazonaws.com (bbuseruploads.s3.amazonaws.com)... 54.231.9.73
Connecting to bbuseruploads.s3.amazonaws.com (bbuseruploads.s3.amazonaws.com)|54.231.9.73|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2016-03-24 06:05:48 ERROR 403: Forbidden.

When I run the script locally everything works:

➜  cache \curl -sSL https://raw.githubusercontent.com/codeship/scripts/master/packages/phantomjs.sh | bash -s
--2016-03-24 07:13:06--  https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-1.9.8-linux-x86_64.tar.bz2
Resolving bitbucket.org... 104.192.143.2, 104.192.143.1, 104.192.143.3
Connecting to bitbucket.org|104.192.143.2|:443... connected.
HTTP request sent, awaiting response... 302 FOUND
Location: https://bbuseruploads.s3.amazonaws.com/ariya/phantomjs/downloads/phantomjs-1.9.8-linux-x86_64.tar.bz2?Signature=2ipt02Vs%2BP1mCqGh9odQ3ucf5RI%3D&Expires=1458801131&AWSAccessKeyId=AKIAIWY5XSVPZPDQYRQQ&versionId=null&response-content-disposition=attachment%3B%20filename%3D%22phantomjs-1.9.8-linux-x86_64.tar.bz2%22 [following]
--2016-03-24 07:13:06--  https://bbuseruploads.s3.amazonaws.com/ariya/phantomjs/downloads/phantomjs-1.9.8-linux-x86_64.tar.bz2?Signature=2ipt02Vs%2BP1mCqGh9odQ3ucf5RI%3D&Expires=1458801131&AWSAccessKeyId=AKIAIWY5XSVPZPDQYRQQ&versionId=null&response-content-disposition=attachment%3B%20filename%3D%22phantomjs-1.9.8-linux-x86_64.tar.bz2%22
Resolving bbuseruploads.s3.amazonaws.com... 54.231.16.169
Connecting to bbuseruploads.s3.amazonaws.com|54.231.16.169|:443... connected.
HTTP request sent, awaiting response... 416 Requested Range Not Satisfiable

    The file is already fully retrieved; nothing to do.

Usage:
  List:    tar -tf <archive-filename>
  Extract: tar -xf <archive-filename>
  Create:  tar -cf <archive-filename> [filenames...]
  Help:    tar --help

Bugsnag Script

Notify Bugsnag about new deployment

Airbrake Script

Notify Airbrake (https://airbrake.io/) about new deployments.

Mongodb script should work for versions 5 and higher

Due to https://github.com/codeship/scripts/blob/master/packages/mongodb.sh#L24 version 5 is tried to be downloaded for ubuntu 14, but it is not available there, only for 18

Hipchat Script

Automated messages for events in Codeship.

Setup script for BrowserStack Local

I think there should be a setup script for BrowserStack Local in the packages folder.

I will be happy to submit a PR for this.

Sentry Script

Notify Sentry (https://getsentry.com/welcome/) about new deployment

Coveralls Script

Allow https://coveralls.io/ to cover code quality running on Codeship.

CodeCov.io Script

Monitor Codeship builds via https://codecov.io/

Uploading artifacts to google cloud compute engine vm

I'm looking into https://github.com/codeship/scripts/blob/google-cloud-sdk/packages/google_cloud_sdk.sh

All these steps worked for me. The next step for me is to upload the artifact and run some commands via ssh. I tried following

gcloud compute copy-files --quiet :/tmp/ --zone us-central1-a
It tries to generate the ssh key and waits for passphrase
gcutil push /tmp
Resp is same as 1

3 .rm -rf ~/.ssh/google_compute_engine*
/usr/bin/ssh-keygen -q -f ~/.ssh/google_compute_engine -N ""
gcloud compute config-ssh --quiet
scp :/tmp

SCP waits forever..

How can I upload artifact and run ssh commands?

Mysql 8.0 script does not handle 206 Partial Content when downloading mysql package

As part of our test setup commands, we run curl -sSL https://raw.githubusercontent.com/codeship/scripts/master/packages/mysql-8.0.sh | bash -s

We are seeing intermittent failures when the mysql CDN responds with a 206 Partial Content and the wget (despite the --continue flag) doesn't complete the download in the default 20 retries. Here's a gist with the output

Let me know if I can provide any additional information!

Kissmetrics Scripts

Allow for Kissmetrics annotations as soon as deployment is done via Codeship

Spelling Error

scripts/languages/go.sh
line: 28

check the correct version is yused

Should be:

check the correct version is used

Customer.io Script

Allow for automated Messages as soon as deployment is done via Codeship.

Librato Script

Provide a shellscript to add deployment annotations to Librato according to their documentation on http://support.metrics.librato.com/knowledgebase/articles/122410-adding-annotations-to-graphs

All required configuration shall be done via environment variables and there should be default values where it makes sense. Include at least a link to the build on Codeship and set the description to the commit message.

Reach out to [email protected] if you have any questions!

Add support for Zeit's now.sh

We use https://now.sh for deploying all our servers, and it'd be great to get a pre-built setup into Codeship!

/cc @leo

eb_deployment_check is missing

eb_deployment_check is used in the script for elastic beanstalk deployment, yet it is not published here.

update node and npm version .

when i triggered my react project , i saw a thousand warnings saying that node version is required >=5 but existing node version is 0.10.
being older version of node . it was not able to install newer packages with new node support .

Datadog Script

Send Codeship metrics to Datadog.

fatal: --unshallow on a complete repository does not make sense

when running this:
https://github.com/codeship/scripts/blob/master/deployments/git_push.sh#L17

Slack Script

Provide a shell script to post custom message to a Slack channel as explained on https://api.slack.com/incoming-webhooks.

Configuration should be done via environment variables, bot name and icon should be configurable as well and default to Codeship Bot & http://bit.ly/1E2uEcn

Automate Elasticsearch 5.x Java Dependency

The elasticsearch.sh script mentions that JDK 8 is necessary for elasticsearch v5, and that steps should be taken to switch to that version of the JDK before launching elasticsearch. However, why not just automatically run jdk_switcher use oraclejdk8 in the check for ES v5?

# Make sure to use Oracle JDK 8 for Elasticsearch 5.x run the following commands in your script.
# source $HOME/bin/jdk/jdk_switcher
# jdk_switcher home oraclejdk8
# jdk_switcher use oraclejdk8
if [ ${ELASTICSEARCH_VERSION:0:1} -eq 5 ]
then
  jdk_switcher use oraclejdk8  # <--- Why not automate this?
  ELASTICSEARCH_DL_URL="https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-${ELASTICSEARCH_VERSION}.tar.gz"
elif [ ${ELASTICSEARCH_VERSION:0:1} -eq 2 ]
#...

Perhaps the complication is switching back to the original JDK version that was in play after the elasticsearch script is finished?

Rollbar Script

Notify rollbar about new deployments.

Graphite Script

Send Codeship metrics to Graphite (http://graphite.wikidot.com/)

Code Climate Script

allow for Code Climate to monitor builds on Codeship.

Suggest excluding tmp directory in zip file

In scripts/deployments/elastic_beanstalk.sh the script packages the tmp directory for Rails projects into the .zip that is pushed to EB. This can deploy test run artifacts and cache fragments to production if they are not completely cleaned after the test run, which I don't think is the default test behavior. It would be wise to exclude this directory from being uploaded.

Chatgrape Script

Automated messages when events happen inside Codeship.

Dynamodb setup scripts consistently runs to error

As part of our setup step, we run \curl -sSL https://raw.githubusercontent.com/codeship/scripts/master/packages/dynamodb.sh | bash -s

It was running fine for a long time, but today we started getting this output and our build is consistently failing:

--2020-08-26 11:43:00--  https://s3-us-west-2.amazonaws.com/dynamodb-local/dynamodb_local_latest.tar.gz
Resolving s3-us-west-2.amazonaws.com (s3-us-west-2.amazonaws.com)... 52.218.249.88
Connecting to s3-us-west-2.amazonaws.com (s3-us-west-2.amazonaws.com)|52.218.249.88|:443... connected.
HTTP request sent, awaiting response... 416 Requested Range Not Satisfiable

The file is already fully retrieved; nothing to do.


gzip: stdin: decompression OK, trailing garbage ignored
tar: Child returned status 2
tar: Error is not recoverable: exiting now

Does anybody know why this is happening and how could we make it work again?

[MongoDB] Faster CI

MongoDB gives some tips to faster its usage in a CI: Webinar: Continuous Integration with Node.JS and MongoDB

I'll try those three parameters and I'll submit a PR. I don't know if it's really efficient, maybe a benchmark could tell us.

Steps name: DOES break on '-' | error creating network during run step: API error (403): Pool overlaps with other one on this address space

I found a bug within the codeship-steps file handling.

If you run something like this:

# ./codeship-steps.yml

...
- name: swarm deploy 1
  service: ssh-helper
  command: /bin/bash -c "echo hello-world"
- name: swarm-deploy-2
  service: ssh-helper
  command: /bin/bash -c "echo hello-world"
...

The step with the name "swarm deploy 1" will be executed as expected.
But step with the name "swarm-deploy-2" will raise an error message like this:

(step: swarm-deploy-2) error ✗
(step: swarm-deploy-2) error creating network during run step: API error (403): Pool overlaps with other one on this address space

It should be clear from the docs that '-' are breaking for step names.
Better '-' should be allowed within step names.
For '-' and any other unsupported character codeship/jet should raise a proper error message.

Solved for me by renaming the step.

Flowdock Script

Provide a script to send custom messages to a specific flows and include default messages for standard events, like tests started, deployment started, ...

[Elixir] Broken build because of IP out of range fetching Elixir.

Running this command:

source /dev/stdin <<< "$(curl -sSL https://raw.githubusercontent.com/codeship/scripts/master/languages/elixir.sh)"

I get this in the Codeship console:

--2017-06-26 19:55:45--  https://s3.amazonaws.com/s3.hex.pm/builds/elixir/v1.4.2.zip
Resolving s3.amazonaws.com (s3.amazonaws.com)... 52.216.65.99
Connecting to s3.amazonaws.com (s3.amazonaws.com)|52.216.65.99|:443... connected.
HTTP request sent, awaiting response... 416 Requested Range Not Satisfiable

The file is already fully retrieved; nothing to do.

error [/home/rof/cache/elixir-v1.4.2.zip]:  start of central directory not found;
zipfile corrupt.
(please check that you have transferred or created the zipfile in the
appropriate BINARY mode and that you have compiled UnZip properly)
bash: elixir: command not found

Elixir script - Mix.Config.LoadError) could not load config ../../config/config.exs

Error when using Elixir script:

** (Mix.Config.LoadError) could not load config ../../config/config.exs
** (File.Error) could not read file "../../config/config.exs": no such file or directory
(elixir) lib/file.ex:244: File.read!/1
(mix) lib/mix/config.ex:180: Mix.Config.read!/2
(mix) lib/mix/tasks/loadconfig.ex:36: Mix.Tasks.Loadconfig.load/1
(mix) lib/mix/tasks/loadconfig.ex:27: Mix.Tasks.Loadconfig.run/1
(mix) lib/mix/task.ex:294: Mix.Task.run_task/3
(mix) lib/mix/cli.ex:57: Mix.CLI.run_task/2

Install a custom version of PhantomJS... 403 Forbidden

The PhantomJS custom version install script is failing with the default PHANTOMJS_HOST environment variable which is set to https://bitbucket.org/ariya/phantomjs/downloads.

https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2 302/redirects to S3 with an expiration token, but the expiration token is already expired once the build sends the request.

--2016-11-14 14:19:33--  https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2
Resolving bitbucket.org (bitbucket.org)... 104.192.143.2, 104.192.143.3, 104.192.143.1, ...
Connecting to bitbucket.org (bitbucket.org)|104.192.143.2|:443... connected.
HTTP request sent, awaiting response... 302 FOUND
Location: https://bbuseruploads.s3.amazonaws.com/fd96ed93-2b32-46a7-9d2b-ecbc0988516a/downloads/396e7977-71fd-4592-8723-495ca4cfa7cc/phantomjs-2.1.1-linux-x86_64.tar.bz2?Signature=8XR8YEDFFVowjdT7Ry3S2ZubZ3Y%3D&Expires=1479132825&AWSAccessKeyId=AKIAIVFPT2YJYYZY3H4A&versionId=null&response-content-disposition=attachment%3B%20filename%3D%22phantomjs-2.1.1-linux-x86_64.tar.bz2%22 [following]
--2016-11-14 14:19:33--  https://bbuseruploads.s3.amazonaws.com/fd96ed93-2b32-46a7-9d2b-ecbc0988516a/downloads/396e7977-71fd-4592-8723-495ca4cfa7cc/phantomjs-2.1.1-linux-x86_64.tar.bz2?Signature=8XR8YEDFFVowjdT7Ry3S2ZubZ3Y%3D&Expires=1479132825&AWSAccessKeyId=AKIAIVFPT2YJYYZY3H4A&versionId=null&response-content-disposition=attachment%3B%20filename%3D%22phantomjs-2.1.1-linux-x86_64.tar.bz2%22
Resolving bbuseruploads.s3.amazonaws.com (bbuseruploads.s3.amazonaws.com)... 52.216.0.160
Connecting to bbuseruploads.s3.amazonaws.com (bbuseruploads.s3.amazonaws.com)|52.216.0.160|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2016-11-14 14:19:33 ERROR 403: Forbidden.

<Error>
<Code>AccessDenied</Code>
<Message>Request has expired</Message>
<Expires>2016-11-14T14:13:45Z</Expires>
<ServerTime>2016-11-14T14:23:41Z</ServerTime>
<RequestId>CCBE9E0690696666</RequestId>
<HostId>
FtiV4Gca58G/U94B0MjpGf/AT3twcz6avQAA1Uo4EexmA1CiWhKlkjQAMICjPCask7fsJYBy5uk=
</HostId>
</Error>

Using the optional export PHANTOMJS_HOST="https://s3.amazonaws.com/codeship-packages" works!

--2016-11-14 14:39:43--  https://s3.amazonaws.com/codeship-packages/phantomjs-2.1.1-linux-x86_64.tar.bz2
Resolving s3.amazonaws.com (s3.amazonaws.com)... 52.216.1.83
Connecting to s3.amazonaws.com (s3.amazonaws.com)|52.216.1.83|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 23415665 (22M) [application/x-bzip2]
Saving to: /home/rof/cache/phantomjs-2.1.1-linux-x86_64.tar.bz2

100%[======================================>] 23,415,665  74.8MB/s   in 0.3s   

2016-11-14 14:39:43 (74.8 MB/s) - /home/rof/cache/phantomjs-2.1.1-linux-x86_64.tar.bz2 saved [23415665/23415665]

Maybe export PHANTOMJS_HOST="https://s3.amazonaws.com/codeship-packages" should be the default? Thanks!

Do not include line breaks or carriage returns. These will cause errors or fail to report.

Do not include a semicolon ; in the character string. Semicolons denote the end of a line in the CLI and your command will be truncated.

which can be included in the commit message and other fields as well.

Pingdom Script

Send Codeship metrics to https://www.pingdom.com/

Python improvements

We're currently making use of the master branch for pyenv in our languages/python.sh script.

We should adapt the script and make the following changes

Switch to using the releases available at https://github.com/yyuu/pyenv/releases
Check if the pyenv folder is already available and don't download & reinstall it if it is. (This would allow somebody to test multiple Python versions serially without having to remove the folder each time.)

Elixir - occasional corrupt ZIP download

Occasionally, we will see the following:

--2017-06-12 18:33:46--  https://s3.amazonaws.com/s3.hex.pm/builds/elixir/v1.4.zip
Resolving s3.amazonaws.com (s3.amazonaws.com)... 52.216.16.195
Connecting to s3.amazonaws.com (s3.amazonaws.com)|52.216.16.195|:443... connected.
HTTP request sent, awaiting response... 416 Requested Range Not Satisfiable

The file is already fully retrieved; nothing to do.

error [/home/rof/cache/elixir-v1.4.zip]:  start of central directory not found;
zipfile corrupt.
(please check that you have transferred or created the zipfile in the
appropriate BINARY mode and that you have compiled UnZip properly)
bash: elixir: command not found

The elixir.sh script won't attempt to re-download the corrupt file until the Cache is cleared.