Giter Club home page Giter Club logo

gate-resource's Introduction

gate-resource

Build

A generic gate resource for Concourse CI.

Allows you to model quality gates and pipeline control flow.

This resource is backed by a Git repository and wraps git-resource.

A public container build of this repo is available at ghcr.io meshcloud/gate-resource.

Contributors Welcome: This resource is new and hot off the press. We welcome your feedback and contributions!

Example

To see an example pipeline using gate-resource, head over to gate-resource-example.

Git Repository Structure

.
├── my-gate
│   ├── 1
│   └── 2
└── my-autogate
    ├── a
    └── b.autogate

note: The gate repository is currently append-only. Files are not expected to be deleted or cleaned.

The gate-resource supports two types of gates: simple-gates and auto-gates.

Simple-gates

Each folder in the repository represents a gate. Files in each gate-folder represent items that successfully passed the gate. The files may be empty or contain any metadata that you whish to track. In the example above, my-gate is a simple-gate

Auto-gates

Auto-gates are gates that automatically close depending on items passing through other gates. An auto-gate contains .autoclose items, which is a simple text file that contains dependant items, one on each line ("autoclose spec"). For example, b.autoclose depends on these two items passing through my-gate:

my-gate/2
my-gate/3
# metadata after this line
arbitrary content goes here

When all dependant items passed, the autoclose item closes and drops the .autoclose extension from its filename. Autoclose files can also contain additional metadata. To separate the autoclose spec from additional metadata, gate-resource looks for a line starting with # and stops interpreting any lines after this.

The "none" version

Unfortunately Concourse does not support emitting an empty set of versions from out. However, this is necessary for gate-resource as it may not find any autoclose items to close in a particular update. As a workaround, gate-resource emits the none version. in will no-op on encountering this version.

Since concourse detects that the none version already exists after the first time it's generated, it will only trigger a single build when an auto-gate is used with trigger: true. You can detect that the none version was fetched when no passed and metadata files were created by the get step (see below).

Source Configuration

  • git: Required. Configuration of the repository. Supports the following subset of git-resource configuration options (review the link for descriptions)
    • Required: uri, branch, private_key
    • Optional: git_config, short_ref_format,
  • gate: Optional. The gate to track.

Behavior

check: Check for changes to the gate

The repository is cloned (or pulled if already present), and any commits made to the specified gate from the given version on are checked for items passing through the gate.

note: If you want to ensure the resource triggers for every item that passed the gate, use the resource with version: every

in: Fetch an item that passed a gate

Outputs 2 files:

  • passed: Contains the name of the item that passed the gate
  • metadata: Contains the contents of whatever was in your gate item. This is useful for environment configuration settings or documenting approval workflows.

note: the git repository cloned during in is a shallow clone and does not track an upstream branch. This is not a problem when using the out step to create or update gates.

out: Pass an item through a gate

Performs one of the following actions to change the state of a gate.

Parameters

One of the following is required.

  • item_file: Path to a file containing an item to pass through the gate. Wildcards are allowed, but should match only a single item. This file may also be a .autogate file.
  • update_autoclose: Process pending autoclose items in the repository

Note: Gate puts are idempotent. When putting a new gate item or .autoclose spec, the resource checks if a matching item already passed the gate. When this is the case, out will emit the version that previously produced this item.

Creating .autoclose items

.autoclose items can be put to the gate just like regular items. However, the version that gate-resource will emmit for this item may vary:

  • when a matching target item (i.e. the filename without .autoclose extension) already exists, emits the version that produced this target item.
  • when the .autoclose item is immediately closable (all dependent items alreday passed their gates), close the item and emit the version
  • otherwise emit a none version (see above)

Development

Prerequisites

  • docker is required - version 17.06.x is tested; earlier versions may also work.

Running the tests

The tests have been embedded with the Dockerfile; ensuring that the testing environment is consistent across any docker enabled platform. When the docker image builds, the test are run inside the docker container, on failure they will stop the build.

Run the tests with the following command:

docker build -t gate-resource .

gate-resource's People

Contributors

florianow avatar henryde avatar johannesrudolph avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

gate-resource's Issues

Git error getting resource

git fetch origin "$ref" $depthflag

Seems to be causing failures when I get gates:

Identity added: /tmp/git-resource-private-key (/tmp/git-resource-private-key)
Initialized empty Git repository in /tmp/build/get/.git
error: Server does not allow request for unadvertised object [sha from version.ref]

I notice that the official git resource uses a different command:
git clone --single-branch $depthflag $uri $branchflag $destination $tagflag

Is it possible to switch that line out?

Consider adding a TTL to autoclose gates

Our CI pipeline builds feature branches using autoclose gates. It is expected that builds on feature branches fail from time to time and may be corrected by developers. This leaves autoclose gates open and "stranded".

When processing autoclose items, the gate resource should probably perform a cleanup of gates after a configurable TTL. Thinking of it, there might be value to having a TTL for gates in general (i.e. reduced git checkout size and less load on the filesystem). This might be best delegated to a "sidecar" task that runs on the gate-repo

Add possibility to "reset" autoclose gates

I'm trying to implement a use-case where I put an autoclose gate and wait for all the dependent items to pass. For the next run of my deployment pipeline, I want to re-enable the same autoclose gate and have the exact same logic. Currently it will just ignore the new autogate because the .autoclose extension has been removed and the file is already there. Instead of generating a new file every time I'd like to reuse the same autoclose gate.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.