Giter Club home page Giter Club logo

cincinnati's Introduction

Cincinnati

Cincinnati is an update protocol designed to facilitate automatic updates. It describes a particular method for representing transitions between releases of a project and allowing a client to perform automatic updates between these releases.

Quick Start

Prepare custom environment variables

# Please change these accordingly
export CINCINNATI_REGISTRY="https://quay.io"
export CINCINNATI_REPO="redhat/openshift-cincinnati-test-public-manual"

Executables on the build host

cargo run --package graph-builder -- --service.address 0.0.0.0 --upstream.registry.url "${CINCINNATI_REGISTRY}" --upstream.registry.repository "${CINCINNATI_REPO}" &
cargo run --package policy-engine -- --service.address 0.0.0.0 &
 curl --verbose --header 'Accept:application/json' http://localhost:8081/graph\?channel=a
{
    "nodes":
    [{
        "version":"0.0.0",
        "payload":"quay.io/redhat/openshift-cincinnati-test-public-manual@sha256:a264db3ac5288c9903dc3db269fca03a0b122fe4af80b57fc5087b329995013d",
        "metadata":{
            "io.openshift.upgrades.graph.release.channels":"a",
            "io.openshift.upgrades.graph.release.manifestref":"sha256:a264db3ac5288c9903dc3db269fca03a0b122fe4af80b57fc5087b329995013d"
        }
    },
    {
        "version":"0.0.1",
        "payload":"quay.io/redhat/openshift-cincinnati-test-public-manual@sha256:73df5efa869eaf57d4125f7655e05e1a72b59d05e55fea06d3701ea5b59234ff",
        "metadata":{
            "io.openshift.upgrades.graph.release.manifestref":"sha256:73df5efa869eaf57d4125f7655e05e1a72b59d05e55fea06d3701ea5b59234ff",
            "io.openshift.upgrades.graph.release.channels":"a",
            "kind":"test"
        }
    }],
    "edges":[[0,1]]
}

Note: the default configuration of the policy-engine requires the channel parameter to be present in each request.

Tests

There are several ways of testing various parts of the Cincinnati stack.

Offline

Language-Level

The language-level tests can be run using cargo --test in the repository's root directory:

cargo test

Online

The online tests for the graph-builder depend on a curated set of repositories to be available on quay.io in the redhat organization. The build instructions for (re-)populating the repositories are available at graph-builder/tests/images/build-n-push.sh. The script must run be run from its directory to function:

cd graph-builder/tests/images
./build-n-push.sh test-*

Language-Level

The graph-builder package currently has network dependent tests which gated behind the feature test-net and test-net-private. The latter requires setting the environment variable CINCINNATI_TEST_CREDENTIALS_PATH which is equivalent to graph-builder's --credentials-path.

Assuming you have access to images under the quay.io/redhat organization, and have an appropriate $HOME/.docker/config.json in place, this might work on your machine:

cd graph-builder
export CINCINNATI_TEST_CREDENTIALS_PATH="$HOME/.docker/config.json"
cargo test --features test-net,test-net-private

CI/CD

The dist/ directory contains various CI/CD related files.

Openshift Dev

  • Uses dist/Dockerfile.build/Dockerfile as the build container image
  • Run the following scripts on PR
    • dist/prow_yaml_lint.sh
    • dist/prow_rustfmt.sh
    • dist/cargo_test.sh

For details please see github.com/openshift/release/(...)/openshift-cincinnati-master.yaml.

App-SRE

  • Uses dist/Dockerfile.build/Dockerfile as the build container image
  • Runs dist/build_deploy.sh for successful merges to the master branch and pushes the result to the staging environment (URL is not yet publicly available)

Development

For developing Cincinnati refer to the developer documentation document.

Updating the Plugin-Interface Scheme

The interface for external plugins is defined as a Protobuf v3 scheme in the file [cincinnati/src/plugins/interface.proto][./cincinnati/src/plugins/interface.proto]. In order to regenerate the files the cincinnati crate must be built with the codegen-protoc feature:

cd cincinnati
cargo build --features=codegen-protoc

The CI/CD system doesn't do this and it relies on the generated code being committed to the repository; please do so after generating new code!

cincinnati's People

Contributors

aditya-konarde avatar arilivigni avatar cblecker avatar crawford avatar dependabot[bot] avatar erdii avatar jeckersb avatar jottofar avatar jupierce avatar lalatendumohanty avatar lisa avatar lucab avatar maorfr avatar openshift-ci[bot] avatar openshift-merge-bot[bot] avatar openshift-merge-robot avatar petr-muller avatar pratikmahajan avatar russellb avatar sdodson avatar smarterclayton avatar steveej avatar stevekuznetsov avatar thiagoalessio avatar vrutkovs avatar wking avatar yprokule avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cincinnati's Issues

Build failure with plugins static thread safety message

Pulled most recent code: 7f68cf5

cargo clean ; cargo build
error[E0277]: *mut (dyn std::ops::Fn() + 'static) cannot be shared between threads safely
--> graph-builder/src/plugins/github_openshift_secondary_metadata_scraper/plugin.rs:261:80

message is

*mut (dyn std::ops::Fn() + 'static) cannot be shared between threads safely

README Instructions fail At running step due to --address parameter

Build on Fedora 30:

Compiling policy-engine v0.1.0 (/home/ayoung/devel/cincinnati/policy-engine)
Finished dev [unoptimized + debuginfo] target(s) in 4m 27s
Running target/debug/policy-engine --address 0.0.0.0
error: Found argument '--address' which wasn't expected, or isn't valid in this context

Could this be an issue with the version of Rust used to build

Steps to repro:

git clone....
cd cincinnati/
export CINCINNATI_REGISTRY="https://quay.io"
cargo run --package graph-builder -- --address 0.0.0.0 --registry "${CINCINNATI_REGISTRY}" --repository "${CINCINNATI_REPO}" &
cargo run --package policy-engine -- --address 0.0.0.0

Its the last command that fails. I got that from the README.

Example run for graph-builder errors out

$ export CINCINNATI_REPO="redhat/openshift-cincinnati-test-public-manual"
$ export CINCINNATI_REGISTRY="https://quay.io"
$ cargo run --package graph-builder -- --service.address 0.0.0.0 --upstream.registry.url "${CINCINNATI_REGISTRY}" --upstream.registry.repository "${CINCINNATI_REPO}"
Finished dev [unoptimized + debuginfo] target(s) in 0.16s
Running target/debug/graph-builder --service.address 0.0.0.0 --upstream.registry.url 'https://quay.io' --upstream.registry.repository redhat/openshift-cincinnati-test-public-manual
Error: Os { code: 2, kind: NotFound, message: "No such file or directory" }

could not open '""'

design/cincinnati: clarify some details on the graph endpoint

A spec clarification request to @crawford on https://github.com/openshift/cincinnati/blob/95db15a823b231636c7a3aaeef25b6c22e6c9a8b/docs/design/cincinnati.md#graph-api.

The current implementation in cincinnati is stricter than the design spec, and these are the points that I'd like to have explicitly noted regarding the JSON API of the graph endpoint:

  • is version a freeform string or a semver label?
  • is a "" allowed as a version value?
  • must a version value be unique in the "nodes" array?
  • is the payload field required and is an empty-string allowed there?
  • is the client supposed to pass additional parameters somehow (e.g. as query parameters)?
  • is the edges "empty object for future use" still a thing? (it isn't in example nor in deserializer code)

Request Add CORS support

Hi,

Our team want to use the api /api/upgrades_info/v1/graph in a frontend app.
but the API not support called by JavaScript from other domain.
Is that possible to support it ?

I think it should be easy to support by add a middleware https://docs.rs/actix-cors/0.5.4/actix_cors/

Then, the frontend developer could use the api with following code

fetch("https://api.openshift.com/api/upgrades_info/v1/graph?channel=fast-4.5")
    .then(res => res.json())
    .then((result)=> console.log(result))

Graph tests fail with http://offline.url.test

[2019-12-02T13:57:42Z ERROR cincinnati::plugins::internal::cincinnati_graph_fetch] error fetching graph: failed to fetch upstream graph: http://offline.url.test/: error trying to connect: failed to lookup address information: Name or service not known
[2019-12-02T13:57:42Z ERROR policy_engine::graph] Error serving request with parameters '""': failed to fetch upstream graph: http://offline.url.test/: error trying to connect: failed to lookup address information: Name or service not known
[2019-12-02T13:57:42Z ERROR actix_http::response] Internal Server Error: FailedUpstreamFetch("http://offline.url.test/: error trying to connect: failed to lookup address information: Name or service not known")

[2019-12-02T13:57:42Z ERROR policy_engine::graph] Error serving request with parameters '"channel=invalid:channel"': invalid client parameters: channel 'invalid:channel' does not match regex '^[0-9a-z-.]+$'
[2019-12-02T13:57:42Z ERROR policy_engine::graph] Error serving request with parameters '"channel=invalid=channel"': invalid client parameters: channel 'invalid=channel' does not match regex '^[0-9a-z-.]+$'
test graph::tests::webservice_graph_json_response ... ok
[2019-12-02T13:57:42Z ERROR policy_engine::graph] Error serving request with parameters '"channel=':'"': invalid client parameters: channel '':'' does not match regex '^[0-9a-z-.]+$'

[RFE] Expose latest versions available

Currently, to check the latest OCP version available this is needed:

curl -sH 'Accept: application/json' "https://api.openshift.com/api/upgrades_info/v1/graph?channel=stable-4.4&amd64" | jq -r '.nodes|map(.version)|.[]' | sort -V | tail -1

You need to know the minor version (4.4) and the channel (stable) beforehand.

It would be nice to expose that value in an API endpoint such as https://api.openshift.com/api/latest or https://api.openshift.com/api/stable-4.4/latest or https://api.openshift.com/api/fast-4.4/latest, etc.

I'm not sure if this RFE belongs here but just in case.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.