Giter Club home page Giter Club logo

online-archivist's Introduction

OpenShift Online Archivist

The Archivist is an application for monitoring the capacity of an OpenShift cluster, and archiving dormant or least active projects. It requires a running instance of Heptio Ark which performs the actual archival/unarchival for a specific namespace. (including all cluster resources, and PV snapshots) Ansible roles are included for deploying both applications in tandem.

Current Status

A cluster monitoring loop is implemented in the Archivist and is successfully submitting requests to archive dormant projects to Ark.

In future we expect to reserve namespaces for a given user

A cluster monitoring loop is implemented, as well as a good start on exporting and importing cluster objects.

S3 upload, unarchival, and the archival reconcile loop are still pending.

Unarchival will initially only support unarchiving to the same cluster and namespace. Provisions will be made to reserve project names while a project is archived. Eventually we do aim to be able to unarchive to another cluster and/or namespace.

Development

Prerequisites

Install golang 1.7 and check this project out in the standard GOPATH. (i.e. ~/go/src/github.com/openshift/online-archivist)

Adding $GOPATH/bin to your $PATH is recommended.

Compile and Run Locally

This method will run the archivist as whatever oc user is currently logged in. This works particularly well with things like minishift, and oc cluster up. In both cases you will want to be logged in as system:admin.

oc cluster up
oc login system:admin

Compile and run:

make build

# Run the cluster monitor component:
$ archivist monitor --config devel.cfg

Running Tests

Unit tests:

$ make test

Integration tests (launches an OpenShift master and etcd internally to work against):

$ make test-integration

Deploy To an OpenShift Cluster

TODO: An ansible role is in progress that should allow deploying to any OpenShift cluster. This role will build the application from this git repository using OpenShift itself.

Ansible is provided to deploy both the Archivist by building from git source, as well as Ark.

  1. Validate settings in ansible/vars/
  2. Create an Ansible inventory matching the standard openshift-ansible format. (as created for online team devenvs with aws-launcher, etc)
  3. ansible-playbook -i hosts ansible/archivist.yml

Manual Unarchival

Until we have API calls to unarchive you can manually initiate one:

  1. 'exec' into the running ark pod (in heptio-ark namespace)
  2. Lookup the backup for username you want to unarchive: /ark backup get -l "openshift.io/requester=username"
  3. Trigger an Ark restore: /ark restore create [backupname]

online-archivist's People

Contributors

dgoodwin avatar vero-tess avatar markturansky avatar damemi avatar tiwillia avatar

Stargazers

Felipe Monteiro avatar chakra j avatar  avatar Prashant Shahi avatar Shubham Minglani avatar

Watchers

Hemant Kumar avatar Brenton Leanhardt avatar Matt Hicks avatar Nick Stielau avatar Glenn West avatar  avatar Mrunal Patel avatar Matthew Barnes avatar Eric Himmelreich avatar Aniket Bhat avatar Iván Chavero avatar  avatar George Adams avatar Tomofumi Hayashi avatar Fabiano Franz avatar Shane Bryzak avatar Masatoshi Hayashi avatar Ian Main avatar Christopher Collins avatar Lukáš Vlček avatar Heiko W. Rupp avatar Liz Blanchard avatar W. Trevor King avatar Mark Lamourine avatar Paul Harvey avatar matthew carleton avatar Grzegorz Grzybek avatar Thomas Heute avatar Filip Brychta avatar Scott M Stark avatar Martin Gencur avatar Edmund Ochieng avatar Gerd Oberlechner avatar Paul Morie avatar Gaurav Chaturvedi avatar Simon Reber avatar Pavol Pitonak avatar Amruta Chitnis avatar RickJWagner avatar Brian Stinson avatar Van Halbert avatar Tafhim Ul Islam avatar Bill DeCoste avatar Kevin Conner avatar Dale Bewley avatar Tomas Dabašinskas avatar Ricardo Martinelli de Oliveira avatar Tony Campbell avatar James Cloos avatar tanaka_733 avatar Mike McGrath avatar Jason Shepherd avatar Udaya Satya Sai Pradeep Amarapalli avatar Juergen Hoffmann avatar Jiaju Zhang avatar Colin Hom avatar James Harrington avatar Ben Miller avatar Matt Clark avatar Andrew Block avatar Abhishek Gupta avatar Radim Hopp avatar Alberto García Lamela avatar Sam Batschelet avatar Ravi Sankar Penta avatar Tzu-Mainn Chen avatar Chris Houseknecht avatar Chris Murphy avatar Eric Rich avatar Honza Horak avatar Miheer Salunke avatar Ilya Buziuk avatar Adam Saleh avatar Mario Abajo Duran avatar Lukas Bednar avatar Matous Jobanek avatar Adam Kaplan avatar Sajeel Irkal avatar Jan-Hendrik Boll avatar Xiaoli Tian avatar Jakub Hadvig avatar Nikhil Mone avatar Ben Pritchett avatar OpenShift Bot avatar Chris Long avatar Karim Boumedhel avatar Chris Morgan avatar Tim Moran avatar John Keck avatar Daniel J Walsh avatar Miciah Dashiel Butler Masters avatar Ameya Sathe avatar Stephen Kitt avatar Alice Hubenko avatar Bo Meng avatar Wei Sun avatar Jooho Lee avatar Chris Callegari avatar Suresh Gaikwad avatar Radek Vokal avatar

online-archivist's Issues

Document a non Ansible way to deploy on OpenShift

The current way to deploy on OpenShift is listed here - https://github.com/openshift/online-archivist#deploy-to-an-openshift-cluster

This README does not cover which resources will be created, and what access levels are required for archivist to run.
Also, what gets deployed using Ansible should be covered, for when I do not desire to deploy using Ansible but just via an OpenShift template, on let's say minishift, to try it out. The template is available at ansible/roles/oso_archivist/files/archivist-template.yaml, and should be documented.

Add examples on how to use archivist

The README lists commands on how to use archivist 👍. It would be nice to have dedicated docs with proper examples on how to use this well.

/kind documentation
/priority important-longterm

Add `archivist install` command

I'm trying to build something similar to online-archivist for https://openshift.io/. It would super cool to install archivist through just a command (similar to ark install, vmware-tanzu/velero#52).

We can have the user fill in the necessary options at the beginning. The docs are great for understanding what's going on step-by-step but it takes a lot of copy-pasting to get a working install going. xref #13

Thoughts on this?

/cc @dgoodwin @vero-tess @damemi
(cc'ing you all here because you have contributed to this repo, not sure who maintains it now 😄)

Use SharedInfomerFactory

Tracks this TODO:

// TODO: deads2k suggests switching to SharedInformerFactory from
// https://github.com/openshift/origin/blob/master/pkg/build/generated/informers/internalversion/factory.go#L29
// Then using .Build().Builds().AddResourceEventHandler()

Creating a separate issue here because I think this is an important fix.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.