Giter Club home page Giter Club logo

pravega-storage-adapters's Introduction

pravega-storage-adapters

TBD

pravega-storage-adapters's People

Contributors

andreipaduroiu avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pravega-storage-adapters's Issues

Consolidate the Jenkinsfile for the Pravega and its bindings

Problem description
Pravega uses HDFS, ExtendedS3, and NFS for tier 2 which is now named long-term-storage. The implementation for this change from binding to binding and requires different build jobs. The implementation is also coming via a separate PDP-34 and the repackaging done in the pravega-storage-adapters repository will be redone via the pull-request in pravega for issue-4907. This issue here tracks the convergence of the build automations into a consolidated template.

Problem location
The Jenkinsfile changes in the nautilus-pipelines repository.

Suggestions for an improvement
Although every repository has similar routines as building jars, images and libraries, the build files are maintained separately for say the server and tools. In this case, the storage adapters are also maintained separately from the main source code but it is in effect a part of the server. Build jobs are for the convenience of development while the script is for the template of steps for one or more code repositories. After the code and builds are stabilized for this repository, there can be a streamlining of steps so that the automation is using the same template.

Infrastructure to publish storage adapter jars

Problem description
Currently there is no infrastructure in the build scripts to publish jars on Maven/BinTray/etc

Problem location
Build scripts

Suggestions for an improvement
Build scripts need to be extended with :publish task. Note that each storage adapter project should handle publishing of itself. The main build.gradle should have only common logic if any.

setup docker file

##Problem description
This repository produces bindings which are used in container deployments using docker images. This issue tracks the setup changes required to put the binding jars in the images so that they can be deployed as enhanced Pravega images.

they should be making a new Pravega image with only additonal jars, little or no additional files if possible

##Problem location
the docker file and associated build script

##Suggestions for an improvement

until this repository goes public, we can use internal image repositories but it should switch to Pravega and Travis will need the credentials to push the images there.

move integration tests for hdfs

Problem description
HDFS and S3 have integration tests in the pravega segmentstore storage project which needs to be moved to this repository

Problem location
move
./segmentstore/server/host/src/test/java/io/pravega/segmentstore/server/host/HDFSIntegrationTest.java
./segmentstore/server/host/src/test/java/io/pravega/segmentstore/server/host/ExtendedS3IntegrationTest.java

Suggestions for an improvement
Put the integration test in the same project as the hdfs or extendeds3 but do not create a new integration project.
A new task called integration task is acceptable.
hdfs and extendeds3 can have their own build.gradle

migrate extendedS3

##Problem description
The extendeds3 binding in the Pravega main source needs to be moved to this repository. Few criteria pertaining to the changes for this purpose:

  1. they should be specific to extendeds3 in pravega main source, little or no additional files if possible
  2. they should preserve history across repositories
  3. they should include all the types of tests in this repository.
  4. they should be as much as in sync with the Pravega main source as possible
  5. they should not require to be imported elsewhere
  6. they should handle all workflows using this binding

##Problem location
replace extendeds3 files in the new repo from those in the Pravega main source

##Suggestions for an improvement

  • separation of pravega version for src,test,docker even though they may point to the same version

migrate hdfs

##Problem description
The hdfs binding in the Pravega main source needs to be moved to this repository. Few criteria pertaining to the changes for this purpose:

  1. they should be specific to hdfs in pravega main source, little or no additional files if possible
  2. they should preserve history across repositories
  3. they should include all the types of tests in this repository.
  4. they should be as much as in sync with the Pravega main source as possible
  5. they should not require to be imported elsewhere
  6. they should handle all workflows using this binding

##Problem location
replace hdfs files in the new repo from those in the Pravega main source

##Suggestions for an improvement

  • separation of pravega version for src,test,docker even though they may point to the same version

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.