Giter Club home page Giter Club logo

vertx-pairtree's Introduction

vertx-pairtree  Build Status Codacy Badge Known Vulnerabilities Maven Javadocs

A file system or S3 backed Pairtree library for use with the Vert.x tool-kit. To use it, include its Jar file in your project's classpath.

Getting Started

To check out and build the project, type on the command line:

git clone https://github.com/ksclarke/vertx-pairtree.git
cd vertx-pairtree
mvn install

This will run the build and the unit tests for the file system back end. Right now, the only tests for the S3 back-end are integration tests. To run a build with them, you will need to put the following properties in your settings.xml file and run the build with the s3it profile:

<vertx.s3.bucket>YOUR_S3_BUCKET_NAME</vertx.s3.bucket>
<vertx.s3.access_key>YOUR_ACCESS_KEY</vertx.s3.access_key>
<vertx.s3.secret_key>YOUR_SECRET_KEY</vertx.s3.secret_key>

Or, you can supply the required properties on the command line (with the same s3it profile) when you build the project:

mvn install -Ps3it -Dvertx.s3.bucket=YOUR_S3_BUCKET_NAME \
  -Dvertx.s3.access_key=YOUR_ACCESS_KEY \
  -Dvertx.s3.secret_key=YOUR_SECRET_KEY

The YOUR_S3_BUCKET_NAME, YOUR_ACCESS_KEY and YOUR_SECRET_KEY values obviously need to be replaced with real values. Within AWS' Identity and Access Management service you can configure a user that has permission to perform actions on the S3 bucket you've created for this purpose. For an example of the IAM inline user policy for an S3 bucket, consult the example JSON file in the project's src/test/resources directory.

You can name the S3 bucket whatever you want (and change its system property to match), but make sure the bucket is only used for these integration tests. The tests will delete all the contents of the bucket as a part of the test tear down. When these tests are run in Travis, a JDK name is appended onto the S3 bucket name so that the tests can be run concurrently (e.g. YOUR_S3_BUCKET_NAME-openjdk11). These buckets also need to be created ahead of time in order for the tests to pass.

If you want to put your test S3 bucket in a region other than the standard us-east-1, you will also need to supply a vertx.s3.region argument. For example:

mvn install -Ps3it -Dvertx.s3.bucket=YOUR_S3_BUCKET_NAME \
  -Dvertx.s3.access_key=YOUR_ACCESS_KEY \
  -Dvertx.s3.secret_key=YOUR_SECRET_KEY \
  -Dvertx.s3.region="us-west-2"

It can also be supplied through your settings.xml file. At this point, only regions that support signature version 2 authentication are supported. To see the valid S3 region endpoints, consult AWS' documentation.

Lastly, if you don't want to build it yourself, the library can be downloaded from the Maven central repository by putting the following in your project's pom.xml file:

<dependency>
  <groupId>info.freelibrary</groupId>
  <artifactId>vertx-pairtree</artifactId>
  <version>${vertx.pairtree.version}</version>
</dependency>

Acknowledgments

In addition to all the dependencies listed in the pom.xml file, this project incorporates a Pairtree implementation from the Library of Congress. It has been modified from its original version.

Contact

If you have questions about vertx-pairtree feel free to ask or, if you encounter a problem, please feel free to open a ticket in the project's issues queue.

vertx-pairtree's People

Contributors

ksclarke avatar snyk-bot avatar

Stargazers

Bill Dueber avatar Johannes Baiter avatar

Watchers

 avatar

vertx-pairtree's Issues

WebKitFormBoundary included in file payload on direct upload to s3

The file upload itself seems to work fine, the files show up in the s3 bucket at the correct file path, however all of the files include something like this wrapped around it

------WebKitFormBoundaryCz04vGJ3w1Zaou1B
Content-Disposition: form-data; name="files[0]"; filename="image-name.png"
Content-Type: image/png

IMAGE CONTENT HERE

------WebKitFormBoundaryMH4lrj8VmFKgt1Ar--

Fix non-standard S3 pairtree structure

S3 doesn't handle '+'s correctly (cf. https://forums.aws.amazon.com/thread.jspa?threadID=55746)

My first workaround for this was to replace '+' with '~'. This made the "pairtree" stored in S3 not really spec compliant and required extra work after copying from S3 down to a local file system using standard S3 tools. I need to fix this to make the S3 pairtrees spec compliant so they're exactly the same as pairtrees created on the file system using this library.

Upgrade Jackson dependencies

There are security issues with the older versions. We use Jackson through vertx-super-s3 so we can update to the latest version of that that has the newest Jackson deps.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.