Giter Club home page Giter Club logo

glusterfs-java-filesystem's Introduction

glusterfs-java-filesystem

This project aims to be a complete implementation of a Java7/NIO.2 File System Provider backed by GlusterFS via libgfapi-jni

Build Status

Test coverage

Test Coverage & Code Quality

Please let me know if you use this project, even if you're just checking it out, I'd like to hear from you.

I prefer to be contacted on IRC, Twitter, or a Github issue. You can find me, semiosis, in #gluster on Freenode IRC. My twitter handle is @pragmaticism.

Thanks!

Use

Adding to your maven project

<dependencies>
    <dependency>
        <groupId>com.peircean.glusterfs</groupId>
        <artifactId>glusterfs-java-filesystem</artifactId>
        <version>1.0.4</version>
    </dependency>
</dependencies>

Adding to your non-maven project

The maven shade plugin can build a unified (shaded) JAR suitable for dropping in to the classpath of any JVM application.

You can build a "shaded" JAR by cloning the project and running the following command in the glusterfs-java-filesystem subdirectory:

cd glusterfs-java-filesystem
mvn package shade:shade

Maven will report the path of this shaded JAR. You can run export CLASSPATH=<path-to-shaded.jar> in a terminal before running your other application.

Contact me (on IRC, Twitter, or in a Github issue) if you need help obtaining a JAR, if you can't, or don't want to, build it with maven yourself.

Access GlusterFS volumes with the NIO.2 API

Once this library is in your classpath all you need to do in your code is access a GlusterFS URI, for example

gluster://server:volume/path

Example usage

A Vagrantfile in the root of this repository sets up a VM with a volume called foo at IP address 172.31.31.31 on a private network.

The Example.java file in the glusterfs-java-filesystem-example project provides a demonstration of the capabilities of this project from a high level consumer's point of view, it connects to the volume on the vagrant VM.

To run:

cd glusterfs-java-filesystem-example
vagrant up
mvn exec:exec

Roadmap

TODO:

  • Replace Example program with formal integration test suite
  • Align project versions with glusterfs (this project & libgfapi-jni)
  • Update watch service to use libgfchangelog (instead of polling)
  • Finish attribute support
    Owner/group names & ability to change
    More ways to set permissions
  • Finish integration testing for advanced synchronous file IO (reading/writing portion of file)
  • Asychronous file I/O
  • Better error reporting & handling (utilize UtilJNI.strerror() as part of IOException throws)
  • Finish readSymbolicLink unit tests
  • Publish test coverage report to Coveralls.io
    Blocked due to use of Atlassian Clover
  • Create hard links

Completed:

  • Connect to a GlusterFS volume using the NIO.2 API
  • Basic synchronous file I/O
    Read the contents of a file all at once
    Write a chunk of bytes to a file all at once
  • File attributes
    See owner/group id, size, permissions, and last modified timestamp on files and directories
    Set permissions
  • Filesystem/volume stats
    See the total, free, and usable bytes in a volume
  • Directory listing (with filtering)
  • Move/rename files
  • Watch files for changes
    Complete except for GlusterWatchKeyTest, in progress
  • Create & Read symlinks (read tests incomplete)
  • Publish test coverage & code quality reports to SonarQube
  • Delete files
  • Copy files
  • Advanced synchronous file IO

Contributing/Development

I'd appreciate your help with this project. If you have any feedback at all please get in touch. I'm interested in everything from gripes to pull requests.

Project License

Until further notice (made here and in LICENSE.txt) this project is licensed under the terms of the 3-clause BSD license, as written in LICENSE.txt.

The licensing is likely to change in the near future as the project matures.

Acknowledgements

  • May G. & Ian H. for their hard work & dedication to improving this project.
  • Atlassian for providing a free license for their most excellent Java code quality analyzer, Clover.
  • All the open source projects we depend on: GlusterFS, HawtJNI, Lombok, JUnit, Mockito, PowerMock, TestNG, sonarqube, Hadoop (whose Glob to Regex converter we borrowed), Maven and all the Maven plugins, and of course Java.

glusterfs-java-filesystem's People

Contributors

iherb001 avatar iherbig avatar louiszuckerman avatar maylem avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

glusterfs-java-filesystem's Issues

cannot find GlusterfsFileSystemProvider

in idea, it can found glusterfsFileSystemProvider,like this:
List fileSystemProviders = FileSystemProvider.installedProviders();
for (FileSystemProvider p: fileSystemProviders
) {
logger.info(String.format("file providers class are:[%s]",p.getClass()));
logger.info(String.format("file providers are:[%s]",p.getScheme()));
}

file providers class are:[class sun.nio.fs.WindowsFileSystemProvider]
file providers are:[file]
file providers class are:[class com.sun.nio.zipfs.ZipFileSystemProvider]
file providers are:[jar]
file providers class are:[class com.peircean.glusterfs.GlusterFileSystemProvider]
file providers are:[gluster]

but,when it was jar on windows or linux, just can find WindowsFileSystemProvider\ZipFileSystemProvider,or LinuxFileSystemProvider\ZipFileSystemProvider.

Cannot create a new filesystem

Hi all, I created a new project by importing glusterfs-java-filesystem-1.0.4.jar, libgfapi-jni-1.0.4.jar and hawtjni-runtime-1.9.jar jars, and by following the Example.java class, but I get the following message error:

Exception in thread "main" java.lang.UnsatisfiedLinkError: Could not load library. Reasons: [no libgfapi-jni in java.library.path]
at org.fusesource.hawtjni.runtime.Library.doLoad(Library.java:182)
at org.fusesource.hawtjni.runtime.Library.load(Library.java:140)
at com.peircean.libgfapi_jni.internal.GLFS.(GLFS.java:52)
at com.peircean.glusterfs.GlusterFileSystemProvider.glfsNew(GlusterFileSystemProvider.java:70)
at com.peircean.glusterfs.GlusterFileSystemProvider.newFileSystem(GlusterFileSystemProvider.java:43)
at java.nio.file.FileSystems.newFileSystem(FileSystems.java:326)
at java.nio.file.FileSystems.newFileSystem(FileSystems.java:276)
at com.gpi.gluster.GlusterConnector.main(GlusterConnector.java:50)

Any idea? Thank you very much in advance!

Having trouble in creating a file

Files.createFile(glusterPath) with or without a set of pre-permissions does not create a file, nor an exception is thrown. What would be the cause of it?

Springboot + GlusterFS : Unable to build DLL from 'glusterfs-java-filesystem/tree/combined-with-libgfapi' on windows 10

Hi Team,

Greetings
We are unable to build the complete source as we see following errors during the build, we used the goal 'mvn package shade:shade'
Kindly guide on how to proceed further with this issue.

Following updates were done on Win10 before ewe reached this step

  1. MVS 2008 was installed to have vcbuild in place for eclipse
  2. We tried to run manually via MVS2008 command prompt but we were stuck with sys/error.h file or directory not found

Not able to proceed further with considerable hours already spent on finding solution to this issue

--- maven-hawtjni-plugin:1.15:build (default) @ libgfapi-linux64 ---
[INFO] executing: cmd.exe /X /C "vcbuild /platform:x64 vs2008.vcproj release"
[INFO] Microsoft (R) Visual C++ Project Builder - Command Line Version 9.00.30729
[INFO] Copyright (C) Microsoft Corporation. All rights reserved.
[INFO]
[INFO] Error: project file 'vs2008.vcproj' was not found or not a valid project file.
[INFO] Project file 'vs2008.vcproj' could not be loaded.
[INFO] The project is either missing from disk or access was denied.
[INFO] rc: -1
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for glusterfs-java-filesystem-project 1.0.5-SNAPSHOT:
[INFO]
[INFO] glusterfs-java-filesystem-project .................. SUCCESS [ 0.472 s]
[INFO] libgfapi-jni ....................................... SUCCESS [ 1.467 s]
[INFO] libgfapi-linux64 ................................... FAILURE [ 0.352 s]
[INFO] glusterfs-java-filesystem .......................... SKIPPED
[INFO] glusterfs-java-filesystem-example .................. SKIPPED
[INFO] ------------------------------------------------------------------------

glusterfs-java-filesystem for output plugin in logstash

Hi Louis,
I cam across this project and saw your presentation from 2014 just yesterday. We currently run a Kafka -> Logstash (File Output Plugin) -> Fuse Client -> GlusterFS setup and seeing your video with the NIO2 input plugin I was wondering how feasible it would be to replace the Logstash File Output Plugin and the flues Client with a Logstash NIO.2 Oputput Plugin using your project.
I seem to be unable to find any nio2 plugins for logstash what so ever.

java.nio.file.FileSystemNotFoundException: Provider \"gluster\" not installed"

I got "java.nio.file.FileSystemNotFoundException: Provider "gluster" not installed"".
I followed README.md in the project.

At first, I added the dependency to my maven project. My project provided a servlet interface to read/write my glisters. I can get my .war, but when i put it into tomcat and startup and called the servlet, I got the Exception.

Then, I tried to cd glusterfs-java-filesystem && mvn package shade:shade, I got the jar successfully. The mvn info is :

Replacing /data2/glusterfs-java-filesystem/target/glusterfs-java-filesystem-1.0.5-SNAPSHOT.jar with /data2/glusterfs-java-filesystem/target/glusterfs-java-filesystem-1.0.5-SNAPSHOT-shaded.jar

I found that the jar path was "/data2/glusterfs-java-filesystem/target/", and I set it in CLASSPATH, but stall the same Exception.

Does the any mistake? Does it need to install libgfapi-jni?

Thanks.

glusterfs-java-filesystem-example exception throws

while run Example.java,exception message is:
Exception in thread "main" java.io.IOException: Unable to close filesystem: dr
at com.peircean.glusterfs.GlusterFileSystem.close(GlusterFileSystem.java:44)
at com.peircean.glusterfs.example.Example.main(Example.java:167)
dr is the gluster server's volume name.the close method is necessary?thx

How to fix the Exception "java.lang.UnsatisfiedLinkError "?

How to fix the Exception "java.lang.UnsatisfiedLinkError: Could not load library. Reasons: [no libgfapi-jni in java.library.path, ", when I run the glusterfs-java-filesystem-example project?

is the reason being short of "libgfapi-jni .dll"?

Detial:
com.peircean.glusterfs.GlusterFileSystemProvider@1a99347
java.library.path: C:\Users\zoujian.m2\repository\com\peircean\libgfapi-jni\libgfapi-jni\1.0.4
Exception in thread "main" java.lang.UnsatisfiedLinkError: Could not load library. Reasons: [no libgfapi-jni32-1.0.4 in java.library.path, no libgfapi-jni-1.0.4 in java.library.path, no libgfapi-jni in java.library.path]
at org.fusesource.hawtjni.runtime.Library.doLoad(Library.java:182)
at org.fusesource.hawtjni.runtime.Library.load(Library.java:140)
at com.peircean.libgfapi_jni.internal.GLFS.(GLFS.java:52)
at com.peircean.glusterfs.GlusterFileSystemProvider.glfsNew(GlusterFileSystemProvider.java:70)
at com.peircean.glusterfs.GlusterFileSystemProvider.newFileSystem(GlusterFileSystemProvider.java:43)
at com.peircean.glusterfs.GlusterFileSystemProvider.getPath(GlusterFileSystemProvider.java:111)
at java.nio.file.Paths.get(Unknown Source)
at com.peircean.glusterfs.example.Example.main(Example.java:43)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.