trevorr / circe Goto Github PK

Hi,

I have created following test-case in CRCTest to verify incremental CRC32C checksum computation.

   @Test
    public void testCRC32CIncremental() {
        IncrementalIntHash crc32c = PROVIDER.getIncrementalInt(CRC32C);
        String data = "data";
        String combine = data + data;

        int combineChecksum = crc32c.calculate(combine.getBytes());
        int dataChecksum = crc32c.calculate(data.getBytes()); 
        int incrementalChecksum = crc32c.resume(dataChecksum, data.getBytes());
        assertEquals(combineChecksum, incrementalChecksum);
    }

We have two separate ByteBuffers and we want to compute combined-checksum on it so, I am trying to use incremental checksum but it seems it is giving incorrect result. @trevorr Can you please advice if I am missing anything here.

hi,

Thanks for providing java solution for crc32c-sse4.2 using native library. I can see you have uploaded SNAPSHOT version of this artifact at : Sonatype-Staging_maven-repo

However, we are not able to use this solution due to following issues:

1. Library not present into maven-repo: The available jni library in maven-repo is only available for windows OS and we can't use it for "Mac OS" or "Linux OS".

Solutions:
1. Upload jni library to maven repo for all well-known OS: Right now, Maven repo only contains jni for windows so other system can't use this library. So, can you also please upload library for MacOSX and Linux as well.

2. One Packaging of all OS specific JNI library using vagrant: Checkout RocksDB Build it uses vagrant to build jni for all OS environments and packages them into one artifact which you can deploy to maven repo

1. Can't use artifact as it as:
   When we build circe-crc32c-sse42 using maven build:

• It internally generates "jni" library based on OS where we build and it packages this generated jni library under .nar artifact. e.g: circe-crc32c-sse42.nar
• Nar Plugin goal also generates NarSystem.java which tries to load this jni library at runtime using System.loadLibrary("circe-crc32c-sse42-0.1-SNAPSHOT"); which fails obviously because it asks ClassLoader to load library from java.library.path or sun.boot.library.path. And as this jni library is packaged under .nar so, definitely it is not present in either library.path and hence, initialization of the interface which provides API to compute checksum Sse42Crc32C.java fails.

Solution

• So, I created a pull request : which generates circe-crc32c-sse42.jar instead circe-crc32c-sse42.nar artifact and internally loads generated jni library without messing up with library.path and user can use this artifact without performing any additional magic.
  I have also added a test case which cleanly uses Sse42Crc32C.java to compute checksum by simply adding dependency of artifact com.scurrilous:circe-crc32c-sse42:${project.version} and without worrying about configuring library.path.