Giter Club home page Giter Club logo

faiss4j's Introduction

faiss4j

java wrapper for facebook faiss

how to build this

#build faiss first

Firstly we need to compile from faiss code, please follow this doc

I started the experiment with a clean centos docker image, so I needed to install something beforehand

yum install -y epel-release
yum install -y git
yum install -y devtoolset-8.x86_64
yum install -y gcc.x86_64 gcc-c++.x86_64 libgcc.x86_64
yum install -y cmake3.x86_64 make
yum groupinstall "Development Tools"  --skip-broken
yum install -y swig3.x86_64
yum install -y java-1.8.0-openjdk-devel.x86_64
yum install -y maven.noarch
yum install -y python-devel.x86_64
yum install -y python34-devel
yum install -y blas64.x86_64 blas64-devel.x86_64 blas64-static.x86_64
yum install -y lapack64.x86_64 lapack64-devel.x86_64 lapack64-static.x86_64

can't do GPU building since I don't have gpu on the server.

enter faiss, run ./configure to check environment

./configure --without-cuda
make
make install

it shows

/usr/bin/mkdir -p /usr/local/lib /usr/local/include/faiss
cp libfaiss.a libfaiss.so /usr/local/lib

then faiss is built

#build java wrapper in root of faiss4j I chose faiss tagged 1.5.0 IT MIGHT NOT WORK WITH OTHER VERSIONS

swig -c++ -java -package com.thenetcircle.services.faiss -o swigfaiss4j.cpp  -outdir src/main/java/com/thenetcircle/services/faiss/ -Doverride= -I../faiss-1.5.0/ swigfaiss4j.swig

this command uses swig to generate a cpp file swigfaiss4j.cpp which works as bridge between jni and faiss code, it also creates correspondent java definitions

now we need to compile this swigfaiss4j.cpp into lib for java to call, (I am just a java guy with little knowledge with compiling c/c++)

g++ -std=c++11 \
 -fPIC \
 -fopenmp \
 -m64 \
 -Wno-sign-compare \
 -g \
 -O3 \
 -Werror \
 -Wextra \
 -msse4 \
 -mpopcnt  \
 -z noexecstack \
 -L /usr/local/lib \
 -I ../faiss-1.5.0/ \
 -I /usr/lib/jvm/java-1.8.0-openjdk/include/ \
 -I /usr/lib/jvm/java-1.8.0-openjdk/include/linux/ \
 -lfaiss \
  swigfaiss4j.cpp \
  -shared -o swigfaiss4j.so

at last, libfaiss.so should be in current path -rwxr-xr-x 1 root root 3594624 Apr 16 09:20 swigfaiss4j.so

somehow, the java stub class has an error at com/thenetcircle/services/faiss/RangeSearchPartialResult.java:[64,14] error: method finalize() is already defined in class RangeSearchPartialResult

then let us run some test

LD_LIBRARY_PATH=. mvn test -Dtest=com.thenetcircle.services.faiss4j.tests.Examples#testLib  

faiss4j's People

Contributors

cfcodefans avatar

Watchers

James Cloos avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.