English | 中文

nebula-algorithm is a Spark Application based on GraphX with the following Algorithm provided for now:

You could submit the entire spark application or invoke algorithms in lib library to apply graph algorithms for DataFrame.

1. Build Nebula Algorithm

   $ git clone https://github.com/vesoft-inc/nebula-algorithm.git
   $ cd nebula-algorithm
   $ mvn clean package -Dgpg.skip -Dmaven.javadoc.skip=true -Dmaven.test.skip=true
   

   After the above buiding process, the target file nebula-algorithm-3.0-SNAPSHOT.jar will be placed under nebula-algorithm/target.

2. Download from Maven repo

   Alternatively, it could be downloaded from the following Maven repo:

   https://repo1.maven.org/maven2/com/vesoft/nebula-algorithm/

• Option 1: Submit nebula-algorithm package

  • Configuration

  Refer to the configuration example.

  • Submit Spark Application

  ${SPARK_HOME}/bin/spark-submit --master <mode> --class com.vesoft.nebula.algorithm.Main nebula-algorithm-3.0—SNAPSHOT.jar -p application.conf
  

  • Limitation

  Due to Nebula Algorithm jar does not encode string id, thus during the algorithm execution, the source and target of edges must be in Type Int (The vid_type in Nebula Space could be String, while data must be in Type Int).

• Option2: Call nebula-algorithm interface

  Now there are 10+ algorithms provided in lib from nebula-algorithm, which could be invoked in a programming fashion as below:

  • Add dependencies in pom.xml.

   <dependency>
        <groupId>com.vesoft</groupId>
        <artifactId>nebula-algorithm</artifactId>
        <version>3.0.0</version>
   </dependency>
  

  • Instantiate algorithm's config, below is an example for PageRank.

  import com.vesoft.nebula.algorithm.config.{Configs, PRConfig, SparkConfig}
  import org.apache.spark.sql.{DataFrame, SparkSession}
  
  val spark = SparkSession.builder().master("local").getOrCreate()
  val data  = spark.read.option("header", true).csv("src/test/resources/edge.csv")
  val prConfig = new PRConfig(5, 1.0)
  val prResult = PageRankAlgo.apply(spark, data, prConfig, false)
  

  If your vertex ids are Strings, see Pagerank Example for how to encoding and decoding them.

  For examples of other algorithms, see examples

  Note: The first column of DataFrame in the application represents the source vertices, the second represents the target vertices and the third represents edges' weight.

If you want to write the algorithm result into Nebula, make sure there is corresponding property name in your tag.
|        Algorithm         |     property name       |property type|
|:------------------------:|:-----------------------:|:-----------:|
|         pagerank         |         pagerank        |double/string|
|          louvain         |          louvain        | int/string  |
|          kcore           |           kcore         | int/string  |
|     labelpropagation     |           lpa           | int/string  |
|   connectedcomponent     |            cc           | int/string  |
|stronglyconnectedcomponent|            scc          | int/string  |
|         betweenness      |         betweenness     |double/string|
|        shortestpath      |        shortestpath     |   string    |
|        degreestatic      |degree,inDegree,outDegree| int/string  |
|        trianglecount     |       trianglecount     | int/string  |
|  clusteringcoefficient   |    clustercoefficient   |double/string|
|         closeness        |         closeness       |double/string|
|            hanp          |            hanp         | int/string  |
|            bfs           |            bfs          |    string   |
|         jaccard          |          jaccard        |    string   |
|        node2vec          |          node2vec       |    string   |

Nebula Algorithm is open source, you are more than welcomed to contribute in the following ways:

• Discuss in the community via the forum or raise issues here.
• Compose or improve our documents.
• Pull Request to help improve the code itself here.