xhendyagsx / textrank Goto Github PK

For ongoing work, see <https://derwen.ai/docs/ptr/>


This is an open source Java implementation of the TextRank algorithm 
by Mihalcea, et al. 

Note that this code only implements key phrase extraction based on 
keyword co-occurance described in section 3 of the Mihalcea-Tarau paper.
This code does not yet implement the sentence extraction described in 
section 4 of that paper.

"Graph-Based Data Science" discussion group on LinkedIn:
  <https://www.linkedin.com/groups/6725785/>


NB: There is a known issue with use of JWNL (Java libraries for
WordNet) such that if the graph size exceeds a particular threshold,
then low-level Java I/O reads to the WordNet database on disk will
cause Java thread to block -- even though JVM tools show no blocked
threads.

A potential remedy is to dump WordNet, or at least the parts of it
used here, into some DBD structure with an in-memory cache.

---------

simple test:
	ant run

test with a specific data file FOO.txt

	ant -Ddata.file=FOO.txt run

build the JAR for export to another project:
	ant jar


Alternatively, if you use Gradle, we have also supplied a build.gradle and
a Gradle wrapper here:

	./gradlew build

The wrapper installs ("bootstraps") Gradle the first time you run it.

The Gradle build step combines all dependent JARs into one, and so you can
just use the resulting JAR:

	java -cp build/libs/ceteri-textrank.jar com.sharethis.textrank.TextRank res/log4j.properties res en test/good.txt graph.tsv

This tells TextRank.main() to use the log4j settings and the data in res/,
use English as the working language, use test/good.txt as input, and serialize
the graph to graph.tsv (this is not implemented in main(), but the argument
is still required).

---------

Sources for third-party JAR files:

commons-logging-1.1.1.jar
    http://commons.apache.org/downloads/download_logging.cgi

commons-math-1.2.jar
    http://commons.apache.org/downloads/download_math.cgi

log4j-1.2.15.jar
    http://logging.apache.org/log4j/1.2/download.html

porterstemmer.jar
    http://snowball.tartarus.org/download.php

opennlp-tools-1.3.0.jar
    http://opennlp.sourceforge.net/

maxent-2.4.0.jar                
    https://sourceforge.net/projects/maxent/

sptoolkit.jar
    http://text0.mib.man.ac.uk:8080/scottpiao/sent_detector

trove-2.0.2.jar
    http://trove4j.sourceforge.net/

jwnl-1.4rc1.jar
    http://sourceforge.net/projects/jwordnet

jdom-1-1.jar
    http://jdom.org/downloads/index.html