firojalam / dp Goto Github PK
View Code? Open in Web Editor NEWThis project forked from esrel/dp
PDTB-style Shallow Discourse Parser (CoNLL 2016 Shared Task)
This project forked from esrel/dp
PDTB-style Shallow Discourse Parser (CoNLL 2016 Shared Task)
UNITN Penn Discourse Treebank Discourse Parser The Discourse Parser was developed as a University of Trento submission to CoNLL 2016 Shared Task on Multilingual Shallow Discourse Parsing and is an extension of the parser developed for the CoNLL 2015 Shared Task on Shallow Discourse Parsing. A. Dependencies & Installation The parser depends on the following 3rd party tools: - CRF++ : https://taku910.github.io/crfpp/ - icsiboost : https://github.com/benob/icsiboost - Tree Tagger: http://www.cis.uni-muenchen.de/~schmid/tools/TreeTagger/ After these tools are installed, the test.sh script might need to be edited with paths to the executables and/or models. B. Execution The parser is run using test.sh script with 2 arguments: input and output directories. The input directory must contain parses.json file (see section C) and raw textual files. C. Input Data Format Originally, the parser was designed to take the raw text and parses json file provided by the task organizers as an input. The json file contains sentence split and tokenized text, where each sentence is represented by: (1) dependency parse tree: list of triplets of dependency functions and dash (-) separated head token sting and ID and token string and its ID. IDs are counted from 1. (2) constituency parse tree string (3) list of words, where each word is represented by: (a) token string (b) character-wise begin and end position of the token in the raw text (c) part-of-speech tag Json file structure. {"DocumentID": {"sentences": [ {"dependencies": [ ["function", "Head_token-Head_tokenID", "token-tokenID"], ...], "parsetree": string, "words": [ [string, { "CharacterOffsetBegin": int, "CharacterOffsetEnd": int, "PartOfSpeech": string } ], ...] }, ... ] } } Both json file and raw text file are required, and the folder with raw documents should be in the same directory with the parses json file. D. Output Data Format The output of the parser is a json file in one json object per line format. Each object represents a discourse relation in PDTB definitions, and consists of 3 span definitions: connective, argument 1 and argument 2, sense and type of a relation. The types and senses are from PDTB definitions. {"DocID": "id_string", "Arg1": {"TokenList": [0, 1, ...]}, "Arg2": {"TokenList": [10, 11, ...]}, "Connective": {"TokenList": []}, "Sense": ["Expansion.Conjunction"], "Type": "Implicit"} E. Processing "Raw" Text Files In order to parse raw text file we provide txt2json.sh script. As dependency and constituency parses are required, the additional dependencies are: - Berkeley Parser: https://github.com/slavpetrov/berkeleyparser - Stanford Parser: http://nlp.stanford.edu/software/lex-parser.shtml The script will produce parser json file.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.