tr-wiktionary-parser's Introduction

TR Wiktionary Parser

TR-wiktionary-parser is a library created based on JWKTL to parse Turkish wiktionary data.

TR-wiktionary-parser can parse 4 different wiktionaries total including 3 languages that JWKTL parses (German, English, Russian) + Turkish.

Dependencies

Project requires Java 8.

Some jar files ara needed to be able to use the library: Apache Ant {ant.jar}, Oracle Berkeley DB Java Edition {je-x.jar}, and JWKTL {jwktl-x.jar}. We currently use the versions apache-ant-1.9.5, je-6.2.31, and jwktl-1.0.2 (it is the one we build from the source code) which are already located under the folder of "lib".

Usage

These are the steps you need to follow to set up the library:

First you need to download the wiktionary data. Careful! You must download not the "wiki" ones but "wiktionary" ones.

After you go to the specified link for downloading wiktionary data (e.g. tr-wiktionary-data), you need to download the one covering "Articles, templates, media/file descriptions, and primary meta-pages." for whole coverage.
Once you have the data (?.xml.bz2), it is suggested to extract it and use the extracted .xml file to parse tha data faster.
Now that you have tha data, to parse it, first you should create the database. util/DataParser.java is the class to perform it.

You need to call its method as in DataParser.parseData(filePath, dbPath, langKey). It is suggested to use different database folders for wiktionaries in different languages.
After the database is ready, you are ready to go. There are examples in class test/DataAccessorTest.java showing how to get a specified data from the database.