Giter Club home page Giter Club logo

reynirpackage's Introduction

image

A fast, efficient natural language parser for Icelandic

image

Overview

Greynir is a Python 3.x package for parsing Icelandic text into sentence trees (i.e. full constituency parse trees). The trees can then be used to extract information from the text, for instance about people, titles, entities, facts, actions and opinions.

Along the way, Greynir tokenizes the text, finds lemmas and assigns part-of-speech (POS) tags to every word.

Full documentation for Greynir is available here.

Greynir is the engine of Greynir.is, a natural-language front end for a database of 9 million sentences parsed from Icelandic news articles, and Embla, a natural-language query app for smart phones.

Greynir uses the Tokenizer package, by the same authors, to tokenize text.

Example

>>> from reynir import Greynir >>> g = Greynir() >>> sent = g.parse_single("Ása sá sól.") >>> print(sent.tree.view) P # Root +-S-MAIN # Main sentence +-IP # Inflected phrase +-NP-SUBJ # Noun phrase, subject +-no_et_nf_kvk: 'Ása' # Noun, singular, nominative, feminine +-VP # Verb phrase containing arguments +-VP # Verb phrase containing verb +-so_1_þf_et_p3: 'sá' # Verb, 1 accusative arg, singular, 3rd p +-NP-OBJ # Noun phrase, object +-no_et_þf_kvk: 'sól' # Noun, singular, accusative, feminine +-'.' # Punctuation >>> sent.tree.nouns ['Ása', 'sól'] >>> sent.tree.verbs ['sjá'] >>> sent.tree.flat 'P S-MAIN IP NP-SUBJ no_et_nf_kvk /NP-SUBJ VP so_1_þf_et_p3 NP-OBJ no_et_þf_kvk /NP-OBJ /VP /IP /S-MAIN p /P' >>> # The subject noun phrase (S.IP.NP also works) >>> sent.tree.S.IP.NP_SUBJ.lemmas ['Ása'] >>> # The verb phrase >>> sent.tree.S.IP.VP.lemmas ['sjá', 'sól'] >>> # The object within the verb phrase (S.IP.VP.NP also works) >>> sent.tree.S.IP.VP.NP_OBJ.lemmas ['sól']

Prerequisites

This package runs on CPython 3.5 or newer, and on PyPy 3.5 or newer.

If a binary wheel package isn't available on PyPi for your system, you may need to have the python3-dev and/or potentially python3.6-dev packages (or their Windows equivalents) installed on your system to set up Greynir successfully. This is because a source distribution install requires a C++ compiler and linker:

# Debian or Ubuntu:
sudo apt-get install python3-dev
sudo apt-get install python3.6-dev

Installation

To install this package:

$ pip3 install reynir   # or pip install reynir if Python3 is your default

If you want to be able to edit the source, do like so (assuming you have git and git-lfs installed):

$ git clone https://github.com/mideind/ReynirPackage
$ cd ReynirPackage
$ # [ Activate your virtualenv here if you have one ]
$ python setup.py develop

The package source code is now in ReynirPackage/src/reynir.

Note that git-lfs is required to clone and pull the full compressed binary files for the Beygingarlýsing íslensks nútímamáls (BÍN) database. If it is missing, you will get assertion errors.

Tests

To run the built-in tests, install pytest, cd to your ReynirPackage subdirectory (and optionally activate your virtualenv), then run:

$ python -m pytest

Documentation

Please consult Greynir's documentation for detailed installation instructions, a quickstart guide, and reference information, as well as important information about copyright and licensing.

Greynir is copyright (C) 2019 by Miðeind ehf. The original author of this software is Vilhjálmur Þorsteinsson.

image

This set of programs is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This set of programs is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

The full text of the GNU General Public License v3 is included here and also available here: https://www.gnu.org/licenses/gpl-3.0.html.

image

If you would like to use this software in ways that are incompatible with the standard GNU GPLv3 license, please contact Miðeind ehf. ([email protected]) to negotiate alternative arrangements.

reynirpackage's People

Contributors

vthorsteinsson avatar holado avatar sveinbjornt avatar

Watchers

James Cloos avatar Sverrir Á. Berg avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.