Giter Club home page Giter Club logo

bengword2vec's Introduction

BengWord2Vec model and it's applications

Our BengWord2Vec is technically a Word2Vec model, which is based on 250 articles. This is based on https://github.com/Kyubyong/wordvectors, implementation but with additional midifications and updates.

Requirements

  • nltk >= 1.11.1
  • regex >= 2016.6.24
  • lxml >= 3.3.3
  • numpy >= 1.11.2
  • gensim > =0.13.1 (for Word2Vec)

References

  • Check this to know how to prepare Word2Vec model for other languages such as Korean, Japanese, French etc.

Workflow of training the BengWord2Vec model

  • STEP 1. Download this raw Bengali texts in case you don't want to collect all the raw text.
  • STEP 2. Extract running texts to data/ folder.
  • STEP 3. Run build_corpus.py using python3 build_corpus.py
  • STEP 4. Run sudo ./make_wordvector.sh to get the BengWord2Vec word vectors but make sure it's already executable.

Pre-trained BengWord2Vec model

Language UTF-8 Vector Size Corpus Size Vocabulary Size
Bengali (BengWord2Vec) | Bengali (f) bn 300 250M 30059
  • Check this for the pre-trained BengWord2Vec model.

bengword2vec's People

Contributors

rezacsedu avatar

Watchers

James Cloos avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.