Giter Club home page Giter Club logo

lda-ruby's Introduction

Latent Dirichlet Allocation โ€“ Ruby Wrapper

What is LDA-Ruby?

This wrapper is based on C-code by David M. Blei. In a nutshell, it can be used to automatically cluster documents into topics. The number of topics are chosen beforehand and the topics found are usually fairly intuitive. Details of the implementation can be found in the paper by Blei, Ng, and Jordan.

The original C code relied on files for the input and output. We felt it was necessary to depart from that model and use Ruby objects for these steps instead. The only file necessary will be the data file (in a format similar to that used by SVMlight). Optionally you may need a vocabulary file to be able to extract the words belonging to topics.

Example usage:

require 'lda-ruby'
corpus = Lda::DataCorpus.new("data/data_file.dat")
lda = Lda::Lda.new(corpus)    # create an Lda object for training
lda.em("random")              # run EM algorithm using random starting points
lda.load_vocabulary("data/vocab.txt")
lda.print_topics(20)          # print all topics with up to 20 words per topic

If you have general questions about Latent Dirichlet Allocation, I urge you to use the topic models mailing list, since the people who monitor that are very knowledgeable. If you encounter bugs specific to lda-ruby, please post an issue on the Github project.

Resources

References

Blei, David M., Ng, Andrew Y., and Jordan, Michael I. 2003. Latent dirichlet allocation. Journal of Machine Learning Research. 3 (Mar. 2003), 993-1022 [pdf].

lda-ruby's People

Contributors

dvisockas avatar ealdent avatar jasmarc avatar lefnord avatar rioleo avatar rishabh-tripathi avatar xxx avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

lda-ruby's Issues

Using Lda::DirectoryCorpus.new('path') does not populate the corpus.documents[0].filename

Hello,

I've noticed that the best way to create a corpus for us is to use the DirectoryCorpus way because we hardly have the .dat files available ahead of time. However, we are looking to be able to identify the documents inside the corpus object after running the analysis and we noticed that the filename does not get populated for each document.

Is this an issue or am I missing something? Thanks!

lda seg faults on em call

offending line: lda.em("random")

ouput:
"
[BUG] Segmentation fault
ruby 1.8.7 (2010-01-10 patchlevel 249) [i686-darwin10]

Abort trap
"

Unit tests segfaulting

Hello-

I'm having some trouble with segfaults and lda-ruby. At first I thought it was something to do with the way I was using it, but it turns out that the unit tests themselves segfault, so I'm guessing it's not in my code--- or, at least, not only in my code. :-) Here's the output of running "rake test":

https://gist.github.com/1051716

As you can see, it looks to be segfaulting in the EM code somewhere.

Any ideas?

Infer topic distribution for new document

So, my research project partner and I were wondering "can it infer the topic distribution of new (unseen) documents?"

If that's not a current feature, do you think it would be difficult to add? Maybe it's something I can help with.

Thanks,
Jason

classify document to topic

At first, thanks for this gem.

By this gem I can create a topics (categories). That is perfect! Is there a way how to classify document by the created topic? What would you suggest?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.