This is an experiment to develop an app which could display news based on user preference. The main Data repo is based on MongoDB
User preference will be decide automatically and that will be obsereved based on User social network clicks and browing history There will be a manual way where user can select specific topics This piece of code is still not exposed.
Currently migrating or researching for better scalabale DB like Cassandra or HBase.
Developing a better data model for building k nearest neighbours and for streaming considering Apache Flink
There is a Twitter crawler which will read tweets from multiple account details from all mentioned Place
cd pyWebCrawler/pyWebCrawler/tweets
python TweetsHandler.py
Crawling Text from various News links
python scripts/MongoSnippet.py "02/20/2016โ
News Crawler
cd pyWebCrawler
scrapy crawl dump
Run Kafka Engine for process data
java -jar news-1.0-SNAPSHOT-jar-with-dependencies.jar