tristanshi / data-gatherer Goto Github PK

Crawlers use selenium webdiver to execute JS of web to scrap data. 

app_rank.py : a  crawler program for app_annie, to gather the data about app download rank of each day include all categories.
app_rank.sql:  the file of MySQL table  for app_rank ( page, rank ,Rebiew_info_pokego)

app_vision_reviews.py : a crawler program for app_annie to gather the data for reviews and vision update.

sensortower.py: a crawler for sensortower to gather the data for downloads and revenue data and save the page_source in the same time.
sensortower.sql: the file to establish MysSQL table for sensortower.py(page, app)