View Code? Open in Web Editor
NEW
Beautiful Soup and Splinter web scraper that pulls content on Mars, stores data in a noSQL mongoDB and hosts webpage on a local Flask server.
Jupyter Notebook 70.38%
Python 16.11%
HTML 13.50%
web_scraper's Introduction
Clone the repository to your local computer, python
If you are using Windows, in the scrape_mars.py file, you will need to uncomment lines 9-10 and comment lines 13-14.
Download the most recent chromedriver (It is important that the version of chrome and the chromedriver match).
Put the Chromedriver in the HD -> usr -> local -> bin directory.
Install requirements into python environment: splinter==0.7.6.
Command: pip install splinter
Open you terminal.
'cd' into the directory that holds the repo.
Run the command "python app.py".
Open your browser and go to "http://localhost:5000/ ".
Click the "Scrap New Data Button" on the web page. It will take about a few moments for the scraping to complete. Once the scraping function is complete, the updated data will be displayed on the webpage.
web_scraper's People
Contributors
Watchers
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent