Giter Club home page Giter Club logo

web-scraping-and-fast-api's Introduction

Web Scraping Project with Fast API

A Web scraping project to scrape the data from the websites which doesn't require login.

Beautiful Soup library is used for extracting my response data. It creates a parse tree from page source code that can be used to extract data in a hierarchical and more readable manner, basically a Python library for pulling data out of HTML and XML files.

I have done scraping on this website: "https://www.onthisday.com"
" On This Day " is the world's largest, most accurate and popular site for on this day in history, it gives all the historical events happened in a day wise frame.
I have scrap the whole bunch of data of all days, filtered it in month wise frame and stored it in json file.
Using Fast API, have assigned endpoints for displaying historical events of today's date, month wise events, a particular day and month event and more...
It's a basic demo, just for understanding purpose.
You can use this code to scrap any website data which doesn't requires login.

Do these installs before running the project,

pip install beautifulsoup4

If anyone got any module error, then install that module like

pip install module_name

For accessing Fast API, run collect_events.py file first ( for creating events.json file ):

python collect_events.py
uvicorn main:app --reload

Then go to the respective url( Ex: http://127.0.0.1:8000/ ), for a better view just add "docs" or "redoc" to your url. ( Ex: http://127.0.0.1:8000/docs or http://127.0.0.1:8000/redoc ) and explore it.

Reference

1). https://beautiful-soup-4.readthedocs.io/
2). https://www.onthisday.com/
3). https://fastapi.tiangolo.com/

For any doubts, raise your issues, willingly waiting to help you and clear your doubts...

web-scraping-and-fast-api's People

Contributors

mysterious-shailendr avatar

Stargazers

 avatar Oshin Sodiq Adeniyi avatar Christina Schweipert avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.