Giter Club home page Giter Club logo

web-scrap-beautiful-soup's Introduction

Web Scraping Beautiful Soup Automation ๐Ÿค–

This project consists of two Python scripts that automate the job search process on a specified website. The scripts utilize the BeautifulSoup library for web scraping.

local.py ๐Ÿ•ต๏ธโ€โ™‚๏ธ

The local.py script is designed to parse an HTML file (index.html) locally and extract information about courses. The main functionalities include:

  • Reading the content of the HTML file.
  • Creating a BeautifulSoup object to parse the HTML content.
  • Searching for specific HTML elements (e.g., <h5> and <div class="card">).
  • Extracting and printing information from the parsed HTML.

You can uncomment specific lines in the script to enable different functionalities, such as printing raw HTML content or searching for specific elements.

main.py ๐Ÿš€

The main.py script automates the job search process on the TimesJobs website. Key features of the script include:

  • Sending an HTTP GET request to the specified job search URL.
  • Extracting job details (company name, required skills, more info URL) using BeautifulSoup.
  • Filtering out jobs that require a specific skill provided by the user.
  • Saving relevant job details to text files for further review.

The script runs in an infinite loop, periodically conducting job searches with a specified time interval between searches.

Instructions ๐Ÿ› ๏ธ

  1. Requirements:

    Install the required Python libraries using:

         pip install beautifulsoup4 requests
    
  2. Run local.py:

    Modify the index.html file or use your own HTML file. Uncomment specific lines to enable different functionalities.

         python local.py
    
  3. Run main.py: Enter a skill that you want to filter out from job listings when prompted. The script will continuously search for Golang-related jobs on TimesJobs.

        python main.py
    

Closing Notes ๐Ÿ“

Feel free to adjust the configuration, and if you encounter any issues or have suggestions for improvement, please open an issue or submit a pull request.

Happy coding! ๐Ÿš€๐Ÿ‘จโ€๐Ÿ’ป

web-scrap-beautiful-soup's People

Contributors

anjasfedo avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.