- The script starts with calling chrome driver to open Google chrome and visit Indeed website. There, the script enters the username and password to login into the Indeed account. Followed by Entering the Job Title to be searched for job posting.
- The script then extracts all job title, job description and location from page 1 and goes to page 2 from extracting the rest. This process continues till it reaches the page 6
- The script stores the job titles and details in the list data types. This list data type then get converted into Dataframe using pandas library and then it get exported to csv ot html tables format, which-so-ever you want
Further work will be storing the job posting into a database like connecting the script with postgre to insert daily postings with links and using them for applying or carrying out analysis. Analysis might include differentiating the job posting from staffing agency to actual companies because every 3rd job posting in the Indeed are from staffing agency rather than from actual companies.