This script scrapes data from a list of websites and extracts the following information:

• Social Media Links
• Tech Stack (MVC, CMS, JS type, etc.)
• Meta Title
• Meta Description
• Payment Gateways (e.g., PayPal, Stripe, Razorpay)
• Website Language
• Category of Website

The extracted data is stored in a MySQL database. Moreover, 1.cvs is the exported data of MySQL database.

• Python 3.x
• requests library
• beautifulsoup4 library
• lxml library
• mysql-connector-python library
• python-Wappalyzer library

1. Clone the repository.

   git clone https://github.com/smileagain6698/scrappy_100.git

2. Create the virtual environment:

   python -m venv venv
   venv\Scripts\activate

3. Install the required Python libraries:

   pip install requests beautifulsoup4 lxml mysql-connector-python python-Wappalyzer

4. Set-up your MySQL database
5. Add extensions to your vscode - SQLTools database management && SQLTools MySQL/MariaDB/TiDB Driver
6. Open SQLTools, select MySQL database and update Connection Settings with similar to the settings.json file and as per your database. *(select MySQL) (password is your MySQL database password)
7. Test your connection and Save Connection.
8. Run in terminal

   python main.py ```

10. To open your database table, click on the plus icon in SQLTools