sam8778 / tedtex-webscraper Goto Github PK

Software Engineer

• Clone the Project using Git Clone
• Change values in .env file
• Create an account in Bright Data and put the automation script in.
• Create an account in Firebase and connect it.
• After all the env values are updated launch the project with the npm run dev command.

In the code example below, we make a request to retrieve the Amazon product page for any product.

settings({ country: input.country || "us" });
navigate(`https://www.amazon.com/s?k=${input.search}`);
let max_page = parse().max_page;
let final_page = Math.min(input.pages || 0, max_page) || 1;
for (let i = 1; i <= final_page; i++) {
  let url = `https://www.amazon.com/s?k=${input.search}`;
  if (i > 1) url += `&page=${i}`;
  next_collector({ url, country: input.country, search: input.search });
}

Can modify this script to run scraper on other websites

⭐️ From @Sam George