tysoncung / recipe-scrapers Goto Github PK

A simple web scraping tool for recipe sites.

pip install recipe-scrapers

then:

from recipe_scrapers import scrape_me

# give the url as a string, it can be url from any site listed below
scraper = scrape_me('https://www.allrecipes.com/recipe/158968/spinach-and-feta-turkey-burgers/')

scraper.title()
scraper.total_time()
scraper.yields()
scraper.ingredients()
scraper.instructions()
scraper.image()
scraper.links()

Note: scraper.links() returns a list of dictionaries containing all of the <a> tag attributes. The attribute names are the dictionary keys.

• https://www.acouplecooks.com
• https://allrecipes.com/
• https://archanaskitchen.com/
• https://averiecooks.com/
• https://bbc.com/
• https://bbc.co.uk/
• https://bbcgoodfood.com/
• https://bettycrocker.com/
• https://bonappetit.com/
• https://bowlofdelicious.com/
• https://budgetbytes.com/
• https://closetcooking.com/
• https://cookieandkate.com/
• https://cookpad.com/
• https://cookstr.com/
• https://copykat.com/
• https://countryliving.com/
• https://cybercook.com.br/
• https://delish.com/
• https://eatsmarter.com/
• https://eatsmarter.de/
• https://epicurious.com/
• https://fifteenspatulas.com/
• https://finedininglovers.com/
• https://fitmencook.com/
• https://food.com/
• https://foodnetwork.com/
• https://foodrepublic.com/
• https://geniuskitchen.com/
• https://giallozafferano.it/
• https://gimmesomeoven.com/
• https://gonnawantseconds.com/
• https://gousto.co.uk/
• https://greatbritishchefs.com/
• https://halfbakedharvest.com/
• https://www.hassanchef.com/
• https://heinzbrasil.com.br/
• https://hellofresh.com/
• https://hellofresh.co.uk/
• https://www.hellofresh.de/
• https://hostthetoast.com/
• https://101cookbooks.com/
• https://receitas.ig.com.br/
• https://inspiralized.com/
• https://jamieoliver.com/
• https://justbento.com/
• https://kennymcgovern.com/
• https://kochbar.de/
• https://lovingitvegan.com/
• https://lecremedelacrumb.com/
• https://marmiton.org/
• https://matprat.no/
• http://mindmegette.hu/
• https://minimalistbaker.com/
• https://misya.info/
• https://momswithcrockpots.com/
• http://motherthyme.com/
• https://mybakingaddiction.com/
• https://myrecipes.com/
• https://healthyeating.nhlbi.nih.gov/
• https://cooking.nytimes.com/
• https://ohsheglows.com/
• https://www.panelinha.com.br/
• https://paninihappy.com/
• https://przepisy.pl/
• https://realsimple.com/
• https://seriouseats.com/
• https://simplyquinoa.com/
• https://simplyrecipes.com/
• https://skinnytaste.com/
• https://southernliving.com/
• https://spendwithpennies.com/
• https://steamykitchen.com/
• https://tastesoflizzyt.com
• https://tasteofhome.com
• https://tasty.co
• https://tastykitchen.com/
• https://thehappyfoodie.co.uk/
• https://thekitchn.com/
• https://thepioneerwoman.com/
• https://thespruceeats.com/
• https://thevintagemixer.com/
• https://thewoksoflife.com/
• https://tine.no/
• https://tudogostoso.com.br/
• https://twopeasandtheirpod.com/
• https://vegolosi.it/
• https://watchwhatueat.com/
• https://whatsgabycooking.com/
• https://en.wikibooks.org/
• https://yummly.com/

Part of the reason I want this open sourced is because if a site makes a design change, the scraper for it should be modified.

If you spot a design change (or something else) that makes the scraper unable to work for a given site - please fire an issue asap.

If you are programmer PRs with fixes are warmly welcomed and acknowledged with a virtual beer.

• Open an Issue providing us the site name, as well as a recipe link from it.
• You are a developer and want to code the scraper on your own:
  • If Schema is available on the site - you can do this
    • How do I know if a schema is available on my site?
  • Otherwise, scrape the HTML - like this

Assuming you have python3 installed, navigate to the directory where you want this project to live in and drop these lines

git clone [email protected]:hhursev/recipe-scrapers.git &&
cd recipe-scrapers &&
python3 -m venv .venv &&
source .venv/bin/activate &&
pip install -r requirements.txt &&
pre-commit install &&
python -m coverage run -m unittest &&
python -m coverage report

• How do I know if a website has a Recipe Schema?
  • Go to a recipe on the website you want to be supported.
  • Hit Ctrl - u on your keyboard
  • Search (Ctrl -f) for application/ld+json. It should be inside a script tag.
  • If you found it then it's highly likely your website supports recipe schemas. Otherwise, you'll need to parse the HTML.

All the contributors that helped improving the package. You are awesome!