zanachka Goto Github PK

dmoz-revisiting

DMOZ Scraper with revisiting using Frontera and Scrapy

domain-discovery-crawler

Broad crawler for domain discovery

domainspider

Simple web crawler that sticks to a set list of domains. Work in progress.

double-agent

A test suite of common scraper detection techniques. See how detectable your scraper stack is.

dragnet

Just the facts -- web page content extraction

dscrapy

distributed scrapy 分布式网络爬虫

dukpy

Simple JavaScript interpreter for Python

dupuis

UI tools for record deduplication and linkage

e-scraper

Collect product and reviews from a different e-commerce stores.

eli5

A library for debugging/inspecting machine learning classifiers and explaining their predictions

estela

estela, an elastic web scraping cluster 🕸

estela-cli

estela Command Line Client 🕸

estela-entrypoint

estela entrypoint for job runner 🕸

ethereum-scraper

UNMAINTAINED! Exporter for Ethereum blocks, transactions, ERC20 transfers, contracts, using Scrapy

eval-deobfuscator

A systematic approach to JavaScript deobfuscation

exporters

Exporters is an extensible export pipeline library that supports filter, transform and several sources and destinations

extract-html-diff

extract difference between two html pages

extraction

A Python library for extracting titles, images, descriptions and canonical urls from HTML.

extruct

Extract embedded metadata from HTML markup

fakebrowser

🤖 Fake fingerprints to bypass anti-bot systems. Simulate mouse and keyboard operations to make behavior like a real person.

fbcrawl

A Facebook crawler

fingerprint-generator

Generates realistic browser fingerprints

fingerprint-injector

Home of fingerprint injector.

fingerprint-suite

fingerprints

Make it easier to compare and cross-reference the names of companies and people by applying strong normalisation.

fk-akamai-solutions

Akamai 1.63 bypass

flaresolverr

Proxy server to bypass Cloudflare protection

flatson

Tool to flatten stream of JSON-like objects, configured via schema

flatten-dict

A flexible utility for flattening and unflattening dict-like objects in Python.

flattering

Flatten, format, and export any JSON-like data to CSV (or any other string output).