This project forked from panziqiang007/tmallspider
天猫商品数据采集,基于 Python3,并非Selenium框架
* 如果是第一次使用,请先运行 setting.py 文件,创建相关表
* 完善 seller 表信息,包含 shop_id,sellerNick,BC_type,url
* 执行 detail_info.py 开始获取数据
可以直接录入商品id到表中,如果表中存在id字段,可以直接执行获取对应的数据,不一定需要依赖 seller 表.
店铺有新上架商品,update_itemID.py 可直接将商品ID更新入库
代码采用的是基本的单线程模式,而且未使用代理,设置默认5秒一次循环;
每小时执行一次,并未发现异常,抓取速度过快容易被限制ip,以及强制登陆;
多次测试过,首先会要求你强制登陆,然后限制ip访问,如果没登陆的情况,不会直接限制ip
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.