lihansunbai / fang_scrapy Goto Github PK
View Code? Open in Web Editor NEW这是一个作者毕业设计的爬虫，爬取58同城、赶集网、链家、安居客、我爱我家网站的房价交易数据。
fang_scrapy's People
Contributors
Stargazers
Watchers
Forkers
zhouzhihubeyond hilllinux xingwen93 hyp528 casper2dd ppjerry chengyanglei xiaosimao awesome-archive wjy816 channieliu onsdk nikwst hujian handspring licyun huangxi668 damoguyan lenswill wacquser lonlonelong muyuwuxin yanchaomars dong138 zhangwc2017 xujun05 luffyhwl hcq908 zhang0102003 kid-geek mhl8002 flytian paulluo qinfamily jy404 knitmesh sunsiyuan hhxx1994 zchenjun liwiliwicode lwchaoyeah ohhmy fwb123 samchen1981 highcoldhc zmjaction laisigo andrew-wei wangneal ziux jiaokang19962017 yangjiaxuan jokerzh book1801 ares5221 crazypassion adamyaoqinglin fulihai jsonbao taoteching atiaboo iniunet alpharootbeta kerwin681 lilingxinyi winghou bearzeng halooooooo fangbo6699 tomcheung12 sayimeyi vincentwang5 840758192 yishuihanhan caowenbei mars531808 changhf soarl jaygith professor98911 gyc567 dalianhome kioco lidapeng520 xufuzhou1201 jinchili 554555645dsa etuios hongyu9000 sj0221 yunchuan537 caviarthea gokhu18 saniaoxiaohei yonglinz yuan-zewei haoshanwei longjingtong 0xf4vul lxngoddess5321fang_scrapy's Issues
Spider for Lianjia is out of date
OUT OF DATE
The HTML structure of Lianjia had changed meaning most code in the project doesn't work any more.
Not found pretty trick for spider coding yet.
It used to be a good script but lost much value for lack of maintaining.
安居客爬虫遇到302重定向问题
你好，最近我也在做一个房地产信息爬虫。在爬取安居客二手房的网页是遇到了302重定向问题，会把我想要爬取的页面重定向到验证码网页。
我已经将DOWNLOAD_DELAY，COOKIES_ENABLED，USER_AGENTS设置好，此外也购买了阿布云代理隧道（代理IP）。
在爬取链家二手房信息时，没有遇到任何问题，成功爬取1500+条信息，但是在爬取安居客二手房信息时，遇到了302重定向问题.
我想问问您之前遇到过这个问题吗？有时如何解决的？
爬虫下来的数据存放在哪里？
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.