fanfank / phpfetcher Goto Github PK
View Code? Open in Web Editor NEWA PHP web crawler framework
phpfetcher's People
Contributors
Stargazers
Watchers
Forkers
zbage ericzju zhaowh ccfeeling loryhuang godofchina faruto buctdarkness onedot truthhun henry-fun gischen fdzcz flatheadman raysail ginkotimberlake stevekevin1005 hedeheng mjyuser yichengming xiangyuwei overcome konggeyuan renzcphp uqiauto redyin khlipeng veaglefly redknox 741968602 andy0010 lnc2014 timelessz tangpanpan23 gushou komong jimchenwyu studyinhub jiangadam yiyuechen zqmath1994 liugu ockor sndnvaps haitao1880 leichaojiayou radishkin mygirld xiaozhuaisnow fordlee wyue0607 lerre 1271281914 williamhe1988 wadecp3 ericrun ikuiki mugun samouly terrygl mahasoa vshangxiao zhangyancoder output suyisun j0bkeeper gongfupanada banxuelai hxzlhby domainvip k12f a4092816 jeeinn2012 jemmy123321 jinjianfeng huangyellow flexxia ckwsens orz1245 stamhe lestatmiao guanzhending falcolee xiaojian001 wyd-asif chuangshu androids7 gitzwt wbluckly jingqianwei chenlu772 fengchengfeng wujunshan suson rxt2012kc changpao5000 cityo bryant1410 wyrover onesevenphpfetcher's Issues
是否考虑支持正则匹配
如题，框架里面是否考虑正则匹配来获取内容
网页charset和实际编码不一致，希望能修改
使用mb_detect_encoding 可检测出实际编码，然后再进行转换。
http://tech.qq.com/ 【腾讯科技频道】 标记的是utf8 实际则是EUC-CN编码
有没有对采集的数据入库的例子？
如插入到wordpress中去，实现发布
Phpfetcher stops when encountering 404 pages
Error logs as follows:
2016-04-07 18:44:54 Default.php Phpfetcher_Page_Default sel 116 Warning: $this->_dom is NULL!
2016-04-07 18:44:54 crawl_baidu_page.php Phpfetcher_Page_Default sel 72 Warning: $this->_dom is NULL! Fatal error: Call to a member function getAttribute() on a non-object in D:\WWW\demo\crawl_baidu_page.php on line 45
大神，需要你的帮助
咨询，如何获取js初始化之后的页面数据
抓取页面有些数据是通过初始化js来追加的html，我发现我使用的程序只能加载html源码的数据，phpfetcher是否实现了抓取js初始化之后的页面？如果有，麻烦告知，不胜感激~
能否排除正则匹配的某些链接?
在 link_rules 中, 写了一个 "排除特定字符" 的正则, 似乎没有生效.
请问是否有别的解决方法吗?
可以可以
可以考虑整合和 phantomjs
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.