Giter Club home page Giter Club logo

contentaggregators's Introduction

说明

特色:
1.前端纯HTML+JS+JSON(链接统计除外),后端python生成标准JSON
2.自带30几个常用网站采集规则
3.多线程抓取,30+网站5秒内采集完毕
4.相同网站放在一个框架内,可局部刷新和滚动
5.更新时间显示(按采集页的自带更新时间或按时间排序的第一条时间采集,没有的为采集完成时间)
6.内容链接防盗链,链接通过base64+字符逆序+大小写反转+base64实现加密(更换大小写转换顺序或增加数字替换可实现不同密码加密)
7.内容链接点击统计
8.部分内容鼠标悬停提示(如appstore排行有更新内容、时间、版本号)

演示:
https://make.mk
https://www.make.mk (cdn镜像站)

安装:
1.python3 caiji.py采集(保持后台运行请用screen或nohup)
2.打开首页访问

注:
caiji.py为后端采集程序,其余为前端需在同一目录
相应设置修改请查看源码中的注释
admin.php为第三方数据库管理程序,用于内容链接统计查看(默认密码admin,请在文件中修改)
点击统计数据库log.db会在首次访问内容链接时生成,如未建立请保证主目录可写
为了安全,后端程序caiji.py和数据库查看程序admin.php请改为复杂文件名,或移动到非web目录(需同时修改文件内的生成json保存目录为网站根目录,默认为当前目录)
默认使用多线程抓取,如有问题请改用单线程
今日头条的采集请更换为自己的cookie,头条短时间采集内容相同的太多(目前1分钟采一次合并10次数据),相同cookie分段采集后几乎不会有重复的,但采集一段时间后推荐的都是杂七杂八无关的垃圾内容(请定期更换cookie或自行加入随机的头条cookie生成规则)
采集站的logo大部分已修改为透明图层png,在非白色背景下可能会很难看

contentaggregators's People

Contributors

asiacny avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

contentaggregators's Issues

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.