handanchen Goto Github PK

article

All articles from website-www.kisspuppet.com

awesome-java-cn

Java资源大全中文版，包括开发库、开发工具、网站、博客、微信、微博等，由伯乐在线持续更新。

awesome-wechat-weapp

微信小程序开发资源汇总 :100:

bad-data-guide

An exhaustive reference to problems seen in real-world data along with suggestions on how to resolve them.

canal

阿里巴巴mysql数据库binlog的增量订阅&消费组件

cdhexample

CDH集群环境Hdfs、MapReduce、Hive、Hbase、Kafka、Solr、Spark、Zookeeper、Mahout示例代码

cds

Content Data Store (HDFS/HBase)

chatbotcourse

自己动手做聊天机器人教程

cloud

云计算之hadoop、hive、hue、oozie、sqoop、hbase、zookeeper环境搭建及配置文件

clouderasizer

Multipurpose tool for discovering and collecting Cloudera Manager metrics.

crawer

各大电商网站数据抓取分析

crawlerqunar

2013年4月：一个爬行去哪儿网（qunar.com）数据的爬虫脚本。提供了一种爬行AJAX类型网站数据的方法。

distribute_crawler

使用scrapy,redis, mongodb,graphite实现的一个分布式网络爬虫,底层存储mongodb集群,分布式使用redis实现,爬虫状态显示使用graphite实现

django-dynamic-scraper

Creating Scrapy scrapers via the Django admin interface

django-redis

Full featured redis cache backend for Django.

douappbook

Crawl book and rating infomations from Douban App

dw_etl

dw etl 工具 mysql 增量、全量抽取 to hive. 合并 hive 数据表, 等数据平台清洗工具

e-business

电商爬虫系统：京东，当当，一号店，国美爬虫（代理使用）

easyrec

easyrec

elite-proxies-scrapy-middleware

Elite Proxies (http://elite.proxies.online) middleware for scrapy http://rev.proxies.online

filecrush

Remedy small files by combining them into larger ones.

filecrusher

financialnewssearchengine

Very simple search engine "specialised" in searching financial news (written using Nutch, Hbase, Solr, SpringBoot, Bootstrap and AngularJS)

hadoop

Mirror of Apache Hadoop

hadoop-combinefileinputformat

Example implementation of hadoop CombineFileInputFormat

hbase-increment-index

hbase+solr实现hbase的二级索引

hbase-indexer

Lily HBase Indexer - indexing HBase, one row at a time

hbase-solr-coprocessor

通过solr实现hbase二级索引，主要通过hbase的coprocessor的Observer实现。

hbasesolrtool

hdfs-file-slurper

Utility to easily copy files into HDFS