Giter Club home page Giter Club logo

cantor's Introduction

Cantor

Introduction

Cantor is a global sequence generator service, which is distributed, stateless and high available.

Cantor generates unique, relatively orderly and inverse decodable 64-bit integer IDs.

This project provides a easy way to use it in your docker environment. In fact, you can only use a few commands to start up all the components the service depends on and the tools, such like monitors and load testing.

This project contains several components:

  • Cantor service: the maintain service of the project
  • Redis: one of the storage Cantor can depends on, optioanl in runtime
  • HBase: one of the storage Cantor can depends on, optioanl in runtime
  • Grafana: used for monitoring Cantor instances
  • InfluxDB: storage Grafana depends on
  • JMXTrans: used for collecting the monitor data of Cantor

Protocol of ID

1 bit 2 bit 2 bit 7 bit 3 bit 28 bit 21 bit
Sign bit as 0, never used Protocol version, supporting 4 versions most Generation sources descriptor, 4 in most Custom spaces, 128 spaces most Cantor service instance number, supporting 4 instances online most Timestamp, which can be used in 8 years from 2018-01-01 00:00:00 Sequence, about 13k ids generated per seconds

In short, Cantor service guarantees that all unique IDs are generated based on its logic clock and a persistent sequence consuming state. When persistence service is down, Cantor service can downgrade to generate ID in local.

User guide

All components are dockerized and we can starts up all the components by docker-compose.

Deployment demonstration:

  • Build Java project
    • $ cd $PROJECT_ROOT
    • $ mvn clean install -N && cd http && mvn clean install && cd ../service && mvn clean package
  • Build Docker images and deploy to docker swarm:
    • $ cd $PROJECT_ROOT
    • $ docker-compose build
    • $ docker stack deploy -c docker-compose.yml cantor

Service RESTful API:

API HTTP Method Parameters Request example Return
/id GET cate: Custom category, range: How many IDs do you want return in a batch http://localhost:8080/id?cate=0&range=100 {"start": "18446744073709551616", "range":"100"}
/info GET id: The ID to decode http://127.0.0.1:8080/info?id=36313111556915201 {"sequence": 1,"descriptor": 1,"category": 2,"timestamp": 21664133}

SDK

A Java SDK to improve the productivity of interacting with Cantor service.

  • Design for thread safety
  • Cache the IDs in memory with a TTL. When IDs are expired or out of stock, SDK will retrieve a new batch from Cantor service.

Monitoring

Monitoring by jmxtrans & influxdb & grafana

Monitoring

  • Build and deploy the docker images
  • Open grafana http://localhost:3000
  • Add data source in grafana
    • Name: influxdb
    • Type: InfluxDB
    • URL: http://influxdb:8086
    • Database: jmxdb
    • User: admin
    • Password: 123456
  • Import grafana.json in grafana

Performance Test

Uses apache-jmeter as performance test client. Jmeter version: apache-jmeter-4.0.

  • Script

    • cantor_sc_100.jmx
    • cantor_sc_200.jmx
    • cantor_sc_400.jmx
    • cantor_sc_1000.jmx
    • cantor_sc_5000.jmx
    • cantor_sc_5000_10.jmx
    • cantor_sc_10000.jmx
    • cantor_sc_10000_10.jmx
    • cantor_sc_10000_300.jmx
  • Start Jmeter

    • docker exec -it <CONTAINER> /jmeter/apache-jmeter-4.0/bin/jmeter -n -t <Script>
  • Jmeter result

    • docker exec -it <CONTAINER> sort r.csv | uniq -c | awk '{print $1}'|sort|uniq -c

cantor's People

Contributors

xmuyoo avatar

Watchers

James Cloos avatar shuangyoung avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.