README.md

Cantor

Introduction

Cantor is a global sequence generator service, which is distributed, stateless and high available.

Cantor generates unique, relatively orderly and inverse decodable 64-bit integer IDs.

This project provides a easy way to use it in your docker environment. In fact, you can only use a few commands to start up all the components the service depends on and the tools, such like monitors and load testing.

This project contains several components:

• Cantor service: the maintain service of the project
• Redis: one of the storage Cantor can depends on, optioanl in runtime
• HBase: one of the storage Cantor can depends on, optioanl in runtime
• Grafana: used for monitoring Cantor instances
• InfluxDB: storage Grafana depends on
• JMXTrans: used for collecting the monitor data of Cantor

Protocol of ID

1 bit 2 bit 2 bit 7 bit 3 bit 28 bit 21 bit Sign bit as 0, never used Protocol version, supporting 4 versions most Generation sources descriptor, 4 in most Custom spaces, 128 spaces most Cantor service instance number, supporting 4 instances online most Timestamp, which can be used in 8 years from 2018-01-01 00:00:00 Sequence, about 13k ids generated per seconds

In short, Cantor service guarantees that all unique IDs are generated based on its logic clock and a persistent sequence consuming state. When persistence service is down, Cantor service can downgrade to generate ID in local.

User guide

All components are dockerized and we can starts up all the components by docker-compose.

Deployment demonstration:

• Build Java project
  • $ cd $PROJECT_ROOT
  • $ mvn clean install -N && cd http && mvn clean install && cd ../service && mvn clean package
• Build Docker images and deploy to docker swarm:
  • $ cd $PROJECT_ROOT
  • $ docker-compose build
  • $ docker stack deploy -c docker-compose.yml cantor

Service RESTful API:

API HTTP Method Parameters Request example Return /id GET cate: Custom category, range: How many IDs do you want return in a batch http://localhost:8080/id?cate=0&range=100 {"start": "18446744073709551616", "range":"100"} /info GET id: The ID to decode http://127.0.0.1:8080/info?id=36313111556915201 {"sequence": 1,"descriptor": 1,"category": 2,"timestamp": 21664133}

SDK

A Java SDK to improve the productivity of interacting with Cantor service.

• Design for thread safety
• Cache the IDs in memory with a TTL. When IDs are expired or out of stock, SDK will retrieve a new batch from Cantor service.

Monitoring

Monitoring by jmxtrans & influxdb & grafana

• Build and deploy the docker images
• Open grafana http://localhost:3000
• Add data source in grafana
  • Name: influxdb
  • Type: InfluxDB
  • URL: http://influxdb:8086
  • Database: jmxdb
  • User: admin
  • Password: 123456
• Import grafana.json in grafana

Performance Test

Uses apache-jmeter as performance test client. Jmeter version: apache-jmeter-4.0.

• Script

  • cantor_sc_100.jmx
  • cantor_sc_200.jmx
  • cantor_sc_400.jmx
  • cantor_sc_1000.jmx
  • cantor_sc_5000.jmx
  • cantor_sc_5000_10.jmx
  • cantor_sc_10000.jmx
  • cantor_sc_10000_10.jmx
  • cantor_sc_10000_300.jmx

• Start Jmeter

  • docker exec -it <CONTAINER> /jmeter/apache-jmeter-4.0/bin/jmeter -n -t <Script>

• Jmeter result

  • docker exec -it <CONTAINER> sort r.csv | uniq -c | awk '{print $1}'|sort|uniq -c