43

GitHub - travisjeffery/jocko: Kafka implemented in Golang with built-in coordina...

 7 years ago
source link: https://github.com/travisjeffery/jocko
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

README.md

JOCKO

Kafka/distributed commit log service in Go.

Build Status Join the chat at https://gitter.im/travisjeffery/jocko codecov

Goals of this project:

  • Implement Kafka in Go
  • Protocol compatible with Kafka so Kafka clients and services work with Jocko
  • Make operating simpler
  • Distribute a single binary
  • Use Serf for discovery, Raft for consensus (and remove the need to run ZooKeeper)
  • Smarter configuration settings
    • Able to use percentages of disk space for retention policies rather than only bytes and time kept
    • Handling size configs when you change the number of partitions or add topics
  • Learn a lot and have fun

TODO

  • Producing
  • Fetching
  • Partition consensus and distribution
  • Protocol
    • Produce
    • Fetch
    • Metadata
    • Create Topics
    • Delete Topics
    • Consumer group [current task]
  • Discovery
  • API versioning [more API versions to implement]
  • Replication [first draft done - testing heavily now]

Reading

Project Layout

├── broker        broker subsystem
├── cmd           commands
│   └── jocko     command to run a Jocko broker and manage topics
├── commitlog     low-level commit log implementation
├── examples      examples running/using Jocko
│   ├── cluster   example booting up a 3-broker Jocko cluster
│   └── sarama    example producing/consuming with Sarama
├── protocol      golang implementation of Kafka's protocol
├── prometheus    wrapper around Prometheus' client lib to handle metrics
├── server        API subsystem
└── testutil      test utils
    └── mock      mocks of the various subsystems

Building

Local

  1. Clone Jocko

    $ go get github.com/travisjeffery/jocko

  2. Build Jocko

    $ cd $GOPATH/src/github.com/travisjeffery/jocko
$ make

Docker

docker build -t travisjeffery/jocko:latest .

License

MIT


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK