474

GitHub - hellobike/amazonriver

 5 years ago
source link: https://github.com/hellobike/amazonriver
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

README.md

amazonriver CircleCI

Go Report Card golang GoDoc GitHub release

amazonriver 是一个将postgresql的实时数据同步到es或kafka的服务

版本支持

  • Postgresql 9.4 or later
  • Kafka 0.8 or later
  • ElasticSearch 6.x

原理

amazonriver 利用pg内部的逻辑复制功能,通过在pg创建逻辑复制槽,接收数据库的逻辑变更,通过解析test_decoding特定格式的消息,得到逻辑数据

安装使用

安装

$git clone https://github.com/hellobike/amazonriver
$cd amazonriver
$glide install
$go install

使用

amazonriver -config config.json

PG 配置

PG数据库需要预先开启逻辑复制pg配置

amazonriver 配置

监控

amazonriver支持使用prometheus来监控同步数据状态,配置Grafana监控

同步到 elasticsearch

同步到elasticsearch

同步到 kafka

同步到kafka

许可

amazonriver 使用 Apache License 2 许可


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK