105

GitHub - confluentinc/ksql: KSQL - the Streaming SQL Engine for Apache Kafka

 7 years ago
source link: https://github.com/confluentinc/ksql
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

README.md

KSQL rocket KSQL - Streaming SQL for Apache Kafka

Important: We recently announced the General Availability of KSQL. It will be available for download in early April at http://confluent.io/ksql. Until then you can download the latest KSQL Developer Preview release at https://github.com/confluentinc/ksql/releases.

KSQL is an open source streaming SQL engine for Apache Kafka. It provides a simple and completely interactive SQL interface for stream processing on Kafka; no need to write code in a programming language such as Java or Python. KSQL is open-source (Apache 2.0 licensed), distributed, scalable, reliable, and real-time. It supports a wide range of powerful stream processing operations including aggregations, joins, windowing, sessionization, and much more.

Click here to watch a screencast of the KSQL demo on YouTube. KSQL screencast

Quick Start

If you are ready to see the power of KSQL, try out these:

  • KSQL Quick Start: Demonstrates a simple workflow using KSQL to write streaming queries against data in Kafka.
  • Clickstream Analysis Demo: Shows how to build an application that performs real-time user analytics.

Use Cases and Examples

Streaming ETL

Apache Kafka is a popular choice for powering data pipelines. KSQL makes it simple to transform data within the pipeline, readying messages to cleanly land in another system.

CREATE STREAM vip_actions AS
  SELECT userid, page, action
  FROM clickstream c
  LEFT JOIN users u ON c.userid = u.user_id
  WHERE u.level = 'Platinum';

Anomaly Detection

KSQL is a good fit for identifying patterns or anomalies on real-time data. By processing the stream as data arrives you can identify and properly surface out of the ordinary events with millisecond latency.

CREATE TABLE possible_fraud AS
  SELECT card_number, count(*)
  FROM authorization_attempts
  WINDOW TUMBLING (SIZE 5 SECONDS)
  GROUP BY card_number
  HAVING count(*) > 3;

Monitoring

Kafka's ability to provide scalable ordered messages with stream processing make it a common solution for log data monitoring and alerting. KSQL lends a familiar syntax for tracking, understanding, and managing alerts.

CREATE TABLE error_counts AS
  SELECT error_code, count(*)
  FROM monitoring_stream
  WINDOW TUMBLING (SIZE 1 MINUTE)
  WHERE  type = 'ERROR'
  GROUP BY error_code;

Latest news

Documentation

You can find the KSQL documentation here

Join the Community

Whether you need help, want to contribute, or are just looking for the latest news, you can find out how to connect with your fellow Confluent community members here.

Contributing

Contributions to the code, examples, documentation, etc, are very much appreciated. For more information, see the contribution guidelines.

License

The project is licensed under the Apache License, version 2.0.

Apache, Apache Kafka, Kafka, and associated open source project names are trademarks of the Apache Software Foundation.


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK