GitHub - hazelcast/hazelcast-jet: A general purpose distributed data processing...

README.md

Hazelcast Jet

Hazelcast Jet is a distributed computing platform built for high-performance stream processing and fast batch processing. It embeds Hazelcast In-Memory Data Grid (IMDG) to provide a lightweight, simple-to-deploy package that includes scalable in-memory storage.

Visit jet.hazelcast.org to learn more about the architecture and use cases.

Features:

• Low-latency, high-throughput distributed data processing framework.
• Highly parallel and distributed stream and batch processing of data.
• Connectors allowing high-velocity ingestion of data from Apache Kafka, HDFS, Hazelcast IMDG, sockets and local data files (such as logs or CSVs).
• API for custom connectors.
• Dynamic node discovery for both on-premise and cloud deployments.
• Virtualization support and resource management via Docker, Apache jclouds, Amazon Web Services, Microsoft Azure, Consul, Heroku, Kubernetes, Pivotal Cloud Foundry and Apache ZooKeeper.

Getting Started

See the Getting Started Guide.

Code Samples

See Hazelcast Jet Code Samples for some examples.

Documentation

See the Jet Reference Manual.

Architecture

See Jet Architecture.

High Performance Design

See the write up on our high performance secret sauce.

Releases

Download from jet.hazelcast.org.

Use Maven snippet:

<dependency>
    <groupId>com.hazelcast.jet</groupId>
    <artifactId>hazelcast-jet</artifactId>
    <version>${hazelcast.jet.version}</version>
</dependency>

Snapshot Releases

Maven snippet:

<repositories>
    <repository>
        <id>snapshot-repository</id>
        <name>Maven2 Snapshot Repository</name>
        <url>https://oss.sonatype.org/content/repositories/snapshots</url>
        <snapshots>
            <enabled>true</enabled>
            <updatePolicy>daily</updatePolicy>
        </snapshots>
    </repository>
</repositories>
<dependencies>
    <dependency>
        <groupId>com.hazelcast.jet</groupId>
        <artifactId>hazelcast-jet</artifactId>
        <version>${hazelcast.jet.version}</version>
    </dependency>
</dependencies>

Building From Source

Pull the latest commits from GitHub (git pull) and use Maven to build (mvn clean install). This will also run all the checks and tests: Checkstyle, FindBugs, and JUnit tests.

Java 8 and Java 9 are supported.

Maven version 3.5.2 is the minimum version required.

Contributing to Hazelcast Jet

We encourage pull requests and process them promptly.

To contribute:

• see Developing with Git for our Git process
• complete the Hazelcast Contributor Agreement

For an enhancement or larger feature, create a GitHub issue first to discuss.

Submit your contribution as a pull request on GitHub. Each pull request is subject to automatic verification, so make sure your contribution passes the mvn clean install build locally before submitting it.

Mail Group

Please join the mail group if you are interested in using or developing Hazelcast Jet.

http://groups.google.com/group/hazelcast-jet

License

Hazelcast Jet is available under the Apache 2 License. Please see the Licensing section for more information.

Copyright

Copyright (c) 2008-2018, Hazelcast, Inc. All Rights Reserved.

Visit www.hazelcast.com for more info.