Apache Flink: Scalable Batch and Stream Data Processing

Apache Flink® is an open source platform for distributed stream and batch data processing.

Flink’s core is a streaming dataflow engine that provides data distribution, communication, and fault tolerance for distributed computations over data streams.

Flink includes several APIs for creating applications that use the Flink engine:

DataStream API for unbounded streams embedded in Java and Scala, and
DataSet API for static data embedded in Java, Scala, and Python,
Table API with a SQL-like expression language embedded in Java and Scala.

Flink also bundles libraries for domain-specific use cases:

CEP, a complex event processing library,
Machine Learning library, and
Gelly, a graph processing API and library.

You can integrate Flink easily with other well-known open source systems both for data input and output as well as deployment.

Streaming First

High throughput and low latency stream processing with exactly-once guarantees.

Batch on Streaming

Batch processing applications run efficiently as special cases of stream processing applications.

APIs, Libraries, and Ecosystem

DataSet, DataStream, and more. Integrated with the Apache Big Data stack.

Check out the Features page to get a tour of all major Flink features and the Powered by Flink page to see real-world Flink use cases.

Getting Started

Download the latest stable release and run Flink on your machine, cluster, or cloud:

Download Apache Flink® 1.1 Apache Flink® on GitHub

The documentation contains a setup guide for all deployment options.

The programming guide contains all information to get you started with writing and testing your Flink programs.

See our list of third-party packages for Flink.

Check out the documentation for the next steps.

Latest blog posts

12 Oct 2016 » Apache Flink 1.1.3 Released
05 Sep 2016 » Apache Flink 1.1.2 Released
24 Aug 2016 » Flink Forward 2016: Announcing Schedule, Keynotes, and Panel Discussion
11 Aug 2016 » Flink 1.1.1 Released
08 Aug 2016 » Announcing Apache Flink 1.1.0

Check out the blog for all posts.

Community

You can post questions to the Flink community on various channels. Pick the one, which suits you best:

User mailing list. Subscribe to the mailing list by sending an empty email to [email protected]. Once the subscription is confirmed, you can send questions to [email protected].
Stack Overflow. Post your questions to Stack Overflow and tag them with #flink.
IRC chat. The IRC channel #flink at irc.freenode.org is dedicated to Apache Flink. Join the channel and chat with the Flink community.

Check out the community page for all community-related information. If you want to contribute, make sure to have a look at the contribution guide.

Tweets by @ApacheFlink

Apache Flink, Apache, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation.