This document provides an overview of building a first Apache Apex application. It describes the main concepts of an Apex application including operators that implement interfaces to process streaming data within windows. The document outlines a "Sorted Word Count" application that uses various operators like LineReader, WordReader, WindowWordCount, and FileWordCount. It also demonstrates wiring these operators together in a directed acyclic graph and running the application to process streaming data.

1. © 2015 DataTorrent
Munagala V. Ramanath (“Ram”) <ram@datatorrent.com>
Dec 15th, 2015
Building your First Apache Apex
Application

2. © 2015 DataTorrent
Outline
Main concepts of an Apex Application.
Brief description of the "Sorted Word Count" application.
Hands on demonstration of cloning the source repository and building
Apex source code.
Hands on demonstration of creating a new application.
Running the application.
Code walk-through.
Questions

3. © 2015 DataTorrent
Main Concepts
Applications are built from Operators which implement the Operator interface;
each operator has input/output ports which are connected by streams to form a
directed acyclic graph (DAG).
A BaseOperator class is provided which provides empty implementations of all the
required methods.
Within an operator, define necessary input and output ports typically using the
DefaultInputPort and DefaultOutputPort classes.
The Application class implements the StreamingApplication interface; need only
implement populateDAG() method which wires the operators together.
Applications process data within time-based windows, typically 0.5s.

4. © 2015 DataTorrent
The Sorted Word Count Application
The following operators are involved:
LineReader: reads file dropped into input directory and outputs lines (on its
output port).
WordReader: splits each line into words using a regex.
WindowWordCount: compute and emit word frequencies for all words in lines
processed in current window.
FileWordCount: accumulates all word counts for current file and emits final
sorted list when EOF is reached.
WordCountWriter: writes list to output file in output directory.

5. © 2015 DataTorrent
The DAG

6. © 2015 DataTorrent
Resources
6
Apache Apex Community Page
Apache Apex LinkedIn Group

7. © 2015 DataTorrent
End
7