86. queue for “raw data” topic
THE KAPPA ARCHITECTURE
events
87. queue for “raw data” topic
THE KAPPA ARCHITECTURE
events
transform
queue for “preprocessed data” topic
88. queue for “raw data” topic
THE KAPPA ARCHITECTURE
events
transform analysis
queue for “preprocessed data” topic
queue for “analysis results” topic
89. queue for “raw data” topic
THE KAPPA ARCHITECTURE
events
transform analysis
queue for “preprocessed data” topic
queue for “analysis results” topic
reporting end-user UI
90. DATA FEDERATION IN THE COMPUTE LAYER
aggregate
trainmodels
archive
events
databases
file, object
storage
management
web and mobile
reporting
developer UItransform
transform
transform
91. DATA FEDERATION IN THE COMPUTE LAYER
aggregate
trainmodels
archive
events
databases
file, object
storage
management
web and mobile
reporting
developer UItransform
transform
transform
92. DATA FEDERATION IN THE COMPUTE LAYER
aggregate
trainmodels
archive
events
databases
file, object
storage
management
web and mobile
reporting
developer UItransform
transform
transform
121. STORAGE
Kubernetes
app 1 app 2
app 5app 4
app 3
app 6
app 1 app 2
app 5app 4
app 3
app 6
object store
✓ interoperability
✓ fine-grained AC
✓ many implementations
✗ consistency model
✗ performance
122. “…in a cloud native architecture, the benefit of
HDFS is actually very small and that is why
many cloud-first organizations no longer run
HDFS, or only run it as a caching layer for S3.”
—Reynold Xin on Quora (http://qr.ae/TAF4cN)