Apache Beam: A unified model for batch and stream processing data
In search of good design...
1. In search of good design...
// When I wrote this code, only God and I understood what I was doing
// Now, God only knows
Monday, October 8, 12
2. ABOUT ME
TWITTER.COM/FAUST45
GITHUB.COM/FAUST45
Monday, October 8, 12
3. What is programing about?
Goal of program convert input to reasonable output.
We calculate something and storing temp results in
vars, until we can produce complete result.
Monday, October 8, 12
4. What is programing about?
temp temp complete
results results result
Calculates Calculates Calculates
Monday, October 8, 12
5. What is programing about?
Program
State
temp temp complete
results results result
Calculates Calculates Calculates
Monday, October 8, 12
6. What is programing about?
Program
State
temp temp complete
results results result
Calculates Calculates Calculates
Layer control order of calculations
Monday, October 8, 12
7. What is programing about?
Program
State
temp temp complete
results results result
Layer manages access to program state
Calculates Calculates Calculates
Layer control order of calculations
Monday, October 8, 12
8. Looks simple?
Program
State
temp temp complete
results results result
Layer manages access to program state
Calculates Calculates Calculates
Layer control order of calculations
Monday, October 8, 12
25. Functional model
Program
State
temp temp complete
results results result
Layer manages access to program state
Calculates Calculates Calculates
Layer control order of calculations
Monday, October 8, 12
27. Units of pure calculations
have no direct access to
program state
Monday, October 8, 12
28. You pass a pice of state in
params
function send_money(sender_account,
receiver_account,
amount_to_send)
Monday, October 8, 12
29. and return result of
calculations, some
new pice of state
function send_money(sender_account, ...)
return
how_sender_account_looks_after_money_was_send
Monday, October 8, 12
31. Functional
Program
state
Pice of program
state
Calculate
return result
Monday, October 8, 12
32. Imperative
Program
state
Pice of program
dir state
ect
acc
ess
Calculate
return result
Monday, October 8, 12
33. Rich Hickey
Creator of clojure, language with immutable
data by default.
Dialect of Lisp on top of Java Virtual Machine
He was inspired by paper “Out of the Tar Pit”
Monday, October 8, 12
34. Model where you don’t
rewrite values but
produce new values
Monday, October 8, 12
39. Resources
★ “Out of the Tar Pit”
★ “The Art of Unix Programming”
★ “Functional programming is a ghetto” (blog bost by
Michael O.Church)
★ Rich Hickey videos
★ Stuart Holloway videos - “Simplicity Ain't Easy”
Monday, October 8, 12