5. v10
⢠Mainly for log forwarding
⢠with good performance
⢠working in many productions reliably
⢠Various plugins are released
⢠There are 200+ plugins!
⢠Mainly for CRuby
5
Sunday, January 26, 14
6. Why v11?
⢠We want more useful features, but...
⢠Changing the internal architecture is hard
⢠Keeping the external API is even harder
⢠Originally, v10 was started as a prototype
⢠Need drastic changes for the future :)
6
Sunday, January 26, 14
7. Whatâs new?
⢠ServerEngine based reliable architecture
⢠Zero-downtime restarting
⢠Filter / Label for ďŹexible record handling
⢠Improved conďŹguration
⢠Improved plugin development
7
Sunday, January 26, 14
8. ServerEngine based
⢠Robust signal handling
⢠Put a signal into Queue ďŹrst
⢠Built-in supervisor
⢠Multiprocess support
⢠No need for in_multiprocess plugin
8
Sunday, January 26, 14
10. Zero downtime restart
⢠SocketManager shares resources with
workers
Supervisor
1. Listen to TCP socket
TCP
10
Sunday, January 26, 14
11. Zero downtime restart
⢠SocketManager shares resources with
workers
Supervisor
TCP
1. Listen to TCP socket
2. Pass its socket to worker
heartbeat
Worker
TCP
11
Sunday, January 26, 14
12. Zero downtime restart
⢠SocketManager shares resources with
workers
Supervisor
TCP
heartbeat
Worker
1. Listen to TCP socket
2. Pass its socket to worker
3. Do same action
at worker restarting
with keeping TCP socket
Worker
TCP
12
Sunday, January 26, 14
13. Filter / Label support
⢠No more tag-related tricks!
⢠add_tag_xxx, remove_tag_xxx, etc...
⢠Redirect events to another group
⢠Much easier to group and share plugins
13
Sunday, January 26, 14
14. Filter
⢠<match> can have nested <match>
⢠ConďŹguration format is not ďŹxed!
v10:
<match access.**>
type ďŹowcounter
add_tag_preďŹx counted
</match>
<match counted.**>
typo growthforecast
</match>
v11:
<match access.** copy>
type ďŹowcounter
<match **>
typo growthforecast
</match>
</match>
14
Sunday, January 26, 14
15. Label
⢠<label> can contain multiple <match>
⢠out_redirect can forward events to <label>
<match access.**>
type rewrite_tag_ďŹlter
...
<match bang.**>
type redirect
to_label blackhole
</match>
...
</match>
<label blackhole>
<match **>
type null
</match>
</label>
bangâs record go away!
15
Sunday, January 26, 14
16. Improved conďŹguration
⢠Can use Hash, Array and others
⢠No need for â,â or similar tricks
⢠You can write Ruby directly
⢠Worker pragma
⢠Separate plugins in each process
⢠Support the v10 compatible mode
16
Sunday, January 26, 14
17. New parameter types
⢠Can write complex values without DSL!
⢠Can use Ruby code for conďŹguration
Hash, Array, etc:
Embedded Ruby code:
<source>
type my_tail
keys ["k1", "k2", "k3"]
</source>
<match **>
typo my_ďŹlter
add_keys {"k1" : "v1"}
</match>
Sunday, January 26, 14
<match ** copy>
type my_ďŹlter
env "#{ENV['KEY']}"
</match>
â˘
â˘
â˘
Socket.gethostname
`command`
etc...
17
18. Improved plugin
⢠Actor
⢠Abstraction layer for several actions
⢠Error stream
⢠with @ERROR label
⢠Log level per plugin
⢠No more global API, Engine.emit, $log, etc...
18
Sunday, January 26, 14
19. Actor
⢠Easy to write popular routines
v10:
v11:
class TimerWatcher <
Coolio::TimerWatcher
...
end
def start
@loop = Coolio::Loop.new
@timer = ...
@loop.attach(@timer)
@thread = ...
end
Sunday, January 26, 14
actor.every(@interval) {
event_router.emit(...)
}
19
20. Error stream
⢠Can handle an error at each record level
Input
{"event":1, ...}
chunk1
{"event":2, ...}
{"event":3, ...}
{"event":4, ...}
Output
OK
ERROR!
OK
OK
âŚ
{"event":5, ...}
{"event":6, ...}
chunk2
ERROR!
OK
Sunday, January 26, 14
Error stream
<label @ERROR>
<match **>
type ďŹle
...
</match>
</label>
Built-in @ERROR is used
when error occurred in âemitâ
20
21. Lastly...
⢠Super alpha status!
⢠Adding / Removing features may occur
⢠There are several âto be implementedâ features
⢠The release date is not ďŹxed yet
⢠The initial release should be useful from day one
⢠Need feedbacks!
21
Sunday, January 26, 14