This document discusses the thread and process safety of the Ruby Logger. It was found that while Logger is thread-safe due to use of a mutex, it is not process-safe as the mutex does not work across processes. The author contributed a fix to Ruby core that uses flock to lock access to logs across processes, ensuring process-safe logging. The fix will be included in Ruby 2.1.0, making the standard Ruby Logger thread and process-safe.
9. Examined Ruby Logger
• 1) Will logs not be mixtured in multi-threads?
• 2) Will logs not be mixtured in multi-processes?
• 3) Does log rotation work safely in multi-threads?
• 4) Does log rotation work safely in multi-processes?
10. 1) Will logs not be mixtured in multi-threads?
Code
require 'logger'
require 'parallel'
logger = Logger.new("/tmp/test.log")
Parallel.map(['a', 'b'], :in_threads => 2) do |letter|
10000.times do
logger.info letter * 5000
end
end
Result
$ egrep -e 'ab' -e 'ba' /tmp/test.log
[empty]
OK
12. 2) Will logs not be mixtured in multi-processes?
Code
require 'logger'
require 'parallel'
logger = Logger.new("/tmp/test.log")
Parallel.map(['a', 'b'], :in_processes => 2) do |letter|
10000.times do
logger.info letter * 5000
end
end
Result
$ egrep -e 'ab' -e 'ba' /tmp/test.log
[empty]
OK
13. Really? Why?
Mutex does not work to exclusively lock in multiprocesses environment.
Atomic/non-atomic: A write is atomic if the whole amount written in one
operation is not interleaved with data from any other process. This is
useful when there are multiple writers sending data to a single reader.
Applications need to know how large a write request can be expected to
be performed atomically. This maximum is called {PIPE_BUF}. This volume
of IEEE Std 1003.1-2001 does not say whether write requests for more
than {PIPE_BUF} bytes are atomic, but requires that writes of {PIPE_BUF} or
fewer bytes shall be atomic.
But, Linux system call write(2) itself is atomic as long
as writing to a local file.
OK
14. 3) Does log rotation work safely multi-threads?
Code
require 'logger'
require 'parallel'
logger = Logger.new("/tmp/test.log", 3, 1024 * 10)
Parallel.map(['a', 'b'], :in_threads => 2) do |letter|
10000.times do
logger.info letter * 5000
end
end
Result
$ ls -l /tmp/test*
-rw-r--r-- 1 sonots sonots 10806
9月 29 23:03 /tmp/test.log
-rw-r--r-- 1 sonots sonots 10806
9月 29 23:03 /tmp/test.log.0
-rw-r--r-- 1 sonots sonots 10806
9月 29 23:03 /tmp/test.log.1
No Error, OK
15. 4) Does log rotation work safely in multi-processes?
Code
require 'logger'
require 'parallel'
logger = Logger.new("/tmp/test.log", 3, 1024 * 10)
Parallel.map(['a', 'b'], :in_processes => 2) do |letter|
10000.times do
logger.info letter * 5000
end
end
Result
log
log
log
log
...
writing failed. closed stream
shifting failed. closed stream
writing failed. closed stream
shifting failed. closed stream
Oops, Bad!