This document contains multiple sections from Anton Chuvakin providing his top 11 reasons for various aspects of log management. The sections cover reasons to collect and preserve logs, look at logs, secure and protect logs, analyze logs, and finally a humorous take on reasons to hate logs. Anton is presented as a recognized security expert in log management and PCI compliance who has published extensively on these topics.

1. Originally posted on my blog “Security Warrior” www.securitywarrior.org –
reposted here all in one place.
DISCLAIMER:
Security is a rapidly changing field of human endeavor. Threats we face literally
change every day; moreover, many security professionals consider the rate of
change to be accelerating. On top of that, to be able to stay in touch with such
ever-changing reality, one has to evolve with the space as well. Thus, even
though I hope that this document will be useful for to my readers, please keep in
mind that is was possibly written years ago. Also, keep in mind that some of the
URL might have gone 404, please Google around.
Anton's "Top 11 Reasons to Collect and Preserve Computer Logs", presented in no
particular order:
1. Before anything else, do you deal with credit cards? Patient info? Are you a
government org under FISMA? A financial org? You have to keep'em - stop
reading further.
2. What if there is a law or a regulation that requires you to retain logs - and you
don't know about it yet? Does the world "compliance" ring a bell?
3. An auditor comes and asks for logs. Do you want to respond "Eh, what do you
mean?"?
4. A system starts crashing and keeps doing so. Where is the answer? Oops, it was in
the logs - you just didn't retain them ...
5. Somebody posts a piece of your future quarterly report online. Did John Smith did
it? How? If not him, who did? Let's see who touched this document, got logs?
6. A malware is rampant on your network. Where it came from? Who spreads it?
Just check the logs - but only if you have them saved.
7. Your boss comes and says 'I emailed you this and you ignored it!!' - 'No, you
didn't!!!' Who is right? Only email logs can tell!
8. Network is slow; somebody is hogging the bandwidth. Let's catch the bastard! Is
your firewall logging? Keep the info at least until you can investigate.
9. Somebody added a table to your database. Maybe he did something else too - no
change control forms were filed. Got database log management? How else would
you know?
10. Disk space is cheap; tape is cheaper still. Save a log! Got SAN or NAS? Save a
few of them!
11. If you plan to throw away a log record, think - are you 100% sure you won't need
it, ever? Exactly! :-) Keep it.
Anton's "Top 11 Reasons to Look at Your Logs", presented in no particular order:

2. 1. The first reason is again disarmingly simple (is it :-)). Read PCI DSS lately?
Glanced at HIPAA? Suffer under FISMA? Yup, all of the above say that you
must not only have, but also review logs periodically.
2. Are you 0wned? How do you know if all your logs are stashed on a tape in a
closet? Look at them! Now!!
3. An incident happens. Really, who needs extra motivation to look at logs in this
case? Duh! Logs for incident response is a "no-brainer" use case for log review.
4. Users - from CEO to a janitor. You might have to know what they do on your IT
systems! How? Read the logs! Everybody leaves tracks.
5. Logged system errors. Sometimes they are stupid, sometimes - benign. However,
often they mean that "stuff" is about to hit the fan. Periodic review of logs reveals
them and saves the day.
6. Network slowed to a crawl? Applications are slooow? Server is not ... well,
serving? :-) Where is the answer? In the logs, but you need to read them and
understand them.
7. That policy you wrote a few months ago. Anybody following that? Anybody
remembers that? Halloooo! Check the logs and you'd know.
8. You know your auditor might check your logs. But did you know they might also
check whether you looked at them? Did'ya? Review the logs and leave the record
of this activity!
9. Change can be good. But then again, it may be the sign that your controls are
lacking. Who changes what and when? From what and to what? Just review the
logs.
10. Now, you hate looking at logs. You have too many! In this case, look at a specific
subset of logs that you never saw before- NBS. Or just deploy log
management that can do it for you.
11. Logs can help you predict the future (if you review, know and love them :-)).
Don't believe it? If you read them for long enough, you develop an ability to -
gasp!- predict the future, albeit mostly future problems :-)
Anton’s "Top 11 Reasons to Secure and Protect Your Logs", presented in no
particular order:
1. Let's review why you are reviewing logs. Will logs that might have been changed
by somebody, somewhere, somehow still be useful for items 1-11 from here? No?
Secure them!
2. Oooh, logs in court? Challenges abound! To respond to them, one needs to protect
the logs so you can claim that they are both authentic and reliable.
3. A human error still beats an evil hacker as the main cause of IT problems. Are
your logs safe from it? Available when needed? Protect them from crashes and
other faults!

3. 4. PCI DSS just says so: "Secure audit trails so they cannot be altered." Wonna do it-
or pay the fines?
5. Do you protect financial records? Identity info? Passwords? Some of it ends up in
logs - thus making them more sensitive. Secure the C-I-A of logs!
6. Do you look at logs during incident investigation? Do you want them to be "true"
or full of random (if creative...) cr*p, inserted by the guilty party? Secure the logs!
7. Think that "attacks vs logging" are theoretical? Think again. Are your logs safe or
vulnerable? Is your logging tool 0wned?
8. Syslog + UDP = log injection. Are you protected (reliable TCP, confirmed
delivery, encryption - SSH, SSL, VPN)?
9. Why change logs? No, really, why change logs? If you never change logs - and
you never should - hash them right away after collection to make them
immutable.
10. Logs are backed up on tape - who will see them? Well, whoever restores the tape,
that's who! Encrypt them to protect them from accidental and malicious disclosure
if tape is lost.
11. Why log access to logs? Same reason why you had the logs in the first place - to
review who did what. Who broke through and stole the logs? Who browsed them
without permission? Only logs will tell - if you have them!
Anton’s "Top 11 Reasons to Analyze Your Logs" Don't just read your logs; analyze
them. Why? Here are the reasons:
1. Seen an obscure log message lately? Me too - in fact, everybody have. How do
you know what it means (and logs usually do mean something) without analysis?
At the very least, you need to bring additional context to know what some logs
mean.
2. Logs often measure in gigabytes and soon will in terabytes; log volume grows all
the time - it passed a limit of what a human can read a long time ago, it then made
simple filtering 'what logs to read' impossible as well: automated log analysis is
the only choice.
3. Do you peruse your logs in real time? This is simply absurd! However, automated
real-time analysis is entirely possible (and some logs do crave for your attention
ASAP!)
4. Can you read multiple logs at the same time? Yes, kind of, if you print them out
on multiple pages to correlate (yes, I've seen this done :-)). Is this efficient? God,
no! Correlation across logs of different types is one of the most useful approaches
to log analysis.
5. A lot of insight hides in "sparse" logs, logs where one record rarely matters, but a
large aggregate does (e.g. from one "connection allowed" firewall log to a scan
pattern). Thus, the only way to extract that insight from a pool of data is through
algorithms (or, as some say, visualization)
6. Ever did a manual log baselining? This is where you read the logs and learn
which ones are normal for your environment. Wonna do it again? :-) Log baseline

4. learning is a useful and simple log analysis technique, but humans can only do it
for so much.
7. OK, let's pick the important logs to review. Which one are those? The right
answer is "we don't know, until we see them." Thus, to even figure out which logs
to read, you need automated analysis.
8. Log analysis for compliance? Why, yes! Compliance is NOT only about log
storage (e.g. see PCI DSS). How to highlight compliance-relevant messages?
How to see which messages will lead to a violation? How do you satisfy those
"daily log review" requirements? Thru automated analysis, of course!
9. Logs allow you to profile your users, your data and your resources/assets.
Really? Yes, really: such profiling can then tell you if those users behave in an
unusual manner (in fact, the oldest log analysis systems worked like that). Such
techniques may help reach the holy grail of log analysis: automatically tell you
what matters for you!
10. Ever tried to hire a log analysis expert? Those are few and far between. What if
your junior analysts can suddenly analyze logs just as well? One log analysis
system creator told me that his log data mining system enabled exactly that. Thus,
saving a lot of money to his organization.
11. Finally, can you predict future with your logs? I hope so! Research on predictive
analytics is ongoing, but you can only do it with automated analysis tools, not
with just your head alone (no matter how big :-)) ...
Finally, HUMOROUS (posted on April 1st)
Anton’s Top 11 Reasons to Hate Logs
You thought I am done with my Top 11 lists? Nah... here is one more, which
actually is designed to bite you in the ass on a certain date. So, "Top 11
Reasons to HATE Logs ... With a Passion."
1. Read any logs lately? Got bored in 5 minutes - or survived for the
whopping 10? Congrats, you score a point! But logs are
still boooooooooooooooooooooooooooooring.
2. One log, two logs, 10 logs.... 1,000,000,000 logs: rabbits and
hamsters cannot match the speed with whichlogs multiply. Don't you
just hate that?
3. You keep hearing people refer to "log data." Then you run 'tail
/var/log/messages' and see text in pidgin English. Where is my data?
Hate it!
4. "Real hackers don't get logged": thus logs are seen as useless - and
hated by some "hard core" security pros!

5. 5. If people lie to you, you hate it. Logs do lie too (see 'false positives')
- and they are hated too.
6. 'Transport error 202 message repeated 3456 times.' Niiiiice. Now go
fix that! Fix what? Ah, hate the log obscurity!
7. Why are there 47 different ways to log that "connection from A to B
was established OK?" Or 21 way to say "user logged in OK?" No,
really? Why? Who can I kill to stop this insanity?
8. You MUST do XYZ with logs for compliance. Or you are going to jail,
buddy! No, sorry, we can't tell you what XYZ is. Maybe in 7 years; for
now, just store everything.
9. 'Critical error: process completed successfully' and
'Operation successfully failed' engender deep and lasting hatred of
logs in most people. They just do ...
10. The book called "Ugliest Logs Ever!" is a fat tome, covering every log
source from a Linux system all the way to databases and CRM. Bad
logs are popular! Bad logs are all the rage among the programmers!
Bad logs are here to stay. Bad logs that mean nothing power the log
hatred.
11. "Logs: can't live with them, can't live without them" :-) Hate them we
might for different reasons, but we still must collect, protect, review,
and analyze them ...
ABOUT AUTHOR:
This is an updated author bio, added to the paper at the time of reposting in
2009.
Dr. Anton Chuvakin (http://www.chuvakin.org) is a recognized security expert in
the field of log management and PCI DSS compliance. He is an author of books
"Security Warrior" and "PCI Compliance" and a contributor to "Know Your Enemy
II", "Information Security Management Handbook" and others. Anton has
published dozens of papers on log management, correlation, data analysis, PCI
DSS, security management (see list www.info-secure.org) . His blog
http://www.securitywarrior.org is one of the most popular in the industry.
In addition, Anton teaches classes and presents at many security conferences
across the world; he recently addressed audiences in United States, UK,
Singapore, Spain, Russia and other countries. He works on emerging security
standards and serves on the advisory boards of several security start-ups.
Currently, Anton is developing his security consulting practice, focusing on
logging and PCI DSS compliance for security vendors and Fortune 500

6. organizations. Dr. Anton Chuvakin was formerly a Director of PCI Compliance
Solutions at Qualys. Previously, Anton worked at LogLogic as a Chief Logging
Evangelist, tasked with educating the world about the importance of logging for
security, compliance and operations. Before LogLogic, Anton was employed by a
security vendor in a strategic product management role. Anton earned his Ph.D.
degree from Stony Brook University.