This presentation will help you understand the importance of logging in applications. Every project is encountered with this aspect, functional or non-functional.This will become more and more important with current innovations: mobile (~offline) applications, SOA-architectures, Cloud integration. An overview of the Java Logging-ecospace is discussed. Eventually, the results are processed and analyzed with facilitating tools. What is logging? Why logging? Who uses logs and what are they used for? What needs to be logged? How to log in Java Log processing and analysis

2. APPLICATION LOGGING

3. About ME Sander De Vos Software Engineer RealDolmen 2008 Past projects MaTMa (M*) E-Metro (FOD-ECO) SP10B (FOD-FIN) MIK (FOD-MOB)

4. QuotesFrom The Field “Whoneeds logs? Isuredon’t!” “Debugger is better! Sysoutworkstoo!” “Isn’tthatwhatinfrastructure does?” “It slows down disk and memory!” “It isn’t in the spec!” “We don’tknowwhatto log – we’lljust log something!” “We don’tknowhow – we’lljust do itsomehow!”

5. Agenda What is logging Definition andChallenges Whylogging Whouses logs What logs are usedfor Functional or non-functional Whatto log Whatwill (not) help you Log messagedecomposed Guidelines How to log in Java Logging in production Log processing and analysis

6. What is logging

7. Definition “Logs are the recordings of one or more events occurring on information systems.” “Logs can be organized based on the program, day, severity, host, or a number of other categories.“ “Logs may be referred to as log files, audit logs, or audit trails. Logs also cover alerts, alarms and event records.“

8. Log challenges Decentralization Cloud SOA Volatility Multiple tiersandlayers Archival Accessability Absence of critical information Random log formats

9. WHY logging

10. Whouses logs Managed Services Security Teams Monitor Detect Investigate Track Analyze Auditors Developers Debugging Customer

11. What logs are usedfor Debugging andForensics Fault monitoring Performance monitoring Troubleshooting Feature usage Security / Incident detection Regulatoryand standards compliance

12. Functional – non-functionallogging Functional Audit Governance Security Non-functional Faults, Errors, Exceptions Execution context Performance Component usage

13. Putting italltogether Functional Logs CustomerAudit Always on Requirements Scope known Years ‘Clutter, yuck, slow’ Non-Functional Logs System operatorDeveloper Sometimes on Errors, Exceptions, Debug Scope notknown Hours, Days ‘Mightcome in handy’

14. WHAT to log

15. Keep in mind ”Whatwouldhelp YOU at 3 AM in the morning? “ Image: Ambro / FreeDigitalPhotos.net

16. Whatwillhelp you Exact notice of whathappened When Where How Who Clearconcise context information Analysis possible without application Manual Semi-automated Automated Remotelycollectible files Low footprint Proven reliableandauthentic

17. Whatwill help youexamples 2011-05-31 08:46:11,308 +0200 INFO [eventlog.security] ACCESS OF ProfileServiceImpl.findProfilesBY [login: some_username]; Password: [PROTECTED]; Authenticated: true; Details: RemoteIpAddress: 192.168.156.17; SessionId: 8XyrNkLWv7FdXS… Granted Authorities:ROLE_USER,ROLE…

18. Whatwon’thelp you No details about the record No context information Inconsistent format or bad use Non-correlatable Subjective/interpretativemessage Too muchuseless log records Loggingtoone single big file

19. Whatwon’t help you – example#1 log4j: setFile called: ./logs/be/mobistar/network-status/log_functional.log, false log4j: setFileended tis nendefault tis nendefault Activations : 15 Deactivations : 0 In straigt to resource filter

20. Whatwon’t help you – example#2 INFO [dao.ProductDao] [] - FOUND 40 INFO [web.listeners.PhaseLogger] [] - +RENDER_RESPONSE INFO [service.ProductFamilyService] [] - product in family is true INFO [.dao.ProfileDao] [] - [OBJECTS] find profiles

21. log messagedecomposed

22. “A log record shouldbeunderstandableby a human, andeasily machine processable.”

23. LoggingGuidelines: Priorities Fatal Application crash&burn Error Exceptions Monitoring team alert Warn Monitor health, performance Info Informative Context Audit trail Debug Extensive Context Development, Testing Troubleshooting

24. LoggingGuidelines: Exceptions Handling Recover Log the details ifcritical Not Handling, Translating Translate Handling logic ifnotpropagating Not Handling Propagate Someoneelsewill handle (and log!) Chooseexceptionsto log Defendfromstacktrace chaos Choosewhatto log foreachexception Catching block has all context details!

25. LoggingGuidelines: Correlation 1 transaction = * log records Correlationnecessary Executiontrail Shared context variable Shared per transaction Reuse of context variables Nested Diagnostic Context Map Diagnostic Context Onlyonecorrelation Use a context uniquevariable Correlate transactions

26. How to log iN

27. How loggingframeworkswork

28. Java Logging Frameworks Implementation System.out – System.err Redirect output Java UtilLogging Log4j Logback Abstraction CommonsLogging SLF4J

29. Java UtilLogging Java 1.4+ Based on Log4J Less features out of the box Console File Stream

30. Log4j First framework Introductionof Logger, Appender, Level Hiearchical Loggers Configurableappenders, e.g. MailAppender FileAppender SMTPAppender SocketAppender SyslogAppender Leader quitandstartedlogback

31. CommonsLogging Apache Project API Bridge API-calls forward to Log4j if on CP Falls back to JUL Plagued Double Configuration Runtime dependency check ClasspathScanning Discontinued v1.1.1 dates from end 2007

32. Slf4j - Logback Twolibrariestorulethemall Slf4j Comparablewithcommons-logging Parameter parsing Framework bridges Much more powerful Logback Evolved log4j Implements Slf4j “Picking up where log4j leaves off” Testing Filters on-need-base

33. SLF4J Binding

34. SLF4J Bridging

35. Keepingit performant String concatenation LOGGER.debug("Hello " + name); Protected block if (logger.isDebugEnabled()) {LOGGER.debug("Hello " + name);} Slf4j LOGGER.debug("Hello {}", name);

36. Context Variables In Practice NDC.push(correlationId); app.logMe("Hello world!"); NDC.pop(); Use %x in Pattern ConversionPattern= %x %d %m %n correlationId 2011-06-21 09:20:41 [INFO] Hello world!

37. Specialized Loggers Logger for class Fine-tune log Executiontrace How distinguish Security Performance Business Specialized Loggers Priorities Output handlers Groupedlogging

38. Logging In Production Onlynecessarylogging Situation -> turn on debuglogging No restart: DynamicReloading ‘Watch’ file for changes Log file management Rolling file Seperation of concerns Clustering Different systems generatemessages Synchronization of timestamps Correlation on host-specific files

39. My Opinion Importance of loggingwillgrow Mobile / offline applications Cloud integration System interaction Logging standards enterprisewide Standardization of logging effort Inexpensiveexercise Draft a standard Build adapters / patterns Enforce standards

40. Log Processing and analysis

41. Log Analysis Bang foryourbuck XpoLog http://www.xpolog.com Licensed Faster Splunk http://www.splunk.com Free Edition ‘Slow’ AJAX feel Nice graphics Modular (http://www.splunkbase.com) Tuningyourpattern is important?

42. Demo

44. For more information: visit our website WWW.REALDOLMEN.COM Follow us on: Selected presentations are available on: Thank You Or scan this QR code with your Smartphone to immediately go to the website