YAPC::EU 2015 Talking about static analysis for perl and Perl::Lint.

1. Static Code Analysis
for Perl
@moznion

2. Taiki Kawakami
a.k.a @moznion
Sever side engineer
(Java and Perl)
Author of
- Perl::Lint
- go-setlock

3. Taiki Kawakami
a.k.a @moznion
Sever side engineer
(Java and Perl)
Author of
- Perl::Lint
- go-setlock

4. Taiki Kawakami
a.k.a @moznion
Sever side engineer
(Java and Perl)
Author of
- Perl::Lint
- go-setlock

5. Fundamental of
Static Analysis

6. Static Analysis
A method of analysis
source code WITHOUT
execution

7. Static Analysis
Example of advantages:
- Easy to detect
- unused vars
- irregular coding styles
- Analyze dependencies
between modules/classes

8. Static Analysis
Example of advantages:
- Easy to detect
- unused vars
- irregular coding styles
- Analyze dependencies
between modules/classes
BORING!

9. Static Analysis
Example of advantages:
- Easy to detect
- unused vars
- irregular coding styles
- Analyze dependencies
between modules/classes
Difficult…

10. Let's Exercise

11. This code has 5 traps

12. This code has 5 traps

13. This code has 5 traps

14. This code has 5 traps

15. This code has 5 traps

16. This code has 5 traps

17. It was fun?

18. This is ridiculous
code ceview

19. Probably
human overlooks

20. We should focus on
advanced topic
on code review

21. How?

22. It is necessary
clean code

23. Destroy these

24. Be maintainable
code!

25. Make computer
analyze them!

26. How to make
static analyzer?

27. Pre-Process
Lexical Analyze
Syntactic Analyze
Source code (String)
Result
Analyze

28. Pre-Process
Lexical Analyze
Syntactic Analyze
Source code (String)
Result
Analyze

30. PPI::Tokenizer

31. Pre-Process
Lexical Analyze
Syntactic Analyze
Source code (String)
Result
Analyze

32. PPI::Document
Provides
PDOM
Structure

33. Pre-Process
Lexical Analyze
Syntactic Analyze
Source code (String)
Result
Analyze

34. “Analyze” phase
checks code with
using AST and tokens
in accordance with
rules

35. Method of some
languages are
different;
they look byte code
(e.g. Java:findbugs)

36. Perl::Critic

37. Perl::Critic is the
great tool!

38. Perl::Critic checks
the code conform
to PBP style or not

39. Perl::Critic uses
PPI as a Lexer
and Parser

40. Pre-Process
Lexical Analyze
Syntactic Analyze
Source code (String)
Result
Analyze
PPI

41. Perl::Lint

42. Perl::Lint is a
yet another static
analyser for perl

43. This project
supported by TPF

44. Perl::Critic is enough.
Why Perl::Lint?

45. I want to make it
faster!!!

46. Mechanism of
Perl::Lint

47. Pre-Process
Lexical Analyze
Syntactic Analyze
Source code (String)
Result
Analyze
Regex
Compiler::Lexer
Perl::Lint::Policy

48. Pre-Process
Lexical Analyze
Syntactic Analyze
Source code (String)
Result
Analyze
Regex
Compiler::Lexer
Perl::Lint::Policy

49. Pre-Processing

50. ## no lint

51. ## no lint
To retrieve this

52. Find where (what line) is
“## no lint” by regex

53. Find where (what line) is
“## no lint” by regex
And compare between
line number of “## no lint”
and violation’s one,
if match them, ignore form result!

54. Compiler::Lexer can retrieve
comments by verbose mode,
but it makes slower about 4 times😢
So using regex

55. Pre-Process
Lexical Analyze
Syntactic Analyze
Source code (String)
Result
Analyze
Regex
Compiler::Lexer
Perl::Lint::Policy

56. Tokenize source code
by Compiler::Lexer

59. Compiler::Lexer made of C++
Really fast!

60. Stable (nowadays)

61. But…

63. Perl-5.22………………

64. Pre-Process
Lexical Analyze
Syntactic Analyze
Source code (String)
Result
Analyze
Regex
Compiler::Lexer
Perl::Lint::Policy

65. Compiler::Parser exists,
but that doesn’t work as expected

66. Pre-Process
Lexical Analyze
Syntactic Analyze
Source code (String)
Result
Analyze
Regex
Compiler::Lexer
Perl::Lint::Policy

67. Read token list sequentially
and evaluate them.
Each policies are responsible
for those.

68. Like this

69. Like this

70. Like this…

71. And it is necessary to analyze
contents of regex (m/here!/)

72. Using Regexp::Lexer
This is a module to tokenize regex

73. Example;

74. Each policies are independent,
so easy to write new policy
(You can write your own policy)

75. Easy and Simple:
Scan tokens and write
validation processing according to
scanned token sequentially

76. Perl::Lint has filter system

77. Perl::Lint executes all of the policies
by default.
Write a black list to ignore
any policy.

78. Current Status

79. Almost policies of Perl::Critic
are available on Perl::Lint

80. 現状のステータス

81. Documentation is lacked…

82. Application

83. Test::Perl::Lint
Testing module like a
Test::Perl::Critic

84. Perl::Lint::Git
Connect git and Perl::Lint to blame the right people for
violations.Connect git and Perl::Lint to blame
the right people for violations.

85. Future works

86. I should have written a parser…
Compiler::Lexer::PP (?)

87. Enhance documentation

88. Bug fix

89. Support new perl notations

90. Support code climate

91. CHEATING:
Run each policies with
pre-fork model

92. Any Q?
(If I can answer…)