SlideShare ist ein Scribd-Unternehmen logo
1 von 91
Downloaden Sie, um offline zu lesen
Analyzing the State of Static Analysis:
A Large-Scale Evaluation
in Open Source Software
Analyzing the State of Static Analysis:
A Large-Scale Evaluation
in Open Source Software
Moritz Beller
@Inventitech
Analyzing the State of Static Analysis:
A Large-Scale Evaluation
in Open Source Software
Moritz Beller
@Inventitech
Radjino Bholanath, Andy Zaidman
Analyzing the State of Static Analysis:
A Large-Scale Evaluation
in Open Source Software
Moritz Beller
@Inventitech
Radjino Bholanath, Andy Zaidman
Shane McIntosh
Automatic Static Analysis Tools (ASATs)
Image: https://pixabay.com/static/uploads/photo/2015/06/12/18/31/cute-807306_960_720.png
Automatic Static Analysis Tools (ASATs)
Image: https://pixabay.com/static/uploads/photo/2015/06/12/18/31/cute-807306_960_720.png
Automatic Static Analysis Tools (ASATs)
Image: https://pixabay.com/static/uploads/photo/2015/06/12/18/31/cute-807306_960_720.png
Automatic Static Analysis Tools (ASATs)
Image: https://pixabay.com/static/uploads/photo/2015/06/12/18/31/cute-807306_960_720.png
Automatic Static Analysis Tools (ASATs)
Image: https://pixabay.com/static/uploads/photo/2015/06/12/18/31/cute-807306_960_720.png
Automatic Static Analysis Tools (ASATs)
Image: https://pixabay.com/static/uploads/photo/2015/06/12/18/31/cute-807306_960_720.png
RQ1: How Prevalent Are ASATs?
Image: http://www.valueinvestasia.com/wp-content/uploads/2015/03/odd-one-out.jpg
122 popular OSS projects
RQ1: How Prevalent Are ASATs?
RQ1: How Prevalent Are ASATs?
122
RQ1: How Prevalent Are ASATs?
122
RQ1: How Prevalent Are ASATs?
122
RQ1: How Prevalent Are ASATs?
122
122
RQ1: How Prevalent Are ASATs?
122
122
RQ1: How Prevalent Are ASATs?
122
36
122
RQ1: How Prevalent Are ASATs?
122
36
122
RQ1: How Prevalent Are ASATs?
122
36
122
RQ1: How Prevalent Are ASATs?
Source Amount of Projects Using ASATs Using >1 ASAT Enforcing ASAT
122 59% 23% -
36 77% 36% 36%
RQ1: How Prevalent Are ASATs?
Source Amount of Projects Using ASATs Using >1 ASAT Enforcing ASAT
122 59% 23% -
36 77% 36% 36%
RQ1: How Prevalent Are ASATs?
Source Amount of Projects Using ASATs Using >1 ASAT Enforcing ASAT
122 59% 23% -
36 77% 36% 36%
RQ1: How Prevalent Are ASATs?
Source Amount of Projects Using ASATs Using >1 ASAT Enforcing ASAT
122 59% 23% -
36 77% 36% 36%
RQ1: How Prevalent Are ASATs?
Source Amount of Projects Using ASATs Using >1 ASAT Enforcing ASAT
122 59% 23% -
36 77% 36% 36%
RQ1: How Prevalent Are ASATs?
Source Amount of Projects Using ASATs Using >1 ASAT Enforcing ASAT
122 59% 23% -
36 77% 36% 36%
1) (Automated) investigation of repository information is an
approximation of real ASAT use
RQ1: How Prevalent Are ASATs?
Source Amount of Projects Using ASATs Using >1 ASAT Enforcing ASAT
122 59% 23% -
36 77% 36% 36%
1) (Automated) investigation of repository information is an
approximation of real ASAT use
2) We cannot infer how a project uses ASATs from a repository
analysis alone
RQ1: How Prevalent Are ASATs?
RQ2: How are ASATs configured?
RQ2: How are ASATs configured?
RQ2: How are ASATs configured?
checkstyle.xml
RQ2: How are ASATs configured?
filename
checkstyle.xml
RQ2: How are ASATs configured?
filename
checkstyle.xml
RQ2: How are ASATs configured?
filename
checkstyle.xml
parse rules
RQ2: How are ASATs configured?
filename
checkstyle.xml
enable
parse rules
RQ2: How are ASATs configured?
filename
checkstyle.xml
enable
re-configure
parse rules
RQ2: How are ASATs configured?
filename
checkstyle.xml
enable
re-configure
parse rules
custom
General Defect Classification (GDC)
General Defect Classification (GDC)
General Defect Classification (GDC)
RQ1:
9
General Defect Classification (GDC)
RQ1:
9
Checkstyle, FindBugs, PMD, ESLint, JSCS, JSHint, JSL, PYLint, RuboCop
General Defect Classification (GDC)
1,825
RQ1:
9
Checkstyle, FindBugs, PMD, ESLint, JSCS, JSHint, JSL, PYLint, RuboCop
RQ2: How are ASATs configured?
RQ2: How are ASATs configured?
RQ2: How are ASATs configured?
168,425
RQ2: How are ASATs configured?
168,425
RQ2: How are ASATs configured?
168,425
RQ2: How are ASATs configured?
168,425
RQ2: How are ASATs configured?
RQ2.1: How Popular Are Certain ASATs?
Tool Language Configuration Files
Checkstyle Java 18,785
FindBugs Java 2,090
PMD Java 7,458
ESLint JavaScript 4,435
JSCS JavaScript 11,677
JSHint JavaScript 108,770
JSL JavaScript 862
Pylint Python 4,071
RuboCop Ruby 10,066
Total - 168,405
RQ2.1: How Popular Are Certain ASATs?
Tool Language Configuration Files
Checkstyle Java 18,785
FindBugs Java 2,090
PMD Java 7,458
ESLint JavaScript 4,435
JSCS JavaScript 11,677
JSHint JavaScript 108,770
JSL JavaScript 862
Pylint Python 4,071
RuboCop Ruby 10,066
Total - 168,405
RQ 2.2: Which Rules Do Developers Enable?
RQ 2.2: Which Rules Do Developers Enable?
65%
RQ 2.2: Which Rules Do Developers Enable?
35% 65%
RQ 2.2: Which Rules Do Developers Enable?
35% 65%
ASATs perform poorly at finding
functional defects. Wagner et al.
RQ 2.2: Which Rules Do Developers Enable?
RQ 2.2: Which Rules Do Developers Enable?
0%
00%
00%
00%
00%
00%
00%
00%
00%
00%
00%
Checkstyle ESLint FindBugs JSCS JSHint JSL PMD Pylint RuboCop
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
0%
00%
00%
00%
00%
00%
00%
00%
00%
00%
00%
Checkstyle ESLint FindBugs JSCS JSHint JSL PMD Pylint RuboCop
RQ 2.2: Which Rules Do Developers Enable?
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
RQ 2.2: Which Rules Do Developers Enable?
We: This is great!
Image: Phil Skipper, The Potentialist,http://i2.wp.com/thepotentiality.com/assets/media/Kurt-Enthusiasm.jpg
RQ 2.2: Which Rules Do Developers Enable?
We: This is great!
Image: Phil Skipper, The Potentialist,http://i2.wp.com/thepotentiality.com/assets/media/Kurt-Enthusiasm.jpg
RQ 2.2: Which Rules Do Developers Enable?
ASAT Developers*: Don't care.
Image: NiBiS, http://bidab.nibis.de/PICT/traurig.jpg
RQ 2.2: Which Rules Do Developers Enable?
ASAT Developers*: Don't care.
Image: NiBiS, http://bidab.nibis.de/PICT/traurig.jpg
RQ 2.3: How Good Is The Default?
RQ 2.3: How Good Is The Default?
Most ASAT configurations
deviate from the default.
RQ 2.3: How Good Is The Default?
Most ASAT configurations
deviate from the default.
RQ 2.3: How Good Is The Default?
Most ASAT configurations
deviate from the default.
RQ 2.3: How Good Is The Default?
But, typically only have one
change from the default …
Most ASAT configurations
deviate from the default.
RQ 2.3: How Good Is The Default?
But, typically only have one change from the
default …
●
- Addition
Most ASAT configurations
deviate from the default.
RQ 2.3: How Good Is The Default?
But, typically only have one change from the default …
●
- Addition
●
- Deletion
Most ASAT configurations
deviate from the default.
RQ 2.3: How Good Is The Default?
But, typically only have one change from the default …
●
- Addition
●
- Deletion
●
- Re-configuration/Custom analysis
Most ASAT configurations
deviate from the default.
RQ2: Open Questions
●
Why do projects favor certain GDC rule categories from
ASATs?
RQ2: Open Questions
●
Why do projects favor certain GDC rule categories from
ASATs?
●
Can ASAT developers better fit their default configurations to
their users' needs?
RQ2: Open Questions
●
Why do projects favor certain GDC rule categories from ASATs?
●
Can ASAT developers better fit their default configurations to
their users' needs?
●
Do 'dynamic' languages require more ASAT use?
RQ2: Open Questions
●
Why do projects favor certain GDC rule categories from ASATs?
●
Can ASAT developers better fit their default configurations to
their users' needs?
●
Do 'dynamic' languages require more ASAT use?
RQ2: Open Questions
RQ3: How Do ASAT Configurations Evolve?
Image: Daimler AG, http://5komma6.mercedes-benz-passion.com/wp-content/uploads/2013/06/s-class-lineup.jpg
RQ 3.1: How Often Do Changes Occur?
RQ 3.1: How Often Do Changes Occur?
RQ 3.1: How Often Do Changes Occur?
>80%
“never”
RQ 3.2: When Do Changes Occur?
RQ 3.2: When Do Changes Occur?
RQ 3.2: When Do Changes Occur?
<20%
of files
RQ 3.3: How Big Are The Changes?
RQ 3: Open Questions
●
Why do ASAT configurations not typically evolve?
RQ 3: Open Questions
●
Why do ASAT configurations not typically evolve?
●
How are ASATs used in a CI-environment?
RQ 3: Open Questions
Moritz Beller
@Inventitech
Analyzing the State of Static Analysis:
A Large-Scale Evaluation
in Open Source Software
Moritz Beller
@Inventitech
Analyzing the State of Static Analysis:
A Large-Scale Evaluation
in Open Source Software
Moritz Beller
@Inventitech
Analyzing the State of Static Analysis:
A Large-Scale Evaluation
in Open Source Software
Moritz Beller
@Inventitech
Analyzing the State of Static Analysis:
A Large-Scale Evaluation
in Open Source Software

Weitere ähnliche Inhalte

Andere mochten auch

Guía metodológica para realizar Consultas Populares en Colombia
Guía metodológica para realizar Consultas Populares en ColombiaGuía metodológica para realizar Consultas Populares en Colombia
Guía metodológica para realizar Consultas Populares en ColombiaCrónicas del despojo
 
Graduationvideo2
Graduationvideo2Graduationvideo2
Graduationvideo2AngieTommy
 
Progrès dans les technologies photovoltaïques: abaissement des coûts et nouve...
Progrès dans les technologies photovoltaïques: abaissement des coûts et nouve...Progrès dans les technologies photovoltaïques: abaissement des coûts et nouve...
Progrès dans les technologies photovoltaïques: abaissement des coûts et nouve...swissolar-romandie
 
professional resume final2
professional resume final2professional resume final2
professional resume final2Sabrina McFadden
 
new_Effect of Magnetic Fields on Coherent Spectroscopy
new_Effect of Magnetic Fields on Coherent Spectroscopynew_Effect of Magnetic Fields on Coherent Spectroscopy
new_Effect of Magnetic Fields on Coherent SpectroscopyDr. Leah Margalit
 

Andere mochten auch (7)

Guía metodológica para realizar Consultas Populares en Colombia
Guía metodológica para realizar Consultas Populares en ColombiaGuía metodológica para realizar Consultas Populares en Colombia
Guía metodológica para realizar Consultas Populares en Colombia
 
Graduationvideo2
Graduationvideo2Graduationvideo2
Graduationvideo2
 
Elena HARCONIŢA, Snejana ZADAINOVA: Particularităţile de dezvoltare a resur...
Elena HARCONIŢA,  Snejana ZADAINOVA: Particularităţile  de dezvoltare a resur...Elena HARCONIŢA,  Snejana ZADAINOVA: Particularităţile  de dezvoltare a resur...
Elena HARCONIŢA, Snejana ZADAINOVA: Particularităţile de dezvoltare a resur...
 
Progrès dans les technologies photovoltaïques: abaissement des coûts et nouve...
Progrès dans les technologies photovoltaïques: abaissement des coûts et nouve...Progrès dans les technologies photovoltaïques: abaissement des coûts et nouve...
Progrès dans les technologies photovoltaïques: abaissement des coûts et nouve...
 
professional resume final2
professional resume final2professional resume final2
professional resume final2
 
Html
HtmlHtml
Html
 
new_Effect of Magnetic Fields on Coherent Spectroscopy
new_Effect of Magnetic Fields on Coherent Spectroscopynew_Effect of Magnetic Fields on Coherent Spectroscopy
new_Effect of Magnetic Fields on Coherent Spectroscopy
 

Ähnlich wie Analyzing the State of Static Analysis: A Large-Scale Evaluation in Open Source Software

A Taxonomy of Clustering, or, No Container is an Island
A Taxonomy of Clustering, or, No Container is an IslandA Taxonomy of Clustering, or, No Container is an Island
A Taxonomy of Clustering, or, No Container is an IslandTed M. Young
 
Automated Identification of On-hold Self-admitted Technical Debt
Automated Identification of On-hold Self-admitted Technical DebtAutomated Identification of On-hold Self-admitted Technical Debt
Automated Identification of On-hold Self-admitted Technical DebtRungrojMaipradit1
 
Speeding up your team with GitOps
Speeding up your team with GitOpsSpeeding up your team with GitOps
Speeding up your team with GitOpsBrice Fernandes
 
JAZOON'13 - Stefan Saasen - True Git: The Great Migration
JAZOON'13 - Stefan Saasen - True Git: The Great MigrationJAZOON'13 - Stefan Saasen - True Git: The Great Migration
JAZOON'13 - Stefan Saasen - True Git: The Great Migrationjazoon13
 
Managing your Black Friday Logs NDC Oslo
Managing your  Black Friday Logs NDC OsloManaging your  Black Friday Logs NDC Oslo
Managing your Black Friday Logs NDC OsloDavid Pilato
 
Managing your black friday logs - Code Europe
Managing your black friday logs - Code EuropeManaging your black friday logs - Code Europe
Managing your black friday logs - Code EuropeDavid Pilato
 
Open stack gbp final sn-4-slideshare
Open stack gbp final sn-4-slideshareOpen stack gbp final sn-4-slideshare
Open stack gbp final sn-4-slideshareSumit Naiksatam
 
[AWS Start-up ゼミ / DevDay 編] よくある課題を一気に解説! 御社の技術レベルがアップする 2018 秋期講習
[AWS Start-up ゼミ / DevDay 編] よくある課題を一気に解説! 御社の技術レベルがアップする 2018 秋期講習[AWS Start-up ゼミ / DevDay 編] よくある課題を一気に解説! 御社の技術レベルがアップする 2018 秋期講習
[AWS Start-up ゼミ / DevDay 編] よくある課題を一気に解説! 御社の技術レベルがアップする 2018 秋期講習Amazon Web Services Japan
 
Alexander Mostovenko "'Devide at impera' with GraphQL and SSR"
Alexander Mostovenko "'Devide at impera' with GraphQL and SSR"Alexander Mostovenko "'Devide at impera' with GraphQL and SSR"
Alexander Mostovenko "'Devide at impera' with GraphQL and SSR"Fwdays
 
IVS CTO Night And Day 2018 Winter - AWS Startup Tech Office Hours
IVS CTO Night And Day 2018 Winter - AWS Startup Tech Office HoursIVS CTO Night And Day 2018 Winter - AWS Startup Tech Office Hours
IVS CTO Night And Day 2018 Winter - AWS Startup Tech Office HoursAmazon Web Services Japan
 
Doctor ZedGe @InsideTrack Rome #sitROME
Doctor ZedGe @InsideTrack Rome #sitROMEDoctor ZedGe @InsideTrack Rome #sitROME
Doctor ZedGe @InsideTrack Rome #sitROMEsergio.ferrari
 
Faceted Search – the 120 Million Documents Story
Faceted Search – the 120 Million Documents StoryFaceted Search – the 120 Million Documents Story
Faceted Search – the 120 Million Documents StorySourcesense
 
Delivering Quality at Speed with GitOps
Delivering Quality at Speed with GitOpsDelivering Quality at Speed with GitOps
Delivering Quality at Speed with GitOpsWeaveworks
 
Spring Boot to Quarkus: A real app migration experience | DevNation Tech Talk
Spring Boot to Quarkus: A real app migration experience | DevNation Tech TalkSpring Boot to Quarkus: A real app migration experience | DevNation Tech Talk
Spring Boot to Quarkus: A real app migration experience | DevNation Tech TalkRed Hat Developers
 
Mobile Software Diagnostics
Mobile Software DiagnosticsMobile Software Diagnostics
Mobile Software DiagnosticsDmitry Vostokov
 
JNation: REST APIs to GraphQL with Express and Apollo
JNation: REST APIs to GraphQL with Express and ApolloJNation: REST APIs to GraphQL with Express and Apollo
JNation: REST APIs to GraphQL with Express and ApolloRoy Derks
 
Keeping Identity Graphs In Sync With Apache Spark
Keeping Identity Graphs In Sync With Apache SparkKeeping Identity Graphs In Sync With Apache Spark
Keeping Identity Graphs In Sync With Apache SparkDatabricks
 
Agile Project with Fixed Budget Scope and Deadline: How is it Possible?
Agile Project with Fixed Budget Scope and Deadline: How is it Possible?Agile Project with Fixed Budget Scope and Deadline: How is it Possible?
Agile Project with Fixed Budget Scope and Deadline: How is it Possible?Vaidas Adomauskas
 

Ähnlich wie Analyzing the State of Static Analysis: A Large-Scale Evaluation in Open Source Software (20)

A Taxonomy of Clustering, or, No Container is an Island
A Taxonomy of Clustering, or, No Container is an IslandA Taxonomy of Clustering, or, No Container is an Island
A Taxonomy of Clustering, or, No Container is an Island
 
Automated Identification of On-hold Self-admitted Technical Debt
Automated Identification of On-hold Self-admitted Technical DebtAutomated Identification of On-hold Self-admitted Technical Debt
Automated Identification of On-hold Self-admitted Technical Debt
 
Speeding up your team with GitOps
Speeding up your team with GitOpsSpeeding up your team with GitOps
Speeding up your team with GitOps
 
JAZOON'13 - Stefan Saasen - True Git: The Great Migration
JAZOON'13 - Stefan Saasen - True Git: The Great MigrationJAZOON'13 - Stefan Saasen - True Git: The Great Migration
JAZOON'13 - Stefan Saasen - True Git: The Great Migration
 
Effective DevSecOps
Effective DevSecOpsEffective DevSecOps
Effective DevSecOps
 
Managing your Black Friday Logs NDC Oslo
Managing your  Black Friday Logs NDC OsloManaging your  Black Friday Logs NDC Oslo
Managing your Black Friday Logs NDC Oslo
 
Managing your black friday logs - Code Europe
Managing your black friday logs - Code EuropeManaging your black friday logs - Code Europe
Managing your black friday logs - Code Europe
 
Open stack gbp final sn-4-slideshare
Open stack gbp final sn-4-slideshareOpen stack gbp final sn-4-slideshare
Open stack gbp final sn-4-slideshare
 
[AWS Start-up ゼミ / DevDay 編] よくある課題を一気に解説! 御社の技術レベルがアップする 2018 秋期講習
[AWS Start-up ゼミ / DevDay 編] よくある課題を一気に解説! 御社の技術レベルがアップする 2018 秋期講習[AWS Start-up ゼミ / DevDay 編] よくある課題を一気に解説! 御社の技術レベルがアップする 2018 秋期講習
[AWS Start-up ゼミ / DevDay 編] よくある課題を一気に解説! 御社の技術レベルがアップする 2018 秋期講習
 
Alexander Mostovenko "'Devide at impera' with GraphQL and SSR"
Alexander Mostovenko "'Devide at impera' with GraphQL and SSR"Alexander Mostovenko "'Devide at impera' with GraphQL and SSR"
Alexander Mostovenko "'Devide at impera' with GraphQL and SSR"
 
IVS CTO Night And Day 2018 Winter - AWS Startup Tech Office Hours
IVS CTO Night And Day 2018 Winter - AWS Startup Tech Office HoursIVS CTO Night And Day 2018 Winter - AWS Startup Tech Office Hours
IVS CTO Night And Day 2018 Winter - AWS Startup Tech Office Hours
 
Doctor ZedGe @InsideTrack Rome #sitROME
Doctor ZedGe @InsideTrack Rome #sitROMEDoctor ZedGe @InsideTrack Rome #sitROME
Doctor ZedGe @InsideTrack Rome #sitROME
 
Faceted Search – the 120 Million Documents Story
Faceted Search – the 120 Million Documents StoryFaceted Search – the 120 Million Documents Story
Faceted Search – the 120 Million Documents Story
 
Delivering Quality at Speed with GitOps
Delivering Quality at Speed with GitOpsDelivering Quality at Speed with GitOps
Delivering Quality at Speed with GitOps
 
Spring Boot to Quarkus: A real app migration experience | DevNation Tech Talk
Spring Boot to Quarkus: A real app migration experience | DevNation Tech TalkSpring Boot to Quarkus: A real app migration experience | DevNation Tech Talk
Spring Boot to Quarkus: A real app migration experience | DevNation Tech Talk
 
Mobile Software Diagnostics
Mobile Software DiagnosticsMobile Software Diagnostics
Mobile Software Diagnostics
 
JNation: REST APIs to GraphQL with Express and Apollo
JNation: REST APIs to GraphQL with Express and ApolloJNation: REST APIs to GraphQL with Express and Apollo
JNation: REST APIs to GraphQL with Express and Apollo
 
Keeping Identity Graphs In Sync With Apache Spark
Keeping Identity Graphs In Sync With Apache SparkKeeping Identity Graphs In Sync With Apache Spark
Keeping Identity Graphs In Sync With Apache Spark
 
Agile Project with Fixed Budget Scope and Deadline: How is it Possible?
Agile Project with Fixed Budget Scope and Deadline: How is it Possible?Agile Project with Fixed Budget Scope and Deadline: How is it Possible?
Agile Project with Fixed Budget Scope and Deadline: How is it Possible?
 
True Git
True Git True Git
True Git
 

Kürzlich hochgeladen

Precise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive GoalPrecise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive GoalLionel Briand
 
Sending Calendar Invites on SES and Calendarsnack.pdf
Sending Calendar Invites on SES and Calendarsnack.pdfSending Calendar Invites on SES and Calendarsnack.pdf
Sending Calendar Invites on SES and Calendarsnack.pdf31events.com
 
Exploring Selenium_Appium Frameworks for Seamless Integration with HeadSpin.pdf
Exploring Selenium_Appium Frameworks for Seamless Integration with HeadSpin.pdfExploring Selenium_Appium Frameworks for Seamless Integration with HeadSpin.pdf
Exploring Selenium_Appium Frameworks for Seamless Integration with HeadSpin.pdfkalichargn70th171
 
Comparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfComparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfDrew Moseley
 
Machine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their EngineeringMachine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their EngineeringHironori Washizaki
 
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxThe Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxRTS corp
 
Understanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM ArchitectureUnderstanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM Architecturerahul_net
 
Patterns for automating API delivery. API conference
Patterns for automating API delivery. API conferencePatterns for automating API delivery. API conference
Patterns for automating API delivery. API conferencessuser9e7c64
 
Osi security architecture in network.pptx
Osi security architecture in network.pptxOsi security architecture in network.pptx
Osi security architecture in network.pptxVinzoCenzo
 
SensoDat: Simulation-based Sensor Dataset of Self-driving Cars
SensoDat: Simulation-based Sensor Dataset of Self-driving CarsSensoDat: Simulation-based Sensor Dataset of Self-driving Cars
SensoDat: Simulation-based Sensor Dataset of Self-driving CarsChristian Birchler
 
How to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion ApplicationHow to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion ApplicationBradBedford3
 
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...Bert Jan Schrijver
 
VK Business Profile - provides IT solutions and Web Development
VK Business Profile - provides IT solutions and Web DevelopmentVK Business Profile - provides IT solutions and Web Development
VK Business Profile - provides IT solutions and Web Developmentvyaparkranti
 
Not a Kubernetes fan? The state of PaaS in 2024
Not a Kubernetes fan? The state of PaaS in 2024Not a Kubernetes fan? The state of PaaS in 2024
Not a Kubernetes fan? The state of PaaS in 2024Anthony Dahanne
 
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...Cizo Technology Services
 
What’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesWhat’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesVictoriaMetrics
 
Leveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + KobitonLeveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + KobitonApplitools
 
Effectively Troubleshoot 9 Types of OutOfMemoryError
Effectively Troubleshoot 9 Types of OutOfMemoryErrorEffectively Troubleshoot 9 Types of OutOfMemoryError
Effectively Troubleshoot 9 Types of OutOfMemoryErrorTier1 app
 
Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptx
Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptxReal-time Tracking and Monitoring with Cargo Cloud Solutions.pptx
Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptxRTS corp
 
Large Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and RepairLarge Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and RepairLionel Briand
 

Kürzlich hochgeladen (20)

Precise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive GoalPrecise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive Goal
 
Sending Calendar Invites on SES and Calendarsnack.pdf
Sending Calendar Invites on SES and Calendarsnack.pdfSending Calendar Invites on SES and Calendarsnack.pdf
Sending Calendar Invites on SES and Calendarsnack.pdf
 
Exploring Selenium_Appium Frameworks for Seamless Integration with HeadSpin.pdf
Exploring Selenium_Appium Frameworks for Seamless Integration with HeadSpin.pdfExploring Selenium_Appium Frameworks for Seamless Integration with HeadSpin.pdf
Exploring Selenium_Appium Frameworks for Seamless Integration with HeadSpin.pdf
 
Comparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfComparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdf
 
Machine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their EngineeringMachine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their Engineering
 
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxThe Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
 
Understanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM ArchitectureUnderstanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM Architecture
 
Patterns for automating API delivery. API conference
Patterns for automating API delivery. API conferencePatterns for automating API delivery. API conference
Patterns for automating API delivery. API conference
 
Osi security architecture in network.pptx
Osi security architecture in network.pptxOsi security architecture in network.pptx
Osi security architecture in network.pptx
 
SensoDat: Simulation-based Sensor Dataset of Self-driving Cars
SensoDat: Simulation-based Sensor Dataset of Self-driving CarsSensoDat: Simulation-based Sensor Dataset of Self-driving Cars
SensoDat: Simulation-based Sensor Dataset of Self-driving Cars
 
How to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion ApplicationHow to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion Application
 
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
 
VK Business Profile - provides IT solutions and Web Development
VK Business Profile - provides IT solutions and Web DevelopmentVK Business Profile - provides IT solutions and Web Development
VK Business Profile - provides IT solutions and Web Development
 
Not a Kubernetes fan? The state of PaaS in 2024
Not a Kubernetes fan? The state of PaaS in 2024Not a Kubernetes fan? The state of PaaS in 2024
Not a Kubernetes fan? The state of PaaS in 2024
 
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...
 
What’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesWhat’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 Updates
 
Leveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + KobitonLeveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
 
Effectively Troubleshoot 9 Types of OutOfMemoryError
Effectively Troubleshoot 9 Types of OutOfMemoryErrorEffectively Troubleshoot 9 Types of OutOfMemoryError
Effectively Troubleshoot 9 Types of OutOfMemoryError
 
Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptx
Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptxReal-time Tracking and Monitoring with Cargo Cloud Solutions.pptx
Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptx
 
Large Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and RepairLarge Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and Repair
 

Analyzing the State of Static Analysis: A Large-Scale Evaluation in Open Source Software

  • 1. Analyzing the State of Static Analysis: A Large-Scale Evaluation in Open Source Software
  • 2. Analyzing the State of Static Analysis: A Large-Scale Evaluation in Open Source Software Moritz Beller @Inventitech
  • 3. Analyzing the State of Static Analysis: A Large-Scale Evaluation in Open Source Software Moritz Beller @Inventitech Radjino Bholanath, Andy Zaidman
  • 4. Analyzing the State of Static Analysis: A Large-Scale Evaluation in Open Source Software Moritz Beller @Inventitech Radjino Bholanath, Andy Zaidman Shane McIntosh
  • 5. Automatic Static Analysis Tools (ASATs) Image: https://pixabay.com/static/uploads/photo/2015/06/12/18/31/cute-807306_960_720.png
  • 6. Automatic Static Analysis Tools (ASATs) Image: https://pixabay.com/static/uploads/photo/2015/06/12/18/31/cute-807306_960_720.png
  • 7. Automatic Static Analysis Tools (ASATs) Image: https://pixabay.com/static/uploads/photo/2015/06/12/18/31/cute-807306_960_720.png
  • 8. Automatic Static Analysis Tools (ASATs) Image: https://pixabay.com/static/uploads/photo/2015/06/12/18/31/cute-807306_960_720.png
  • 9. Automatic Static Analysis Tools (ASATs) Image: https://pixabay.com/static/uploads/photo/2015/06/12/18/31/cute-807306_960_720.png
  • 10. Automatic Static Analysis Tools (ASATs) Image: https://pixabay.com/static/uploads/photo/2015/06/12/18/31/cute-807306_960_720.png
  • 11. RQ1: How Prevalent Are ASATs? Image: http://www.valueinvestasia.com/wp-content/uploads/2015/03/odd-one-out.jpg
  • 12. 122 popular OSS projects
  • 13. RQ1: How Prevalent Are ASATs?
  • 14. RQ1: How Prevalent Are ASATs? 122
  • 15. RQ1: How Prevalent Are ASATs? 122
  • 16. RQ1: How Prevalent Are ASATs? 122
  • 17. RQ1: How Prevalent Are ASATs? 122 122
  • 18. RQ1: How Prevalent Are ASATs? 122 122
  • 19. RQ1: How Prevalent Are ASATs? 122 36 122
  • 20. RQ1: How Prevalent Are ASATs? 122 36 122
  • 21. RQ1: How Prevalent Are ASATs? 122 36 122
  • 22. RQ1: How Prevalent Are ASATs?
  • 23. Source Amount of Projects Using ASATs Using >1 ASAT Enforcing ASAT 122 59% 23% - 36 77% 36% 36% RQ1: How Prevalent Are ASATs?
  • 24. Source Amount of Projects Using ASATs Using >1 ASAT Enforcing ASAT 122 59% 23% - 36 77% 36% 36% RQ1: How Prevalent Are ASATs?
  • 25. Source Amount of Projects Using ASATs Using >1 ASAT Enforcing ASAT 122 59% 23% - 36 77% 36% 36% RQ1: How Prevalent Are ASATs?
  • 26. Source Amount of Projects Using ASATs Using >1 ASAT Enforcing ASAT 122 59% 23% - 36 77% 36% 36% RQ1: How Prevalent Are ASATs?
  • 27. Source Amount of Projects Using ASATs Using >1 ASAT Enforcing ASAT 122 59% 23% - 36 77% 36% 36% RQ1: How Prevalent Are ASATs?
  • 28. Source Amount of Projects Using ASATs Using >1 ASAT Enforcing ASAT 122 59% 23% - 36 77% 36% 36% 1) (Automated) investigation of repository information is an approximation of real ASAT use RQ1: How Prevalent Are ASATs?
  • 29. Source Amount of Projects Using ASATs Using >1 ASAT Enforcing ASAT 122 59% 23% - 36 77% 36% 36% 1) (Automated) investigation of repository information is an approximation of real ASAT use 2) We cannot infer how a project uses ASATs from a repository analysis alone RQ1: How Prevalent Are ASATs?
  • 30. RQ2: How are ASATs configured?
  • 31. RQ2: How are ASATs configured?
  • 32. RQ2: How are ASATs configured? checkstyle.xml
  • 33. RQ2: How are ASATs configured? filename checkstyle.xml
  • 34. RQ2: How are ASATs configured? filename checkstyle.xml
  • 35. RQ2: How are ASATs configured? filename checkstyle.xml parse rules
  • 36. RQ2: How are ASATs configured? filename checkstyle.xml enable parse rules
  • 37. RQ2: How are ASATs configured? filename checkstyle.xml enable re-configure parse rules
  • 38. RQ2: How are ASATs configured? filename checkstyle.xml enable re-configure parse rules custom
  • 42. General Defect Classification (GDC) RQ1: 9 Checkstyle, FindBugs, PMD, ESLint, JSCS, JSHint, JSL, PYLint, RuboCop
  • 43. General Defect Classification (GDC) 1,825 RQ1: 9 Checkstyle, FindBugs, PMD, ESLint, JSCS, JSHint, JSL, PYLint, RuboCop
  • 44. RQ2: How are ASATs configured?
  • 45. RQ2: How are ASATs configured?
  • 46. RQ2: How are ASATs configured?
  • 47. 168,425 RQ2: How are ASATs configured?
  • 48. 168,425 RQ2: How are ASATs configured?
  • 49. 168,425 RQ2: How are ASATs configured?
  • 50. 168,425 RQ2: How are ASATs configured?
  • 51. RQ2.1: How Popular Are Certain ASATs? Tool Language Configuration Files Checkstyle Java 18,785 FindBugs Java 2,090 PMD Java 7,458 ESLint JavaScript 4,435 JSCS JavaScript 11,677 JSHint JavaScript 108,770 JSL JavaScript 862 Pylint Python 4,071 RuboCop Ruby 10,066 Total - 168,405
  • 52. RQ2.1: How Popular Are Certain ASATs? Tool Language Configuration Files Checkstyle Java 18,785 FindBugs Java 2,090 PMD Java 7,458 ESLint JavaScript 4,435 JSCS JavaScript 11,677 JSHint JavaScript 108,770 JSL JavaScript 862 Pylint Python 4,071 RuboCop Ruby 10,066 Total - 168,405
  • 53. RQ 2.2: Which Rules Do Developers Enable?
  • 54. RQ 2.2: Which Rules Do Developers Enable?
  • 55. 65% RQ 2.2: Which Rules Do Developers Enable?
  • 56. 35% 65% RQ 2.2: Which Rules Do Developers Enable?
  • 57. 35% 65% ASATs perform poorly at finding functional defects. Wagner et al. RQ 2.2: Which Rules Do Developers Enable?
  • 58. RQ 2.2: Which Rules Do Developers Enable? 0% 00% 00% 00% 00% 00% 00% 00% 00% 00% 00% Checkstyle ESLint FindBugs JSCS JSHint JSL PMD Pylint RuboCop 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%
  • 59. 0% 00% 00% 00% 00% 00% 00% 00% 00% 00% 00% Checkstyle ESLint FindBugs JSCS JSHint JSL PMD Pylint RuboCop RQ 2.2: Which Rules Do Developers Enable? 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%
  • 60. RQ 2.2: Which Rules Do Developers Enable? We: This is great! Image: Phil Skipper, The Potentialist,http://i2.wp.com/thepotentiality.com/assets/media/Kurt-Enthusiasm.jpg
  • 61. RQ 2.2: Which Rules Do Developers Enable? We: This is great! Image: Phil Skipper, The Potentialist,http://i2.wp.com/thepotentiality.com/assets/media/Kurt-Enthusiasm.jpg
  • 62. RQ 2.2: Which Rules Do Developers Enable? ASAT Developers*: Don't care. Image: NiBiS, http://bidab.nibis.de/PICT/traurig.jpg
  • 63. RQ 2.2: Which Rules Do Developers Enable? ASAT Developers*: Don't care. Image: NiBiS, http://bidab.nibis.de/PICT/traurig.jpg
  • 64. RQ 2.3: How Good Is The Default?
  • 65. RQ 2.3: How Good Is The Default? Most ASAT configurations deviate from the default.
  • 66. RQ 2.3: How Good Is The Default? Most ASAT configurations deviate from the default.
  • 67. RQ 2.3: How Good Is The Default? Most ASAT configurations deviate from the default.
  • 68. RQ 2.3: How Good Is The Default? But, typically only have one change from the default … Most ASAT configurations deviate from the default.
  • 69. RQ 2.3: How Good Is The Default? But, typically only have one change from the default … ● - Addition Most ASAT configurations deviate from the default.
  • 70. RQ 2.3: How Good Is The Default? But, typically only have one change from the default … ● - Addition ● - Deletion Most ASAT configurations deviate from the default.
  • 71. RQ 2.3: How Good Is The Default? But, typically only have one change from the default … ● - Addition ● - Deletion ● - Re-configuration/Custom analysis Most ASAT configurations deviate from the default.
  • 73. ● Why do projects favor certain GDC rule categories from ASATs? RQ2: Open Questions
  • 74. ● Why do projects favor certain GDC rule categories from ASATs? ● Can ASAT developers better fit their default configurations to their users' needs? RQ2: Open Questions
  • 75. ● Why do projects favor certain GDC rule categories from ASATs? ● Can ASAT developers better fit their default configurations to their users' needs? ● Do 'dynamic' languages require more ASAT use? RQ2: Open Questions
  • 76. ● Why do projects favor certain GDC rule categories from ASATs? ● Can ASAT developers better fit their default configurations to their users' needs? ● Do 'dynamic' languages require more ASAT use? RQ2: Open Questions
  • 77. RQ3: How Do ASAT Configurations Evolve? Image: Daimler AG, http://5komma6.mercedes-benz-passion.com/wp-content/uploads/2013/06/s-class-lineup.jpg
  • 78. RQ 3.1: How Often Do Changes Occur?
  • 79. RQ 3.1: How Often Do Changes Occur?
  • 80. RQ 3.1: How Often Do Changes Occur? >80% “never”
  • 81. RQ 3.2: When Do Changes Occur?
  • 82. RQ 3.2: When Do Changes Occur?
  • 83. RQ 3.2: When Do Changes Occur? <20% of files
  • 84. RQ 3.3: How Big Are The Changes?
  • 85. RQ 3: Open Questions
  • 86. ● Why do ASAT configurations not typically evolve? RQ 3: Open Questions
  • 87. ● Why do ASAT configurations not typically evolve? ● How are ASATs used in a CI-environment? RQ 3: Open Questions
  • 88. Moritz Beller @Inventitech Analyzing the State of Static Analysis: A Large-Scale Evaluation in Open Source Software
  • 89. Moritz Beller @Inventitech Analyzing the State of Static Analysis: A Large-Scale Evaluation in Open Source Software
  • 90. Moritz Beller @Inventitech Analyzing the State of Static Analysis: A Large-Scale Evaluation in Open Source Software
  • 91. Moritz Beller @Inventitech Analyzing the State of Static Analysis: A Large-Scale Evaluation in Open Source Software