SlideShare ist ein Scribd-Unternehmen logo
1 von 31
Downloaden Sie, um offline zu lesen
A survey on software quality practice -
     Pilot study in the Walloon region




Javier Perez, Tom Mens, Service de Génie Logiciel, Université de Mons
           Flora Kamseu, Naji Habra, PRECISE lab, FUNDP




                 Presented at SATToSE seminar
                University of Koblenz, August 2012
 SATTOSE 2012
Context
• Portefeuille TIC
  – ERDF project lead by CETIC (2007-2013)
      • CEIQS: Center of expertise in engineering and
        quality of systems
            – aimed at developing a portfolio of innovative techniques
              allowing local companies to master the diversity,
              complexity, quality and rapid evolution of information
              systems
      • workpackage QUALGEN
            – collaboration between FUNDP and UMONS since 2010
  – Supported by Wallonia
   SATTOSE 2012                                                          2
Objectives

• Explore how quality-related software development
  practice is being performed in industry
• Compare this across different regions and
  countries
• Relates this to what is being taught in academia
   – Is there a gap between teaching and industry needs?
   – What good methodologies, practices, tools are not
     being used and why?




    SATTOSE 2012                                           3
About the survey
• Online survey carried out in Walloon region
   – Using LimeSurvey, from 29/5 till 30/6 2012
   – Companies involved in software development or
     software maintenance
• Addressed topics
   – Use of processes during software development and
     maintenance
   – Use of software quality measurement and improvement
   – Use of quality models and quality standards
   – Use of testing
   – Organisational support of development teams
    SATTOSE 2012                                           4
Structure of the questionnaire
• Introductory questions (6)
    – Details of respondent and company
• General development questions (5)
    – Perspective on dev. practices carried out by company
•   Structural software quality (5)
•   Software testing and maintenance (5)
•   Quality models and quality standards (5)
•   Organisational support of development teams (5)



      SATTOSE 2012                                           5
Respondents – Number
– Initial mailing sent out to 145 companies
– 71 responses out of 188 contacted respondents
    • Response rate 37,8%
– Responses from 47 different companies
    • Multiple responses from same companies were
      aggregated into a single one
– Incomplete responses were ignored
– 44 fully completed questionnaires kept for analysis
    • Corresponds to 62% of received responses


 SATTOSE 2012                                           6
Respondents - Origin
Mainly from Wallonia and Brussels (the target of our pilot study)




      SATTOSE 2012                                                  7
Respondents – Company size
Good balance between company size (number of employees) of respondents




     SATTOSE 2012                                                        8
Respondents – Training Level
Many at master+engineer level (18+17) and bachelor (20) level
Few or none with PhD (10+26), other degree (10+16) or no degree (3+27)




        SATTOSE 2012                                                     9
Process – Dev. process support
• Use of a well-defined and well-documented
  development process? 45,5%
  – No: 19
  – Yes: 20 (agile or scrum, Prince 2, RUP, ISO
    certification, proprietary)
  – Don’t know: 5
• Use of agile practices or methods? 63,6%
  – No: 12
  – Yes: 28 (17 mention SCRUM, 2 mention Prince2)
  – Don’t know: 4

   SATTOSE 2012                                     10
Process - change or configuration
      management process
Use of change or configuration management process is highly popular
73,8% (31/42)




      SATTOSE 2012                                                    11
Process - perceived importance factors
     for software project success
• Rated from (1) not important to (5) essential




                                     Average : 4   4,2
       SATTOSE 2012                                      12
Process - Generation of
artefacts/documents/deliverables
             during




  SATTOSE 2012                     13
Process - Creation and modification of
          software artefacts
Are arch. descr. (36,4%) and design models (39,5%) being evolved?




          SATTOSE 2012                                              14
Tools - Use of integrated
                    platform for
Version control (97,6%) and bug tracking (92,7%) well established
Platforms for continuous integration (57,1%), configuration (55,6%)
and testing (60,5%) a bit less




          SATTOSE 2012                                                15
Tools - Programming languages
                              used followed by scripting languages.
OO languages most popular (Java, C#, C++),
SQL and Cobol legacy also remains important.




       SATTOSE 2012                                                   16
Tools - Development
              environments used
Dichotomy between Java and .Net visible at IDE level.
Many others but much less frequently used.




   SATTOSE 2012                                         17
Tools - support for design models,
documentation and code synchro
• Very basic
  – Visio (10), Word (8), Sparx Enterprise Architect
    (5), Doxygen (3), StarUML (2), Confluence (2)
    and many others
  – Little use of UML modeling tools
• Little or no support for model-code
  synchronisation


   SATTOSE 2012                                        18
Quality - Use of design patterns
Use of design patterns is highly popular 72,7% (32/44)




       SATTOSE 2012                                      19
Quality - Use of quality
              improvement techniques
Only moderately popular (35,7%<x<43%) except for refactoring
                                                               Often or
(58,5%)                                                        continuously

                                                               36,6% (15/41)


                                                               42,9% (18/42)


                                                               35,7% (15/42)


                                                               58,5% (24/41)




         SATTOSE 2012                                                    20
Use of quality support continued
Poor support for quality (no quality tools, processes or models)




                                                                   36,8% (14/38)

                                                                   32,4% (12/37)

                                                                   19,4% (7/36)




          SATTOSE 2012                                                       21
Popularity of quality
           improvement techniques
•   Most popular
      Version control 97,6% and bug tracking 92,7% platforms
      Change and configuration management: 73,8%
      Design patterns: 72,7%
      Refactoring: 58,5%

•   Less popular
      Design improvement (e.g. code smell reduction): 42,9%
      Bad quality detection tools: 36,8%
      Metrics and visualisation tools: 35,7%
      Dynamic analysis tools (profiling etc.): 36,6%
      Quality support or improvement process: 32,4%

•   Unpopular
      Use of a quality model: 19,4%

      SATTOSE 2012                                              22
Reuse of libraries, components
               and platforms
Reuse is highly successful
 86% from own company
 75% from open source
 60% from other companies
 (excluding “don’t know” results)




          SATTOSE 2012                  23
Testing - Use of test process
Testing is done by nearly all respondents (97,7%), but
test process used by only 46,5% of the respondents




       SATTOSE 2012                                      24
Testing – development time spent




  SATTOSE 2012                     25
Testing - tool support
• Very varied
  – Mostly unit testing frameworks (8)
  – Others: Mantis (4), HP Quality Center (4),
    Selenium (3), Hudson (2), Jenkins (2), Quick
    Test Pro (2), and many more




   SATTOSE 2012                                    26
Preliminary conclusions
• Wide range of respondents, from very small to big
  companies, using many different programming
  languages and development environments

• Strong points
   – All respondents believe that quality assurance and
     testing are very important for project success
   – Wide use of testing (97,7%), agile practices (63,6%),
     design patterns (72,9%) and refactoring (56,5%)
   – High level of reuse of components/libraries/platforms
     (60% to 86% depending on source of reuse)


    SATTOSE 2012                                             27
Preliminary conclusions
• Weak points
  – Mitigated success of processes
     • development processes (45,5%), test processes (46,5%),
       quality support/improvement process (32,4%), quality
       models (19,4%)
     • Exception: change management process (73,8%)




      SATTOSE 2012                                              28
Preliminary conclusions
• Weak points
  – Mitigated success of static and dynamic code analysis
    tools for detecting quality issues, visualisation,
    computing metrics, profiling, etc… (popularity
    between 35% and 43%)
  – Does not reflect the high
    perceived importance of
    quality assurance for
    software project success

      SATTOSE 2012                                          29
What’s next?
• Analyse results in more detail
  – Correlate results to company size and training level
     • Are bigger companies more process-driven and smaller
       ones more agile? Does training level play a role?
• Report on the results
• Repeat the study in other countries
  – Identify regional or national trends
  – Compare differences and commonalities
  – We need your help here!


                                                         studies
        SATTOSE 2012                                               30
Collaborative study
• Research 2.0
  – Carry out follow-up / more narrow studies
  – Carry out this study as a collaborative
    community effort?
• Store / share / reuse our data with others
  – Which format/platform/… to use?
• Have a working session on this topic during
  SATTOSE/SOTESOLA?
    SATTOSE 2012                                31

Weitere ähnliche Inhalte

Was ist angesagt?

1010 guide–a simple framework for
1010 guide–a simple framework for1010 guide–a simple framework for
1010 guide–a simple framework forijseajournal
 
Ch24-Software Engineering 9
Ch24-Software Engineering 9Ch24-Software Engineering 9
Ch24-Software Engineering 9Ian Sommerville
 
NITC-2016 - Effectiveness of Agile Test Planning
NITC-2016 - Effectiveness of Agile Test Planning NITC-2016 - Effectiveness of Agile Test Planning
NITC-2016 - Effectiveness of Agile Test Planning Udayantha de Silva
 
WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability tes...
WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability tes...WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability tes...
WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability tes...WCIT 2014
 
Quality Assurance is Not Testing
Quality Assurance is Not TestingQuality Assurance is Not Testing
Quality Assurance is Not TestingTom Walton
 
Obstacle Driven Development
Obstacle Driven Development Obstacle Driven Development
Obstacle Driven Development Jonathan Herring
 
Gap Survey, Assessment and Analysis for DevSecOps
Gap Survey, Assessment and Analysis for DevSecOpsGap Survey, Assessment and Analysis for DevSecOps
Gap Survey, Assessment and Analysis for DevSecOpsMarc Hornbeek
 
The Quest for Quality at Speed
The Quest for Quality at SpeedThe Quest for Quality at Speed
The Quest for Quality at SpeedMarc Hornbeek
 
How Do You Measure The KM Maturity Of Your Organization Final Ver.
How Do You Measure The KM Maturity Of Your Organization Final Ver.How Do You Measure The KM Maturity Of Your Organization Final Ver.
How Do You Measure The KM Maturity Of Your Organization Final Ver.Art Schlussel
 
Using the test process improvement models. Case study based on TPI Next model...
Using the test process improvement models. Case study based on TPI Next model...Using the test process improvement models. Case study based on TPI Next model...
Using the test process improvement models. Case study based on TPI Next model...Sigma Software
 
IDG MarketPulse: Virtual Graphics Processing Unit (vGPU)
IDG MarketPulse: Virtual Graphics Processing Unit (vGPU)IDG MarketPulse: Virtual Graphics Processing Unit (vGPU)
IDG MarketPulse: Virtual Graphics Processing Unit (vGPU)jmariani14
 
Systems Engineering training brochure
Systems Engineering training brochureSystems Engineering training brochure
Systems Engineering training brochureShaun West
 
Quality Engineering Material
Quality Engineering Material Quality Engineering Material
Quality Engineering Material TeluguSudhakar3
 
Quality Engineering and Testing with TMAP in DevOps IT delivery
Quality Engineering and Testing with TMAP in DevOps IT deliveryQuality Engineering and Testing with TMAP in DevOps IT delivery
Quality Engineering and Testing with TMAP in DevOps IT deliveryRik Marselis
 
Gap Assessment for DevOps
Gap Assessment   for DevOpsGap Assessment   for DevOps
Gap Assessment for DevOpsMarc Hornbeek
 

Was ist angesagt? (20)

1010 guide–a simple framework for
1010 guide–a simple framework for1010 guide–a simple framework for
1010 guide–a simple framework for
 
Ch24-Software Engineering 9
Ch24-Software Engineering 9Ch24-Software Engineering 9
Ch24-Software Engineering 9
 
NITC-2016 - Effectiveness of Agile Test Planning
NITC-2016 - Effectiveness of Agile Test Planning NITC-2016 - Effectiveness of Agile Test Planning
NITC-2016 - Effectiveness of Agile Test Planning
 
WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability tes...
WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability tes...WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability tes...
WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability tes...
 
Cen6070 chapter2
Cen6070 chapter2Cen6070 chapter2
Cen6070 chapter2
 
Quality Assurance is Not Testing
Quality Assurance is Not TestingQuality Assurance is Not Testing
Quality Assurance is Not Testing
 
Obstacle Driven Development
Obstacle Driven Development Obstacle Driven Development
Obstacle Driven Development
 
Reliable SAP
Reliable SAPReliable SAP
Reliable SAP
 
Gap Survey, Assessment and Analysis for DevSecOps
Gap Survey, Assessment and Analysis for DevSecOpsGap Survey, Assessment and Analysis for DevSecOps
Gap Survey, Assessment and Analysis for DevSecOps
 
ODD + Project Control 1.0
ODD + Project Control 1.0ODD + Project Control 1.0
ODD + Project Control 1.0
 
The Quest for Quality at Speed
The Quest for Quality at SpeedThe Quest for Quality at Speed
The Quest for Quality at Speed
 
How Do You Measure The KM Maturity Of Your Organization Final Ver.
How Do You Measure The KM Maturity Of Your Organization Final Ver.How Do You Measure The KM Maturity Of Your Organization Final Ver.
How Do You Measure The KM Maturity Of Your Organization Final Ver.
 
209
209209
209
 
Using the test process improvement models. Case study based on TPI Next model...
Using the test process improvement models. Case study based on TPI Next model...Using the test process improvement models. Case study based on TPI Next model...
Using the test process improvement models. Case study based on TPI Next model...
 
IDG MarketPulse: Virtual Graphics Processing Unit (vGPU)
IDG MarketPulse: Virtual Graphics Processing Unit (vGPU)IDG MarketPulse: Virtual Graphics Processing Unit (vGPU)
IDG MarketPulse: Virtual Graphics Processing Unit (vGPU)
 
Systems Engineering training brochure
Systems Engineering training brochureSystems Engineering training brochure
Systems Engineering training brochure
 
Quality Engineering Material
Quality Engineering Material Quality Engineering Material
Quality Engineering Material
 
QA standards
QA standardsQA standards
QA standards
 
Quality Engineering and Testing with TMAP in DevOps IT delivery
Quality Engineering and Testing with TMAP in DevOps IT deliveryQuality Engineering and Testing with TMAP in DevOps IT delivery
Quality Engineering and Testing with TMAP in DevOps IT delivery
 
Gap Assessment for DevOps
Gap Assessment   for DevOpsGap Assessment   for DevOps
Gap Assessment for DevOps
 

Ähnlich wie A survey on software quality practice - Pilot study in the Walloon region

Presentation - Rational Unified Process
Presentation - Rational Unified ProcessPresentation - Rational Unified Process
Presentation - Rational Unified ProcessSharad Srivastava
 
Software Engineering The Multiview Approach And Wisdm
Software Engineering   The Multiview Approach And WisdmSoftware Engineering   The Multiview Approach And Wisdm
Software Engineering The Multiview Approach And Wisdmguestc990b6
 
REDEFINING ASSUMPTIONS Accessibility and Its Stakeholders
REDEFINING ASSUMPTIONS Accessibility and Its StakeholdersREDEFINING ASSUMPTIONS Accessibility and Its Stakeholders
REDEFINING ASSUMPTIONS Accessibility and Its StakeholdersAEGIS-ACCESSIBLE Projects
 
Recent and-future-trends spm
Recent and-future-trends spmRecent and-future-trends spm
Recent and-future-trends spmPrakash Poudel
 
Cheat-Sheet-Lean-Manufacturing-EN.pdf
Cheat-Sheet-Lean-Manufacturing-EN.pdfCheat-Sheet-Lean-Manufacturing-EN.pdf
Cheat-Sheet-Lean-Manufacturing-EN.pdfChristoph Hachen
 
software-process-project-managementr17a0539.pdf
software-process-project-managementr17a0539.pdfsoftware-process-project-managementr17a0539.pdf
software-process-project-managementr17a0539.pdfSUDHEER REDDY BANDI
 
Model-Based Software Engineering: A Multiple-Case Study on Challenges and Dev...
Model-Based Software Engineering: A Multiple-Case Study on Challenges and Dev...Model-Based Software Engineering: A Multiple-Case Study on Challenges and Dev...
Model-Based Software Engineering: A Multiple-Case Study on Challenges and Dev...Rodi Jolak
 
A Method for Evaluating End-User Development Technologies
A Method for Evaluating End-User Development TechnologiesA Method for Evaluating End-User Development Technologies
A Method for Evaluating End-User Development TechnologiesClaudia Melo
 
Learn Key Insights from The State of Web Application Testing Research Report
Learn Key Insights from The State of Web Application Testing Research ReportLearn Key Insights from The State of Web Application Testing Research Report
Learn Key Insights from The State of Web Application Testing Research ReportSencha
 
Beit 381 se lec 5, 6, 7 & 8 - 69 - 12 feb21,22,28,29 - sw process 1-3 sdlc m...
Beit 381 se lec 5, 6, 7 & 8 - 69 - 12 feb21,22,28,29  - sw process 1-3 sdlc m...Beit 381 se lec 5, 6, 7 & 8 - 69 - 12 feb21,22,28,29  - sw process 1-3 sdlc m...
Beit 381 se lec 5, 6, 7 & 8 - 69 - 12 feb21,22,28,29 - sw process 1-3 sdlc m...babak danyal
 
ProductCamp Amsterdam Session 9 Rudy Katchow
ProductCamp Amsterdam Session 9 Rudy KatchowProductCamp Amsterdam Session 9 Rudy Katchow
ProductCamp Amsterdam Session 9 Rudy KatchowProductCamp Amsterdam
 
Lean for Competitive Advantage and Customer Delight
Lean for Competitive Advantage and Customer DelightLean for Competitive Advantage and Customer Delight
Lean for Competitive Advantage and Customer DelightLean India Summit
 
Diana M. Arias Resume - 2017
Diana M. Arias Resume - 2017Diana M. Arias Resume - 2017
Diana M. Arias Resume - 2017Diana M. Arias
 

Ähnlich wie A survey on software quality practice - Pilot study in the Walloon region (20)

Presentation - Rational Unified Process
Presentation - Rational Unified ProcessPresentation - Rational Unified Process
Presentation - Rational Unified Process
 
Software Engineering The Multiview Approach And Wisdm
Software Engineering   The Multiview Approach And WisdmSoftware Engineering   The Multiview Approach And Wisdm
Software Engineering The Multiview Approach And Wisdm
 
REDEFINING ASSUMPTIONS Accessibility and Its Stakeholders
REDEFINING ASSUMPTIONS Accessibility and Its StakeholdersREDEFINING ASSUMPTIONS Accessibility and Its Stakeholders
REDEFINING ASSUMPTIONS Accessibility and Its Stakeholders
 
Recent and-future-trends spm
Recent and-future-trends spmRecent and-future-trends spm
Recent and-future-trends spm
 
The Journey to DevOps
The Journey to DevOpsThe Journey to DevOps
The Journey to DevOps
 
Dynamic System Development Method
Dynamic System Development MethodDynamic System Development Method
Dynamic System Development Method
 
reaserch ppt.pptx
reaserch ppt.pptxreaserch ppt.pptx
reaserch ppt.pptx
 
Ch 11(spi)relationship pa
Ch 11(spi)relationship paCh 11(spi)relationship pa
Ch 11(spi)relationship pa
 
RUP
RUPRUP
RUP
 
Agile Test Transformation
Agile Test TransformationAgile Test Transformation
Agile Test Transformation
 
Cheat-Sheet-Lean-Manufacturing-EN.pdf
Cheat-Sheet-Lean-Manufacturing-EN.pdfCheat-Sheet-Lean-Manufacturing-EN.pdf
Cheat-Sheet-Lean-Manufacturing-EN.pdf
 
software project management
software project managementsoftware project management
software project management
 
software-process-project-managementr17a0539.pdf
software-process-project-managementr17a0539.pdfsoftware-process-project-managementr17a0539.pdf
software-process-project-managementr17a0539.pdf
 
Model-Based Software Engineering: A Multiple-Case Study on Challenges and Dev...
Model-Based Software Engineering: A Multiple-Case Study on Challenges and Dev...Model-Based Software Engineering: A Multiple-Case Study on Challenges and Dev...
Model-Based Software Engineering: A Multiple-Case Study on Challenges and Dev...
 
A Method for Evaluating End-User Development Technologies
A Method for Evaluating End-User Development TechnologiesA Method for Evaluating End-User Development Technologies
A Method for Evaluating End-User Development Technologies
 
Learn Key Insights from The State of Web Application Testing Research Report
Learn Key Insights from The State of Web Application Testing Research ReportLearn Key Insights from The State of Web Application Testing Research Report
Learn Key Insights from The State of Web Application Testing Research Report
 
Beit 381 se lec 5, 6, 7 & 8 - 69 - 12 feb21,22,28,29 - sw process 1-3 sdlc m...
Beit 381 se lec 5, 6, 7 & 8 - 69 - 12 feb21,22,28,29  - sw process 1-3 sdlc m...Beit 381 se lec 5, 6, 7 & 8 - 69 - 12 feb21,22,28,29  - sw process 1-3 sdlc m...
Beit 381 se lec 5, 6, 7 & 8 - 69 - 12 feb21,22,28,29 - sw process 1-3 sdlc m...
 
ProductCamp Amsterdam Session 9 Rudy Katchow
ProductCamp Amsterdam Session 9 Rudy KatchowProductCamp Amsterdam Session 9 Rudy Katchow
ProductCamp Amsterdam Session 9 Rudy Katchow
 
Lean for Competitive Advantage and Customer Delight
Lean for Competitive Advantage and Customer DelightLean for Competitive Advantage and Customer Delight
Lean for Competitive Advantage and Customer Delight
 
Diana M. Arias Resume - 2017
Diana M. Arias Resume - 2017Diana M. Arias Resume - 2017
Diana M. Arias Resume - 2017
 

Mehr von Tom Mens

How to be(come) a successful PhD student
How to be(come) a successful PhD studentHow to be(come) a successful PhD student
How to be(come) a successful PhD studentTom Mens
 
Recognising bot activity in collaborative software development
Recognising bot activity in collaborative software developmentRecognising bot activity in collaborative software development
Recognising bot activity in collaborative software developmentTom Mens
 
A Dataset of Bot and Human Activities in GitHub
A Dataset of Bot and Human Activities in GitHubA Dataset of Bot and Human Activities in GitHub
A Dataset of Bot and Human Activities in GitHubTom Mens
 
The (r)evolution of CI/CD on GitHub
 The (r)evolution of CI/CD on GitHub The (r)evolution of CI/CD on GitHub
The (r)evolution of CI/CD on GitHubTom Mens
 
Nurturing the Software Ecosystems of the Future
Nurturing the Software Ecosystems of the FutureNurturing the Software Ecosystems of the Future
Nurturing the Software Ecosystems of the FutureTom Mens
 
Comment programmer un robot en 30 minutes?
Comment programmer un robot en 30 minutes?Comment programmer un robot en 30 minutes?
Comment programmer un robot en 30 minutes?Tom Mens
 
On the rise and fall of CI services in GitHub
On the rise and fall of CI services in GitHubOn the rise and fall of CI services in GitHub
On the rise and fall of CI services in GitHubTom Mens
 
On backporting practices in package dependency networks
On backporting practices in package dependency networksOn backporting practices in package dependency networks
On backporting practices in package dependency networksTom Mens
 
Comparing semantic versioning practices in Cargo, npm, Packagist and Rubygems
Comparing semantic versioning practices in Cargo, npm, Packagist and RubygemsComparing semantic versioning practices in Cargo, npm, Packagist and Rubygems
Comparing semantic versioning practices in Cargo, npm, Packagist and RubygemsTom Mens
 
Lost in Zero Space
Lost in Zero SpaceLost in Zero Space
Lost in Zero SpaceTom Mens
 
Evaluating a bot detection model on git commit messages
Evaluating a bot detection model on git commit messagesEvaluating a bot detection model on git commit messages
Evaluating a bot detection model on git commit messagesTom Mens
 
Is my software ecosystem healthy? It depends!
Is my software ecosystem healthy? It depends!Is my software ecosystem healthy? It depends!
Is my software ecosystem healthy? It depends!Tom Mens
 
Bot or not? Detecting bots in GitHub pull request activity based on comment s...
Bot or not? Detecting bots in GitHub pull request activity based on comment s...Bot or not? Detecting bots in GitHub pull request activity based on comment s...
Bot or not? Detecting bots in GitHub pull request activity based on comment s...Tom Mens
 
On the fragility of open source software packaging ecosystems
On the fragility of open source software packaging ecosystemsOn the fragility of open source software packaging ecosystems
On the fragility of open source software packaging ecosystemsTom Mens
 
How magic is zero? An Empirical Analysis of Initial Development Releases in S...
How magic is zero? An Empirical Analysis of Initial Development Releases in S...How magic is zero? An Empirical Analysis of Initial Development Releases in S...
How magic is zero? An Empirical Analysis of Initial Development Releases in S...Tom Mens
 
Comparing dependency issues across software package distributions (FOSDEM 2020)
Comparing dependency issues across software package distributions (FOSDEM 2020)Comparing dependency issues across software package distributions (FOSDEM 2020)
Comparing dependency issues across software package distributions (FOSDEM 2020)Tom Mens
 
Measuring Technical Lag in Software Deployments (CHAOSScon 2020)
Measuring Technical Lag in Software Deployments (CHAOSScon 2020)Measuring Technical Lag in Software Deployments (CHAOSScon 2020)
Measuring Technical Lag in Software Deployments (CHAOSScon 2020)Tom Mens
 
SecoHealth 2019 Research Achievements
SecoHealth 2019 Research AchievementsSecoHealth 2019 Research Achievements
SecoHealth 2019 Research AchievementsTom Mens
 
SECO-Assist 2019 research seminar
SECO-Assist 2019 research seminarSECO-Assist 2019 research seminar
SECO-Assist 2019 research seminarTom Mens
 
Empirically Analysing the Socio-Technical Health of Software Package Managers
Empirically Analysing the Socio-Technical Health of Software Package ManagersEmpirically Analysing the Socio-Technical Health of Software Package Managers
Empirically Analysing the Socio-Technical Health of Software Package ManagersTom Mens
 

Mehr von Tom Mens (20)

How to be(come) a successful PhD student
How to be(come) a successful PhD studentHow to be(come) a successful PhD student
How to be(come) a successful PhD student
 
Recognising bot activity in collaborative software development
Recognising bot activity in collaborative software developmentRecognising bot activity in collaborative software development
Recognising bot activity in collaborative software development
 
A Dataset of Bot and Human Activities in GitHub
A Dataset of Bot and Human Activities in GitHubA Dataset of Bot and Human Activities in GitHub
A Dataset of Bot and Human Activities in GitHub
 
The (r)evolution of CI/CD on GitHub
 The (r)evolution of CI/CD on GitHub The (r)evolution of CI/CD on GitHub
The (r)evolution of CI/CD on GitHub
 
Nurturing the Software Ecosystems of the Future
Nurturing the Software Ecosystems of the FutureNurturing the Software Ecosystems of the Future
Nurturing the Software Ecosystems of the Future
 
Comment programmer un robot en 30 minutes?
Comment programmer un robot en 30 minutes?Comment programmer un robot en 30 minutes?
Comment programmer un robot en 30 minutes?
 
On the rise and fall of CI services in GitHub
On the rise and fall of CI services in GitHubOn the rise and fall of CI services in GitHub
On the rise and fall of CI services in GitHub
 
On backporting practices in package dependency networks
On backporting practices in package dependency networksOn backporting practices in package dependency networks
On backporting practices in package dependency networks
 
Comparing semantic versioning practices in Cargo, npm, Packagist and Rubygems
Comparing semantic versioning practices in Cargo, npm, Packagist and RubygemsComparing semantic versioning practices in Cargo, npm, Packagist and Rubygems
Comparing semantic versioning practices in Cargo, npm, Packagist and Rubygems
 
Lost in Zero Space
Lost in Zero SpaceLost in Zero Space
Lost in Zero Space
 
Evaluating a bot detection model on git commit messages
Evaluating a bot detection model on git commit messagesEvaluating a bot detection model on git commit messages
Evaluating a bot detection model on git commit messages
 
Is my software ecosystem healthy? It depends!
Is my software ecosystem healthy? It depends!Is my software ecosystem healthy? It depends!
Is my software ecosystem healthy? It depends!
 
Bot or not? Detecting bots in GitHub pull request activity based on comment s...
Bot or not? Detecting bots in GitHub pull request activity based on comment s...Bot or not? Detecting bots in GitHub pull request activity based on comment s...
Bot or not? Detecting bots in GitHub pull request activity based on comment s...
 
On the fragility of open source software packaging ecosystems
On the fragility of open source software packaging ecosystemsOn the fragility of open source software packaging ecosystems
On the fragility of open source software packaging ecosystems
 
How magic is zero? An Empirical Analysis of Initial Development Releases in S...
How magic is zero? An Empirical Analysis of Initial Development Releases in S...How magic is zero? An Empirical Analysis of Initial Development Releases in S...
How magic is zero? An Empirical Analysis of Initial Development Releases in S...
 
Comparing dependency issues across software package distributions (FOSDEM 2020)
Comparing dependency issues across software package distributions (FOSDEM 2020)Comparing dependency issues across software package distributions (FOSDEM 2020)
Comparing dependency issues across software package distributions (FOSDEM 2020)
 
Measuring Technical Lag in Software Deployments (CHAOSScon 2020)
Measuring Technical Lag in Software Deployments (CHAOSScon 2020)Measuring Technical Lag in Software Deployments (CHAOSScon 2020)
Measuring Technical Lag in Software Deployments (CHAOSScon 2020)
 
SecoHealth 2019 Research Achievements
SecoHealth 2019 Research AchievementsSecoHealth 2019 Research Achievements
SecoHealth 2019 Research Achievements
 
SECO-Assist 2019 research seminar
SECO-Assist 2019 research seminarSECO-Assist 2019 research seminar
SECO-Assist 2019 research seminar
 
Empirically Analysing the Socio-Technical Health of Software Package Managers
Empirically Analysing the Socio-Technical Health of Software Package ManagersEmpirically Analysing the Socio-Technical Health of Software Package Managers
Empirically Analysing the Socio-Technical Health of Software Package Managers
 

Kürzlich hochgeladen

NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?SANGHEE SHIN
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdfJamie (Taka) Wang
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncObject Automation
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 

Kürzlich hochgeladen (20)

NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation Inc
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 

A survey on software quality practice - Pilot study in the Walloon region

  • 1. A survey on software quality practice - Pilot study in the Walloon region Javier Perez, Tom Mens, Service de Génie Logiciel, Université de Mons Flora Kamseu, Naji Habra, PRECISE lab, FUNDP Presented at SATToSE seminar University of Koblenz, August 2012 SATTOSE 2012
  • 2. Context • Portefeuille TIC – ERDF project lead by CETIC (2007-2013) • CEIQS: Center of expertise in engineering and quality of systems – aimed at developing a portfolio of innovative techniques allowing local companies to master the diversity, complexity, quality and rapid evolution of information systems • workpackage QUALGEN – collaboration between FUNDP and UMONS since 2010 – Supported by Wallonia SATTOSE 2012 2
  • 3. Objectives • Explore how quality-related software development practice is being performed in industry • Compare this across different regions and countries • Relates this to what is being taught in academia – Is there a gap between teaching and industry needs? – What good methodologies, practices, tools are not being used and why? SATTOSE 2012 3
  • 4. About the survey • Online survey carried out in Walloon region – Using LimeSurvey, from 29/5 till 30/6 2012 – Companies involved in software development or software maintenance • Addressed topics – Use of processes during software development and maintenance – Use of software quality measurement and improvement – Use of quality models and quality standards – Use of testing – Organisational support of development teams SATTOSE 2012 4
  • 5. Structure of the questionnaire • Introductory questions (6) – Details of respondent and company • General development questions (5) – Perspective on dev. practices carried out by company • Structural software quality (5) • Software testing and maintenance (5) • Quality models and quality standards (5) • Organisational support of development teams (5) SATTOSE 2012 5
  • 6. Respondents – Number – Initial mailing sent out to 145 companies – 71 responses out of 188 contacted respondents • Response rate 37,8% – Responses from 47 different companies • Multiple responses from same companies were aggregated into a single one – Incomplete responses were ignored – 44 fully completed questionnaires kept for analysis • Corresponds to 62% of received responses SATTOSE 2012 6
  • 7. Respondents - Origin Mainly from Wallonia and Brussels (the target of our pilot study) SATTOSE 2012 7
  • 8. Respondents – Company size Good balance between company size (number of employees) of respondents SATTOSE 2012 8
  • 9. Respondents – Training Level Many at master+engineer level (18+17) and bachelor (20) level Few or none with PhD (10+26), other degree (10+16) or no degree (3+27) SATTOSE 2012 9
  • 10. Process – Dev. process support • Use of a well-defined and well-documented development process? 45,5% – No: 19 – Yes: 20 (agile or scrum, Prince 2, RUP, ISO certification, proprietary) – Don’t know: 5 • Use of agile practices or methods? 63,6% – No: 12 – Yes: 28 (17 mention SCRUM, 2 mention Prince2) – Don’t know: 4 SATTOSE 2012 10
  • 11. Process - change or configuration management process Use of change or configuration management process is highly popular 73,8% (31/42) SATTOSE 2012 11
  • 12. Process - perceived importance factors for software project success • Rated from (1) not important to (5) essential Average : 4 4,2 SATTOSE 2012 12
  • 13. Process - Generation of artefacts/documents/deliverables during SATTOSE 2012 13
  • 14. Process - Creation and modification of software artefacts Are arch. descr. (36,4%) and design models (39,5%) being evolved? SATTOSE 2012 14
  • 15. Tools - Use of integrated platform for Version control (97,6%) and bug tracking (92,7%) well established Platforms for continuous integration (57,1%), configuration (55,6%) and testing (60,5%) a bit less SATTOSE 2012 15
  • 16. Tools - Programming languages used followed by scripting languages. OO languages most popular (Java, C#, C++), SQL and Cobol legacy also remains important. SATTOSE 2012 16
  • 17. Tools - Development environments used Dichotomy between Java and .Net visible at IDE level. Many others but much less frequently used. SATTOSE 2012 17
  • 18. Tools - support for design models, documentation and code synchro • Very basic – Visio (10), Word (8), Sparx Enterprise Architect (5), Doxygen (3), StarUML (2), Confluence (2) and many others – Little use of UML modeling tools • Little or no support for model-code synchronisation SATTOSE 2012 18
  • 19. Quality - Use of design patterns Use of design patterns is highly popular 72,7% (32/44) SATTOSE 2012 19
  • 20. Quality - Use of quality improvement techniques Only moderately popular (35,7%<x<43%) except for refactoring Often or (58,5%) continuously 36,6% (15/41) 42,9% (18/42) 35,7% (15/42) 58,5% (24/41) SATTOSE 2012 20
  • 21. Use of quality support continued Poor support for quality (no quality tools, processes or models) 36,8% (14/38) 32,4% (12/37) 19,4% (7/36) SATTOSE 2012 21
  • 22. Popularity of quality improvement techniques • Most popular  Version control 97,6% and bug tracking 92,7% platforms  Change and configuration management: 73,8%  Design patterns: 72,7%  Refactoring: 58,5% • Less popular  Design improvement (e.g. code smell reduction): 42,9%  Bad quality detection tools: 36,8%  Metrics and visualisation tools: 35,7%  Dynamic analysis tools (profiling etc.): 36,6%  Quality support or improvement process: 32,4% • Unpopular  Use of a quality model: 19,4% SATTOSE 2012 22
  • 23. Reuse of libraries, components and platforms Reuse is highly successful 86% from own company 75% from open source 60% from other companies (excluding “don’t know” results) SATTOSE 2012 23
  • 24. Testing - Use of test process Testing is done by nearly all respondents (97,7%), but test process used by only 46,5% of the respondents SATTOSE 2012 24
  • 25. Testing – development time spent SATTOSE 2012 25
  • 26. Testing - tool support • Very varied – Mostly unit testing frameworks (8) – Others: Mantis (4), HP Quality Center (4), Selenium (3), Hudson (2), Jenkins (2), Quick Test Pro (2), and many more SATTOSE 2012 26
  • 27. Preliminary conclusions • Wide range of respondents, from very small to big companies, using many different programming languages and development environments • Strong points – All respondents believe that quality assurance and testing are very important for project success – Wide use of testing (97,7%), agile practices (63,6%), design patterns (72,9%) and refactoring (56,5%) – High level of reuse of components/libraries/platforms (60% to 86% depending on source of reuse) SATTOSE 2012 27
  • 28. Preliminary conclusions • Weak points – Mitigated success of processes • development processes (45,5%), test processes (46,5%), quality support/improvement process (32,4%), quality models (19,4%) • Exception: change management process (73,8%) SATTOSE 2012 28
  • 29. Preliminary conclusions • Weak points – Mitigated success of static and dynamic code analysis tools for detecting quality issues, visualisation, computing metrics, profiling, etc… (popularity between 35% and 43%) – Does not reflect the high perceived importance of quality assurance for software project success SATTOSE 2012 29
  • 30. What’s next? • Analyse results in more detail – Correlate results to company size and training level • Are bigger companies more process-driven and smaller ones more agile? Does training level play a role? • Report on the results • Repeat the study in other countries – Identify regional or national trends – Compare differences and commonalities – We need your help here! studies SATTOSE 2012 30
  • 31. Collaborative study • Research 2.0 – Carry out follow-up / more narrow studies – Carry out this study as a collaborative community effort? • Store / share / reuse our data with others – Which format/platform/… to use? • Have a working session on this topic during SATTOSE/SOTESOLA? SATTOSE 2012 31

Hinweis der Redaktion

  1. 188 potential respondents were identified and invited by mail to respond to survey
  2. Pay attention that this does not necessarily represent the real distribution of number of companies per size in considered region. This would mean that, in principle, all charts and results shown hereafter should be “weighted” to reflect the real distribution of sizes in the considered region.
  3. Where are PhDs working? SMEs? Bigger companies? Agile-driven companies?
  4. Correlate this to the company size !
  5. multiple answers were possible Responses were very varied Singletons: ASP; Flex; HTML5; PERL; Eiffel; Pascal; Objective C; Delphi; Mumps; Groovy; Scala; Matlab; JCL; CL; CICS; DDS; RPG; …