SlideShare ist ein Scribd-Unternehmen logo

Make the Most of Your Time: How Should the Analyst Work with Automated Traceability Tools

•Als PPT, PDF herunterladen•
•
Tim Menzies
Tim Menzies

Alexander Dekhytar, Jane Hayes, and Jody Larson

TechnologieNews & Politik
1 von 27
Make the Most of Your Time: How Should the Analyst Work with Automated Traceability Tools?* Alex Dekhtyar Jane Huffman Hayes Jody Larsen *Funded by NASA
Introduction ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
What is Requirements Traceability? ,[object Object],[object Object]
Why is Traceability Important? ,[object Object],[object Object],[object Object],[object Object]
Design Document Information Retrieval and Traceability Representation Requirements Document Matching algorithm Analyst
What’s Different about Traceability? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
IR Based Traceability Frameworks ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
RETRO.NET ,[object Object],[object Object],[object Object],[object Object]
Research Questions ,[object Object],[object Object]
Experimental Design ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Measures N - number of low-level requirements; M - number of high-level requirements; Hits - number of confirmed candidate links Strikes - number of false positives Misses - number of missed links Precision = Hits Hits + Strikes Recall = Hits Hits +Misses Selectivity = Hits + Strikes M * N
Global Ordering Without Feedback ,[object Object],[object Object],[object Object]
Local Ordering Without Feedback ,[object Object],[object Object],[object Object]
Global Ordering With Feedback ,[object Object],[object Object],[object Object],[object Object]
Local Ordering With Feedback ,[object Object],[object Object],[object Object],[object Object]
Dataset ,[object Object],[object Object],[object Object],[object Object],[object Object],36,556 Total # of retrieved candidate links 70.7% Selectivity 358 Total # of correct links retrieved by basic Vector Space 99% Recall 0.1% Precision # of elements in requirements (high-level) 235 # of elements in design specification (low-level) 220 # of correct links 361
Experimental Design Detail ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Random? ,[object Object],[object Object]
Random Methods ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Experiment Operation ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Experiment Validity ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Study 1 Results, Fixed Recall Observed Links Precision Selectivity Local, Feedback 1595 20% 3% Local, No Feedback 1713 19% 3% Global, Feedback 5399 6% 10% Global, No Feedback 6149 5% 12%
Study 2 Results, Fixed Effort Confirmed Links Precision Recall Local, Feedback 321 20% 89% Local, No Feedback 326 20% 87.5% Global, Feedback 236 15% 65% Global, No Feedback 227 14% 63% Local, Random 65 4% 18% Global, Random 58 3.6% 16%
Study 2 Intermediate Results
Analysis ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Future Work ,[object Object],[object Object],[object Object]
Questions?

Empfohlen

Project Data Incorporating Qualitative Factors for Improved Software Defect P...
Project Data Incorporating Qualitative Factors for Improved Software Defect P...Project Data Incorporating Qualitative Factors for Improved Software Defect P...
Project Data Incorporating Qualitative Factors for Improved Software Defect P...Tim Menzies
 
Experiments on Design Pattern Discovery
Experiments on Design Pattern DiscoveryExperiments on Design Pattern Discovery
Experiments on Design Pattern DiscoveryTim Menzies
 
Promise Keynote
Promise KeynotePromise Keynote
Promise KeynoteTim Menzies
 
Complexity Measures for Secure Service-Orieted Software Architectures
Complexity Measures for Secure Service-Orieted Software ArchitecturesComplexity Measures for Secure Service-Orieted Software Architectures
Complexity Measures for Secure Service-Orieted Software ArchitecturesTim Menzies
 
Assessing the Reliability of a Human Estimator
Assessing the Reliability of a Human EstimatorAssessing the Reliability of a Human Estimator
Assessing the Reliability of a Human EstimatorTim Menzies
 
Using Developer Information as a Prediction Factor
Using Developer Information as a Prediction FactorUsing Developer Information as a Prediction Factor
Using Developer Information as a Prediction FactorTim Menzies
 
Testing 2 - Thinking Like A Tester
Testing 2 - Thinking Like A TesterTesting 2 - Thinking Like A Tester
Testing 2 - Thinking Like A TesterArleneAndrews2
 
Model-Driven Run-Time Enforcement of Complex Role-Based Access Control Policies
Model-Driven Run-Time Enforcement of Complex Role-Based Access Control PoliciesModel-Driven Run-Time Enforcement of Complex Role-Based Access Control Policies
Model-Driven Run-Time Enforcement of Complex Role-Based Access Control PoliciesLionel Briand
 

Empfohlen

Project Data Incorporating Qualitative Factors for Improved Software Defect P...
Project Data Incorporating Qualitative Factors for Improved Software Defect P...Project Data Incorporating Qualitative Factors for Improved Software Defect P...
Project Data Incorporating Qualitative Factors for Improved Software Defect P...Tim Menzies
 
Experiments on Design Pattern Discovery
Experiments on Design Pattern DiscoveryExperiments on Design Pattern Discovery
Experiments on Design Pattern DiscoveryTim Menzies
 
Promise Keynote
Promise KeynotePromise Keynote
Promise KeynoteTim Menzies
 
Complexity Measures for Secure Service-Orieted Software Architectures
Complexity Measures for Secure Service-Orieted Software ArchitecturesComplexity Measures for Secure Service-Orieted Software Architectures
Complexity Measures for Secure Service-Orieted Software ArchitecturesTim Menzies
 
Assessing the Reliability of a Human Estimator
Assessing the Reliability of a Human EstimatorAssessing the Reliability of a Human Estimator
Assessing the Reliability of a Human EstimatorTim Menzies
 
Using Developer Information as a Prediction Factor
Using Developer Information as a Prediction FactorUsing Developer Information as a Prediction Factor
Using Developer Information as a Prediction FactorTim Menzies
 
Testing 2 - Thinking Like A Tester
Testing 2 - Thinking Like A TesterTesting 2 - Thinking Like A Tester
Testing 2 - Thinking Like A TesterArleneAndrews2
 
Model-Driven Run-Time Enforcement of Complex Role-Based Access Control Policies
Model-Driven Run-Time Enforcement of Complex Role-Based Access Control PoliciesModel-Driven Run-Time Enforcement of Complex Role-Based Access Control Policies
Model-Driven Run-Time Enforcement of Complex Role-Based Access Control PoliciesLionel Briand
 
Instance Space Analysis for Search Based Software Engineering
Instance Space Analysis for Search Based Software EngineeringInstance Space Analysis for Search Based Software Engineering
Instance Space Analysis for Search Based Software EngineeringAldeida Aleti
 
Cukic Promise08 V3
Cukic Promise08 V3Cukic Promise08 V3
Cukic Promise08 V3gregoryg
 
Generation of Search Based Test Data on Acceptability Testing Principle
Generation of Search Based Test Data on Acceptability Testing PrincipleGeneration of Search Based Test Data on Acceptability Testing Principle
Generation of Search Based Test Data on Acceptability Testing Principleiosrjce
 
[Tho Quan] Fault Localization - Where is the root cause of a bug?
[Tho Quan] Fault Localization - Where is the root cause of a bug?[Tho Quan] Fault Localization - Where is the root cause of a bug?
[Tho Quan] Fault Localization - Where is the root cause of a bug?Ho Chi Minh City Software Testing Club
 
A software fault localization technique based on program mutations
A software fault localization technique based on program mutationsA software fault localization technique based on program mutations
A software fault localization technique based on program mutationsTao He
 
StratCel: A Strategy-Centric Approach to the Design of End-User Debugging Tools
StratCel: A Strategy-Centric Approach to the Design of End-User Debugging ToolsStratCel: A Strategy-Centric Approach to the Design of End-User Debugging Tools
StratCel: A Strategy-Centric Approach to the Design of End-User Debugging Toolshciresearch
 
The Tester’s Dashboard: Release Decision Support
The Tester’s Dashboard: Release Decision SupportThe Tester’s Dashboard: Release Decision Support
The Tester’s Dashboard: Release Decision SupportBob Binder
 
It Doesn't Do What You Think It Does
It Doesn't Do What You Think It DoesIt Doesn't Do What You Think It Does
It Doesn't Do What You Think It DoesJustin Stoller
 
Real-world Strategies for Debugging Machine Learning Systems
Real-world Strategies for Debugging Machine Learning SystemsReal-world Strategies for Debugging Machine Learning Systems
Real-world Strategies for Debugging Machine Learning SystemsDatabricks
 
The adoption of machine learning techniques for software defect prediction: A...
The adoption of machine learning techniques for software defect prediction: A...The adoption of machine learning techniques for software defect prediction: A...
The adoption of machine learning techniques for software defect prediction: A...RAKESH RANA
 
Bugs as Deviant Behavior: A General Approach to Inferring Errors in Systems Code
Bugs as Deviant Behavior: A General Approach to Inferring Errors in Systems CodeBugs as Deviant Behavior: A General Approach to Inferring Errors in Systems Code
Bugs as Deviant Behavior: A General Approach to Inferring Errors in Systems CodeMiro Cupak
 
Test design techniques
Test design techniquesTest design techniques
Test design techniquesMohamed Elshenawy
 
AI-Driven Software Quality Assurance in the Age of DevOps
AI-Driven Software Quality Assurance in the Age of DevOpsAI-Driven Software Quality Assurance in the Age of DevOps
AI-Driven Software Quality Assurance in the Age of DevOpsChakkrit (Kla) Tantithamthavorn
 
Towards a Better Understanding of the Impact of Experimental Components on De...
Towards a Better Understanding of the Impact of Experimental Components on De...Towards a Better Understanding of the Impact of Experimental Components on De...
Towards a Better Understanding of the Impact of Experimental Components on De...Chakkrit (Kla) Tantithamthavorn
 
Software Analytics In Action: A Hands-on Tutorial on Mining, Analyzing, Model...
Software Analytics In Action: A Hands-on Tutorial on Mining, Analyzing, Model...Software Analytics In Action: A Hands-on Tutorial on Mining, Analyzing, Model...
Software Analytics In Action: A Hands-on Tutorial on Mining, Analyzing, Model...Chakkrit (Kla) Tantithamthavorn
 
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...Abdel Salam Sayyad
 
A Review on Parameter Estimation Techniques of Software Reliability Growth Mo...
A Review on Parameter Estimation Techniques of Software Reliability Growth Mo...A Review on Parameter Estimation Techniques of Software Reliability Growth Mo...
A Review on Parameter Estimation Techniques of Software Reliability Growth Mo...Editor IJCATR
 
MSR End of Internship Talk
MSR End of Internship TalkMSR End of Internship Talk
MSR End of Internship TalkRay Buse
 
Odin2018_Minh_ML_Risk_Prediction
Odin2018_Minh_ML_Risk_PredictionOdin2018_Minh_ML_Risk_Prediction
Odin2018_Minh_ML_Risk_PredictionMinh Nguyen
 
Automated parameter optimization should be included in future 
defect predict...
Automated parameter optimization should be included in future 
defect predict...Automated parameter optimization should be included in future 
defect predict...
Automated parameter optimization should be included in future 
defect predict...Chakkrit (Kla) Tantithamthavorn
 
Thesis Presentation
Thesis PresentationThesis Presentation
Thesis Presentationnirvdrum
 
SRECon-Europe-2017: Reducing MTTR and False Escalations: Event Correlation at...
SRECon-Europe-2017: Reducing MTTR and False Escalations: Event Correlation at...SRECon-Europe-2017: Reducing MTTR and False Escalations: Event Correlation at...
SRECon-Europe-2017: Reducing MTTR and False Escalations: Event Correlation at...Michael Kehoe
 

Weitere ähnliche Inhalte

Was ist angesagt?

Instance Space Analysis for Search Based Software Engineering
Instance Space Analysis for Search Based Software EngineeringInstance Space Analysis for Search Based Software Engineering
Instance Space Analysis for Search Based Software EngineeringAldeida Aleti
 
Cukic Promise08 V3
Cukic Promise08 V3Cukic Promise08 V3
Cukic Promise08 V3gregoryg
 
Generation of Search Based Test Data on Acceptability Testing Principle
Generation of Search Based Test Data on Acceptability Testing PrincipleGeneration of Search Based Test Data on Acceptability Testing Principle
Generation of Search Based Test Data on Acceptability Testing Principleiosrjce
 
[Tho Quan] Fault Localization - Where is the root cause of a bug?
[Tho Quan] Fault Localization - Where is the root cause of a bug?[Tho Quan] Fault Localization - Where is the root cause of a bug?
[Tho Quan] Fault Localization - Where is the root cause of a bug?Ho Chi Minh City Software Testing Club
 
A software fault localization technique based on program mutations
A software fault localization technique based on program mutationsA software fault localization technique based on program mutations
A software fault localization technique based on program mutationsTao He
 
StratCel: A Strategy-Centric Approach to the Design of End-User Debugging Tools
StratCel: A Strategy-Centric Approach to the Design of End-User Debugging ToolsStratCel: A Strategy-Centric Approach to the Design of End-User Debugging Tools
StratCel: A Strategy-Centric Approach to the Design of End-User Debugging Toolshciresearch
 
The Tester’s Dashboard: Release Decision Support
The Tester’s Dashboard: Release Decision SupportThe Tester’s Dashboard: Release Decision Support
The Tester’s Dashboard: Release Decision SupportBob Binder
 
It Doesn't Do What You Think It Does
It Doesn't Do What You Think It DoesIt Doesn't Do What You Think It Does
It Doesn't Do What You Think It DoesJustin Stoller
 
Real-world Strategies for Debugging Machine Learning Systems
Real-world Strategies for Debugging Machine Learning SystemsReal-world Strategies for Debugging Machine Learning Systems
Real-world Strategies for Debugging Machine Learning SystemsDatabricks
 
The adoption of machine learning techniques for software defect prediction: A...
The adoption of machine learning techniques for software defect prediction: A...The adoption of machine learning techniques for software defect prediction: A...
The adoption of machine learning techniques for software defect prediction: A...RAKESH RANA
 
Bugs as Deviant Behavior: A General Approach to Inferring Errors in Systems Code
Bugs as Deviant Behavior: A General Approach to Inferring Errors in Systems CodeBugs as Deviant Behavior: A General Approach to Inferring Errors in Systems Code
Bugs as Deviant Behavior: A General Approach to Inferring Errors in Systems CodeMiro Cupak
 
Test design techniques
Test design techniquesTest design techniques
Test design techniquesMohamed Elshenawy
 
AI-Driven Software Quality Assurance in the Age of DevOps
AI-Driven Software Quality Assurance in the Age of DevOpsAI-Driven Software Quality Assurance in the Age of DevOps
AI-Driven Software Quality Assurance in the Age of DevOpsChakkrit (Kla) Tantithamthavorn
 
Towards a Better Understanding of the Impact of Experimental Components on De...
Towards a Better Understanding of the Impact of Experimental Components on De...Towards a Better Understanding of the Impact of Experimental Components on De...
Towards a Better Understanding of the Impact of Experimental Components on De...Chakkrit (Kla) Tantithamthavorn
 
Software Analytics In Action: A Hands-on Tutorial on Mining, Analyzing, Model...
Software Analytics In Action: A Hands-on Tutorial on Mining, Analyzing, Model...Software Analytics In Action: A Hands-on Tutorial on Mining, Analyzing, Model...
Software Analytics In Action: A Hands-on Tutorial on Mining, Analyzing, Model...Chakkrit (Kla) Tantithamthavorn
 
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...Abdel Salam Sayyad
 
A Review on Parameter Estimation Techniques of Software Reliability Growth Mo...
A Review on Parameter Estimation Techniques of Software Reliability Growth Mo...A Review on Parameter Estimation Techniques of Software Reliability Growth Mo...
A Review on Parameter Estimation Techniques of Software Reliability Growth Mo...Editor IJCATR
 
MSR End of Internship Talk
MSR End of Internship TalkMSR End of Internship Talk
MSR End of Internship TalkRay Buse
 
Odin2018_Minh_ML_Risk_Prediction
Odin2018_Minh_ML_Risk_PredictionOdin2018_Minh_ML_Risk_Prediction
Odin2018_Minh_ML_Risk_PredictionMinh Nguyen
 
Automated parameter optimization should be included in future 
defect predict...
Automated parameter optimization should be included in future 
defect predict...Automated parameter optimization should be included in future 
defect predict...
Automated parameter optimization should be included in future 
defect predict...Chakkrit (Kla) Tantithamthavorn
 

Was ist angesagt? (20)

Instance Space Analysis for Search Based Software Engineering
Instance Space Analysis for Search Based Software EngineeringInstance Space Analysis for Search Based Software Engineering
Instance Space Analysis for Search Based Software Engineering
 
Cukic Promise08 V3
Cukic Promise08 V3Cukic Promise08 V3
Cukic Promise08 V3
 
Generation of Search Based Test Data on Acceptability Testing Principle
Generation of Search Based Test Data on Acceptability Testing PrincipleGeneration of Search Based Test Data on Acceptability Testing Principle
Generation of Search Based Test Data on Acceptability Testing Principle
 
[Tho Quan] Fault Localization - Where is the root cause of a bug?
[Tho Quan] Fault Localization - Where is the root cause of a bug?[Tho Quan] Fault Localization - Where is the root cause of a bug?
[Tho Quan] Fault Localization - Where is the root cause of a bug?
 
A software fault localization technique based on program mutations
A software fault localization technique based on program mutationsA software fault localization technique based on program mutations
A software fault localization technique based on program mutations
 
StratCel: A Strategy-Centric Approach to the Design of End-User Debugging Tools
StratCel: A Strategy-Centric Approach to the Design of End-User Debugging ToolsStratCel: A Strategy-Centric Approach to the Design of End-User Debugging Tools
StratCel: A Strategy-Centric Approach to the Design of End-User Debugging Tools
 
The Tester’s Dashboard: Release Decision Support
The Tester’s Dashboard: Release Decision SupportThe Tester’s Dashboard: Release Decision Support
The Tester’s Dashboard: Release Decision Support
 
It Doesn't Do What You Think It Does
It Doesn't Do What You Think It DoesIt Doesn't Do What You Think It Does
It Doesn't Do What You Think It Does
 
Real-world Strategies for Debugging Machine Learning Systems
Real-world Strategies for Debugging Machine Learning SystemsReal-world Strategies for Debugging Machine Learning Systems
Real-world Strategies for Debugging Machine Learning Systems
 
The adoption of machine learning techniques for software defect prediction: A...
The adoption of machine learning techniques for software defect prediction: A...The adoption of machine learning techniques for software defect prediction: A...
The adoption of machine learning techniques for software defect prediction: A...
 
Bugs as Deviant Behavior: A General Approach to Inferring Errors in Systems Code
Bugs as Deviant Behavior: A General Approach to Inferring Errors in Systems CodeBugs as Deviant Behavior: A General Approach to Inferring Errors in Systems Code
Bugs as Deviant Behavior: A General Approach to Inferring Errors in Systems Code
 
Test design techniques
Test design techniquesTest design techniques
Test design techniques
 
AI-Driven Software Quality Assurance in the Age of DevOps
AI-Driven Software Quality Assurance in the Age of DevOpsAI-Driven Software Quality Assurance in the Age of DevOps
AI-Driven Software Quality Assurance in the Age of DevOps
 
Towards a Better Understanding of the Impact of Experimental Components on De...
Towards a Better Understanding of the Impact of Experimental Components on De...Towards a Better Understanding of the Impact of Experimental Components on De...
Towards a Better Understanding of the Impact of Experimental Components on De...
 
Software Analytics In Action: A Hands-on Tutorial on Mining, Analyzing, Model...
Software Analytics In Action: A Hands-on Tutorial on Mining, Analyzing, Model...Software Analytics In Action: A Hands-on Tutorial on Mining, Analyzing, Model...
Software Analytics In Action: A Hands-on Tutorial on Mining, Analyzing, Model...
 
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...
 
A Review on Parameter Estimation Techniques of Software Reliability Growth Mo...
A Review on Parameter Estimation Techniques of Software Reliability Growth Mo...A Review on Parameter Estimation Techniques of Software Reliability Growth Mo...
A Review on Parameter Estimation Techniques of Software Reliability Growth Mo...
 
MSR End of Internship Talk
MSR End of Internship TalkMSR End of Internship Talk
MSR End of Internship Talk
 
Odin2018_Minh_ML_Risk_Prediction
Odin2018_Minh_ML_Risk_PredictionOdin2018_Minh_ML_Risk_Prediction
Odin2018_Minh_ML_Risk_Prediction
 
Automated parameter optimization should be included in future 
defect predict...
Automated parameter optimization should be included in future 
defect predict...Automated parameter optimization should be included in future 
defect predict...
Automated parameter optimization should be included in future 
defect predict...
 

Ă„hnlich wie Make the Most of Your Time: How Should the Analyst Work with Automated Traceability Tools

Thesis Presentation
Thesis PresentationThesis Presentation
Thesis Presentationnirvdrum
 
SRECon-Europe-2017: Reducing MTTR and False Escalations: Event Correlation at...
SRECon-Europe-2017: Reducing MTTR and False Escalations: Event Correlation at...SRECon-Europe-2017: Reducing MTTR and False Escalations: Event Correlation at...
SRECon-Europe-2017: Reducing MTTR and False Escalations: Event Correlation at...Michael Kehoe
 
How to improve your system monitoring
How to improve your system monitoringHow to improve your system monitoring
How to improve your system monitoringAndrew White
 
Search Quality Evaluation to Help Reproducibility : an Open Source Approach
Search Quality Evaluation to Help Reproducibility : an Open Source ApproachSearch Quality Evaluation to Help Reproducibility : an Open Source Approach
Search Quality Evaluation to Help Reproducibility : an Open Source ApproachAlessandro Benedetti
 
Search Quality Evaluation to Help Reproducibility: An Open-source Approach
Search Quality Evaluation to Help Reproducibility: An Open-source ApproachSearch Quality Evaluation to Help Reproducibility: An Open-source Approach
Search Quality Evaluation to Help Reproducibility: An Open-source ApproachAlessandro Benedetti
 
IRJET-Fake Product Review Monitoring
IRJET-Fake Product Review MonitoringIRJET-Fake Product Review Monitoring
IRJET-Fake Product Review MonitoringIRJET Journal
 
Multi-method Evaluation in Scientific Paper Recommender Systems
Multi-method Evaluation in Scientific Paper Recommender SystemsMulti-method Evaluation in Scientific Paper Recommender Systems
Multi-method Evaluation in Scientific Paper Recommender SystemsAravind Sesagiri Raamkumar
 
Optimising Clinical Trials Monitoring Data review - Neill Barron
Optimising Clinical Trials Monitoring Data review - Neill BarronOptimising Clinical Trials Monitoring Data review - Neill Barron
Optimising Clinical Trials Monitoring Data review - Neill BarronNeill Barron
 
Data analysis
Data analysisData analysis
Data analysisNursing Path
 
Presentation in ACL Connections in Atlanta - April 2013
Presentation in ACL Connections in Atlanta - April 2013Presentation in ACL Connections in Atlanta - April 2013
Presentation in ACL Connections in Atlanta - April 2013mcoello
 
Cser13.ppt
Cser13.pptCser13.ppt
Cser13.pptPtidej Team
 
requirement analysis characteristics
requirement analysis characteristics requirement analysis characteristics
requirement analysis characteristics Helmy Faisal
 
1440 track 2 boire_using our laptop
1440 track 2 boire_using our laptop1440 track 2 boire_using our laptop
1440 track 2 boire_using our laptopRising Media, Inc.
 
information technology materrailas paper
information technology materrailas paperinformation technology materrailas paper
information technology materrailas papermelkamutesfay1
 
AIRG Presentation
AIRG PresentationAIRG Presentation
AIRG Presentationnirvdrum
 

Ă„hnlich wie Make the Most of Your Time: How Should the Analyst Work with Automated Traceability Tools (20)

Thesis Presentation
Thesis PresentationThesis Presentation
Thesis Presentation
 
SRECon-Europe-2017: Reducing MTTR and False Escalations: Event Correlation at...
SRECon-Europe-2017: Reducing MTTR and False Escalations: Event Correlation at...SRECon-Europe-2017: Reducing MTTR and False Escalations: Event Correlation at...
SRECon-Europe-2017: Reducing MTTR and False Escalations: Event Correlation at...
 
How to improve your system monitoring
How to improve your system monitoringHow to improve your system monitoring
How to improve your system monitoring
 
Search Quality Evaluation to Help Reproducibility : an Open Source Approach
Search Quality Evaluation to Help Reproducibility : an Open Source ApproachSearch Quality Evaluation to Help Reproducibility : an Open Source Approach
Search Quality Evaluation to Help Reproducibility : an Open Source Approach
 
Search Quality Evaluation to Help Reproducibility: An Open-source Approach
Search Quality Evaluation to Help Reproducibility: An Open-source ApproachSearch Quality Evaluation to Help Reproducibility: An Open-source Approach
Search Quality Evaluation to Help Reproducibility: An Open-source Approach
 
IRJET-Fake Product Review Monitoring
IRJET-Fake Product Review MonitoringIRJET-Fake Product Review Monitoring
IRJET-Fake Product Review Monitoring
 
Multi-method Evaluation in Scientific Paper Recommender Systems
Multi-method Evaluation in Scientific Paper Recommender SystemsMulti-method Evaluation in Scientific Paper Recommender Systems
Multi-method Evaluation in Scientific Paper Recommender Systems
 
Optimising Clinical Trials Monitoring Data review - Neill Barron
Optimising Clinical Trials Monitoring Data review - Neill BarronOptimising Clinical Trials Monitoring Data review - Neill Barron
Optimising Clinical Trials Monitoring Data review - Neill Barron
 
Thesis
ThesisThesis
Thesis
 
Data analysis
Data analysisData analysis
Data analysis
 
Presentation in ACL Connections in Atlanta - April 2013
Presentation in ACL Connections in Atlanta - April 2013Presentation in ACL Connections in Atlanta - April 2013
Presentation in ACL Connections in Atlanta - April 2013
 
Quality key users
Quality key usersQuality key users
Quality key users
 
D017642026
D017642026D017642026
D017642026
 
Cser13.ppt
Cser13.pptCser13.ppt
Cser13.ppt
 
requirement analysis characteristics
requirement analysis characteristics requirement analysis characteristics
requirement analysis characteristics
 
-linkedin
-linkedin-linkedin
-linkedin
 
1440 track 2 boire_using our laptop
1440 track 2 boire_using our laptop1440 track 2 boire_using our laptop
1440 track 2 boire_using our laptop
 
Text Analytics for Legal work
Text Analytics for Legal workText Analytics for Legal work
Text Analytics for Legal work
 
information technology materrailas paper
information technology materrailas paperinformation technology materrailas paper
information technology materrailas paper
 
AIRG Presentation
AIRG PresentationAIRG Presentation
AIRG Presentation
 

KĂĽrzlich hochgeladen

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 3652toLead Limited
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdfhans926745
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersThousandEyes
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonetsnaman860154
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Paola De la Torre
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesSinan KOZAK
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
WhatsApp 9892124323 âś“Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 âś“Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 âś“Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 âś“Call Girls In Kalyan ( Mumbai ) secure servicePooja Nehwal
 
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | DelhiFULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhisoniya singh
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking MenDelhi Call girls
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...HostedbyConfluent
 
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...gurkirankumar98700
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slidevu2urc
 
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Alan Dix
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia
 

KĂĽrzlich hochgeladen (20)

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen Frames
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
WhatsApp 9892124323 âś“Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 âś“Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 âś“Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 âś“Call Girls In Kalyan ( Mumbai ) secure service
 
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | DelhiFULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
 
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 

Make the Most of Your Time: How Should the Analyst Work with Automated Traceability Tools

  • 1. Make the Most of Your Time: How Should the Analyst Work with Automated Traceability Tools?* Alex Dekhtyar Jane Huffman Hayes Jody Larsen *Funded by NASA
  • 2.
  • 3.
  • 4.
  • 5. Design Document Information Retrieval and Traceability Representation Requirements Document Matching algorithm Analyst
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11. Measures N - number of low-level requirements; M - number of high-level requirements; Hits - number of confirmed candidate links Strikes - number of false positives Misses - number of missed links Precision = Hits Hits + Strikes Recall = Hits Hits +Misses Selectivity = Hits + Strikes M * N
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.
  • 22. Study 1 Results, Fixed Recall Observed Links Precision Selectivity Local, Feedback 1595 20% 3% Local, No Feedback 1713 19% 3% Global, Feedback 5399 6% 10% Global, No Feedback 6149 5% 12%
  • 23. Study 2 Results, Fixed Effort Confirmed Links Precision Recall Local, Feedback 321 20% 89% Local, No Feedback 326 20% 87.5% Global, Feedback 236 15% 65% Global, No Feedback 227 14% 63% Local, Random 65 4% 18% Global, Random 58 3.6% 16%
  • 25.
  • 26.