SlideShare ist ein Scribd-Unternehmen logo
1 von 36
Downloaden Sie, um offline zu lesen
Combinatorial
Interaction Testing (CIT)
An introduction
Xavier Devroey - x.d.m.devroey@tudelft.nl
public class Message {

    private long id = -1;
    private String author;
    private String message;

    public Message(String author, String message) {
        this.author = author;
        this.message = message;
    }
    // Gets and Sets ...
    /**
     * Appends the given text at the end of this' message.
     * A space is added before the given text.
     *
     * @param text The test to append.
     */
    public void compose(String text) {
        this.message = message + " " + text;
    }
}
@Test
    public void testComposeString() {
        Message msg = new Message("Me", "despicable");
        msg.compose("me");
    }
Code coverage
Line coverage: percentage of lines covered by tests
https://github.com/cobertura/cobertura
Achieving line coverage 

= 

executing each line at least once
@Test
    public void testComposeString() {
        Message msg = new Message("Me", "despicable");
        msg.compose("me");
    }
Is the compose method covered by the test case?
@Test
    public void testComposeString() {
        Message msg = new Message("Me", "despicable");
        msg.compose("me");
    }
Is the compose method covered by the test case?
Yes
@Test
    public void testComposeString() {
        Message msg = new Message("Me", "despicable");
        msg.compose("me");
    }
Is the compose method covered by the test case?
Is it enough?
Yes
@Test
    public void testComposeString() {
        Message msg = new Message("Me", "despicable");
        msg.compose("me");
    }
Is the compose method covered by the test case?
Is it enough?
Yes
No
@Test
    public void testComposeString() {
        Message msg = new Message("Me", "despicable");
        msg.compose("me");
    }
Is the compose method covered by the test case?
Is it enough?
Yes
No
There are no assertions on the result/effect of the method
Input/output domain coverage
public void compose(String text);
(based on pre/post conditions)
Input/output domain coverage
public void compose(String text);
• Equivalence partitioning
(based on pre/post conditions)
Input/output domain coverage
public void compose(String text);
• Equivalence partitioning
Normal case: "Despicable you"
(based on pre/post conditions)
Input/output domain coverage
public void compose(String text);
• Equivalence partitioning
Normal case: "Despicable you"
Limit cases: "" null "azrtyuiop^$ù m;21@98#3!"
(based on pre/post conditions)
Input/output domain coverage
public void compose(String text);
public void compose(String t1, String t2);
• Equivalence partitioning
Normal case: "Despicable you"
Limit cases: "" null "azrtyuiop^$ù m;21@98#3!"
(based on pre/post conditions)
Input/output domain coverage
public void compose(String text);
public void compose(String t1, String t2);
• Equivalence partitioning
Normal case: "Despicable you"
Limit cases: "" null "azrtyuiop^$ù m;21@98#3!"
• t1
• t2
Normal case: "Despicable you"
Limit cases: "" null "azrtyuiop^$ù m;21@98#3!"
Normal case: "Despicable you"
Limit cases: "" null "azrtyuiop^$ù m;21@98#3!"
(based on pre/post conditions)
Input/output domain coverage
public void compose(String text);
public void compose(String t1, String t2);
• Equivalence partitioning
Normal case: "Despicable you"
Limit cases: "" null "azrtyuiop^$ù m;21@98#3!"
• t1
• t2
Normal case: "Despicable you"
Limit cases: "" null "azrtyuiop^$ù m;21@98#3!"
Normal case: "Despicable you"
Limit cases: "" null "azrtyuiop^$ù m;21@98#3!"
(based on pre/post conditions)
public void compose(String t1, String t2, String t3);
Input/output domain coverage
public void compose(String text);
public void compose(String t1, String t2);
• Equivalence partitioning
Normal case: "Despicable you"
Limit cases: "" null "azrtyuiop^$ù m;21@98#3!"
• t1
• t2
Normal case: "Despicable you"
Limit cases: "" null "azrtyuiop^$ù m;21@98#3!"
Normal case: "Despicable you"
Limit cases: "" null "azrtyuiop^$ù m;21@98#3!"
☛ Combinatorial explosion with the number of parameters
(based on pre/post conditions)
public void compose(String t1, String t2, String t3);
Combinatorial Interaction Testing (CIT)
Hyp.: Most of the faults are due to undesired
interactions of a limited number (N) of factors
(here, equivalent classes of parameter values).
D. Kuhn et al, Software fault interactions and implications for software testing. TSE 2004
• In practice: N ≤ 4
• Usually: N = 2 (pairwise)
Combinatorial Interaction Testing (CIT)
Hyp.: Most of the faults are due to undesired
interactions of a limited number (N) of factors
(here, equivalent classes of parameter values).
D. Kuhn et al, Software fault interactions and implications for software testing. TSE 2004
• In practice: N ≤ 4
• Usually: N = 2 (pairwise)
Idea: Generate test cases such that all
combinations of N values are covered at least
once.
Combinatorial Interaction Testing (CIT)
t1 t2 t3
"Despicable you" "Despicable you" "Despicable you"
"" "" ""
null null null
"azrtyuiop^$ù" "azrtyuiop^$ù" "azrtyuiop^$ù"
Hyp.: most faults are caused by interactions of at most two factors
Pairwise testing: cover all pairs of values
Combinatorial Interaction Testing (CIT)
t1 t2 t3
"Despicable you" "Despicable you" "Despicable you"
"" "" ""
null null null
"azrtyuiop^$ù" "azrtyuiop^$ù" "azrtyuiop^$ù"
Hyp.: most faults are caused by interactions of at most two factors
Pairwise testing: cover all pairs of values
"Despicable you" "Despicable you" "Despicable you"
"Despicable you" "" ""
"Despicable you" null null
"Despicable you" "azrtyuiop^$ù" "azrtyuiop^$ù"
"" "Despicable you" ""
"" "" "Despicable you"
"" null "azrtyuiop^$ù"
"" "azrtyuiop^$ù" null
null "Despicable you" null
null "" "azrtyuiop^$ù"
null null "Despicable you"
null "azrtyuiop^$ù" ""
"azrtyuiop^$ù" "Despicable you" "azrtyuiop^$ù"
"azrtyuiop^$ù" "" null
"azrtyuiop^$ù" null ""
"azrtyuiop^$ù" "azrtyuiop^$ù" "Despicable you"
16 test cases
instead of 64
Combinatorial Interaction Testing (CIT)
t1 t2 t3
"Despicable you" "Despicable you" "Despicable you"
"" "" ""
null null null
"azrtyuiop^$ù" "azrtyuiop^$ù" "azrtyuiop^$ù"
Hyp.: most faults are caused by interactions of at most two factors
Pairwise testing: cover all pairs of values
"Despicable you" "Despicable you" "Despicable you"
"Despicable you" "" ""
"Despicable you" null null
"Despicable you" "azrtyuiop^$ù" "azrtyuiop^$ù"
"" "Despicable you" ""
"" "" "Despicable you"
"" null "azrtyuiop^$ù"
"" "azrtyuiop^$ù" null
null "Despicable you" null
null "" "azrtyuiop^$ù"
null null "Despicable you"
null "azrtyuiop^$ù" ""
"azrtyuiop^$ù" "Despicable you" "azrtyuiop^$ù"
"azrtyuiop^$ù" "" null
"azrtyuiop^$ù" null ""
"azrtyuiop^$ù" "azrtyuiop^$ù" "Despicable you"
16 test cases
instead of 64
Combinatorial Interaction Testing (CIT)
t1 t2 t3
"Despicable you" "Despicable you" "Despicable you"
"" "" ""
null null null
"azrtyuiop^$ù" "azrtyuiop^$ù" "azrtyuiop^$ù"
Hyp.: most faults are caused by interactions of at most two factors
Pairwise testing: cover all pairs of values
"Despicable you" "Despicable you" "Despicable you"
"Despicable you" "" ""
"Despicable you" null null
"Despicable you" "azrtyuiop^$ù" "azrtyuiop^$ù"
"" "Despicable you" ""
"" "" "Despicable you"
"" null "azrtyuiop^$ù"
"" "azrtyuiop^$ù" null
null "Despicable you" null
null "" "azrtyuiop^$ù"
null null "Despicable you"
null "azrtyuiop^$ù" ""
"azrtyuiop^$ù" "Despicable you" "azrtyuiop^$ù"
"azrtyuiop^$ù" "" null
"azrtyuiop^$ù" null ""
"azrtyuiop^$ù" "azrtyuiop^$ù" "Despicable you"
16 test cases
instead of 64
Combinatorial Interaction Testing (CIT)
t1 t2 t3
"Despicable you" "Despicable you" "Despicable you"
"" "" ""
null null null
"azrtyuiop^$ù" "azrtyuiop^$ù" "azrtyuiop^$ù"
Hyp.: most faults are caused by interactions of at most two factors
Pairwise testing: cover all pairs of values
"Despicable you" "Despicable you" "Despicable you"
"Despicable you" "" ""
"Despicable you" null null
"Despicable you" "azrtyuiop^$ù" "azrtyuiop^$ù"
"" "Despicable you" ""
"" "" "Despicable you"
"" null "azrtyuiop^$ù"
"" "azrtyuiop^$ù" null
null "Despicable you" null
null "" "azrtyuiop^$ù"
null null "Despicable you"
null "azrtyuiop^$ù" ""
"azrtyuiop^$ù" "Despicable you" "azrtyuiop^$ù"
"azrtyuiop^$ù" "" null
"azrtyuiop^$ù" null ""
"azrtyuiop^$ù" "azrtyuiop^$ù" "Despicable you"
16 test cases
instead of 64
Combinatorial Interaction Testing (CIT)
t1 t2 t3
"Despicable you" "Despicable you" "Despicable you"
"" "" ""
null null null
"azrtyuiop^$ù" "azrtyuiop^$ù" "azrtyuiop^$ù"
Hyp.: most faults are caused by interactions of at most two factors
Pairwise testing: cover all pairs of values
"Despicable you" "Despicable you" "Despicable you"
"Despicable you" "" ""
"Despicable you" null null
"Despicable you" "azrtyuiop^$ù" "azrtyuiop^$ù"
"" "Despicable you" ""
"" "" "Despicable you"
"" null "azrtyuiop^$ù"
"" "azrtyuiop^$ù" null
null "Despicable you" null
null "" "azrtyuiop^$ù"
null null "Despicable you"
null "azrtyuiop^$ù" ""
"azrtyuiop^$ù" "Despicable you" "azrtyuiop^$ù"
"azrtyuiop^$ù" "" null
"azrtyuiop^$ù" null ""
"azrtyuiop^$ù" "azrtyuiop^$ù" "Despicable you"
16 test cases
instead of 64
Combinatorial Interaction Testing (CIT)
t1 t2 t3
"Despicable you" "Despicable you" "Despicable you"
"" "" ""
null null null
"azrtyuiop^$ù" "azrtyuiop^$ù" "azrtyuiop^$ù"
Hyp.: most faults are caused by interactions of at most two factors
Pairwise testing: cover all pairs of values
"Despicable you" "Despicable you" "Despicable you"
"Despicable you" "" ""
"Despicable you" null null
"Despicable you" "azrtyuiop^$ù" "azrtyuiop^$ù"
"" "Despicable you" ""
"" "" "Despicable you"
"" null "azrtyuiop^$ù"
"" "azrtyuiop^$ù" null
null "Despicable you" null
null "" "azrtyuiop^$ù"
null null "Despicable you"
null "azrtyuiop^$ù" ""
"azrtyuiop^$ù" "Despicable you" "azrtyuiop^$ù"
"azrtyuiop^$ù" "" null
"azrtyuiop^$ù" null ""
"azrtyuiop^$ù" "azrtyuiop^$ù" "Despicable you"
16 test cases
instead of 64
Combinatorial Interaction Testing (CIT)
t1 t2 t3
"Despicable you" "Despicable you" "Despicable you"
"" "" ""
null null null
"azrtyuiop^$ù" "azrtyuiop^$ù" "azrtyuiop^$ù"
Hyp.: most faults are caused by interactions of at most two factors
Pairwise testing: cover all pairs of values
"Despicable you" "Despicable you" "Despicable you"
"Despicable you" "" ""
"Despicable you" null null
"Despicable you" "azrtyuiop^$ù" "azrtyuiop^$ù"
"" "Despicable you" ""
"" "" "Despicable you"
"" null "azrtyuiop^$ù"
"" "azrtyuiop^$ù" null
null "Despicable you" null
null "" "azrtyuiop^$ù"
null null "Despicable you"
null "azrtyuiop^$ù" ""
"azrtyuiop^$ù" "Despicable you" "azrtyuiop^$ù"
"azrtyuiop^$ù" "" null
"azrtyuiop^$ù" null ""
"azrtyuiop^$ù" "azrtyuiop^$ù" "Despicable you"
16 test cases
instead of 64
http://cse.unl.edu/~citportal/
http://ctweb.abstracta.com.uy/combinatorial.jsp
Hyp.: most faults are caused by interactions of at most t factors
T-wise testing: cover all t-uples of values
http://cse.unl.edu/~citportal/
http://ctweb.abstracta.com.uy/combinatorial.jsp
Hyp.: most faults are caused by interactions of at most t factors
T-wise testing: cover all t-uples of values
Integration and System testing
app.
http://cse.unl.edu/~citportal/
http://ctweb.abstracta.com.uy/combinatorial.jsp
Hyp.: most faults are caused by interactions of at most t factors
T-wise testing: cover all t-uples of values
Integration and System testing
app.
http://cse.unl.edu/~citportal/
http://ctweb.abstracta.com.uy/combinatorial.jsp
Hyp.: most faults are caused by interactions of at most t factors
T-wise testing: cover all t-uples of values
Integration and System testing
app.
http://cse.unl.edu/~citportal/
http://ctweb.abstracta.com.uy/combinatorial.jsp
Hyp.: most faults are caused by interactions of at most t factors
T-wise testing: cover all t-uples of values
Integration and System testing
Which
configuration
should I use
for my tests?
type
Android
version
Screen Processor
tablet 4.0.4 XLarge ARM Cortex
smartphone 4.1 Large Intel x86
phablet 4.1.1 Normal
4.1.2 Small
type
Android
version
Screen Processor
tablet 4.0.4 XLarge ARM Cortex
smartphone 4.1 Large Intel x86
phablet 4.1.1 Normal
4.1.2 Small
… or using a feature model
http://familiar-project.github.io
https://featureide.github.io
http://www.skalup.com/
http://research.henard.net/SPL/PLEDGE/
http://martinfjohansen.com/models2011/spltool/
http://cse.unl.edu/~citportal/
app.
Hyp.: most faults are caused by interactions of at most t factors
T-wise testing: cover all t-uples of values
Integration and System testing
Which
configuration
should I use
for my tests?
Under …
• tests requirements
• $$$ budget
• test execution time
budget
… constraints
https://console.firebase.google.com

Weitere ähnliche Inhalte

Ähnlich wie Combinatorial Interaction Testing, an Introduction - 2018

Дмитрий Селиванов, OK.RU. Finding Similar Items in high-dimensional spaces: L...
Дмитрий Селиванов, OK.RU. Finding Similar Items in high-dimensional spaces: L...Дмитрий Селиванов, OK.RU. Finding Similar Items in high-dimensional spaces: L...
Дмитрий Селиванов, OK.RU. Finding Similar Items in high-dimensional spaces: L...Mail.ru Group
 
Test antipatterns
Test antipatternsTest antipatterns
Test antipatternsJiří Kiml
 
property-based testing (FrOsCon 9, 2014, August 23)
property-based testing (FrOsCon 9, 2014, August 23)property-based testing (FrOsCon 9, 2014, August 23)
property-based testing (FrOsCon 9, 2014, August 23)Christoph Neuroth
 
Graph Methods for Generating Test Cases with Universal and Existential Constr...
Graph Methods for Generating Test Cases with Universal and Existential Constr...Graph Methods for Generating Test Cases with Universal and Existential Constr...
Graph Methods for Generating Test Cases with Universal and Existential Constr...Sylvain Hallé
 
Rotten Green Tests
Rotten Green TestsRotten Green Tests
Rotten Green TestsESUG
 
1.Buffer Overflows
1.Buffer Overflows1.Buffer Overflows
1.Buffer Overflowsphanleson
 
Testing Code and Assuring Quality
Testing Code and Assuring QualityTesting Code and Assuring Quality
Testing Code and Assuring QualityKent Cowgill
 
Arrays string handling java packages
Arrays string handling java packagesArrays string handling java packages
Arrays string handling java packagesSardar Alam
 
Perl training-in-navi mumbai
Perl training-in-navi mumbaiPerl training-in-navi mumbai
Perl training-in-navi mumbaivibrantuser
 
The Art of Identifying Vulnerabilities - CascadiaFest 2015
The Art of Identifying Vulnerabilities  - CascadiaFest 2015The Art of Identifying Vulnerabilities  - CascadiaFest 2015
The Art of Identifying Vulnerabilities - CascadiaFest 2015Adam Baldwin
 
Seven Ineffective Coding Habits of Many Programmers
Seven Ineffective Coding Habits of Many ProgrammersSeven Ineffective Coding Habits of Many Programmers
Seven Ineffective Coding Habits of Many ProgrammersKevlin Henney
 
From OCaml To Javascript At Skydeck
From OCaml To Javascript At SkydeckFrom OCaml To Javascript At Skydeck
From OCaml To Javascript At SkydeckJake Donham
 
Java Generics for Dummies
Java Generics for DummiesJava Generics for Dummies
Java Generics for Dummiesknutmork
 
The Power of Probabilistic Thinking (keynote talk at ASE 2016)
The Power of Probabilistic Thinking (keynote talk at ASE 2016)The Power of Probabilistic Thinking (keynote talk at ASE 2016)
The Power of Probabilistic Thinking (keynote talk at ASE 2016)David Rosenblum
 
Final project powerpoint template (fndprg) (1)
Final project powerpoint template (fndprg) (1)Final project powerpoint template (fndprg) (1)
Final project powerpoint template (fndprg) (1)heoff
 
Seven Ineffective Coding Habits of Many Programmers
Seven Ineffective Coding Habits of Many ProgrammersSeven Ineffective Coding Habits of Many Programmers
Seven Ineffective Coding Habits of Many ProgrammersKevlin Henney
 

Ähnlich wie Combinatorial Interaction Testing, an Introduction - 2018 (20)

Дмитрий Селиванов, OK.RU. Finding Similar Items in high-dimensional spaces: L...
Дмитрий Селиванов, OK.RU. Finding Similar Items in high-dimensional spaces: L...Дмитрий Селиванов, OK.RU. Finding Similar Items in high-dimensional spaces: L...
Дмитрий Селиванов, OK.RU. Finding Similar Items in high-dimensional spaces: L...
 
Stop that!
Stop that!Stop that!
Stop that!
 
Test antipatterns
Test antipatternsTest antipatterns
Test antipatterns
 
property-based testing (FrOsCon 9, 2014, August 23)
property-based testing (FrOsCon 9, 2014, August 23)property-based testing (FrOsCon 9, 2014, August 23)
property-based testing (FrOsCon 9, 2014, August 23)
 
Graph Methods for Generating Test Cases with Universal and Existential Constr...
Graph Methods for Generating Test Cases with Universal and Existential Constr...Graph Methods for Generating Test Cases with Universal and Existential Constr...
Graph Methods for Generating Test Cases with Universal and Existential Constr...
 
Pythonintro
PythonintroPythonintro
Pythonintro
 
Rotten Green Tests
Rotten Green TestsRotten Green Tests
Rotten Green Tests
 
1.Buffer Overflows
1.Buffer Overflows1.Buffer Overflows
1.Buffer Overflows
 
The Rule of Three
The Rule of ThreeThe Rule of Three
The Rule of Three
 
Testing Code and Assuring Quality
Testing Code and Assuring QualityTesting Code and Assuring Quality
Testing Code and Assuring Quality
 
Arrays string handling java packages
Arrays string handling java packagesArrays string handling java packages
Arrays string handling java packages
 
Perl training-in-navi mumbai
Perl training-in-navi mumbaiPerl training-in-navi mumbai
Perl training-in-navi mumbai
 
The Art of Identifying Vulnerabilities - CascadiaFest 2015
The Art of Identifying Vulnerabilities  - CascadiaFest 2015The Art of Identifying Vulnerabilities  - CascadiaFest 2015
The Art of Identifying Vulnerabilities - CascadiaFest 2015
 
Seven Ineffective Coding Habits of Many Programmers
Seven Ineffective Coding Habits of Many ProgrammersSeven Ineffective Coding Habits of Many Programmers
Seven Ineffective Coding Habits of Many Programmers
 
From OCaml To Javascript At Skydeck
From OCaml To Javascript At SkydeckFrom OCaml To Javascript At Skydeck
From OCaml To Javascript At Skydeck
 
Java Generics for Dummies
Java Generics for DummiesJava Generics for Dummies
Java Generics for Dummies
 
The Power of Probabilistic Thinking (keynote talk at ASE 2016)
The Power of Probabilistic Thinking (keynote talk at ASE 2016)The Power of Probabilistic Thinking (keynote talk at ASE 2016)
The Power of Probabilistic Thinking (keynote talk at ASE 2016)
 
Final project powerpoint template (fndprg) (1)
Final project powerpoint template (fndprg) (1)Final project powerpoint template (fndprg) (1)
Final project powerpoint template (fndprg) (1)
 
Seven Ineffective Coding Habits of Many Programmers
Seven Ineffective Coding Habits of Many ProgrammersSeven Ineffective Coding Habits of Many Programmers
Seven Ineffective Coding Habits of Many Programmers
 
Stats chapter 15
Stats chapter 15Stats chapter 15
Stats chapter 15
 

Mehr von XavierDevroey

Software Variability Management
Software Variability ManagementSoftware Variability Management
Software Variability ManagementXavierDevroey
 
Effective and Efficient API Misuse Detection via Exception Propagation and Se...
Effective and Efficient API Misuse Detection via Exception Propagation and Se...Effective and Efficient API Misuse Detection via Exception Propagation and Se...
Effective and Efficient API Misuse Detection via Exception Propagation and Se...XavierDevroey
 
Software variability management - 2019
Software variability management - 2019Software variability management - 2019
Software variability management - 2019XavierDevroey
 
Testing Variability-Intensive Systems, tutorial SPLC 2017, part Ii
Testing Variability-Intensive Systems, tutorial SPLC 2017, part IiTesting Variability-Intensive Systems, tutorial SPLC 2017, part Ii
Testing Variability-Intensive Systems, tutorial SPLC 2017, part IiXavierDevroey
 
Testing Variability-Intensive Systems, tutorial SPLC 2017, part I
Testing Variability-Intensive Systems, tutorial SPLC 2017, part ITesting Variability-Intensive Systems, tutorial SPLC 2017, part I
Testing Variability-Intensive Systems, tutorial SPLC 2017, part IXavierDevroey
 
Lorentz workshop - 2018
Lorentz workshop - 2018Lorentz workshop - 2018
Lorentz workshop - 2018XavierDevroey
 
Software variability management - 2017
Software variability management - 2017Software variability management - 2017
Software variability management - 2017XavierDevroey
 
Software testing: an introduction - 2017
Software testing: an introduction - 2017Software testing: an introduction - 2017
Software testing: an introduction - 2017XavierDevroey
 
Software testing: an introduction - 2016
Software testing: an introduction - 2016Software testing: an introduction - 2016
Software testing: an introduction - 2016XavierDevroey
 
Software testing: an introduction - 2015
Software testing: an introduction - 2015Software testing: an introduction - 2015
Software testing: an introduction - 2015XavierDevroey
 
Research overview Oct. 2018
Research overview Oct. 2018Research overview Oct. 2018
Research overview Oct. 2018XavierDevroey
 

Mehr von XavierDevroey (11)

Software Variability Management
Software Variability ManagementSoftware Variability Management
Software Variability Management
 
Effective and Efficient API Misuse Detection via Exception Propagation and Se...
Effective and Efficient API Misuse Detection via Exception Propagation and Se...Effective and Efficient API Misuse Detection via Exception Propagation and Se...
Effective and Efficient API Misuse Detection via Exception Propagation and Se...
 
Software variability management - 2019
Software variability management - 2019Software variability management - 2019
Software variability management - 2019
 
Testing Variability-Intensive Systems, tutorial SPLC 2017, part Ii
Testing Variability-Intensive Systems, tutorial SPLC 2017, part IiTesting Variability-Intensive Systems, tutorial SPLC 2017, part Ii
Testing Variability-Intensive Systems, tutorial SPLC 2017, part Ii
 
Testing Variability-Intensive Systems, tutorial SPLC 2017, part I
Testing Variability-Intensive Systems, tutorial SPLC 2017, part ITesting Variability-Intensive Systems, tutorial SPLC 2017, part I
Testing Variability-Intensive Systems, tutorial SPLC 2017, part I
 
Lorentz workshop - 2018
Lorentz workshop - 2018Lorentz workshop - 2018
Lorentz workshop - 2018
 
Software variability management - 2017
Software variability management - 2017Software variability management - 2017
Software variability management - 2017
 
Software testing: an introduction - 2017
Software testing: an introduction - 2017Software testing: an introduction - 2017
Software testing: an introduction - 2017
 
Software testing: an introduction - 2016
Software testing: an introduction - 2016Software testing: an introduction - 2016
Software testing: an introduction - 2016
 
Software testing: an introduction - 2015
Software testing: an introduction - 2015Software testing: an introduction - 2015
Software testing: an introduction - 2015
 
Research overview Oct. 2018
Research overview Oct. 2018Research overview Oct. 2018
Research overview Oct. 2018
 

Kürzlich hochgeladen

online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdfMeon Technology
 
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.Sharon Liu
 
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Jaydeep Chhasatia
 
IA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeIA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeNeo4j
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesShyamsundar Das
 
How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?AmeliaSmith90
 
JS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIJS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIIvo Andreev
 
Streamlining Your Application Builds with Cloud Native Buildpacks
Streamlining Your Application Builds  with Cloud Native BuildpacksStreamlining Your Application Builds  with Cloud Native Buildpacks
Streamlining Your Application Builds with Cloud Native BuildpacksVish Abrams
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampVICTOR MAESTRE RAMIREZ
 
Fields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxFields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxJoão Esperancinha
 
Generative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilGenerative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilVICTOR MAESTRE RAMIREZ
 
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfTobias Schneck
 
Top Software Development Trends in 2024
Top Software Development Trends in  2024Top Software Development Trends in  2024
Top Software Development Trends in 2024Mind IT Systems
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorShane Coughlan
 
Growing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesGrowing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesSoftwareMill
 
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageSales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageDista
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxAutus Cyber Tech
 
Cybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadCybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadIvo Andreev
 
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsJaydeep Chhasatia
 

Kürzlich hochgeladen (20)

online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdf
 
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
 
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
 
IA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeIA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG time
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security Challenges
 
How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?
 
JS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIJS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AI
 
Streamlining Your Application Builds with Cloud Native Buildpacks
Streamlining Your Application Builds  with Cloud Native BuildpacksStreamlining Your Application Builds  with Cloud Native Buildpacks
Streamlining Your Application Builds with Cloud Native Buildpacks
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - Datacamp
 
Fields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxFields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptx
 
Generative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilGenerative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-Council
 
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
 
Top Software Development Trends in 2024
Top Software Development Trends in  2024Top Software Development Trends in  2024
Top Software Development Trends in 2024
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS Calculator
 
Growing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesGrowing Oxen: channel operators and retries
Growing Oxen: channel operators and retries
 
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageSales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in Trivandrum
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptx
 
Cybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadCybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and Bad
 
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
 

Combinatorial Interaction Testing, an Introduction - 2018

  • 1. Combinatorial Interaction Testing (CIT) An introduction Xavier Devroey - x.d.m.devroey@tudelft.nl
  • 2. public class Message {
     private long id = -1;     private String author;     private String message;
     public Message(String author, String message) {         this.author = author;         this.message = message;     }     // Gets and Sets ...     /**      * Appends the given text at the end of this' message.      * A space is added before the given text.      *      * @param text The test to append.      */     public void compose(String text) {         this.message = message + " " + text;     } }
  • 3. @Test     public void testComposeString() {         Message msg = new Message("Me", "despicable");         msg.compose("me");     }
  • 4. Code coverage Line coverage: percentage of lines covered by tests https://github.com/cobertura/cobertura Achieving line coverage 
 = 
 executing each line at least once
  • 5. @Test     public void testComposeString() {         Message msg = new Message("Me", "despicable");         msg.compose("me");     } Is the compose method covered by the test case?
  • 6. @Test     public void testComposeString() {         Message msg = new Message("Me", "despicable");         msg.compose("me");     } Is the compose method covered by the test case? Yes
  • 7. @Test     public void testComposeString() {         Message msg = new Message("Me", "despicable");         msg.compose("me");     } Is the compose method covered by the test case? Is it enough? Yes
  • 8. @Test     public void testComposeString() {         Message msg = new Message("Me", "despicable");         msg.compose("me");     } Is the compose method covered by the test case? Is it enough? Yes No
  • 9. @Test     public void testComposeString() {         Message msg = new Message("Me", "despicable");         msg.compose("me");     } Is the compose method covered by the test case? Is it enough? Yes No There are no assertions on the result/effect of the method
  • 10. Input/output domain coverage public void compose(String text); (based on pre/post conditions)
  • 11. Input/output domain coverage public void compose(String text); • Equivalence partitioning (based on pre/post conditions)
  • 12. Input/output domain coverage public void compose(String text); • Equivalence partitioning Normal case: "Despicable you" (based on pre/post conditions)
  • 13. Input/output domain coverage public void compose(String text); • Equivalence partitioning Normal case: "Despicable you" Limit cases: "" null "azrtyuiop^$ù m;21@98#3!" (based on pre/post conditions)
  • 14. Input/output domain coverage public void compose(String text); public void compose(String t1, String t2); • Equivalence partitioning Normal case: "Despicable you" Limit cases: "" null "azrtyuiop^$ù m;21@98#3!" (based on pre/post conditions)
  • 15. Input/output domain coverage public void compose(String text); public void compose(String t1, String t2); • Equivalence partitioning Normal case: "Despicable you" Limit cases: "" null "azrtyuiop^$ù m;21@98#3!" • t1 • t2 Normal case: "Despicable you" Limit cases: "" null "azrtyuiop^$ù m;21@98#3!" Normal case: "Despicable you" Limit cases: "" null "azrtyuiop^$ù m;21@98#3!" (based on pre/post conditions)
  • 16. Input/output domain coverage public void compose(String text); public void compose(String t1, String t2); • Equivalence partitioning Normal case: "Despicable you" Limit cases: "" null "azrtyuiop^$ù m;21@98#3!" • t1 • t2 Normal case: "Despicable you" Limit cases: "" null "azrtyuiop^$ù m;21@98#3!" Normal case: "Despicable you" Limit cases: "" null "azrtyuiop^$ù m;21@98#3!" (based on pre/post conditions) public void compose(String t1, String t2, String t3);
  • 17. Input/output domain coverage public void compose(String text); public void compose(String t1, String t2); • Equivalence partitioning Normal case: "Despicable you" Limit cases: "" null "azrtyuiop^$ù m;21@98#3!" • t1 • t2 Normal case: "Despicable you" Limit cases: "" null "azrtyuiop^$ù m;21@98#3!" Normal case: "Despicable you" Limit cases: "" null "azrtyuiop^$ù m;21@98#3!" ☛ Combinatorial explosion with the number of parameters (based on pre/post conditions) public void compose(String t1, String t2, String t3);
  • 18. Combinatorial Interaction Testing (CIT) Hyp.: Most of the faults are due to undesired interactions of a limited number (N) of factors (here, equivalent classes of parameter values). D. Kuhn et al, Software fault interactions and implications for software testing. TSE 2004 • In practice: N ≤ 4 • Usually: N = 2 (pairwise)
  • 19. Combinatorial Interaction Testing (CIT) Hyp.: Most of the faults are due to undesired interactions of a limited number (N) of factors (here, equivalent classes of parameter values). D. Kuhn et al, Software fault interactions and implications for software testing. TSE 2004 • In practice: N ≤ 4 • Usually: N = 2 (pairwise) Idea: Generate test cases such that all combinations of N values are covered at least once.
  • 20. Combinatorial Interaction Testing (CIT) t1 t2 t3 "Despicable you" "Despicable you" "Despicable you" "" "" "" null null null "azrtyuiop^$ù" "azrtyuiop^$ù" "azrtyuiop^$ù" Hyp.: most faults are caused by interactions of at most two factors Pairwise testing: cover all pairs of values
  • 21. Combinatorial Interaction Testing (CIT) t1 t2 t3 "Despicable you" "Despicable you" "Despicable you" "" "" "" null null null "azrtyuiop^$ù" "azrtyuiop^$ù" "azrtyuiop^$ù" Hyp.: most faults are caused by interactions of at most two factors Pairwise testing: cover all pairs of values "Despicable you" "Despicable you" "Despicable you" "Despicable you" "" "" "Despicable you" null null "Despicable you" "azrtyuiop^$ù" "azrtyuiop^$ù" "" "Despicable you" "" "" "" "Despicable you" "" null "azrtyuiop^$ù" "" "azrtyuiop^$ù" null null "Despicable you" null null "" "azrtyuiop^$ù" null null "Despicable you" null "azrtyuiop^$ù" "" "azrtyuiop^$ù" "Despicable you" "azrtyuiop^$ù" "azrtyuiop^$ù" "" null "azrtyuiop^$ù" null "" "azrtyuiop^$ù" "azrtyuiop^$ù" "Despicable you" 16 test cases instead of 64
  • 22. Combinatorial Interaction Testing (CIT) t1 t2 t3 "Despicable you" "Despicable you" "Despicable you" "" "" "" null null null "azrtyuiop^$ù" "azrtyuiop^$ù" "azrtyuiop^$ù" Hyp.: most faults are caused by interactions of at most two factors Pairwise testing: cover all pairs of values "Despicable you" "Despicable you" "Despicable you" "Despicable you" "" "" "Despicable you" null null "Despicable you" "azrtyuiop^$ù" "azrtyuiop^$ù" "" "Despicable you" "" "" "" "Despicable you" "" null "azrtyuiop^$ù" "" "azrtyuiop^$ù" null null "Despicable you" null null "" "azrtyuiop^$ù" null null "Despicable you" null "azrtyuiop^$ù" "" "azrtyuiop^$ù" "Despicable you" "azrtyuiop^$ù" "azrtyuiop^$ù" "" null "azrtyuiop^$ù" null "" "azrtyuiop^$ù" "azrtyuiop^$ù" "Despicable you" 16 test cases instead of 64
  • 23. Combinatorial Interaction Testing (CIT) t1 t2 t3 "Despicable you" "Despicable you" "Despicable you" "" "" "" null null null "azrtyuiop^$ù" "azrtyuiop^$ù" "azrtyuiop^$ù" Hyp.: most faults are caused by interactions of at most two factors Pairwise testing: cover all pairs of values "Despicable you" "Despicable you" "Despicable you" "Despicable you" "" "" "Despicable you" null null "Despicable you" "azrtyuiop^$ù" "azrtyuiop^$ù" "" "Despicable you" "" "" "" "Despicable you" "" null "azrtyuiop^$ù" "" "azrtyuiop^$ù" null null "Despicable you" null null "" "azrtyuiop^$ù" null null "Despicable you" null "azrtyuiop^$ù" "" "azrtyuiop^$ù" "Despicable you" "azrtyuiop^$ù" "azrtyuiop^$ù" "" null "azrtyuiop^$ù" null "" "azrtyuiop^$ù" "azrtyuiop^$ù" "Despicable you" 16 test cases instead of 64
  • 24. Combinatorial Interaction Testing (CIT) t1 t2 t3 "Despicable you" "Despicable you" "Despicable you" "" "" "" null null null "azrtyuiop^$ù" "azrtyuiop^$ù" "azrtyuiop^$ù" Hyp.: most faults are caused by interactions of at most two factors Pairwise testing: cover all pairs of values "Despicable you" "Despicable you" "Despicable you" "Despicable you" "" "" "Despicable you" null null "Despicable you" "azrtyuiop^$ù" "azrtyuiop^$ù" "" "Despicable you" "" "" "" "Despicable you" "" null "azrtyuiop^$ù" "" "azrtyuiop^$ù" null null "Despicable you" null null "" "azrtyuiop^$ù" null null "Despicable you" null "azrtyuiop^$ù" "" "azrtyuiop^$ù" "Despicable you" "azrtyuiop^$ù" "azrtyuiop^$ù" "" null "azrtyuiop^$ù" null "" "azrtyuiop^$ù" "azrtyuiop^$ù" "Despicable you" 16 test cases instead of 64
  • 25. Combinatorial Interaction Testing (CIT) t1 t2 t3 "Despicable you" "Despicable you" "Despicable you" "" "" "" null null null "azrtyuiop^$ù" "azrtyuiop^$ù" "azrtyuiop^$ù" Hyp.: most faults are caused by interactions of at most two factors Pairwise testing: cover all pairs of values "Despicable you" "Despicable you" "Despicable you" "Despicable you" "" "" "Despicable you" null null "Despicable you" "azrtyuiop^$ù" "azrtyuiop^$ù" "" "Despicable you" "" "" "" "Despicable you" "" null "azrtyuiop^$ù" "" "azrtyuiop^$ù" null null "Despicable you" null null "" "azrtyuiop^$ù" null null "Despicable you" null "azrtyuiop^$ù" "" "azrtyuiop^$ù" "Despicable you" "azrtyuiop^$ù" "azrtyuiop^$ù" "" null "azrtyuiop^$ù" null "" "azrtyuiop^$ù" "azrtyuiop^$ù" "Despicable you" 16 test cases instead of 64
  • 26. Combinatorial Interaction Testing (CIT) t1 t2 t3 "Despicable you" "Despicable you" "Despicable you" "" "" "" null null null "azrtyuiop^$ù" "azrtyuiop^$ù" "azrtyuiop^$ù" Hyp.: most faults are caused by interactions of at most two factors Pairwise testing: cover all pairs of values "Despicable you" "Despicable you" "Despicable you" "Despicable you" "" "" "Despicable you" null null "Despicable you" "azrtyuiop^$ù" "azrtyuiop^$ù" "" "Despicable you" "" "" "" "Despicable you" "" null "azrtyuiop^$ù" "" "azrtyuiop^$ù" null null "Despicable you" null null "" "azrtyuiop^$ù" null null "Despicable you" null "azrtyuiop^$ù" "" "azrtyuiop^$ù" "Despicable you" "azrtyuiop^$ù" "azrtyuiop^$ù" "" null "azrtyuiop^$ù" null "" "azrtyuiop^$ù" "azrtyuiop^$ù" "Despicable you" 16 test cases instead of 64
  • 27. Combinatorial Interaction Testing (CIT) t1 t2 t3 "Despicable you" "Despicable you" "Despicable you" "" "" "" null null null "azrtyuiop^$ù" "azrtyuiop^$ù" "azrtyuiop^$ù" Hyp.: most faults are caused by interactions of at most two factors Pairwise testing: cover all pairs of values "Despicable you" "Despicable you" "Despicable you" "Despicable you" "" "" "Despicable you" null null "Despicable you" "azrtyuiop^$ù" "azrtyuiop^$ù" "" "Despicable you" "" "" "" "Despicable you" "" null "azrtyuiop^$ù" "" "azrtyuiop^$ù" null null "Despicable you" null null "" "azrtyuiop^$ù" null null "Despicable you" null "azrtyuiop^$ù" "" "azrtyuiop^$ù" "Despicable you" "azrtyuiop^$ù" "azrtyuiop^$ù" "" null "azrtyuiop^$ù" null "" "azrtyuiop^$ù" "azrtyuiop^$ù" "Despicable you" 16 test cases instead of 64
  • 28. http://cse.unl.edu/~citportal/ http://ctweb.abstracta.com.uy/combinatorial.jsp Hyp.: most faults are caused by interactions of at most t factors T-wise testing: cover all t-uples of values
  • 29. http://cse.unl.edu/~citportal/ http://ctweb.abstracta.com.uy/combinatorial.jsp Hyp.: most faults are caused by interactions of at most t factors T-wise testing: cover all t-uples of values Integration and System testing
  • 30. app. http://cse.unl.edu/~citportal/ http://ctweb.abstracta.com.uy/combinatorial.jsp Hyp.: most faults are caused by interactions of at most t factors T-wise testing: cover all t-uples of values Integration and System testing
  • 31. app. http://cse.unl.edu/~citportal/ http://ctweb.abstracta.com.uy/combinatorial.jsp Hyp.: most faults are caused by interactions of at most t factors T-wise testing: cover all t-uples of values Integration and System testing
  • 32. app. http://cse.unl.edu/~citportal/ http://ctweb.abstracta.com.uy/combinatorial.jsp Hyp.: most faults are caused by interactions of at most t factors T-wise testing: cover all t-uples of values Integration and System testing Which configuration should I use for my tests?
  • 33. type Android version Screen Processor tablet 4.0.4 XLarge ARM Cortex smartphone 4.1 Large Intel x86 phablet 4.1.1 Normal 4.1.2 Small
  • 34. type Android version Screen Processor tablet 4.0.4 XLarge ARM Cortex smartphone 4.1 Large Intel x86 phablet 4.1.1 Normal 4.1.2 Small … or using a feature model http://familiar-project.github.io https://featureide.github.io http://www.skalup.com/ http://research.henard.net/SPL/PLEDGE/ http://martinfjohansen.com/models2011/spltool/ http://cse.unl.edu/~citportal/
  • 35. app. Hyp.: most faults are caused by interactions of at most t factors T-wise testing: cover all t-uples of values Integration and System testing Which configuration should I use for my tests? Under … • tests requirements • $$$ budget • test execution time budget … constraints