SlideShare ist ein Scribd-Unternehmen logo
1 von 26
Downloaden Sie, um offline zu lesen
Test & evaluation = the knowledge factory
Knowledge-centered perspective on T&E

Shifting focus to the people

Some observations

Glimpse to theory and its applications
Systems


Discovery               Action
T&E, Reqs                SE


            Knowledge
Make predictions based on theory/design


Verify/Falsify predictions by experimenting

Search for unpredicted behavior


Reporting findings formally
Test & evaluation = the knowledge factory
Maximal coverage of phenomena


Scheduled delivery of knowledge


Immediate usefulness of knowledge
Test & evaluation = the knowledge factory
Project / Organization




Test Team




              Test Engineer
Test & evaluation = the knowledge factory
Creating a mental model




                                        In order to be able
Using all available                       to “tell a story”
     sources                             about the system

    Without
 documentation
The evidence
                  Need to Learn
contradicts the
 mental model      arises when




            The production of
           required documents
               is impeded
Test & evaluation = the knowledge factory
Test & evaluation = the knowledge factory
Knowledge is experience.
Everything else is just information.
Albert Einstein
Documented        Undocumented
                  Knowledge             Tacit
                                     Knowledge
 Planning T&E         Plans

                         Planner Experience

Designing Tests   Test Procedures

                         Designer Experience

Executing Tests   Test Protocols

                          Tester Experience

  Evaluating       Test Reports

                         Analyst Experience
Test & evaluation = the knowledge factory
Developers on-site
i   i

        Customer on-site



        Testing in teams
i
i       i   Talking with developers
    g
i       i   Talking with customers/users
    i


            Meetings and discussions


            Writing reports
g
            Analyzing requirements specs
g   o   g
            Analyzing design documents
    g

            Designing test plans


            Designing test procedures



             Complex Adaptive Integrator
o   Building “mental models” of the
    g       system

i
            “Playing” with the system simulations


            Unstructured and exploratory testing


            “Playing around” planned tests
Actively question the developers, customers and
users to externalize their tacit knowledge

Encourage active participation of developers,
customers and fellow testers in running tests

Take meetings and non-formal discussions
seriously and use them to learn and share
knowledge
Learning Preferences
Know your learning preferences and use
them while studying the SUTs

Augment reports with multimedia
Organize presentations of the results for
relevant audiences
Test & Evaluation is a knowledge creating
and sharing process


Test Engineers are professional learners


T&E should use the insights and tools of
knowledge and learning theory and practice
How the knowledge is created and shared by
test engineers and test teams?

How do the knowledge processes integrate
with testing activities?

What are the knowledge-related factors that
contribute to or impede better testing?

What factors facilitate more effective and
efficient learning of Test Engineers?
Test & evaluation = the knowledge factory

Weitere ähnliche Inhalte

Andere mochten auch

introduction to test measure and evaluation
introduction to test measure and evaluationintroduction to test measure and evaluation
introduction to test measure and evaluationTahir Ramzan Bhat
 
Language testing and evaluation validity and reliability.
Language testing and evaluation validity and reliability.Language testing and evaluation validity and reliability.
Language testing and evaluation validity and reliability.Vadher Ankita
 
Language testing and evaluation
Language testing and evaluationLanguage testing and evaluation
Language testing and evaluationesra66
 
Formative & summative evaluation
Formative & summative evaluationFormative & summative evaluation
Formative & summative evaluationBuxoo Abdullah
 
Design Chapter 7 - Testing and Evaluation Techniques
Design Chapter 7 - Testing and Evaluation TechniquesDesign Chapter 7 - Testing and Evaluation Techniques
Design Chapter 7 - Testing and Evaluation Techniquesguest01bdf1
 
Summative and formative evaluation
Summative and formative evaluationSummative and formative evaluation
Summative and formative evaluationKing Ayapana
 
Measurement,evaluation,assessment(upload)
Measurement,evaluation,assessment(upload)Measurement,evaluation,assessment(upload)
Measurement,evaluation,assessment(upload)Dr.Shazia Zamir
 
Measurement, Evaluation, Assessment, and Tests
Measurement, Evaluation, Assessment, and TestsMeasurement, Evaluation, Assessment, and Tests
Measurement, Evaluation, Assessment, and TestsMonica P
 
Guiding Principles for the Assessment of Student Learning
Guiding Principles for the Assessment of Student LearningGuiding Principles for the Assessment of Student Learning
Guiding Principles for the Assessment of Student LearningSheila Lavapie
 
Language Testing: Approaches and Techniques
Language Testing: Approaches and TechniquesLanguage Testing: Approaches and Techniques
Language Testing: Approaches and TechniquesMonica Angeles
 
Formative Assessment vs. Summative Assessment
Formative Assessment vs. Summative AssessmentFormative Assessment vs. Summative Assessment
Formative Assessment vs. Summative Assessmentjcheek2008
 
Assessment of Learning
Assessment of LearningAssessment of Learning
Assessment of LearningYaKu Loveria
 
Evaluation in Education
Evaluation in EducationEvaluation in Education
Evaluation in EducationKusum Gaur
 
Evaluation – concepts and principles
Evaluation – concepts and principlesEvaluation – concepts and principles
Evaluation – concepts and principlesAruna Ap
 

Andere mochten auch (20)

Cerias talk on testing and evaluation
Cerias talk on testing and evaluationCerias talk on testing and evaluation
Cerias talk on testing and evaluation
 
Test and measurement
Test and measurement Test and measurement
Test and measurement
 
introduction to test measure and evaluation
introduction to test measure and evaluationintroduction to test measure and evaluation
introduction to test measure and evaluation
 
Language testing and evaluation validity and reliability.
Language testing and evaluation validity and reliability.Language testing and evaluation validity and reliability.
Language testing and evaluation validity and reliability.
 
Language testing and evaluation
Language testing and evaluationLanguage testing and evaluation
Language testing and evaluation
 
Formative & summative evaluation
Formative & summative evaluationFormative & summative evaluation
Formative & summative evaluation
 
Design Chapter 7 - Testing and Evaluation Techniques
Design Chapter 7 - Testing and Evaluation TechniquesDesign Chapter 7 - Testing and Evaluation Techniques
Design Chapter 7 - Testing and Evaluation Techniques
 
Teaching and testing
Teaching and testingTeaching and testing
Teaching and testing
 
Summative and formative evaluation
Summative and formative evaluationSummative and formative evaluation
Summative and formative evaluation
 
Language Testing
Language TestingLanguage Testing
Language Testing
 
Measurement,evaluation,assessment(upload)
Measurement,evaluation,assessment(upload)Measurement,evaluation,assessment(upload)
Measurement,evaluation,assessment(upload)
 
Types of evaluation
Types of evaluationTypes of evaluation
Types of evaluation
 
Measurement, Evaluation, Assessment, and Tests
Measurement, Evaluation, Assessment, and TestsMeasurement, Evaluation, Assessment, and Tests
Measurement, Evaluation, Assessment, and Tests
 
Guiding Principles for the Assessment of Student Learning
Guiding Principles for the Assessment of Student LearningGuiding Principles for the Assessment of Student Learning
Guiding Principles for the Assessment of Student Learning
 
Language Testing: Approaches and Techniques
Language Testing: Approaches and TechniquesLanguage Testing: Approaches and Techniques
Language Testing: Approaches and Techniques
 
Formative Assessment vs. Summative Assessment
Formative Assessment vs. Summative AssessmentFormative Assessment vs. Summative Assessment
Formative Assessment vs. Summative Assessment
 
Assessment of Learning
Assessment of LearningAssessment of Learning
Assessment of Learning
 
Evaluation in Education
Evaluation in EducationEvaluation in Education
Evaluation in Education
 
Evaluation – concepts and principles
Evaluation – concepts and principlesEvaluation – concepts and principles
Evaluation – concepts and principles
 
Assessment of learning outcomes
Assessment of learning outcomesAssessment of learning outcomes
Assessment of learning outcomes
 

Ähnlich wie Test & evaluation = the knowledge factory

Right Experience, Right People - ConveyUX 2014
Right Experience, Right People - ConveyUX 2014Right Experience, Right People - ConveyUX 2014
Right Experience, Right People - ConveyUX 2014Phillip Hunter
 
A New Model For Testing
A New Model For TestingA New Model For Testing
A New Model For TestingTEST Huddle
 
Automatic for the People
Automatic for the PeopleAutomatic for the People
Automatic for the PeopleAndy Zaidman
 
Planning your analytics journey - webinar slides
Planning your analytics journey  - webinar slidesPlanning your analytics journey  - webinar slides
Planning your analytics journey - webinar slidesSprout Labs
 
Create the Foundation of an App - UX Wannabe 5
Create the Foundation of an App - UX Wannabe 5Create the Foundation of an App - UX Wannabe 5
Create the Foundation of an App - UX Wannabe 5Daeng Muhammad Feisal
 
Managing Data Science | Lessons from the Field
Managing Data Science | Lessons from the Field Managing Data Science | Lessons from the Field
Managing Data Science | Lessons from the Field Domino Data Lab
 
Dare to Explore: Discover ET!
Dare to Explore: Discover ET!Dare to Explore: Discover ET!
Dare to Explore: Discover ET!Raj Indugula
 
Staying research led with almost no resources (UXcamp 2019)
Staying research led with almost no resources (UXcamp 2019)Staying research led with almost no resources (UXcamp 2019)
Staying research led with almost no resources (UXcamp 2019)Kea Zhang
 
Knowledge Management
Knowledge ManagementKnowledge Management
Knowledge ManagementElaine Aitken
 
Y3 ICT Lecture 6 Planning
Y3 ICT Lecture 6 PlanningY3 ICT Lecture 6 Planning
Y3 ICT Lecture 6 PlanningMiles Berry
 
Advanced Project Data Analytics for Improved Project Delivery
Advanced Project Data Analytics for Improved Project DeliveryAdvanced Project Data Analytics for Improved Project Delivery
Advanced Project Data Analytics for Improved Project DeliveryMark Constable
 
Creating a culture of collaborative learning
Creating a culture of collaborative learningCreating a culture of collaborative learning
Creating a culture of collaborative learningCory Banks
 
11 - Evaluating Framework in Interaction Design_new.pptx
11 - Evaluating Framework in Interaction Design_new.pptx11 - Evaluating Framework in Interaction Design_new.pptx
11 - Evaluating Framework in Interaction Design_new.pptxZahirahZairul2
 
Digital transformation testing.
Digital transformation testing. Digital transformation testing.
Digital transformation testing. Deepak Daniel
 
Commercial Ethnography Euro Ia 2008 Kalbach
Commercial Ethnography Euro Ia 2008 KalbachCommercial Ethnography Euro Ia 2008 Kalbach
Commercial Ethnography Euro Ia 2008 KalbachJim Kalbach
 

Ähnlich wie Test & evaluation = the knowledge factory (20)

Right Experience, Right People - ConveyUX 2014
Right Experience, Right People - ConveyUX 2014Right Experience, Right People - ConveyUX 2014
Right Experience, Right People - ConveyUX 2014
 
Agile testing
Agile testingAgile testing
Agile testing
 
New model
New modelNew model
New model
 
A New Model For Testing
A New Model For TestingA New Model For Testing
A New Model For Testing
 
Automatic for the People
Automatic for the PeopleAutomatic for the People
Automatic for the People
 
Prototyping.pptx
Prototyping.pptxPrototyping.pptx
Prototyping.pptx
 
Planning your analytics journey - webinar slides
Planning your analytics journey  - webinar slidesPlanning your analytics journey  - webinar slides
Planning your analytics journey - webinar slides
 
Create the Foundation of an App - UX Wannabe 5
Create the Foundation of an App - UX Wannabe 5Create the Foundation of an App - UX Wannabe 5
Create the Foundation of an App - UX Wannabe 5
 
Managing Data Science | Lessons from the Field
Managing Data Science | Lessons from the Field Managing Data Science | Lessons from the Field
Managing Data Science | Lessons from the Field
 
Dare to Explore: Discover ET!
Dare to Explore: Discover ET!Dare to Explore: Discover ET!
Dare to Explore: Discover ET!
 
Staying research led with almost no resources (UXcamp 2019)
Staying research led with almost no resources (UXcamp 2019)Staying research led with almost no resources (UXcamp 2019)
Staying research led with almost no resources (UXcamp 2019)
 
Knowledge Management
Knowledge ManagementKnowledge Management
Knowledge Management
 
Choosing the Right UX Method
Choosing the Right UX MethodChoosing the Right UX Method
Choosing the Right UX Method
 
Hype research
Hype researchHype research
Hype research
 
Y3 ICT Lecture 6 Planning
Y3 ICT Lecture 6 PlanningY3 ICT Lecture 6 Planning
Y3 ICT Lecture 6 Planning
 
Advanced Project Data Analytics for Improved Project Delivery
Advanced Project Data Analytics for Improved Project DeliveryAdvanced Project Data Analytics for Improved Project Delivery
Advanced Project Data Analytics for Improved Project Delivery
 
Creating a culture of collaborative learning
Creating a culture of collaborative learningCreating a culture of collaborative learning
Creating a culture of collaborative learning
 
11 - Evaluating Framework in Interaction Design_new.pptx
11 - Evaluating Framework in Interaction Design_new.pptx11 - Evaluating Framework in Interaction Design_new.pptx
11 - Evaluating Framework in Interaction Design_new.pptx
 
Digital transformation testing.
Digital transformation testing. Digital transformation testing.
Digital transformation testing.
 
Commercial Ethnography Euro Ia 2008 Kalbach
Commercial Ethnography Euro Ia 2008 KalbachCommercial Ethnography Euro Ia 2008 Kalbach
Commercial Ethnography Euro Ia 2008 Kalbach
 

Mehr von Sergey Tozik

Ghost in the machine
Ghost in the machineGhost in the machine
Ghost in the machineSergey Tozik
 
Pcdf framework for teaching practicing professionals
Pcdf framework for teaching practicing professionalsPcdf framework for teaching practicing professionals
Pcdf framework for teaching practicing professionalsSergey Tozik
 
Systems engineering in the light of knowledge management
Systems engineering in the light of knowledge managementSystems engineering in the light of knowledge management
Systems engineering in the light of knowledge managementSergey Tozik
 
System integration complexity
System integration complexitySystem integration complexity
System integration complexitySergey Tozik
 
Fielding Systems-of-Systems, Riding the agile sw tiger
Fielding Systems-of-Systems, Riding the agile sw tigerFielding Systems-of-Systems, Riding the agile sw tiger
Fielding Systems-of-Systems, Riding the agile sw tigerSergey Tozik
 
A need for creativity in systems integration
A need for creativity in systems integrationA need for creativity in systems integration
A need for creativity in systems integrationSergey Tozik
 

Mehr von Sergey Tozik (6)

Ghost in the machine
Ghost in the machineGhost in the machine
Ghost in the machine
 
Pcdf framework for teaching practicing professionals
Pcdf framework for teaching practicing professionalsPcdf framework for teaching practicing professionals
Pcdf framework for teaching practicing professionals
 
Systems engineering in the light of knowledge management
Systems engineering in the light of knowledge managementSystems engineering in the light of knowledge management
Systems engineering in the light of knowledge management
 
System integration complexity
System integration complexitySystem integration complexity
System integration complexity
 
Fielding Systems-of-Systems, Riding the agile sw tiger
Fielding Systems-of-Systems, Riding the agile sw tigerFielding Systems-of-Systems, Riding the agile sw tiger
Fielding Systems-of-Systems, Riding the agile sw tiger
 
A need for creativity in systems integration
A need for creativity in systems integrationA need for creativity in systems integration
A need for creativity in systems integration
 

Test & evaluation = the knowledge factory

  • 2. Knowledge-centered perspective on T&E Shifting focus to the people Some observations Glimpse to theory and its applications
  • 3. Systems Discovery Action T&E, Reqs SE Knowledge
  • 4. Make predictions based on theory/design Verify/Falsify predictions by experimenting Search for unpredicted behavior Reporting findings formally
  • 6. Maximal coverage of phenomena Scheduled delivery of knowledge Immediate usefulness of knowledge
  • 8. Project / Organization Test Team Test Engineer
  • 10. Creating a mental model In order to be able Using all available to “tell a story” sources about the system Without documentation
  • 11. The evidence Need to Learn contradicts the mental model arises when The production of required documents is impeded
  • 14. Knowledge is experience. Everything else is just information. Albert Einstein
  • 15. Documented Undocumented Knowledge Tacit Knowledge Planning T&E Plans Planner Experience Designing Tests Test Procedures Designer Experience Executing Tests Test Protocols Tester Experience Evaluating Test Reports Analyst Experience
  • 17. Developers on-site i i Customer on-site Testing in teams
  • 18. i i i Talking with developers g i i Talking with customers/users i Meetings and discussions Writing reports
  • 19. g Analyzing requirements specs g o g Analyzing design documents g Designing test plans Designing test procedures Complex Adaptive Integrator
  • 20. o Building “mental models” of the g system i “Playing” with the system simulations Unstructured and exploratory testing “Playing around” planned tests
  • 21. Actively question the developers, customers and users to externalize their tacit knowledge Encourage active participation of developers, customers and fellow testers in running tests Take meetings and non-formal discussions seriously and use them to learn and share knowledge
  • 23. Know your learning preferences and use them while studying the SUTs Augment reports with multimedia Organize presentations of the results for relevant audiences
  • 24. Test & Evaluation is a knowledge creating and sharing process Test Engineers are professional learners T&E should use the insights and tools of knowledge and learning theory and practice
  • 25. How the knowledge is created and shared by test engineers and test teams? How do the knowledge processes integrate with testing activities? What are the knowledge-related factors that contribute to or impede better testing? What factors facilitate more effective and efficient learning of Test Engineers?