SlideShare ist ein Scribd-Unternehmen logo
1 von 26
Downloaden Sie, um offline zu lesen
WISSEN
TECHNIK
LEIDENSCHAFT
Automated system testing for
a learning management
system
31.10.2019
Lukas Krisper
Agenda
• Motivation
• Test Automation
• Test Cases for an LMS
• Test Data for an LMS
• Implementation and Infrastructure
• Conclusion
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
2
Agenda
• Motivation
• Test Automation
• Test Cases for an LMS
• Test Data for an LMS
• Implementation and Infrastructure
• Conclusion
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
3
A shift towards Agile Software Development
• Over the last two decades …
• Emerge of agile methodologies in software development
• Acceleration of software delivery and feedback cycles
• Automation of manual processes
• Quality Assurance?
• Only 20% of IT organisations have automated more than half of their
test cases (State of Testing Report, 2018)
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
4
Motivation
New Releases in Summer 2019
• TeachCenter 3.0
• Learning Management System (LMS)
• Migration of whole LMS to new Moodle Version (3.5)
• New GUI for all users
• Set of Unit-Tests existed
• Sparely documented manual tests
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
5
Motivation
Research Questions
• (RQ1) How can the existing core functionality of a large software system
be assured automatically in new versions of the system?
• (RQ2) Which test cases and test suites need to be designed to test a
learning management system effectively?
• (RQ3) Which test data is needed to test a learning management system
and how can the data be provided to automated test cases?
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
6
Motivation
Agenda
• Motivation
• Test Automation
• Test Cases for an LMS
• Test Data for an LMS
• Implementation and Infrastructure
• Conclusion
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
7
Focus of Automation Activities
• Software Quality Characteristic*: Functional Suitability
• Testing Level: System Testing
• Testing Type: Regression Tests
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
8
Test Automation
*According to ISO-25010
Software Quality Characteristic: Functional Suitability
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
9
Test Automation
• Does the software product do what it is supposed to do?
• Functional testing as basis of all testing activities
• A software product that does not provide functional suitability is rather
worthless
• E.g. calculator that calculates very fast
and has a nice UI, but the calculations are wrong
2 + 2 = 5
Testing Level: System Testing
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
10
Test Automation
• Software product is considered in its entirety
• Test environment has close resemblance to production environment
• Tests conducted from user’s perspective
• Typically using the systems GUI for interaction
• Automated system tests as second line of defense
• If tests fail at this level → malfunctioning system + wrong or missing
test at lower testing level
Testing Type: Regression Tests
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
11
Test Automation
• Is already existing functionality affected by the changes applied to a software?
• The first type of tests to be considered when automating tests
• Quickly run a set of tests after applying changes:
• Extensions to a feature
• Bug fixes
• Refactorings
• …
• Especially useful when changes happen late in the project and time for
manual testing is short
Agenda
• Motivation
• Test Automation
• Test Cases for an LMS
• Test Data for an LMS
• Implementation and Infrastructure
• Conclusion
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
12
What Test Cases should be automated?
• Important consideration!
• Interview with first level support of TeachCenter 2.0
• Research in literature on usages of LMS
• Results:
• Knowledge on most important use cases for students and teachers
E.g. reading the course’s contents, communicating with other
participants, submitting material to the course, …
• > 30 test cases combined to different test suites
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
13
Test Cases for a LMS
Agenda
• Motivation
• Test Automation
• Test Cases for an LMS
• Test Data for an LMS
• Implementation and Infrastructure
• Conclusion
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
14
Real Data vs Synthetic Data
• Real Data
• Transfer data from production environment to test environment
• Data has to be edited (according to laws on data protection/privacy*)
• Important: Keep consistency in data set!
• Synthetic Data
• Newly created data for the purpose of testing
• Always GDPR compliant
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
15
Test Data for a LMS
* e.g. General Data Protection Regulation (GDPR)
Data for a LMS
• A LMS typically contains data on
• Courses (course descriptions, learning material, …)
• Data on people involved in the courses (names, email-addresses, …)
• Data from interaction of people with the courses (submissions, …)
• TeachCenter 3.0 is tested with both synthetic and real data
• Data used for automated tests is labeled as such
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
16
Test Data for a LMS
Agenda
• Motivation
• Test Automation
• Test Cases for an LMS
• Test Data for an LMS
• Implementation and Infrastructure
• Conclusion
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
17
Overview on Infrastructure
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
18
Implementation and Infrastructure
Test Suite 1
Test Case 1
Test Case n
Test Case 2
…
Test Suite 1
Test Case 1
Test Case n
Test Case 2
…
Overview on Implementation
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
19
Implementation and Infrastructure
Test Suite 1
Test Case 1
Test Case n
Test Case 2
…
Test Data
Manager
Test Driver
Utility
e.g. initialise headless Chrome
browser with cleared cache
e.g. get PDF files for
submissions, references to
test data stored in JSON
files, …
Page Object 1Page Object 1Page Object 1
TestNG
Page Object 1Page Object 1
Helper Classes
e.g. taking screenshots on errors
See following slides
PageObject Design Pattern
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
20
Implementation and Infrastructure
• Components of a website are modelled into
reusable objects → “Page Objects”
• Page Objects offer functionality to interact with
website (click links, enter input fields, …)
• Tests use Page Objects to interact with website
(perform action → assert result of action)
Identification of Objects in UI
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
21
Implementation and Infrastructure
• Tests should be executable regardless …
• screen resolution
• window sizes
• language settings
• Identification of objects using stable features (IDs or XPath expressions)
DOM Element <button type= "button" id= "btn1" data-role= "next "> Next </button >
ID btn1
XPath expression //button[@data-role=’next’]
Agenda
• Motivation
• Test Automation
• Test Cases for an LMS
• Test Data for an LMS
• Implementation and Infrastructure
• Conclusion
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
22
(RQ1) How can the existing core functionality of a large software
system be assured automatically in new versions of the system?
• Regression Tests for „Functional Suitabilty“
• System Testing Level
• Large software system = many integrated components that interact
with each other → “high” testing level to ensure stability and detect
side effects
• Continuous Integration Environment (Automation Server)
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
23
Conclusion
(RQ2) Which test cases and test suites need to be designed to test
a learning management system effectively?
• Identification of use cases
• Interview with 1st level support + literature research
• Prioritisation of use cases
• Implementation of test cases based on use case
• Execution of suites of test cases according to prioritisation of use cases
• Multiple times a day ~ once a week
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
24
Conclusion
(RQ3) Which test data is needed to test a learning management
system and how can the data be provided to automated test cases?
• Synthetic and real test data
• 1 configured course with various activities and resources
• 1 student‘s account (enrolled to course)
• 1 teacher‘s account (enrolled to course)
• 1 PDF file
• Test data is persisted on test system (references in JSON-files) …
• Or provided to the test cases on demand
• Separation of test data and automated test cases
• No hardcoded test data
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
25
Conclusion
Thank you for your attention!
Lukas Krisper, 31.10.2019
Automated system testing for a learning management system
26
Conclusion

Weitere ähnliche Inhalte

Was ist angesagt?

Mis and erp
Mis and erpMis and erp
Mis and erpsumit235
 
Openeducat Features product brochure | Education Software Development
Openeducat Features product brochure | Education Software DevelopmentOpeneducat Features product brochure | Education Software Development
Openeducat Features product brochure | Education Software DevelopmentTech Receptives
 
Learning Management System
Learning Management SystemLearning Management System
Learning Management SystemParth Acharya
 
Material requirement planning - Production and Operation Management
Material requirement planning - Production and Operation ManagementMaterial requirement planning - Production and Operation Management
Material requirement planning - Production and Operation ManagementSuryakumar Thangarasu
 
Pull system of manufacturing
Pull system of manufacturingPull system of manufacturing
Pull system of manufacturingMohit Singla
 
Management Information System
Management Information System Management Information System
Management Information System Ajilal
 
Management information system (MIS)
Management information system (MIS)Management information system (MIS)
Management information system (MIS)Bhupen Meena
 
Clarity First: What it is. Why you need it. How to get it.
Clarity First: What it is. Why you need it. How to get it.Clarity First: What it is. Why you need it. How to get it.
Clarity First: What it is. Why you need it. How to get it.TKMG, Inc.
 
Information systems in Organizations
Information systems in OrganizationsInformation systems in Organizations
Information systems in Organizationsmulugetaa
 
MIS concepts
MIS conceptsMIS concepts
MIS conceptssajappy
 
introduction to management information systems (MIS)
introduction to management information systems (MIS)introduction to management information systems (MIS)
introduction to management information systems (MIS)Sujan Oli
 
Harnessing The Power of 5S and Visual Management with GoLeanSixSigma.com
Harnessing The Power of 5S and Visual Management with GoLeanSixSigma.comHarnessing The Power of 5S and Visual Management with GoLeanSixSigma.com
Harnessing The Power of 5S and Visual Management with GoLeanSixSigma.comGoLeanSixSigma.com
 
Enterprise resource planning ppt
Enterprise resource planning pptEnterprise resource planning ppt
Enterprise resource planning pptBabasab Patil
 
Business information system with explaination
Business information system with explainationBusiness information system with explaination
Business information system with explainationAlana Abraham
 
Knowledge Management System(KMS)
Knowledge Management System(KMS)Knowledge Management System(KMS)
Knowledge Management System(KMS)ayush goyal
 
IS740 Chapter 09
IS740 Chapter 09IS740 Chapter 09
IS740 Chapter 09iDocs
 

Was ist angesagt? (20)

Mis and erp
Mis and erpMis and erp
Mis and erp
 
Openeducat Features product brochure | Education Software Development
Openeducat Features product brochure | Education Software DevelopmentOpeneducat Features product brochure | Education Software Development
Openeducat Features product brochure | Education Software Development
 
Learning Management System
Learning Management SystemLearning Management System
Learning Management System
 
Material requirement planning - Production and Operation Management
Material requirement planning - Production and Operation ManagementMaterial requirement planning - Production and Operation Management
Material requirement planning - Production and Operation Management
 
Pull system of manufacturing
Pull system of manufacturingPull system of manufacturing
Pull system of manufacturing
 
Mis lecture 2
Mis lecture 2Mis lecture 2
Mis lecture 2
 
Recommender Systems
Recommender SystemsRecommender Systems
Recommender Systems
 
Value Stream Mapping (VSM)
Value Stream Mapping (VSM)Value Stream Mapping (VSM)
Value Stream Mapping (VSM)
 
Management Information System
Management Information System Management Information System
Management Information System
 
Management information system (MIS)
Management information system (MIS)Management information system (MIS)
Management information system (MIS)
 
Clarity First: What it is. Why you need it. How to get it.
Clarity First: What it is. Why you need it. How to get it.Clarity First: What it is. Why you need it. How to get it.
Clarity First: What it is. Why you need it. How to get it.
 
Information systems in Organizations
Information systems in OrganizationsInformation systems in Organizations
Information systems in Organizations
 
MIS concepts
MIS conceptsMIS concepts
MIS concepts
 
introduction to management information systems (MIS)
introduction to management information systems (MIS)introduction to management information systems (MIS)
introduction to management information systems (MIS)
 
Harnessing The Power of 5S and Visual Management with GoLeanSixSigma.com
Harnessing The Power of 5S and Visual Management with GoLeanSixSigma.comHarnessing The Power of 5S and Visual Management with GoLeanSixSigma.com
Harnessing The Power of 5S and Visual Management with GoLeanSixSigma.com
 
Enterprise resource planning ppt
Enterprise resource planning pptEnterprise resource planning ppt
Enterprise resource planning ppt
 
Benefits of ERP
Benefits of ERPBenefits of ERP
Benefits of ERP
 
Business information system with explaination
Business information system with explainationBusiness information system with explaination
Business information system with explaination
 
Knowledge Management System(KMS)
Knowledge Management System(KMS)Knowledge Management System(KMS)
Knowledge Management System(KMS)
 
IS740 Chapter 09
IS740 Chapter 09IS740 Chapter 09
IS740 Chapter 09
 

Ähnlich wie Automated system testing for a learning management system

Computer Based Assessment.pptx
Computer Based Assessment.pptxComputer Based Assessment.pptx
Computer Based Assessment.pptxsalah327743
 
Quality Assurance in Modern Software Development
Quality Assurance in Modern Software DevelopmentQuality Assurance in Modern Software Development
Quality Assurance in Modern Software DevelopmentZahra Sadeghi
 
Project Management for Information System Development
Project Management for Information System DevelopmentProject Management for Information System Development
Project Management for Information System DevelopmentNabilaNuzhat
 
Software Development Methodologies
Software Development MethodologiesSoftware Development Methodologies
Software Development MethodologiesNicholas Davis
 
Development of admission management system
Development of admission management systemDevelopment of admission management system
Development of admission management systemAlokDatta76
 
OOAD - System Analysis and Design
OOAD - System Analysis and Design OOAD - System Analysis and Design
OOAD - System Analysis and Design Murugeswari Ravi
 
Usability evaluation methods (part 2) and performance metrics
Usability evaluation methods (part 2) and performance metricsUsability evaluation methods (part 2) and performance metrics
Usability evaluation methods (part 2) and performance metricsAndres Baravalle
 
A practical guide for using Statistical Tests to assess Randomized Algorithms...
A practical guide for using Statistical Tests to assess Randomized Algorithms...A practical guide for using Statistical Tests to assess Randomized Algorithms...
A practical guide for using Statistical Tests to assess Randomized Algorithms...Lionel Briand
 
manajemen komputer dan sistem informasi.
manajemen komputer dan sistem informasi.manajemen komputer dan sistem informasi.
manajemen komputer dan sistem informasi.HermanTusiadi
 
Structured NERC CIP Process Improvement Using Six Sigma
Structured NERC CIP Process Improvement Using Six SigmaStructured NERC CIP Process Improvement Using Six Sigma
Structured NERC CIP Process Improvement Using Six SigmaEnergySec
 
Governance Strategies for Office 365
Governance Strategies for Office 365Governance Strategies for Office 365
Governance Strategies for Office 365Montrium
 
Software Testing - Introduction
Software Testing - IntroductionSoftware Testing - Introduction
Software Testing - IntroductionAjeng Savitri
 
Pressman ch-22-process-and-project-metrics
Pressman ch-22-process-and-project-metricsPressman ch-22-process-and-project-metrics
Pressman ch-22-process-and-project-metricsSeema Kamble
 
1 sad-01-introduction-june2015-rev
1 sad-01-introduction-june2015-rev1 sad-01-introduction-june2015-rev
1 sad-01-introduction-june2015-revFajar Baskoro
 
7 Tips from Siemens Energy for Success with Automation
7 Tips from Siemens Energy for Success with Automation7 Tips from Siemens Energy for Success with Automation
7 Tips from Siemens Energy for Success with AutomationWorksoft
 
ASMUG February 2015 Knowledge Event
ASMUG February 2015 Knowledge EventASMUG February 2015 Knowledge Event
ASMUG February 2015 Knowledge Eventjmustac
 
Lecture 01 Software Process Maturity.pptx
Lecture 01 Software Process Maturity.pptxLecture 01 Software Process Maturity.pptx
Lecture 01 Software Process Maturity.pptxGetahuntigistu5
 

Ähnlich wie Automated system testing for a learning management system (20)

Computer Based Assessment.pptx
Computer Based Assessment.pptxComputer Based Assessment.pptx
Computer Based Assessment.pptx
 
Quality Assurance in Modern Software Development
Quality Assurance in Modern Software DevelopmentQuality Assurance in Modern Software Development
Quality Assurance in Modern Software Development
 
Project Management for Information System Development
Project Management for Information System DevelopmentProject Management for Information System Development
Project Management for Information System Development
 
Building information systems
Building information systemsBuilding information systems
Building information systems
 
Software Development Methodologies
Software Development MethodologiesSoftware Development Methodologies
Software Development Methodologies
 
SDLC 21.11.2022.pdf
SDLC 21.11.2022.pdfSDLC 21.11.2022.pdf
SDLC 21.11.2022.pdf
 
Development of admission management system
Development of admission management systemDevelopment of admission management system
Development of admission management system
 
OOAD - System Analysis and Design
OOAD - System Analysis and Design OOAD - System Analysis and Design
OOAD - System Analysis and Design
 
Usability evaluation methods (part 2) and performance metrics
Usability evaluation methods (part 2) and performance metricsUsability evaluation methods (part 2) and performance metrics
Usability evaluation methods (part 2) and performance metrics
 
A practical guide for using Statistical Tests to assess Randomized Algorithms...
A practical guide for using Statistical Tests to assess Randomized Algorithms...A practical guide for using Statistical Tests to assess Randomized Algorithms...
A practical guide for using Statistical Tests to assess Randomized Algorithms...
 
manajemen komputer dan sistem informasi.
manajemen komputer dan sistem informasi.manajemen komputer dan sistem informasi.
manajemen komputer dan sistem informasi.
 
Structured NERC CIP Process Improvement Using Six Sigma
Structured NERC CIP Process Improvement Using Six SigmaStructured NERC CIP Process Improvement Using Six Sigma
Structured NERC CIP Process Improvement Using Six Sigma
 
Governance Strategies for Office 365
Governance Strategies for Office 365Governance Strategies for Office 365
Governance Strategies for Office 365
 
Software Testing - Introduction
Software Testing - IntroductionSoftware Testing - Introduction
Software Testing - Introduction
 
Pressman ch-22-process-and-project-metrics
Pressman ch-22-process-and-project-metricsPressman ch-22-process-and-project-metrics
Pressman ch-22-process-and-project-metrics
 
1 sad-01-introduction-june2015-rev
1 sad-01-introduction-june2015-rev1 sad-01-introduction-june2015-rev
1 sad-01-introduction-june2015-rev
 
7 Tips from Siemens Energy for Success with Automation
7 Tips from Siemens Energy for Success with Automation7 Tips from Siemens Energy for Success with Automation
7 Tips from Siemens Energy for Success with Automation
 
ASMUG February 2015 Knowledge Event
ASMUG February 2015 Knowledge EventASMUG February 2015 Knowledge Event
ASMUG February 2015 Knowledge Event
 
Lecture3
Lecture3Lecture3
Lecture3
 
Lecture 01 Software Process Maturity.pptx
Lecture 01 Software Process Maturity.pptxLecture 01 Software Process Maturity.pptx
Lecture 01 Software Process Maturity.pptx
 

Mehr von Educational Technology

The use of programming tasks in interactive videos to increase learning effec...
The use of programming tasks in interactive videos to increase learning effec...The use of programming tasks in interactive videos to increase learning effec...
The use of programming tasks in interactive videos to increase learning effec...Educational Technology
 
Analysis of students' behavior watching iMooX courses with interactive elements
Analysis of students' behavior watching iMooX courses with interactive elementsAnalysis of students' behavior watching iMooX courses with interactive elements
Analysis of students' behavior watching iMooX courses with interactive elementsEducational Technology
 
Erhebung von Lernaktivitäten in einem Pop-Up-Makerspace mit einer technischen...
Erhebung von Lernaktivitäten in einem Pop-Up-Makerspace mit einer technischen...Erhebung von Lernaktivitäten in einem Pop-Up-Makerspace mit einer technischen...
Erhebung von Lernaktivitäten in einem Pop-Up-Makerspace mit einer technischen...Educational Technology
 
Mixed Reality im Distance Learning in der Hochschullehre
Mixed Reality im Distance Learning in der HochschullehreMixed Reality im Distance Learning in der Hochschullehre
Mixed Reality im Distance Learning in der HochschullehreEducational Technology
 
Development of a WCAG theme for a learning management system
Development of a WCAG theme for a learning management systemDevelopment of a WCAG theme for a learning management system
Development of a WCAG theme for a learning management systemEducational Technology
 
Math trainer as a chatbot via system(push) messages for Android
Math trainer as a chatbot via system(push) messages for AndroidMath trainer as a chatbot via system(push) messages for Android
Math trainer as a chatbot via system(push) messages for AndroidEducational Technology
 
Empirical Analysis of Automated Editing of Raw Learning Video Footage
Empirical Analysis of Automated Editing of Raw Learning Video FootageEmpirical Analysis of Automated Editing of Raw Learning Video Footage
Empirical Analysis of Automated Editing of Raw Learning Video FootageEducational Technology
 
Fächerintegrativer Unterricht am Beispiel der Leichtathletik
Fächerintegrativer Unterricht am Beispiel der LeichtathletikFächerintegrativer Unterricht am Beispiel der Leichtathletik
Fächerintegrativer Unterricht am Beispiel der LeichtathletikEducational Technology
 
DENKEN UND TECHNIK Über manipulative Auswirkungen von Internettechnologien
DENKEN UND TECHNIK Über manipulative Auswirkungen von InternettechnologienDENKEN UND TECHNIK Über manipulative Auswirkungen von Internettechnologien
DENKEN UND TECHNIK Über manipulative Auswirkungen von InternettechnologienEducational Technology
 
Empfehlungen für den Unterricht im Fach Informatik für Menschen mit Autismus-...
Empfehlungen für den Unterricht im Fach Informatik für Menschen mit Autismus-...Empfehlungen für den Unterricht im Fach Informatik für Menschen mit Autismus-...
Empfehlungen für den Unterricht im Fach Informatik für Menschen mit Autismus-...Educational Technology
 
Entwicklung eines Online-Kurses für digitale Kompetenzen für Studienanfänger:...
Entwicklung eines Online-Kurses für digitale Kompetenzen für Studienanfänger:...Entwicklung eines Online-Kurses für digitale Kompetenzen für Studienanfänger:...
Entwicklung eines Online-Kurses für digitale Kompetenzen für Studienanfänger:...Educational Technology
 
Development of a mobile French language learning platform
Development of a mobile French language learning platformDevelopment of a mobile French language learning platform
Development of a mobile French language learning platformEducational Technology
 
Learning Analytics and Spelling Acquisition in German - the Path to Indivdual...
Learning Analytics and Spelling Acquisition in German - the Path to Indivdual...Learning Analytics and Spelling Acquisition in German - the Path to Indivdual...
Learning Analytics and Spelling Acquisition in German - the Path to Indivdual...Educational Technology
 
Fächerintegrativer Unterricht am Beispiel des Lernroboters Thymio
Fächerintegrativer Unterricht am Beispiel des Lernroboters ThymioFächerintegrativer Unterricht am Beispiel des Lernroboters Thymio
Fächerintegrativer Unterricht am Beispiel des Lernroboters ThymioEducational Technology
 
Einsatz von Mixed Reality im Klassenzimmer
Einsatz von Mixed Reality im KlassenzimmerEinsatz von Mixed Reality im Klassenzimmer
Einsatz von Mixed Reality im KlassenzimmerEducational Technology
 
Chatbots for Brand Representation in Comparison with Traditional Websites
Chatbots for Brand Representation in Comparison with Traditional WebsitesChatbots for Brand Representation in Comparison with Traditional Websites
Chatbots for Brand Representation in Comparison with Traditional WebsitesEducational Technology
 
Development of a learning diary for a MOOC platform
Development of a learning diary for a MOOC platformDevelopment of a learning diary for a MOOC platform
Development of a learning diary for a MOOC platformEducational Technology
 

Mehr von Educational Technology (20)

The use of programming tasks in interactive videos to increase learning effec...
The use of programming tasks in interactive videos to increase learning effec...The use of programming tasks in interactive videos to increase learning effec...
The use of programming tasks in interactive videos to increase learning effec...
 
Analysis of students' behavior watching iMooX courses with interactive elements
Analysis of students' behavior watching iMooX courses with interactive elementsAnalysis of students' behavior watching iMooX courses with interactive elements
Analysis of students' behavior watching iMooX courses with interactive elements
 
Portability of Mobile Applications
Portability of Mobile ApplicationsPortability of Mobile Applications
Portability of Mobile Applications
 
Erhebung von Lernaktivitäten in einem Pop-Up-Makerspace mit einer technischen...
Erhebung von Lernaktivitäten in einem Pop-Up-Makerspace mit einer technischen...Erhebung von Lernaktivitäten in einem Pop-Up-Makerspace mit einer technischen...
Erhebung von Lernaktivitäten in einem Pop-Up-Makerspace mit einer technischen...
 
Mixed Reality im Distance Learning in der Hochschullehre
Mixed Reality im Distance Learning in der HochschullehreMixed Reality im Distance Learning in der Hochschullehre
Mixed Reality im Distance Learning in der Hochschullehre
 
Development of a WCAG theme for a learning management system
Development of a WCAG theme for a learning management systemDevelopment of a WCAG theme for a learning management system
Development of a WCAG theme for a learning management system
 
Math trainer as a chatbot via system(push) messages for Android
Math trainer as a chatbot via system(push) messages for AndroidMath trainer as a chatbot via system(push) messages for Android
Math trainer as a chatbot via system(push) messages for Android
 
Empirical Analysis of Automated Editing of Raw Learning Video Footage
Empirical Analysis of Automated Editing of Raw Learning Video FootageEmpirical Analysis of Automated Editing of Raw Learning Video Footage
Empirical Analysis of Automated Editing of Raw Learning Video Footage
 
Fächerintegrativer Unterricht am Beispiel der Leichtathletik
Fächerintegrativer Unterricht am Beispiel der LeichtathletikFächerintegrativer Unterricht am Beispiel der Leichtathletik
Fächerintegrativer Unterricht am Beispiel der Leichtathletik
 
DENKEN UND TECHNIK Über manipulative Auswirkungen von Internettechnologien
DENKEN UND TECHNIK Über manipulative Auswirkungen von InternettechnologienDENKEN UND TECHNIK Über manipulative Auswirkungen von Internettechnologien
DENKEN UND TECHNIK Über manipulative Auswirkungen von Internettechnologien
 
Empfehlungen für den Unterricht im Fach Informatik für Menschen mit Autismus-...
Empfehlungen für den Unterricht im Fach Informatik für Menschen mit Autismus-...Empfehlungen für den Unterricht im Fach Informatik für Menschen mit Autismus-...
Empfehlungen für den Unterricht im Fach Informatik für Menschen mit Autismus-...
 
Entwicklung eines Online-Kurses für digitale Kompetenzen für Studienanfänger:...
Entwicklung eines Online-Kurses für digitale Kompetenzen für Studienanfänger:...Entwicklung eines Online-Kurses für digitale Kompetenzen für Studienanfänger:...
Entwicklung eines Online-Kurses für digitale Kompetenzen für Studienanfänger:...
 
School Start Screening Tool
School Start Screening ToolSchool Start Screening Tool
School Start Screening Tool
 
Development of a mobile French language learning platform
Development of a mobile French language learning platformDevelopment of a mobile French language learning platform
Development of a mobile French language learning platform
 
Learning Analytics and Spelling Acquisition in German - the Path to Indivdual...
Learning Analytics and Spelling Acquisition in German - the Path to Indivdual...Learning Analytics and Spelling Acquisition in German - the Path to Indivdual...
Learning Analytics and Spelling Acquisition in German - the Path to Indivdual...
 
Learning Analytics and MOOCs
Learning Analytics and MOOCsLearning Analytics and MOOCs
Learning Analytics and MOOCs
 
Fächerintegrativer Unterricht am Beispiel des Lernroboters Thymio
Fächerintegrativer Unterricht am Beispiel des Lernroboters ThymioFächerintegrativer Unterricht am Beispiel des Lernroboters Thymio
Fächerintegrativer Unterricht am Beispiel des Lernroboters Thymio
 
Einsatz von Mixed Reality im Klassenzimmer
Einsatz von Mixed Reality im KlassenzimmerEinsatz von Mixed Reality im Klassenzimmer
Einsatz von Mixed Reality im Klassenzimmer
 
Chatbots for Brand Representation in Comparison with Traditional Websites
Chatbots for Brand Representation in Comparison with Traditional WebsitesChatbots for Brand Representation in Comparison with Traditional Websites
Chatbots for Brand Representation in Comparison with Traditional Websites
 
Development of a learning diary for a MOOC platform
Development of a learning diary for a MOOC platformDevelopment of a learning diary for a MOOC platform
Development of a learning diary for a MOOC platform
 

Kürzlich hochgeladen

Indexing Structures in Database Management system.pdf
Indexing Structures in Database Management system.pdfIndexing Structures in Database Management system.pdf
Indexing Structures in Database Management system.pdfChristalin Nelson
 
ICS 2208 Lecture Slide Notes for Topic 6
ICS 2208 Lecture Slide Notes for Topic 6ICS 2208 Lecture Slide Notes for Topic 6
ICS 2208 Lecture Slide Notes for Topic 6Vanessa Camilleri
 
Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4JOYLYNSAMANIEGO
 
Mythology Quiz-4th April 2024, Quiz Club NITW
Mythology Quiz-4th April 2024, Quiz Club NITWMythology Quiz-4th April 2024, Quiz Club NITW
Mythology Quiz-4th April 2024, Quiz Club NITWQuiz Club NITW
 
Oppenheimer Film Discussion for Philosophy and Film
Oppenheimer Film Discussion for Philosophy and FilmOppenheimer Film Discussion for Philosophy and Film
Oppenheimer Film Discussion for Philosophy and FilmStan Meyer
 
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...DhatriParmar
 
Active Learning Strategies (in short ALS).pdf
Active Learning Strategies (in short ALS).pdfActive Learning Strategies (in short ALS).pdf
Active Learning Strategies (in short ALS).pdfPatidar M
 
MS4 level being good citizen -imperative- (1) (1).pdf
MS4 level   being good citizen -imperative- (1) (1).pdfMS4 level   being good citizen -imperative- (1) (1).pdf
MS4 level being good citizen -imperative- (1) (1).pdfMr Bounab Samir
 
ICS2208 Lecture6 Notes for SL spaces.pdf
ICS2208 Lecture6 Notes for SL spaces.pdfICS2208 Lecture6 Notes for SL spaces.pdf
ICS2208 Lecture6 Notes for SL spaces.pdfVanessa Camilleri
 
Grade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptxGrade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptxkarenfajardo43
 
DIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptx
DIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptxDIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptx
DIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptxMichelleTuguinay1
 
CLASSIFICATION OF ANTI - CANCER DRUGS.pptx
CLASSIFICATION OF ANTI - CANCER DRUGS.pptxCLASSIFICATION OF ANTI - CANCER DRUGS.pptx
CLASSIFICATION OF ANTI - CANCER DRUGS.pptxAnupam32727
 
CHEST Proprioceptive neuromuscular facilitation.pptx
CHEST Proprioceptive neuromuscular facilitation.pptxCHEST Proprioceptive neuromuscular facilitation.pptx
CHEST Proprioceptive neuromuscular facilitation.pptxAneriPatwari
 
4.16.24 Poverty and Precarity--Desmond.pptx
4.16.24 Poverty and Precarity--Desmond.pptx4.16.24 Poverty and Precarity--Desmond.pptx
4.16.24 Poverty and Precarity--Desmond.pptxmary850239
 
Q-Factor General Quiz-7th April 2024, Quiz Club NITW
Q-Factor General Quiz-7th April 2024, Quiz Club NITWQ-Factor General Quiz-7th April 2024, Quiz Club NITW
Q-Factor General Quiz-7th April 2024, Quiz Club NITWQuiz Club NITW
 
Reading and Writing Skills 11 quarter 4 melc 1
Reading and Writing Skills 11 quarter 4 melc 1Reading and Writing Skills 11 quarter 4 melc 1
Reading and Writing Skills 11 quarter 4 melc 1GloryAnnCastre1
 
ARTERIAL BLOOD GAS ANALYSIS........pptx
ARTERIAL BLOOD  GAS ANALYSIS........pptxARTERIAL BLOOD  GAS ANALYSIS........pptx
ARTERIAL BLOOD GAS ANALYSIS........pptxAneriPatwari
 

Kürzlich hochgeladen (20)

Indexing Structures in Database Management system.pdf
Indexing Structures in Database Management system.pdfIndexing Structures in Database Management system.pdf
Indexing Structures in Database Management system.pdf
 
ICS 2208 Lecture Slide Notes for Topic 6
ICS 2208 Lecture Slide Notes for Topic 6ICS 2208 Lecture Slide Notes for Topic 6
ICS 2208 Lecture Slide Notes for Topic 6
 
Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4
 
Mythology Quiz-4th April 2024, Quiz Club NITW
Mythology Quiz-4th April 2024, Quiz Club NITWMythology Quiz-4th April 2024, Quiz Club NITW
Mythology Quiz-4th April 2024, Quiz Club NITW
 
Oppenheimer Film Discussion for Philosophy and Film
Oppenheimer Film Discussion for Philosophy and FilmOppenheimer Film Discussion for Philosophy and Film
Oppenheimer Film Discussion for Philosophy and Film
 
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...
Beauty Amidst the Bytes_ Unearthing Unexpected Advantages of the Digital Wast...
 
Active Learning Strategies (in short ALS).pdf
Active Learning Strategies (in short ALS).pdfActive Learning Strategies (in short ALS).pdf
Active Learning Strategies (in short ALS).pdf
 
MS4 level being good citizen -imperative- (1) (1).pdf
MS4 level   being good citizen -imperative- (1) (1).pdfMS4 level   being good citizen -imperative- (1) (1).pdf
MS4 level being good citizen -imperative- (1) (1).pdf
 
ICS2208 Lecture6 Notes for SL spaces.pdf
ICS2208 Lecture6 Notes for SL spaces.pdfICS2208 Lecture6 Notes for SL spaces.pdf
ICS2208 Lecture6 Notes for SL spaces.pdf
 
Grade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptxGrade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptx
 
prashanth updated resume 2024 for Teaching Profession
prashanth updated resume 2024 for Teaching Professionprashanth updated resume 2024 for Teaching Profession
prashanth updated resume 2024 for Teaching Profession
 
DIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptx
DIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptxDIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptx
DIFFERENT BASKETRY IN THE PHILIPPINES PPT.pptx
 
CLASSIFICATION OF ANTI - CANCER DRUGS.pptx
CLASSIFICATION OF ANTI - CANCER DRUGS.pptxCLASSIFICATION OF ANTI - CANCER DRUGS.pptx
CLASSIFICATION OF ANTI - CANCER DRUGS.pptx
 
Paradigm shift in nursing research by RS MEHTA
Paradigm shift in nursing research by RS MEHTAParadigm shift in nursing research by RS MEHTA
Paradigm shift in nursing research by RS MEHTA
 
CHEST Proprioceptive neuromuscular facilitation.pptx
CHEST Proprioceptive neuromuscular facilitation.pptxCHEST Proprioceptive neuromuscular facilitation.pptx
CHEST Proprioceptive neuromuscular facilitation.pptx
 
4.16.24 Poverty and Precarity--Desmond.pptx
4.16.24 Poverty and Precarity--Desmond.pptx4.16.24 Poverty and Precarity--Desmond.pptx
4.16.24 Poverty and Precarity--Desmond.pptx
 
Q-Factor General Quiz-7th April 2024, Quiz Club NITW
Q-Factor General Quiz-7th April 2024, Quiz Club NITWQ-Factor General Quiz-7th April 2024, Quiz Club NITW
Q-Factor General Quiz-7th April 2024, Quiz Club NITW
 
Reading and Writing Skills 11 quarter 4 melc 1
Reading and Writing Skills 11 quarter 4 melc 1Reading and Writing Skills 11 quarter 4 melc 1
Reading and Writing Skills 11 quarter 4 melc 1
 
Faculty Profile prashantha K EEE dept Sri Sairam college of Engineering
Faculty Profile prashantha K EEE dept Sri Sairam college of EngineeringFaculty Profile prashantha K EEE dept Sri Sairam college of Engineering
Faculty Profile prashantha K EEE dept Sri Sairam college of Engineering
 
ARTERIAL BLOOD GAS ANALYSIS........pptx
ARTERIAL BLOOD  GAS ANALYSIS........pptxARTERIAL BLOOD  GAS ANALYSIS........pptx
ARTERIAL BLOOD GAS ANALYSIS........pptx
 

Automated system testing for a learning management system

  • 1. WISSEN TECHNIK LEIDENSCHAFT Automated system testing for a learning management system 31.10.2019 Lukas Krisper
  • 2. Agenda • Motivation • Test Automation • Test Cases for an LMS • Test Data for an LMS • Implementation and Infrastructure • Conclusion Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 2
  • 3. Agenda • Motivation • Test Automation • Test Cases for an LMS • Test Data for an LMS • Implementation and Infrastructure • Conclusion Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 3
  • 4. A shift towards Agile Software Development • Over the last two decades … • Emerge of agile methodologies in software development • Acceleration of software delivery and feedback cycles • Automation of manual processes • Quality Assurance? • Only 20% of IT organisations have automated more than half of their test cases (State of Testing Report, 2018) Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 4 Motivation
  • 5. New Releases in Summer 2019 • TeachCenter 3.0 • Learning Management System (LMS) • Migration of whole LMS to new Moodle Version (3.5) • New GUI for all users • Set of Unit-Tests existed • Sparely documented manual tests Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 5 Motivation
  • 6. Research Questions • (RQ1) How can the existing core functionality of a large software system be assured automatically in new versions of the system? • (RQ2) Which test cases and test suites need to be designed to test a learning management system effectively? • (RQ3) Which test data is needed to test a learning management system and how can the data be provided to automated test cases? Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 6 Motivation
  • 7. Agenda • Motivation • Test Automation • Test Cases for an LMS • Test Data for an LMS • Implementation and Infrastructure • Conclusion Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 7
  • 8. Focus of Automation Activities • Software Quality Characteristic*: Functional Suitability • Testing Level: System Testing • Testing Type: Regression Tests Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 8 Test Automation *According to ISO-25010
  • 9. Software Quality Characteristic: Functional Suitability Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 9 Test Automation • Does the software product do what it is supposed to do? • Functional testing as basis of all testing activities • A software product that does not provide functional suitability is rather worthless • E.g. calculator that calculates very fast and has a nice UI, but the calculations are wrong 2 + 2 = 5
  • 10. Testing Level: System Testing Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 10 Test Automation • Software product is considered in its entirety • Test environment has close resemblance to production environment • Tests conducted from user’s perspective • Typically using the systems GUI for interaction • Automated system tests as second line of defense • If tests fail at this level → malfunctioning system + wrong or missing test at lower testing level
  • 11. Testing Type: Regression Tests Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 11 Test Automation • Is already existing functionality affected by the changes applied to a software? • The first type of tests to be considered when automating tests • Quickly run a set of tests after applying changes: • Extensions to a feature • Bug fixes • Refactorings • … • Especially useful when changes happen late in the project and time for manual testing is short
  • 12. Agenda • Motivation • Test Automation • Test Cases for an LMS • Test Data for an LMS • Implementation and Infrastructure • Conclusion Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 12
  • 13. What Test Cases should be automated? • Important consideration! • Interview with first level support of TeachCenter 2.0 • Research in literature on usages of LMS • Results: • Knowledge on most important use cases for students and teachers E.g. reading the course’s contents, communicating with other participants, submitting material to the course, … • > 30 test cases combined to different test suites Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 13 Test Cases for a LMS
  • 14. Agenda • Motivation • Test Automation • Test Cases for an LMS • Test Data for an LMS • Implementation and Infrastructure • Conclusion Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 14
  • 15. Real Data vs Synthetic Data • Real Data • Transfer data from production environment to test environment • Data has to be edited (according to laws on data protection/privacy*) • Important: Keep consistency in data set! • Synthetic Data • Newly created data for the purpose of testing • Always GDPR compliant Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 15 Test Data for a LMS * e.g. General Data Protection Regulation (GDPR)
  • 16. Data for a LMS • A LMS typically contains data on • Courses (course descriptions, learning material, …) • Data on people involved in the courses (names, email-addresses, …) • Data from interaction of people with the courses (submissions, …) • TeachCenter 3.0 is tested with both synthetic and real data • Data used for automated tests is labeled as such Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 16 Test Data for a LMS
  • 17. Agenda • Motivation • Test Automation • Test Cases for an LMS • Test Data for an LMS • Implementation and Infrastructure • Conclusion Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 17
  • 18. Overview on Infrastructure Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 18 Implementation and Infrastructure
  • 19. Test Suite 1 Test Case 1 Test Case n Test Case 2 … Test Suite 1 Test Case 1 Test Case n Test Case 2 … Overview on Implementation Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 19 Implementation and Infrastructure Test Suite 1 Test Case 1 Test Case n Test Case 2 … Test Data Manager Test Driver Utility e.g. initialise headless Chrome browser with cleared cache e.g. get PDF files for submissions, references to test data stored in JSON files, … Page Object 1Page Object 1Page Object 1 TestNG Page Object 1Page Object 1 Helper Classes e.g. taking screenshots on errors See following slides
  • 20. PageObject Design Pattern Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 20 Implementation and Infrastructure • Components of a website are modelled into reusable objects → “Page Objects” • Page Objects offer functionality to interact with website (click links, enter input fields, …) • Tests use Page Objects to interact with website (perform action → assert result of action)
  • 21. Identification of Objects in UI Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 21 Implementation and Infrastructure • Tests should be executable regardless … • screen resolution • window sizes • language settings • Identification of objects using stable features (IDs or XPath expressions) DOM Element <button type= "button" id= "btn1" data-role= "next "> Next </button > ID btn1 XPath expression //button[@data-role=’next’]
  • 22. Agenda • Motivation • Test Automation • Test Cases for an LMS • Test Data for an LMS • Implementation and Infrastructure • Conclusion Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 22
  • 23. (RQ1) How can the existing core functionality of a large software system be assured automatically in new versions of the system? • Regression Tests for „Functional Suitabilty“ • System Testing Level • Large software system = many integrated components that interact with each other → “high” testing level to ensure stability and detect side effects • Continuous Integration Environment (Automation Server) Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 23 Conclusion
  • 24. (RQ2) Which test cases and test suites need to be designed to test a learning management system effectively? • Identification of use cases • Interview with 1st level support + literature research • Prioritisation of use cases • Implementation of test cases based on use case • Execution of suites of test cases according to prioritisation of use cases • Multiple times a day ~ once a week Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 24 Conclusion
  • 25. (RQ3) Which test data is needed to test a learning management system and how can the data be provided to automated test cases? • Synthetic and real test data • 1 configured course with various activities and resources • 1 student‘s account (enrolled to course) • 1 teacher‘s account (enrolled to course) • 1 PDF file • Test data is persisted on test system (references in JSON-files) … • Or provided to the test cases on demand • Separation of test data and automated test cases • No hardcoded test data Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 25 Conclusion
  • 26. Thank you for your attention! Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 26 Conclusion