SlideShare ist ein Scribd-Unternehmen logo
1 von 15
Using Kirkpatrick Model and Success Case Model.
Majd Al Rammah and Kathryn E. Patrick
NITTP Objective:
Teach students a different aspect of Nissan/Infiniti new vehicles
technology.
Evaluation:
Conducted through Kirkpatrick model.
Level one: Reaction
The reaction was positive, we knew this after using different
methods to evaluate how much the learners were satisfied with
the program, methods such as surveys, questionnaires and direct
contact.
The most important thing the learners mentioned was: we
enjoyed the combination of seven and a half weeks of lessons
followed by seven and a half weeks of practicing the skills that we
learned.
Level two: Learning
In the first level, we didn’t measure the skills acquired, but
in this level, our aim was to compare the actual skills with
the skills the learners had before the program, the results
came out positively upon the tests, now they are able to
repair Nissan and Infinity cars in an outstanding time
records and less mistakes.
Level three: Behavior
As we noticed during the implementation of the
program/training that learners were comfortable with
the process. Their behavior was outstanding because
they were helping each other through the
transformation of knowledge. The high quality in
performance reflected how effective the skills learned.
Level four: Results
ROI or the Crops was unexpected, the high quality
performance in the repairing service that is due to the
learning program resulted an increase in sales, easy
and simple, customers became satisfied due to the
good repairing service and this help growing a good
reputation, good reputation means trust, and trust
means more customers
NITTP Objective:
Teach students a different aspect of Nissan/Infiniti new vehicles
technology.
Evaluation:
Conducted through the success-case model.
Most Successful:
The most successful students of the NITTP reported
that they used the skills learned every day in their jobs,
allowing them to work more efficiently and effectively.
The results are proving highly valuable—fixing more
cars with fewer mistakes each day increases profit, and
the increased reputation is bringing in more business.
Most Successful:
The successful students reported that they were able to
implement what they learned because of the effective
hands-on teaching in the second half of the program. They
reported that they reviewed what they had learned in the
first seven and a half weeks before each hands-on session,
so that they would get the most out of the program. They
also reported that their workplaces followed the standards
and procedures suggested by NITTP.
Least Successful:
The least successful students reported that they did
not make much use of the training they received. They
did not notice any difference in their work on
Nissan/Infiniti vehicles, and did not think that the
program had provided a valuable result.
Least Successful:
The unsuccessful students reported that they were unable
to implement what they learned because the training was
too specific. Their workplaces service a wide variety of
vehicles, meaning that they are not always working on
Nissans and Infinities, or they work on models that are
older than was covered by the training. They also reported
that their workplaces had very different operating
procedures than the NITTP had prepared them for, and
that they were held to different standards.
Comparison of the Kirkpatrick Model and the Success
Case Method
Both methods:
•Use surveys, questionnaires, and direct
communication to gather information.
•Measure results, including changes in behavior.
•Measure students’ value gained from the training.
•Allow a trainer to determine what is and is not
working in their program.
Contrast of the Kirkpatrick Model and the Success
Case Method
Kirkpatrick Model
•Four “levels” divided by
what is examined
•Conclusions from data
•Focuses on positive
aspects
Success Case Method
•Two parts divided by
who is examined
•Narrative data
•Shows positive and
negative aspects

Weitere ähnliche Inhalte

Ähnlich wie Al Rammah and Patrick ch10

AkdjcijdjcBusiness card 8 Dec 2023.pdfhjdhbcjkewhcjkejckljecjdj
AkdjcijdjcBusiness card 8 Dec 2023.pdfhjdhbcjkewhcjkejckljecjdjAkdjcijdjcBusiness card 8 Dec 2023.pdfhjdhbcjkewhcjkejckljecjdj
AkdjcijdjcBusiness card 8 Dec 2023.pdfhjdhbcjkewhcjkejckljecjdjRagaviS16
 
Kirkpatrick 4 level evaluation model
Kirkpatrick 4 level evaluation modelKirkpatrick 4 level evaluation model
Kirkpatrick 4 level evaluation modelzhumin
 
Measuring learning impact & effectiveness
Measuring learning impact & effectivenessMeasuring learning impact & effectiveness
Measuring learning impact & effectivenessJoko Suseno
 
evaluation of program.pptx
evaluation of program.pptxevaluation of program.pptx
evaluation of program.pptxMahwishBukhari3
 
Training evaluation models
Training evaluation modelsTraining evaluation models
Training evaluation modelsMegha Anilkumar
 
Customer Service detailed overview
Customer Service detailed overviewCustomer Service detailed overview
Customer Service detailed overviewNick Pascoe
 
Internship in Georgia 2022 - INKOMPASS
Internship in Georgia 2022 - INKOMPASSInternship in Georgia 2022 - INKOMPASS
Internship in Georgia 2022 - INKOMPASSINKOMPASS
 
Instructional module training plan i, ii, & iii
Instructional module training plan i, ii, & iiiInstructional module training plan i, ii, & iii
Instructional module training plan i, ii, & iiiEmma Avila
 
ROI on Learning and Development _long_dec_6_2007
ROI on Learning and Development _long_dec_6_2007ROI on Learning and Development _long_dec_6_2007
ROI on Learning and Development _long_dec_6_2007Ricky Kujawa
 
Evaluation of training
Evaluation of trainingEvaluation of training
Evaluation of trainingSelf-employed
 
anjana kumari project.pdf
anjana kumari project.pdfanjana kumari project.pdf
anjana kumari project.pdfAnjanaKumari96
 
Interviews Module 05 – Live ChatEvaluating Tr.docx
Interviews  Module 05 – Live ChatEvaluating Tr.docxInterviews  Module 05 – Live ChatEvaluating Tr.docx
Interviews Module 05 – Live ChatEvaluating Tr.docxnormanibarber20063
 
Return on Learning from Every Angle: ROI and the Triple Bottom Line of Learning
Return on Learning from Every Angle: ROI and the Triple Bottom Line of LearningReturn on Learning from Every Angle: ROI and the Triple Bottom Line of Learning
Return on Learning from Every Angle: ROI and the Triple Bottom Line of LearningAggregage
 

Ähnlich wie Al Rammah and Patrick ch10 (20)

AkdjcijdjcBusiness card 8 Dec 2023.pdfhjdhbcjkewhcjkejckljecjdj
AkdjcijdjcBusiness card 8 Dec 2023.pdfhjdhbcjkewhcjkejckljecjdjAkdjcijdjcBusiness card 8 Dec 2023.pdfhjdhbcjkewhcjkejckljecjdj
AkdjcijdjcBusiness card 8 Dec 2023.pdfhjdhbcjkewhcjkejckljecjdj
 
Kirkpatrick Evaluation Model.ppt
Kirkpatrick Evaluation Model.pptKirkpatrick Evaluation Model.ppt
Kirkpatrick Evaluation Model.ppt
 
Kirkpatrick 4 level evaluation model
Kirkpatrick 4 level evaluation modelKirkpatrick 4 level evaluation model
Kirkpatrick 4 level evaluation model
 
Measuring learning impact & effectiveness
Measuring learning impact & effectivenessMeasuring learning impact & effectiveness
Measuring learning impact & effectiveness
 
evaluation of program.pptx
evaluation of program.pptxevaluation of program.pptx
evaluation of program.pptx
 
Unique file 8
Unique file 8Unique file 8
Unique file 8
 
Training Evaluation
Training EvaluationTraining Evaluation
Training Evaluation
 
MED07_joycepagkatipunan.pdf
MED07_joycepagkatipunan.pdfMED07_joycepagkatipunan.pdf
MED07_joycepagkatipunan.pdf
 
4DX and Mobile Learning
4DX and Mobile Learning4DX and Mobile Learning
4DX and Mobile Learning
 
Training evaluation models
Training evaluation modelsTraining evaluation models
Training evaluation models
 
Customer Service detailed overview
Customer Service detailed overviewCustomer Service detailed overview
Customer Service detailed overview
 
Training- Classroom Vs Online
Training- Classroom Vs OnlineTraining- Classroom Vs Online
Training- Classroom Vs Online
 
Training-Online Vs Offline
Training-Online Vs OfflineTraining-Online Vs Offline
Training-Online Vs Offline
 
Internship in Georgia 2022 - INKOMPASS
Internship in Georgia 2022 - INKOMPASSInternship in Georgia 2022 - INKOMPASS
Internship in Georgia 2022 - INKOMPASS
 
Instructional module training plan i, ii, & iii
Instructional module training plan i, ii, & iiiInstructional module training plan i, ii, & iii
Instructional module training plan i, ii, & iii
 
ROI on Learning and Development _long_dec_6_2007
ROI on Learning and Development _long_dec_6_2007ROI on Learning and Development _long_dec_6_2007
ROI on Learning and Development _long_dec_6_2007
 
Evaluation of training
Evaluation of trainingEvaluation of training
Evaluation of training
 
anjana kumari project.pdf
anjana kumari project.pdfanjana kumari project.pdf
anjana kumari project.pdf
 
Interviews Module 05 – Live ChatEvaluating Tr.docx
Interviews  Module 05 – Live ChatEvaluating Tr.docxInterviews  Module 05 – Live ChatEvaluating Tr.docx
Interviews Module 05 – Live ChatEvaluating Tr.docx
 
Return on Learning from Every Angle: ROI and the Triple Bottom Line of Learning
Return on Learning from Every Angle: ROI and the Triple Bottom Line of LearningReturn on Learning from Every Angle: ROI and the Triple Bottom Line of Learning
Return on Learning from Every Angle: ROI and the Triple Bottom Line of Learning
 

Mehr von kpatric1

802 presentation
802 presentation802 presentation
802 presentationkpatric1
 
849 research paper
849 research paper849 research paper
849 research paperkpatric1
 
837 lesson
837 lesson837 lesson
837 lessonkpatric1
 
815 - technology plan
815 - technology plan815 - technology plan
815 - technology plankpatric1
 
805 strategic plan
805 strategic plan805 strategic plan
805 strategic plankpatric1
 
813 client diagnosis
813 client diagnosis813 client diagnosis
813 client diagnosiskpatric1
 
810 research proposal
810 research proposal810 research proposal
810 research proposalkpatric1
 
804 annotated bibliography
804 annotated bibliography804 annotated bibliography
804 annotated bibliographykpatric1
 

Mehr von kpatric1 (8)

802 presentation
802 presentation802 presentation
802 presentation
 
849 research paper
849 research paper849 research paper
849 research paper
 
837 lesson
837 lesson837 lesson
837 lesson
 
815 - technology plan
815 - technology plan815 - technology plan
815 - technology plan
 
805 strategic plan
805 strategic plan805 strategic plan
805 strategic plan
 
813 client diagnosis
813 client diagnosis813 client diagnosis
813 client diagnosis
 
810 research proposal
810 research proposal810 research proposal
810 research proposal
 
804 annotated bibliography
804 annotated bibliography804 annotated bibliography
804 annotated bibliography
 

Kürzlich hochgeladen

Cybersecurity Awareness Training Presentation v2024.03
Cybersecurity Awareness Training Presentation v2024.03Cybersecurity Awareness Training Presentation v2024.03
Cybersecurity Awareness Training Presentation v2024.03DallasHaselhorst
 
8447779800, Low rate Call girls in Shivaji Enclave Delhi NCR
8447779800, Low rate Call girls in Shivaji Enclave Delhi NCR8447779800, Low rate Call girls in Shivaji Enclave Delhi NCR
8447779800, Low rate Call girls in Shivaji Enclave Delhi NCRashishs7044
 
Memorándum de Entendimiento (MoU) entre Codelco y SQM
Memorándum de Entendimiento (MoU) entre Codelco y SQMMemorándum de Entendimiento (MoU) entre Codelco y SQM
Memorándum de Entendimiento (MoU) entre Codelco y SQMVoces Mineras
 
(Best) ENJOY Call Girls in Faridabad Ex | 8377087607
(Best) ENJOY Call Girls in Faridabad Ex | 8377087607(Best) ENJOY Call Girls in Faridabad Ex | 8377087607
(Best) ENJOY Call Girls in Faridabad Ex | 8377087607dollysharma2066
 
Independent Call Girls Andheri Nightlaila 9967584737
Independent Call Girls Andheri Nightlaila 9967584737Independent Call Girls Andheri Nightlaila 9967584737
Independent Call Girls Andheri Nightlaila 9967584737Riya Pathan
 
Organizational Structure Running A Successful Business
Organizational Structure Running A Successful BusinessOrganizational Structure Running A Successful Business
Organizational Structure Running A Successful BusinessSeta Wicaksana
 
Traction part 2 - EOS Model JAX Bridges.
Traction part 2 - EOS Model JAX Bridges.Traction part 2 - EOS Model JAX Bridges.
Traction part 2 - EOS Model JAX Bridges.Anamaria Contreras
 
Guide Complete Set of Residential Architectural Drawings PDF
Guide Complete Set of Residential Architectural Drawings PDFGuide Complete Set of Residential Architectural Drawings PDF
Guide Complete Set of Residential Architectural Drawings PDFChandresh Chudasama
 
Investment in The Coconut Industry by Nancy Cheruiyot
Investment in The Coconut Industry by Nancy CheruiyotInvestment in The Coconut Industry by Nancy Cheruiyot
Investment in The Coconut Industry by Nancy Cheruiyotictsugar
 
8447779800, Low rate Call girls in Kotla Mubarakpur Delhi NCR
8447779800, Low rate Call girls in Kotla Mubarakpur Delhi NCR8447779800, Low rate Call girls in Kotla Mubarakpur Delhi NCR
8447779800, Low rate Call girls in Kotla Mubarakpur Delhi NCRashishs7044
 
Kenya’s Coconut Value Chain by Gatsby Africa
Kenya’s Coconut Value Chain by Gatsby AfricaKenya’s Coconut Value Chain by Gatsby Africa
Kenya’s Coconut Value Chain by Gatsby Africaictsugar
 
Call Us 📲8800102216📞 Call Girls In DLF City Gurgaon
Call Us 📲8800102216📞 Call Girls In DLF City GurgaonCall Us 📲8800102216📞 Call Girls In DLF City Gurgaon
Call Us 📲8800102216📞 Call Girls In DLF City Gurgaoncallgirls2057
 
8447779800, Low rate Call girls in Rohini Delhi NCR
8447779800, Low rate Call girls in Rohini Delhi NCR8447779800, Low rate Call girls in Rohini Delhi NCR
8447779800, Low rate Call girls in Rohini Delhi NCRashishs7044
 
Cyber Security Training in Office Environment
Cyber Security Training in Office EnvironmentCyber Security Training in Office Environment
Cyber Security Training in Office Environmentelijahj01012
 
Annual General Meeting Presentation Slides
Annual General Meeting Presentation SlidesAnnual General Meeting Presentation Slides
Annual General Meeting Presentation SlidesKeppelCorporation
 
Digital Transformation in the PLM domain - distrib.pdf
Digital Transformation in the PLM domain - distrib.pdfDigital Transformation in the PLM domain - distrib.pdf
Digital Transformation in the PLM domain - distrib.pdfJos Voskuil
 
Unlocking the Future: Explore Web 3.0 Workshop to Start Earning Today!
Unlocking the Future: Explore Web 3.0 Workshop to Start Earning Today!Unlocking the Future: Explore Web 3.0 Workshop to Start Earning Today!
Unlocking the Future: Explore Web 3.0 Workshop to Start Earning Today!Doge Mining Website
 
8447779800, Low rate Call girls in Saket Delhi NCR
8447779800, Low rate Call girls in Saket Delhi NCR8447779800, Low rate Call girls in Saket Delhi NCR
8447779800, Low rate Call girls in Saket Delhi NCRashishs7044
 
Market Sizes Sample Report - 2024 Edition
Market Sizes Sample Report - 2024 EditionMarket Sizes Sample Report - 2024 Edition
Market Sizes Sample Report - 2024 EditionMintel Group
 
Global Scenario On Sustainable and Resilient Coconut Industry by Dr. Jelfina...
Global Scenario On Sustainable  and Resilient Coconut Industry by Dr. Jelfina...Global Scenario On Sustainable  and Resilient Coconut Industry by Dr. Jelfina...
Global Scenario On Sustainable and Resilient Coconut Industry by Dr. Jelfina...ictsugar
 

Kürzlich hochgeladen (20)

Cybersecurity Awareness Training Presentation v2024.03
Cybersecurity Awareness Training Presentation v2024.03Cybersecurity Awareness Training Presentation v2024.03
Cybersecurity Awareness Training Presentation v2024.03
 
8447779800, Low rate Call girls in Shivaji Enclave Delhi NCR
8447779800, Low rate Call girls in Shivaji Enclave Delhi NCR8447779800, Low rate Call girls in Shivaji Enclave Delhi NCR
8447779800, Low rate Call girls in Shivaji Enclave Delhi NCR
 
Memorándum de Entendimiento (MoU) entre Codelco y SQM
Memorándum de Entendimiento (MoU) entre Codelco y SQMMemorándum de Entendimiento (MoU) entre Codelco y SQM
Memorándum de Entendimiento (MoU) entre Codelco y SQM
 
(Best) ENJOY Call Girls in Faridabad Ex | 8377087607
(Best) ENJOY Call Girls in Faridabad Ex | 8377087607(Best) ENJOY Call Girls in Faridabad Ex | 8377087607
(Best) ENJOY Call Girls in Faridabad Ex | 8377087607
 
Independent Call Girls Andheri Nightlaila 9967584737
Independent Call Girls Andheri Nightlaila 9967584737Independent Call Girls Andheri Nightlaila 9967584737
Independent Call Girls Andheri Nightlaila 9967584737
 
Organizational Structure Running A Successful Business
Organizational Structure Running A Successful BusinessOrganizational Structure Running A Successful Business
Organizational Structure Running A Successful Business
 
Traction part 2 - EOS Model JAX Bridges.
Traction part 2 - EOS Model JAX Bridges.Traction part 2 - EOS Model JAX Bridges.
Traction part 2 - EOS Model JAX Bridges.
 
Guide Complete Set of Residential Architectural Drawings PDF
Guide Complete Set of Residential Architectural Drawings PDFGuide Complete Set of Residential Architectural Drawings PDF
Guide Complete Set of Residential Architectural Drawings PDF
 
Investment in The Coconut Industry by Nancy Cheruiyot
Investment in The Coconut Industry by Nancy CheruiyotInvestment in The Coconut Industry by Nancy Cheruiyot
Investment in The Coconut Industry by Nancy Cheruiyot
 
8447779800, Low rate Call girls in Kotla Mubarakpur Delhi NCR
8447779800, Low rate Call girls in Kotla Mubarakpur Delhi NCR8447779800, Low rate Call girls in Kotla Mubarakpur Delhi NCR
8447779800, Low rate Call girls in Kotla Mubarakpur Delhi NCR
 
Kenya’s Coconut Value Chain by Gatsby Africa
Kenya’s Coconut Value Chain by Gatsby AfricaKenya’s Coconut Value Chain by Gatsby Africa
Kenya’s Coconut Value Chain by Gatsby Africa
 
Call Us 📲8800102216📞 Call Girls In DLF City Gurgaon
Call Us 📲8800102216📞 Call Girls In DLF City GurgaonCall Us 📲8800102216📞 Call Girls In DLF City Gurgaon
Call Us 📲8800102216📞 Call Girls In DLF City Gurgaon
 
8447779800, Low rate Call girls in Rohini Delhi NCR
8447779800, Low rate Call girls in Rohini Delhi NCR8447779800, Low rate Call girls in Rohini Delhi NCR
8447779800, Low rate Call girls in Rohini Delhi NCR
 
Cyber Security Training in Office Environment
Cyber Security Training in Office EnvironmentCyber Security Training in Office Environment
Cyber Security Training in Office Environment
 
Annual General Meeting Presentation Slides
Annual General Meeting Presentation SlidesAnnual General Meeting Presentation Slides
Annual General Meeting Presentation Slides
 
Digital Transformation in the PLM domain - distrib.pdf
Digital Transformation in the PLM domain - distrib.pdfDigital Transformation in the PLM domain - distrib.pdf
Digital Transformation in the PLM domain - distrib.pdf
 
Unlocking the Future: Explore Web 3.0 Workshop to Start Earning Today!
Unlocking the Future: Explore Web 3.0 Workshop to Start Earning Today!Unlocking the Future: Explore Web 3.0 Workshop to Start Earning Today!
Unlocking the Future: Explore Web 3.0 Workshop to Start Earning Today!
 
8447779800, Low rate Call girls in Saket Delhi NCR
8447779800, Low rate Call girls in Saket Delhi NCR8447779800, Low rate Call girls in Saket Delhi NCR
8447779800, Low rate Call girls in Saket Delhi NCR
 
Market Sizes Sample Report - 2024 Edition
Market Sizes Sample Report - 2024 EditionMarket Sizes Sample Report - 2024 Edition
Market Sizes Sample Report - 2024 Edition
 
Global Scenario On Sustainable and Resilient Coconut Industry by Dr. Jelfina...
Global Scenario On Sustainable  and Resilient Coconut Industry by Dr. Jelfina...Global Scenario On Sustainable  and Resilient Coconut Industry by Dr. Jelfina...
Global Scenario On Sustainable and Resilient Coconut Industry by Dr. Jelfina...
 

Al Rammah and Patrick ch10

  • 1. Using Kirkpatrick Model and Success Case Model. Majd Al Rammah and Kathryn E. Patrick
  • 2. NITTP Objective: Teach students a different aspect of Nissan/Infiniti new vehicles technology. Evaluation: Conducted through Kirkpatrick model.
  • 3. Level one: Reaction The reaction was positive, we knew this after using different methods to evaluate how much the learners were satisfied with the program, methods such as surveys, questionnaires and direct contact. The most important thing the learners mentioned was: we enjoyed the combination of seven and a half weeks of lessons followed by seven and a half weeks of practicing the skills that we learned.
  • 4. Level two: Learning In the first level, we didn’t measure the skills acquired, but in this level, our aim was to compare the actual skills with the skills the learners had before the program, the results came out positively upon the tests, now they are able to repair Nissan and Infinity cars in an outstanding time records and less mistakes.
  • 5. Level three: Behavior As we noticed during the implementation of the program/training that learners were comfortable with the process. Their behavior was outstanding because they were helping each other through the transformation of knowledge. The high quality in performance reflected how effective the skills learned.
  • 6. Level four: Results ROI or the Crops was unexpected, the high quality performance in the repairing service that is due to the learning program resulted an increase in sales, easy and simple, customers became satisfied due to the good repairing service and this help growing a good reputation, good reputation means trust, and trust means more customers
  • 7. NITTP Objective: Teach students a different aspect of Nissan/Infiniti new vehicles technology. Evaluation: Conducted through the success-case model.
  • 8. Most Successful: The most successful students of the NITTP reported that they used the skills learned every day in their jobs, allowing them to work more efficiently and effectively. The results are proving highly valuable—fixing more cars with fewer mistakes each day increases profit, and the increased reputation is bringing in more business.
  • 9. Most Successful: The successful students reported that they were able to implement what they learned because of the effective hands-on teaching in the second half of the program. They reported that they reviewed what they had learned in the first seven and a half weeks before each hands-on session, so that they would get the most out of the program. They also reported that their workplaces followed the standards and procedures suggested by NITTP.
  • 10. Least Successful: The least successful students reported that they did not make much use of the training they received. They did not notice any difference in their work on Nissan/Infiniti vehicles, and did not think that the program had provided a valuable result.
  • 11. Least Successful: The unsuccessful students reported that they were unable to implement what they learned because the training was too specific. Their workplaces service a wide variety of vehicles, meaning that they are not always working on Nissans and Infinities, or they work on models that are older than was covered by the training. They also reported that their workplaces had very different operating procedures than the NITTP had prepared them for, and that they were held to different standards.
  • 12. Comparison of the Kirkpatrick Model and the Success Case Method
  • 13. Both methods: •Use surveys, questionnaires, and direct communication to gather information. •Measure results, including changes in behavior. •Measure students’ value gained from the training. •Allow a trainer to determine what is and is not working in their program.
  • 14. Contrast of the Kirkpatrick Model and the Success Case Method
  • 15. Kirkpatrick Model •Four “levels” divided by what is examined •Conclusions from data •Focuses on positive aspects Success Case Method •Two parts divided by who is examined •Narrative data •Shows positive and negative aspects

Hinweis der Redaktion

  1. - Exactly what they used, when they used it, how, when etc. - What results they accomplished - How valuable the results are (e.g., in dollars) - What environmental factors enabled their application and results.
  2. - Exactly what they used, when they used it, how, when etc. - What results they accomplished - How valuable the results are (e.g., in dollars) - What environmental factors enabled their application and results.
  3. - Exactly what they used, when they used it, how, when etc. - What results they accomplished - How valuable the results are (e.g., in dollars) - What environmental factors enabled their application and results.
  4. - Exactly what they used, when they used it, how, when etc. - What results they accomplished - How valuable the results are (e.g., in dollars) - What environmental factors enabled their application and results.