SlideShare ist ein Scribd-Unternehmen logo
1 von 4
Downloaden Sie, um offline zu lesen
Assessing Transformational Learning
in Online Professional Programs:
Methodological Approaches and Challenges
Swapna Kumar1/Patricia Arnold2
1 College of Education, University of Florida, USA
2 E-Learning Center, Munich University of Applied Sciences,
_Germany
EDENRW8, Oxford, England
Transformational Learning in Online Professional Programs
• Transformational Learning (TL): “a transformed way of
understanding, or interpreting, or viewing something” (Meyer
& Land, 2003, p. 4) that can be facilitated by critical reflection
and critical discourse (Mezirow, 1998; 2003)
• Focus of Paper: Methodological challenges assessing TL
• Two approaches: How TL was assessed in two online
professional programs
• Methodological Challenges: Shortcomings and benefits of
each approach
• Discussion: Challenges assessing TL
• at different points of an online program
• when using embedded assignments within an online program.
TwoApproachestoAssessingTransformationalLearning
Online Bachelor Social Work Online EdD in Educational Technology
Students: Professionals in Social Work Students: Professionals in Education
Educational Design: Connecting Theory
and Practice.
Activities: Discourse and Reflection.
Educational Design: Connecting Theory,
Research and Practice.
Activities: Discourse, Reflection, Research
Methodology:
• Internal Approach
• “Embedded assessments”
• Students represented TL in e-
portfolios
• Content Analysis of e-portfolios
Methodology:
• External Approach
• Research plan to assess TL at various
points. External Researchers.
• Interviews, focus groups, surveys
• Quantitative and qualitative analysis
Findings: TL occurred Findings: TL occurred
Methodological Challenges:
• Student self-report, as part of graded
assignment.
• Student affinity with e-portfolio,
Researcher affinity with media
Methodological Challenges:
• Student self-report, inability to follow
up at workplace.
• Focus on external indicators of TL,
internal struggles not considered, but
in results.
• Virtual data collection
Partially funded by Prof. Dr. Patricia Arnold
Munich Univ of AppliedSciences
arnold@hm.edu
patriciaarnold.wikispaces.com/
Prof. Dr. Swapna Kumar
University of Florida
swapnak@ufl.edu
http://www.swapnakumar.com
Discussion: Assessing Transformational Learning
Our conclusion: Best to combine
internal and external approaches
Our questions:
• How to avoid possible bias in methods
used?
• How to design a methodology that is
viable in the long run, i.e. without third
party funding, for documenting impact
and quality in online programs?

Weitere ähnliche Inhalte

Was ist angesagt?

Transfer of Learning research proposal
Transfer of Learning research proposalTransfer of Learning research proposal
Transfer of Learning research proposal
johnpegan
 
No significant deffirent
No significant deffirentNo significant deffirent
No significant deffirent
u082934
 
Dissertation Proposal Final
Dissertation Proposal FinalDissertation Proposal Final
Dissertation Proposal Final
heathercarpenter
 
C:\fakepath\applied and participatory paradigm
C:\fakepath\applied and participatory paradigmC:\fakepath\applied and participatory paradigm
C:\fakepath\applied and participatory paradigm
Robyn
 
Denduang 606 Final Pp Ready
Denduang 606 Final Pp ReadyDenduang 606 Final Pp Ready
Denduang 606 Final Pp Ready
guesta536f75
 

Was ist angesagt? (20)

Mission impossible?
Mission impossible?Mission impossible?
Mission impossible?
 
Transfer of Learning research proposal
Transfer of Learning research proposalTransfer of Learning research proposal
Transfer of Learning research proposal
 
Relieving feedback frustrations
Relieving feedback frustrationsRelieving feedback frustrations
Relieving feedback frustrations
 
A Discussion of Learning Analytics on Big Data 基于大数据的学习分析研究
A Discussion of Learning Analytics on Big Data 基于大数据的学习分析研究A Discussion of Learning Analytics on Big Data 基于大数据的学习分析研究
A Discussion of Learning Analytics on Big Data 基于大数据的学习分析研究
 
Student-centered learning analytics development in higher education: initial ...
Student-centered learning analytics development in higher education: initial ...Student-centered learning analytics development in higher education: initial ...
Student-centered learning analytics development in higher education: initial ...
 
No significant deffirent
No significant deffirentNo significant deffirent
No significant deffirent
 
Dissertation Proposal Final
Dissertation Proposal FinalDissertation Proposal Final
Dissertation Proposal Final
 
Doctoral Consortium ECTEL presentation Learning Analytics Learning Design
Doctoral Consortium ECTEL presentation Learning Analytics Learning DesignDoctoral Consortium ECTEL presentation Learning Analytics Learning Design
Doctoral Consortium ECTEL presentation Learning Analytics Learning Design
 
Hatch, Match and Dispatch MRI13 Presentation
Hatch, Match and Dispatch MRI13 PresentationHatch, Match and Dispatch MRI13 Presentation
Hatch, Match and Dispatch MRI13 Presentation
 
Pres 116 leonard walletzký jaroslav zelený march 9 2016
Pres 116 leonard walletzký jaroslav zelený march 9 2016Pres 116 leonard walletzký jaroslav zelený march 9 2016
Pres 116 leonard walletzký jaroslav zelený march 9 2016
 
Understanding, predicting and optimizing learning with Learning Analytics
Understanding, predicting and optimizing learning with Learning AnalyticsUnderstanding, predicting and optimizing learning with Learning Analytics
Understanding, predicting and optimizing learning with Learning Analytics
 
C:\fakepath\applied and participatory paradigm
C:\fakepath\applied and participatory paradigmC:\fakepath\applied and participatory paradigm
C:\fakepath\applied and participatory paradigm
 
Designing effective assessment
Designing effective assessment Designing effective assessment
Designing effective assessment
 
Ectel2017 paper schmitz_mjwm_presentation
Ectel2017 paper schmitz_mjwm_presentationEctel2017 paper schmitz_mjwm_presentation
Ectel2017 paper schmitz_mjwm_presentation
 
Discussant SRHE Symposium "A cross-institutional perspective on merits and ch...
Discussant SRHE Symposium "A cross-institutional perspective on merits and ch...Discussant SRHE Symposium "A cross-institutional perspective on merits and ch...
Discussant SRHE Symposium "A cross-institutional perspective on merits and ch...
 
CDE Conference 09/02/2009. S Hatzinpanagos. Closing the loop: identifying eff...
CDE Conference 09/02/2009. S Hatzinpanagos. Closing the loop: identifying eff...CDE Conference 09/02/2009. S Hatzinpanagos. Closing the loop: identifying eff...
CDE Conference 09/02/2009. S Hatzinpanagos. Closing the loop: identifying eff...
 
Supporting Students in Generating and Using Feedback
Supporting Students in Generating and Using FeedbackSupporting Students in Generating and Using Feedback
Supporting Students in Generating and Using Feedback
 
Redefining dropping out in online higher education
Redefining dropping out in online higher educationRedefining dropping out in online higher education
Redefining dropping out in online higher education
 
Denduang 606 Final Pp Ready
Denduang 606 Final Pp ReadyDenduang 606 Final Pp Ready
Denduang 606 Final Pp Ready
 
Affective behaviour cognition learning gains project presentation
Affective behaviour cognition learning gains project presentationAffective behaviour cognition learning gains project presentation
Affective behaviour cognition learning gains project presentation
 

Ähnlich wie Kumar arnold eden_2014_final

Lamar coehd regional_research_presentation_final_3-23-12
Lamar coehd regional_research_presentation_final_3-23-12Lamar coehd regional_research_presentation_final_3-23-12
Lamar coehd regional_research_presentation_final_3-23-12
Lamar University
 
14.30 kurilovas bireniene
14.30 kurilovas bireniene14.30 kurilovas bireniene
14.30 kurilovas bireniene
progettoacariss
 
Presentation of ola erstad's paper of changing assessment practices and the ...
Presentation of ola erstad's paper of  changing assessment practices and the ...Presentation of ola erstad's paper of  changing assessment practices and the ...
Presentation of ola erstad's paper of changing assessment practices and the ...
Dongyi Guo
 
Julia Parra Research for VSS 2009
Julia Parra Research for VSS 2009Julia Parra Research for VSS 2009
Julia Parra Research for VSS 2009
Julia Parra
 
Ecer2014 mc cartney&marwick-st-researchengagement
Ecer2014 mc cartney&marwick-st-researchengagementEcer2014 mc cartney&marwick-st-researchengagement
Ecer2014 mc cartney&marwick-st-researchengagement
EERA-Network10
 
Monaghan DET-CHE 2013
Monaghan DET-CHE 2013Monaghan DET-CHE 2013
Monaghan DET-CHE 2013
Jim Monaghan
 

Ähnlich wie Kumar arnold eden_2014_final (20)

LAK15 - SCALABILITY AND FLEXIBILITY THROUGH OPEN RESEARCH
LAK15 - SCALABILITY AND FLEXIBILITY THROUGH OPEN RESEARCHLAK15 - SCALABILITY AND FLEXIBILITY THROUGH OPEN RESEARCH
LAK15 - SCALABILITY AND FLEXIBILITY THROUGH OPEN RESEARCH
 
Exploring learners’ motivations on Assessment in a Massive Open Online Course...
Exploring learners’ motivations on Assessment in a Massive Open Online Course...Exploring learners’ motivations on Assessment in a Massive Open Online Course...
Exploring learners’ motivations on Assessment in a Massive Open Online Course...
 
Education System and Employ-ability: From multicultural team perspective
Education System and Employ-ability: From multicultural team perspectiveEducation System and Employ-ability: From multicultural team perspective
Education System and Employ-ability: From multicultural team perspective
 
Lamar coehd regional_research_presentation_final_3-23-12
Lamar coehd regional_research_presentation_final_3-23-12Lamar coehd regional_research_presentation_final_3-23-12
Lamar coehd regional_research_presentation_final_3-23-12
 
Enduring Impact: What can we learn about ePortfolios by listening to program ...
Enduring Impact: What can we learn about ePortfolios by listening to program ...Enduring Impact: What can we learn about ePortfolios by listening to program ...
Enduring Impact: What can we learn about ePortfolios by listening to program ...
 
Enduring Impact What can we learn about ePortfolios by listening to program g...
Enduring Impact What can we learn about ePortfolios by listening to program g...Enduring Impact What can we learn about ePortfolios by listening to program g...
Enduring Impact What can we learn about ePortfolios by listening to program g...
 
What can we learn about ePortfolio programs by listening to graduates?
What can we learn about ePortfolio programs by listening to graduates?What can we learn about ePortfolio programs by listening to graduates?
What can we learn about ePortfolio programs by listening to graduates?
 
14.30 kurilovas bireniene
14.30 kurilovas bireniene14.30 kurilovas bireniene
14.30 kurilovas bireniene
 
Internationalizing the Curriculum.
Internationalizing the Curriculum.Internationalizing the Curriculum.
Internationalizing the Curriculum.
 
The Convenience Factor in Continuing Education and Adult Learning
The Convenience Factor in Continuing Education and Adult LearningThe Convenience Factor in Continuing Education and Adult Learning
The Convenience Factor in Continuing Education and Adult Learning
 
Presentation of ola erstad's paper of changing assessment practices and the ...
Presentation of ola erstad's paper of  changing assessment practices and the ...Presentation of ola erstad's paper of  changing assessment practices and the ...
Presentation of ola erstad's paper of changing assessment practices and the ...
 
Julia Parra Research for VSS 2009
Julia Parra Research for VSS 2009Julia Parra Research for VSS 2009
Julia Parra Research for VSS 2009
 
Ecer2014 mc cartney&marwick-st-researchengagement
Ecer2014 mc cartney&marwick-st-researchengagementEcer2014 mc cartney&marwick-st-researchengagement
Ecer2014 mc cartney&marwick-st-researchengagement
 
Ma samuel what the ph d for (uj) (170316) (1)
Ma samuel what the ph d for (uj) (170316) (1)Ma samuel what the ph d for (uj) (170316) (1)
Ma samuel what the ph d for (uj) (170316) (1)
 
Program evaluation and outdoor education: An overview
Program evaluation and outdoor education: An overviewProgram evaluation and outdoor education: An overview
Program evaluation and outdoor education: An overview
 
Keynote speech: The promises and pitfalls of personalised eLearning
Keynote speech: The promises and pitfalls of personalised eLearningKeynote speech: The promises and pitfalls of personalised eLearning
Keynote speech: The promises and pitfalls of personalised eLearning
 
Reaching Students Where They Are
Reaching Students Where They AreReaching Students Where They Are
Reaching Students Where They Are
 
Monaghan DET-CHE 2013
Monaghan DET-CHE 2013Monaghan DET-CHE 2013
Monaghan DET-CHE 2013
 
Self directed learning in future learn courses using the Bouchard framework
Self directed learning in future learn courses using the Bouchard frameworkSelf directed learning in future learn courses using the Bouchard framework
Self directed learning in future learn courses using the Bouchard framework
 
Feeling supported: Enabling students in diverse cohorts through personalised ...
Feeling supported: Enabling students in diverse cohorts through personalised ...Feeling supported: Enabling students in diverse cohorts through personalised ...
Feeling supported: Enabling students in diverse cohorts through personalised ...
 

Mehr von Patricia Arnold

Arnold wortmann edulearn_150706
Arnold wortmann edulearn_150706Arnold wortmann edulearn_150706
Arnold wortmann edulearn_150706
Patricia Arnold
 
Arnold online educa_131205
Arnold online educa_131205Arnold online educa_131205
Arnold online educa_131205
Patricia Arnold
 
Arnold_E-Learning_Tag_TUM_2013
Arnold_E-Learning_Tag_TUM_2013Arnold_E-Learning_Tag_TUM_2013
Arnold_E-Learning_Tag_TUM_2013
Patricia Arnold
 
Arnold_INTED_2013_Valencia
Arnold_INTED_2013_ValenciaArnold_INTED_2013_Valencia
Arnold_INTED_2013_Valencia
Patricia Arnold
 
Zukunftswerkstatt salzburg arnold_100510
Zukunftswerkstatt salzburg arnold_100510Zukunftswerkstatt salzburg arnold_100510
Zukunftswerkstatt salzburg arnold_100510
Patricia Arnold
 

Mehr von Patricia Arnold (20)

15 wissensmanagement josefstal
15 wissensmanagement josefstal15 wissensmanagement josefstal
15 wissensmanagement josefstal
 
Arnold wortmann edulearn_150706
Arnold wortmann edulearn_150706Arnold wortmann edulearn_150706
Arnold wortmann edulearn_150706
 
Arnold kumar hci_2014
Arnold kumar hci_2014Arnold kumar hci_2014
Arnold kumar hci_2014
 
Wissensarbeit20 141018
Wissensarbeit20 141018Wissensarbeit20 141018
Wissensarbeit20 141018
 
CoP_und_Lernraeume_GMW2014_Arnold_Kumar
CoP_und_Lernraeume_GMW2014_Arnold_KumarCoP_und_Lernraeume_GMW2014_Arnold_Kumar
CoP_und_Lernraeume_GMW2014_Arnold_Kumar
 
Arnold sew 140425
Arnold sew 140425Arnold sew 140425
Arnold sew 140425
 
Arnold_Kumar_NLC2014
Arnold_Kumar_NLC2014Arnold_Kumar_NLC2014
Arnold_Kumar_NLC2014
 
Arnold online educa_131205
Arnold online educa_131205Arnold online educa_131205
Arnold online educa_131205
 
Wissensarbeit20 131101
Wissensarbeit20 131101Wissensarbeit20 131101
Wissensarbeit20 131101
 
Arnold el tag_jena_131022
Arnold el tag_jena_131022Arnold el tag_jena_131022
Arnold el tag_jena_131022
 
Arnold el tag_brig_130927_2
Arnold el tag_brig_130927_2Arnold el tag_brig_130927_2
Arnold el tag_brig_130927_2
 
Arnold ag1 bremen_2013
Arnold ag1 bremen_2013Arnold ag1 bremen_2013
Arnold ag1 bremen_2013
 
Arnold poster oslo_2013
Arnold poster oslo_2013Arnold poster oslo_2013
Arnold poster oslo_2013
 
Arnold_E-Learning_Tag_TUM_2013
Arnold_E-Learning_Tag_TUM_2013Arnold_E-Learning_Tag_TUM_2013
Arnold_E-Learning_Tag_TUM_2013
 
Arnold_INTED_2013_Valencia
Arnold_INTED_2013_ValenciaArnold_INTED_2013_Valencia
Arnold_INTED_2013_Valencia
 
Arnold prato cirn12
Arnold prato cirn12Arnold prato cirn12
Arnold prato cirn12
 
Arnold oer 120615
Arnold oer 120615Arnold oer 120615
Arnold oer 120615
 
Online educa 2011-arnold
Online educa 2011-arnoldOnline educa 2011-arnold
Online educa 2011-arnold
 
Zukunftswerkstatt salzburg arnold_100510
Zukunftswerkstatt salzburg arnold_100510Zukunftswerkstatt salzburg arnold_100510
Zukunftswerkstatt salzburg arnold_100510
 
one more tool
one more toolone more tool
one more tool
 

Kürzlich hochgeladen

1029-Danh muc Sach Giao Khoa khoi 6.pdf
1029-Danh muc Sach Giao Khoa khoi  6.pdf1029-Danh muc Sach Giao Khoa khoi  6.pdf
1029-Danh muc Sach Giao Khoa khoi 6.pdf
QucHHunhnh
 

Kürzlich hochgeladen (20)

On National Teacher Day, meet the 2024-25 Kenan Fellows
On National Teacher Day, meet the 2024-25 Kenan FellowsOn National Teacher Day, meet the 2024-25 Kenan Fellows
On National Teacher Day, meet the 2024-25 Kenan Fellows
 
Kodo Millet PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
Kodo Millet  PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...Kodo Millet  PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
Kodo Millet PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
 
PROCESS RECORDING FORMAT.docx
PROCESS      RECORDING        FORMAT.docxPROCESS      RECORDING        FORMAT.docx
PROCESS RECORDING FORMAT.docx
 
1029-Danh muc Sach Giao Khoa khoi 6.pdf
1029-Danh muc Sach Giao Khoa khoi  6.pdf1029-Danh muc Sach Giao Khoa khoi  6.pdf
1029-Danh muc Sach Giao Khoa khoi 6.pdf
 
ICT role in 21st century education and it's challenges.
ICT role in 21st century education and it's challenges.ICT role in 21st century education and it's challenges.
ICT role in 21st century education and it's challenges.
 
Micro-Scholarship, What it is, How can it help me.pdf
Micro-Scholarship, What it is, How can it help me.pdfMicro-Scholarship, What it is, How can it help me.pdf
Micro-Scholarship, What it is, How can it help me.pdf
 
Food safety_Challenges food safety laboratories_.pdf
Food safety_Challenges food safety laboratories_.pdfFood safety_Challenges food safety laboratories_.pdf
Food safety_Challenges food safety laboratories_.pdf
 
Making communications land - Are they received and understood as intended? we...
Making communications land - Are they received and understood as intended? we...Making communications land - Are they received and understood as intended? we...
Making communications land - Are they received and understood as intended? we...
 
This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.
 
Magic bus Group work1and 2 (Team 3).pptx
Magic bus Group work1and 2 (Team 3).pptxMagic bus Group work1and 2 (Team 3).pptx
Magic bus Group work1and 2 (Team 3).pptx
 
SOC 101 Demonstration of Learning Presentation
SOC 101 Demonstration of Learning PresentationSOC 101 Demonstration of Learning Presentation
SOC 101 Demonstration of Learning Presentation
 
How to Give a Domain for a Field in Odoo 17
How to Give a Domain for a Field in Odoo 17How to Give a Domain for a Field in Odoo 17
How to Give a Domain for a Field in Odoo 17
 
Mehran University Newsletter Vol-X, Issue-I, 2024
Mehran University Newsletter Vol-X, Issue-I, 2024Mehran University Newsletter Vol-X, Issue-I, 2024
Mehran University Newsletter Vol-X, Issue-I, 2024
 
Application orientated numerical on hev.ppt
Application orientated numerical on hev.pptApplication orientated numerical on hev.ppt
Application orientated numerical on hev.ppt
 
Unit-IV- Pharma. Marketing Channels.pptx
Unit-IV- Pharma. Marketing Channels.pptxUnit-IV- Pharma. Marketing Channels.pptx
Unit-IV- Pharma. Marketing Channels.pptx
 
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
 
Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...
Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...
Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...
 
Unit-IV; Professional Sales Representative (PSR).pptx
Unit-IV; Professional Sales Representative (PSR).pptxUnit-IV; Professional Sales Representative (PSR).pptx
Unit-IV; Professional Sales Representative (PSR).pptx
 
2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx
2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx
2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx
 
Asian American Pacific Islander Month DDSD 2024.pptx
Asian American Pacific Islander Month DDSD 2024.pptxAsian American Pacific Islander Month DDSD 2024.pptx
Asian American Pacific Islander Month DDSD 2024.pptx
 

Kumar arnold eden_2014_final

  • 1. Assessing Transformational Learning in Online Professional Programs: Methodological Approaches and Challenges Swapna Kumar1/Patricia Arnold2 1 College of Education, University of Florida, USA 2 E-Learning Center, Munich University of Applied Sciences, _Germany EDENRW8, Oxford, England
  • 2. Transformational Learning in Online Professional Programs • Transformational Learning (TL): “a transformed way of understanding, or interpreting, or viewing something” (Meyer & Land, 2003, p. 4) that can be facilitated by critical reflection and critical discourse (Mezirow, 1998; 2003) • Focus of Paper: Methodological challenges assessing TL • Two approaches: How TL was assessed in two online professional programs • Methodological Challenges: Shortcomings and benefits of each approach • Discussion: Challenges assessing TL • at different points of an online program • when using embedded assignments within an online program.
  • 3. TwoApproachestoAssessingTransformationalLearning Online Bachelor Social Work Online EdD in Educational Technology Students: Professionals in Social Work Students: Professionals in Education Educational Design: Connecting Theory and Practice. Activities: Discourse and Reflection. Educational Design: Connecting Theory, Research and Practice. Activities: Discourse, Reflection, Research Methodology: • Internal Approach • “Embedded assessments” • Students represented TL in e- portfolios • Content Analysis of e-portfolios Methodology: • External Approach • Research plan to assess TL at various points. External Researchers. • Interviews, focus groups, surveys • Quantitative and qualitative analysis Findings: TL occurred Findings: TL occurred Methodological Challenges: • Student self-report, as part of graded assignment. • Student affinity with e-portfolio, Researcher affinity with media Methodological Challenges: • Student self-report, inability to follow up at workplace. • Focus on external indicators of TL, internal struggles not considered, but in results. • Virtual data collection
  • 4. Partially funded by Prof. Dr. Patricia Arnold Munich Univ of AppliedSciences arnold@hm.edu patriciaarnold.wikispaces.com/ Prof. Dr. Swapna Kumar University of Florida swapnak@ufl.edu http://www.swapnakumar.com Discussion: Assessing Transformational Learning Our conclusion: Best to combine internal and external approaches Our questions: • How to avoid possible bias in methods used? • How to design a methodology that is viable in the long run, i.e. without third party funding, for documenting impact and quality in online programs?