Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Needs Assessment Webinar to AFC of ISPI

970 Aufrufe

Veröffentlicht am

A webinar on needs assessment presented to the Armed Forces Chapter of ISPI

Veröffentlicht in: Technologie, Business
  • Als Erste(r) kommentieren

Needs Assessment Webinar to AFC of ISPI

  1. 1. Needs Assessment Methods and Tools Brett Christensen Kayleen Grage Mike Hajba Barb Spice
  2. 2. Outline <ul><li>Background of the project </li></ul><ul><li>Methods employed </li></ul><ul><ul><ul><li>Archival Data </li></ul></ul></ul><ul><ul><ul><li>Interviews </li></ul></ul></ul><ul><ul><ul><li>Cause Analysis </li></ul></ul></ul><ul><ul><ul><li>Surveys </li></ul></ul></ul><ul><li>Lessons Learned </li></ul><ul><li>Q & A </li></ul><ul><li>Your evaluation of this webinar </li></ul>
  3. 3. Dilbert on Data Collection
  4. 4. Background <ul><li>Sep 2008: Client implements Project Management Process </li></ul><ul><li>Project status reports inconsistent </li></ul><ul><li>Frequency of reports is reduced </li></ul><ul><ul><ul><li>Project reporting still drops </li></ul></ul></ul><ul><li>Project reporting suspended </li></ul><ul><li>Needs Assessment requested </li></ul>
  5. 5. Background <ul><li>Client requests assistance from BSU IPT 529 (Needs Assessment) class </li></ul><ul><li>Project Plan drafted, reviewed, and approved by client and professor </li></ul>
  6. 6. Data Collection Methods
  7. 7. Archival Method <ul><li>Definition </li></ul><ul><ul><ul><li>“ Qualitative or quantitative data collected for governmental, research, education, or service purposes and available to researchers in usable raw data forms and formats” (Schensul et al., 1999) </li></ul></ul></ul><ul><li>Purpose </li></ul><ul><ul><ul><li>To find already existing information that relates to the need being assessed </li></ul></ul></ul>
  8. 8. Using Archival Data <ul><li>Ways you can use archival data </li></ul><ul><ul><ul><li>If primary data is not available/too expensive </li></ul></ul></ul><ul><ul><ul><li>To identify trends among other groups </li></ul></ul></ul><ul><li>How we used archival data </li></ul><ul><ul><ul><li>To determine actual reporting frequency by project officer and by project </li></ul></ul></ul><ul><ul><ul><li>To determine exemplar vs. average vs. poor performers </li></ul></ul></ul><ul><ul><ul><li>To assist in creation of Behaviour Over Time Graph (BOTG) </li></ul></ul></ul>
  9. 9. Archival Data Pros & Cons <ul><li>Pros </li></ul><ul><ul><ul><li>Cost effective </li></ul></ul></ul><ul><ul><ul><li>Good for trend analysis and identification </li></ul></ul></ul><ul><ul><ul><li>More and more data is available electronically </li></ul></ul></ul><ul><li>Cons </li></ul><ul><ul><ul><li>Data requires manipulation to make it relevant </li></ul></ul></ul><ul><ul><ul><li>Currency of data can be an issue </li></ul></ul></ul>
  10. 10. Dogbert on Data Mining
  11. 11. Dogbert on Qualitative Data
  12. 12. Interview Method <ul><li>Definition </li></ul><ul><ul><ul><li>“ A data-collection method used with individuals that can be conducted in person, by phone, or by computer technology (such as online cameras, or videoconferencing)” (Gupta et al., 2007) </li></ul></ul></ul><ul><li>Purpose </li></ul><ul><ul><ul><li>To get information and opinions about some issue in the current needs assessment </li></ul></ul></ul>
  13. 13. Using Interviews <ul><li>Ways you can use interviews </li></ul><ul><ul><ul><li>Collect data for a needs assessment, study, paper, etc. </li></ul></ul></ul><ul><ul><ul><li>Source of more in-depth data (follow-up) </li></ul></ul></ul><ul><ul><ul><li>Source to triangulate other data </li></ul></ul></ul><ul><li>How we used interviews </li></ul><ul><ul><ul><li>Source to triangulate survey and archived data </li></ul></ul></ul><ul><ul><ul><li>Via phone and web conferencing software </li></ul></ul></ul><ul><ul><ul><li>Semi-structured approach, with probes </li></ul></ul></ul>
  14. 14. Interview Pros and Cons <ul><li>Pros </li></ul><ul><ul><ul><li>Allows for a good build of rapport </li></ul></ul></ul><ul><ul><ul><li>Can use/see body language if in-person </li></ul></ul></ul><ul><ul><ul><li>Easy, wide reach with technology </li></ul></ul></ul><ul><li>Cons </li></ul><ul><ul><ul><li>Resource intensive (people, travel, time, etc.) </li></ul></ul></ul><ul><ul><ul><li>Strong interviewee/weak interviewer can derail an interview </li></ul></ul></ul><ul><ul><ul><li>Differences in responses due to different interviewers </li></ul></ul></ul>
  15. 15. Dogbert on Interviews
  16. 16. Cause Analysis Worksheet <ul><li>ORIGINS </li></ul><ul><li>Behavior Engineering Model (BEM) (Gilbert, 2007) </li></ul><ul><li>Force Field Analysis (Lewin, 1947) </li></ul><ul><li>Cause Analysis Worksheet (Chevalier, 2007) </li></ul>WHAT IT IS HOW TO USE IT BENEFITS
  17. 17. Cause Analysis Worksheet <ul><li>ORIGINS </li></ul><ul><li>Behavior Engineering Model (BEM) (Gilbert, 2007) </li></ul><ul><li>Force Field Analysis (Lewin, 1947) </li></ul><ul><li>Cause Analysis Worksheet (Chevalier, 2007) </li></ul><ul><li>WHAT IT IS </li></ul><ul><li>Identifies current and desired performance with a defined reasonable goal </li></ul><ul><li>Structured tool to guide the development of information for data gathering and subsequent analysis using BEM factors and sub factors </li></ul><ul><li>Incorporates weighting of data </li></ul>HOW TO USE IT BENEFITS
  18. 18. Cause Analysis Worksheet Example BEM Factor Force Field Weighting BEM Sub Factors
  19. 19. Cause Analysis Worksheet Adaptation for Project BEM Factor BEM Sub Factors FACTORS DRIVING FORCES RESTRAINING FORCES INFORMATION Clear Expectations 1a. Expectations for the project status reporting process are clear. Relevant Feedback 1b. Feedback given to me on my project status reports is timely. 1c. I receive feedback on my project status reports for good performance. 1d. I receive feedback on my project status reports for not so good performance. 1e. Feedback I have received on my performance on my project status reports is relevant. Relevant guides/job aids 1f. The handout (containing job aids, templates, and examples) for project status reporting is useful. Performance coaching 1g. I receive coaching regarding the quality of my performance on project status reports. Purpose/use of the reports 1h. I understand how my project status reports will be used by others.
  20. 20. Cause Analysis Worksheet <ul><li>ORIGINS </li></ul><ul><li>Behavior Engineering Model (BEM) (Gilbert, 2007) </li></ul><ul><li>Force Field Analysis (Lewin, 1947) </li></ul><ul><li>Cause Analysis Worksheet (Chevalier, 2007) </li></ul><ul><li>WHAT IT IS </li></ul><ul><li>Identifies current and desired performance with a defined reasonable goal </li></ul><ul><li>Structured tool to guide the development of information for data gathering and subsequent analysis using BEM factors and sub factors </li></ul><ul><li>Incorporates weighting of data </li></ul><ul><li>HOW TO USE IT </li></ul><ul><li>Identify gap </li></ul><ul><li>Using BEM factors, revise sub factors relative to problem </li></ul><ul><li>Place results of data on weighted scale for analysis </li></ul>BENEFITS
  21. 21. Cause Analysis Worksheet <ul><li>ORIGINS </li></ul><ul><li>Behavior Engineering Model (BEM) (Gilbert, 2007) </li></ul><ul><li>Force Field Analysis (Lewin, 1947) </li></ul><ul><li>Cause Analysis Worksheet (Chevalier, 2007) </li></ul><ul><li>WHAT IT IS </li></ul><ul><li>Identifies current and desired performance with a defined reasonable goal </li></ul><ul><li>Structured tool to guide the development of information for data gathering and subsequent analysis using BEM factors and sub factors </li></ul><ul><li>Incorporates weighting of data </li></ul><ul><li>HOW TO USE IT </li></ul><ul><li>Identify gap </li></ul><ul><li>Using BEM factors, revise sub factors relative to problem </li></ul><ul><li>Place results of data on weighted scale for analysis </li></ul><ul><li>BENEFITS </li></ul><ul><li>Systematic approach </li></ul><ul><li>Links data gathering directly with analysis </li></ul><ul><li>Data is logically grouped by factor </li></ul>
  22. 22. Dogbert on Cause Analysis
  23. 23. Surveys <ul><li>Can be used before, after, or with other data collection methods </li></ul><ul><li>Collect quantitative and qualitative data </li></ul><ul><li>Collect opinions, knowledge, feelings </li></ul><ul><li>Written, face-to-face, phone, electronic </li></ul>
  24. 24. Using Surveys <ul><li>Collaboratively written with Interview data </li></ul><ul><li>Structured around Gilbert’s BEM and Chevalier’s Cause Analysis worksheet </li></ul><ul><li>Reviewed and revised several times </li></ul><ul><li>Pilot survey, revisions, final survey </li></ul><ul><li>Electronic via Opinio </li></ul><ul><li>Emailed invitation with link </li></ul>
  25. 25. Surveys: Pros & Cons <ul><li>PROS </li></ul><ul><ul><ul><li>Inexpensive </li></ul></ul></ul><ul><ul><ul><li>Consistency in questions and presentation </li></ul></ul></ul><ul><ul><ul><li>Participants can fill out surveys on their own time </li></ul></ul></ul><ul><ul><ul><li>Can send to a larger participant pool than needed </li></ul></ul></ul><ul><ul><ul><li>Can be anonymous </li></ul></ul></ul><ul><li>CONS </li></ul><ul><ul><ul><li>Completion rate is generally low </li></ul></ul></ul><ul><ul><ul><li>Questions may be misinterpreted </li></ul></ul></ul><ul><ul><ul><li>Participants can’t ask for clarification </li></ul></ul></ul><ul><ul><ul><li>Participants’ motivation/mood can impact </li></ul></ul></ul><ul><ul><ul><li>Unable to probe for additional information </li></ul></ul></ul>
  26. 26. Dogbert on Surveys
  27. 27. Lessons Learned Key Points Things to Consider Planning <ul><li>Factor in time constraints </li></ul><ul><li>Be thorough </li></ul><ul><li>Develop clear understanding of data to be collected </li></ul><ul><li>Data gathering tools feed data analysis tools </li></ul>Instrument Design <ul><li>Keep it simple </li></ul><ul><li>Design for user friendly administration </li></ul><ul><li>Pilot test instruments </li></ul>
  28. 28. Lessons Learned Key Points Things to Consider Interviews <ul><li>Keep interview focused </li></ul><ul><li>Determine if recording will enhance or inhibit data </li></ul><ul><li>Provide feedback for in-process changes as additional interviews are conducted </li></ul>Surveys <ul><li>Simple is better </li></ul><ul><li>Comments provide good information </li></ul><ul><li>Know your purpose </li></ul><ul><li>Allow extra time to write and rewrite questions </li></ul><ul><li>Plan to pilot your survey </li></ul><ul><li>Keep the survey as short and concise as possible </li></ul>
  29. 29. Questions
  30. 30. How Did We Do? <ul><li>Six quick questions… </li></ul>
  31. 31. References <ul><li>BFMartin.ca. The Dilbert search works. Retrieved June 18, 2009, from http://bgmartin.ca/finder </li></ul><ul><li>Chevalier, R. (2007). A manager’s guide to improving workplace performance. New York: American Management Association. </li></ul><ul><li>Gilbert, T. (2007). Human competence: Engineering worthy performance (Tribute ed.). San Francisco, CA: Pfeiffer. </li></ul><ul><li>Gupta, K., Sleezer, C., & Russ-Eft, D. (2007). A practical guide to needs assessment (2nd ed.). San Francisco, CA: Pfeiffer/ASTD. </li></ul><ul><li>Lewin, K. (1947). Frontiers in group dynamics: Concept, method, and reality in social science, equilibria and change. Human Relations 1 (1), 5-41. </li></ul><ul><li>Schensul, S., Schensul, J., & LeCompte, M. (1999). Essential ethnographic methods: Observations, interviews and questionnaires (Vol. 2). Walnut Creek, CA: Alta Mira Press. </li></ul>

×