Mattingly "AI & Prompt Design: The Basics of Prompt Design"
Assessing student learning
1. Assessing Student Learning
What Do Students Already Know?
Why check students’ background knowledge?
Doing so is grounded in learning theories and is supported byresearch on
the learning process .
Determining what students already know allows you to :
Target specific knowledge gaps .
Become aware of the diversity of backgrounds in your classroom.
Create a bridge between students’ previous knowledge and new material.
Check for misconceptions that may hinder student learning of new material .
For students, understanding their starting point will make it easier for them
to see what they have learned by the end of the course. They can better recall
past learning and construct“bridges” between old and new knowledge .
How can you check students’ backgroundknowledge ?
Plan your background knowledge assessment by asking the following
questions:
What do you assume students already know?
What kinds of questions will help you confirm your assumptions?
What are some common misconceptions or myths related to your subject?
How are you going to analyze and respond to the data your pre-assessment
provides?
2. Some Strategies :
Common Sense Inventory :
Make a list of 10-15 statements related to course content, including
commonly held misconceptions.
Have students mark "true" or "false" next to each statement.
BackgroundKnowledge Probe
Prepare two or three open-ended or multiple-choice questions.
Write questions on the board.
Ask students to respond in two or three sentences to each question or circle a
response.
Example: The Golden Triangle
Have never heard of this place.
Have heard of it, but don’treally know where it is.
Have some idea where this is, but not too clear.
Have a clear idea where this is and can explain.
Let students know these will not be graded and that thoughtful answers help
you make effective instructional decisions.
Share results with students during the next class.
Gallery Walk
Place images, graphs and excerpts from upcoming coursecontent in the
middle of a posterpaper. This leaves room around the material for students
to write.
Hang images around the room.
Create groups of two to four students.
3. Place one group in front of each poster. Give them five minutes to write
observations, what they know or what they are wondering about the
material.
Give each group one posterand sheet of paper to synthesize comments.
Review themes of comments.
Articulate impact of comments on coursedesign to students during the next
class.
Minute Paper :
Decide what prior knowledge is necessary for the course.
Write a Minute Paper prompt (or question) that hits on the above. Try
answering it yourself before using it in class.
Set aside 5-10 minutes of class time to use the technique, as well as time to
explain what you are doing beforehand.
Hand out index cards or half-sheets of paper.
Ask students not to write their names on the cards.
Let students know how much time they will have (two to three minutes per
question), what kinds of answers you want (words, short sentences, a list),
and when they will receive feedback.
Give students feedback. Explain how this information informed course
design.
BlackboardQuizzes and Qualtrics Surveys
Create a series of multiple-choice questions.
Postto Blackboard as an assignment for the first class.
Explain that you will track who responded, butnot how they responded.
Use report results calculated by the software to inform course design.
4. Share results and impact on coursedesign with students.
Some considerations :
When using background knowledge assessments:
Communicate that the assessmentis not graded.
Do not require students to put their name on the assessment.
Use technology. Blackboard, Qualtrics and classroom responsesystems will
quantify some of the data for you and provide graphs that you can then share with
students.
Take the assessment yourself to confirm the questions make sense.
2. Measuring Student Learning :
Assessment is the systematic collection of information about student
learning, using the time, knowledge, expertise, and resources available, in
order to inform decisions that affect student learning (Walvoord, 2010, p.
23).
How can you measure student learning?
What are some practical assessmentstrategies?
How can you incorporate an assessment plan into a course?
Resources
How can you measure student learning ?
Summative assessments are tests, quizzes, and other graded course activities
that are used to measure student performance. They are cumulative and often
reveal what students have learned at the end of a course. Within a course,
summative assessmentincludes the system for calculating individual student
grades.
5. In contrast to summative assessment, formative assessment is any means by
which students receive input and guiding feedback on their relative
performance to help them improve, absent their grade. Formative assessment
can be provided face-to-face in office hours, in written comments on papers,
projects and problem sets, and through e-mails.
Formative assessments can be used to measure student learning on a daily,
ongoing basis. These assessments reveal how and what students are learning
during the courseand often inform next steps in teaching and learning.
Rather than relying on questions such as “Do you understand?” or “Are
there any questions?”, you can be more systematic and intentional asking
students at the end of the class period to write the most important points or
the most confusing aspectof the lecture on index cards. Collecting and
reviewing the responses provides insight into what themes students have
gleaned from your lecture and what your next teaching steps might be.
Providing feedback on these themes to students gives them insight into their
own learning.
You can also ask students to report on their own learning. Surveying
students about their learning is called indirect assessment. Asking students to
rate their knowledge about a topic after taking your course as compared to
what they believe they knew before taking your courseis an example of
indirect assessment. Direct assessments, on the other hand, assess a student’s
direct application of knowledge or skill. Some examples of direct assessment
6. are evaluating students’ abilities to summarize a process,apply a theory,
solve a problem, or synthesize literature.
What are some practicalassessmentstrategies?
Summative assessments :
Asking Good TestQuestions
Using Rubrics
Formative assessments :
For formative assessments, include students in the process:
Tell them what and how you are assessing.
Share your findings with them.
Based on your findings, explain what changes you will make, if any, and
help students focus or redirect their learning.
Classroom ResponseSystems are useful for formative assessments as they
allow for quick collection of data and instant feedback.
Indirect and direct assessments
Course-level Assessment Guide- Assessment Methods
How can you incorporate an assessmentplan into a course ?
Measuring is the third step in a five-part process (Walvoord, 2010, p. 26):
Outcomes. What do you want students to know, be able to do, or value as a
result of taking a course?
Identify. Where in the curriculum are the outcomes addressed?
Measure. How well are students achieving the outcomes?
Revision. What changes can be made to the course to improve student
achievement?
7. Re-measure. Did the revision to the curriculum work?
Some considerations :
Include indirect and direct assessments as well as formative and summative
assessments for a comprehensive assessment plan.
Evaluate whether or not the assessment aligns directly with a learning
outcome.
Make sure the measurement is sustainable in terms of time and resources.
Acknowledge assessments you are already doing through weekly
assignments or course projects.
Use mid-term and end-of-semester student evaluations to assess student
learning.
Using Rubrics
A rubric is a type of scoring guide that assesses and articulates specific
components and expectations for an assignment. Rubrics can be used for a variety
of assignments: research papers, group projects, portfolios and presentations.
Why use rubrics?
Rubrics help instructors :
Assess assignments consistently from student-to-student.
Save time in grading, both short-term and long-term.
Give timely, effective feedback and promote student learning in a
sustainable way.
Clarify expectations and components of an assignment for both students and
courseTAs.
Refine teaching skills by evaluating rubric results.
Rubrics help students :
8. Understand expectations and components of an assignment.
Become more aware of their learning process and progress.
Improve work through timely and detailed feedback.
How can you develop a rubric ?
Getting Started
Start small by creating one rubric for one assignment in a semester.
Ask colleagues if they have developed rubrics for similar assignments.
Although it takes time to build a rubric, time will be saved in the long run as
grading and providing feedback on student work will become more
streamlined.
Rubric DevelopmentGuidelines :
Examine an assignment for your course.
Outline the elements or critical attributes to be evaluated (these attributes
must be objectively measurable).
Create an evaluative range for performance quality under each element; for
instance, “excellent,” “good,”“unsatisfactory.”
You can reinforce a developmental approach by students by using a
developmental scale in your rubric, like “Beginning”, “Emerging” and
“Exemplary.”
Add descriptors that qualify each level of performance:
Avoid using subjective or vague criteria such as “interesting” or “creative”;
instead, outline objective indicators that would fall under these categories.
The criteria must clearly differentiate one performance level from another.
Assign a numerical scale to each level.
9. Give a draft of the rubric to your colleagues and/or TAs for feedback.
Train students to use your rubric and solicit feedback; this will help you
judge whether the rubric is clear to them and will identify any weaknesses.
Rework the rubric based on the feedback.
When developing rubrics considerthe following :
A rubric can be a fillable pdfthat can easily be e-mailed to students.
How much class time is required for teaching and re-teaching the rubric.
How can you incorporate rubrics in a course?
Rubrics are most often used to grade written assignments, but they have
many other uses.
They can be used for oral presentations.
They are a great tool to evaluate teamwork and individual contribution to
group tasks.
Rubrics facilitate peer-review by setting evaluation standards.
Students can use them for self-assessment to improve personalperformance
and learning.
For larger assignments, have students use the rubric to provide peer
assessment on various drafts.
Encourage students to use the rubrics to assess their own work.
Motivate students to improve their work by using rubric feedback to
resubmit their work incorporating the feedback.
Here is a sample strategy for introducing rubrics to students:
Provide samples, or smaller sections of samples, of a complete assignment
(consider asking previous students for permission to use their assignments as
samples, provided that you remove their names).
10. Have students evaluate the assignments individually using the rubric.
Have students share their results with a partner and justify their evaluation
by explaining how they used the rubric.
Ask a few pairs to share their responses with the class. (Paying attention to
students’ reactions/interpretations of the rubric is useful and may inform
rubric adjustments).
Provide your own evaluation of the sample assignments and explain how
you used the rubric to assess the work.
Asking GoodTestQuestions
Questions not only measure what students know, but can reveal different levels of
knowledge and learning.
1. What are good test questions?
2. How can you develop good test questions?
1. What are goodtest questions ?
Goodtest questions :
Assess what you intend for them to assess.
Allow students to demonstrate what they have learned.
Elicit answers that reveal students’ intellectual progress.
Motivate students and help them structure their academic efforts.
Can discriminate between students who have learned what you had intended
them to learn and those who have not.
2. How canyou develop goodtest questions ?
Some basic steps :
11. Determine what you want students to know. Revisit the learning outcomes
you may have articulated at the beginning of the course.
Use Bloom’s Taxonomy to review verbs that could inform test questions,
such as apply, compare, describe, etc.
Examine the different types of questions: fill-in the blank, short answer,
multiple choice, matching, essay question, etc. Use multiple question types,
as different students may be better or worse at answering various types of
questions.
Examine whether the questions ask, “How much do you remember of what
has been covered?” rather than “What can you do with what you have
learned?” (Dressel, 1976).
Come up with a few possible questions after each lecture; this may increase
the quality of test questions because they are more likely to be more
representative of your instruction.
Consider having students create and submit test questions after a class.
Review them for possible use.
Have a colleague or TA review and provide feedback on test questions.
Self-Assessment
1. What is self-assessment?
The ability to be a realistic judge of one’s own performance.
2. Why employ self-assessment?
Provides timely and effective feedback and allows for quick assessmentof
student learning.
Allows instructors to understand and provide quick feedback on learning.
Promotes academic integrity through student self-reporting of learning
progress.
12. Promotes the skills of reflective practice and self-monitoring.
Develops self-directed learning.
Increases student motivation.
Improves satisfaction from participating in a collaborative learning
environment.
Helps students develop a range of personal, transferrable skills to meet the
expectations of future employers.
3. How can you incorporate self-assessment ?
Identify which assignments and criteria are to be assessed.
Articulate expectations and clear criteria for the task; this can be
accomplished with a rubric.
Motivate students by framing the assignment as an opportunity to reflect
objectively on their work, determine how this work aligns with the
assignment criteria, and determine ways for improvement.
Provide an opportunity for students to agree upon and take ownership of the
assessment criteria.
Draw attention to the inner dialogue that people engage in as they producea
piece of work. You can model this by talking out loud as you solve a
problem, or by explaining the types of decisions you had to think about and
make as you moved along through a project.
Some Self-Assessment Tasks
Assignment cover sheet
Require students to submit a cover sheet with their assignment.
On the cover sheet, students should respond self-assessment prompts (for
example):
What is strong, or what went well with this assignment? Provide examples.
13. What do you think is weak about this assignment?
Small Feedback Groups
Provide students with feedback on an assignment.
Have students work in pairs or small groups.
Have them or orally explain and discuss the feedback they received.
4. What else should you consider when incorporating self-assessment?
The difference between self-assessment and self-grading will need
clarification.
The process ofeffective self-assessment will require instruction and
sufficient time for students to learn.
Students are used to a system where they have little or no input in how they
are assessed, and are often unaware of assessment criteria.
Students will want to know how much self-assessed assignments will count
toward their final grade in the course.
Incorporating self-assessment can motivate students to engage with the
material more deeply.
Self-assessment assignments can take more time.
Research shows that students can be more stringent in their self-assessment
than the instructor.
Traditional, instructor assessmentcan result in “backwash” where the
assessment determines what and how students learn more than the
curriculum.
Upper-level, science-oriented courses are better suited for self-assessment.
Peer-Assessment
1. What is peer-assessment ?
14. Peer assessment allows instructors to share the evaluation of assignments with their
students. It is grounded in theories of active learning , adult learning and social
constructionism .
2. Why employ peer-assessment?
Peer assessment can:
Empower students to take responsibility for, and manage, their own learning.
Enable students to learn to assess and to develop life-long assessmentskills.
Enhance students' learning through knowledge diffusion and exchange of
ideas.
Motivate students to engage with coursematerial more deeply.
3. How canyou incorporate peer-assessment ?
Identify assignments or activities for which students might benefit from peer
feedback.
Consider breaking a larger assignment into smaller pieces and incorporating
peer assessmentopportunities at each stage. For example, assignment
outline, first draft, second draft, etc.
Design guidelines or rubrics with clearly defined tasks for the reviewer.
Introduce rubrics through learning exercises to ensure students have the
ability to apply the rubric effectively.
Determine whether peer review activities will be conducted as in-class or
out-of-class assignments; for out-of-class assignments, peer assessments can
be facilitated online by Blackboard.
Help students learn to carry out peer assessment by modeling appropriate,
constructive criticism and descriptive feedback through your own comments
on student work and well-constructed rubrics.
15. Incorporate small feedback groups where written comments on assignments
can be explained and discussed with the receiver.
4. What else should you considerwhen incorporating peer-assessment?
Let students know the rationale for doing peer review; explain the
expectations and benefits of engaging in a peer review process.
Consider having students evaluate anonymous assignments for more
objective feedback.
Be prepared to give feedback on students’ feedback to each other. Display
some examples of feedback of varying quality and discuss which kind of
feedback is useful and why.
Give clear directions and time limits for in-class peer review sessions and set
defined deadlines for out-of-class peer review assignments.
Listen to group feedback discussions and provide guidance and input when
necessary.
Student familiarity and ownership of criteria tend to enhance peer
assessment validity; therefore, involve students in a discussion of the criteria
used.
Students have more experience with academic tasks; therefore, be cautious
about having them peer-assess professionaltasks: choosetasks that lie
within their relative experiential base.
Encourage students to take more individual responsibility by not having
multiple peers assess the same task.
16. Teaching with technology (E-learning)
E-learning (or eLearning) is the use of electronic media, educational
technology and information and communication technologies (ICT) in
education. E-learning includes numerous types of media that deliver text,
audio, images, animation, and streaming video, and includes technology
applications and processessuchas audio or video tape, satellite TV, CD-
ROM, and computer-based learning, as well as local intranet/extranet and
web-based learning. Information and communication systems, whether free-
standing or based on either local networks or the Internet in networked
learning, underlie many e-learning processes.
E-learning can occurin or out of the classroom. It can be self-paced,
asynchronous learning or may be instructor-led, synchronous learning. E-
learning is suited to distance learning and flexible learning, but it can also be
used in conjunction with face-to-face teaching, in which case the term
blended learning is commonly used.
E-learning includes, and is broadly synonymous with multimedia learning,
technology-enhanced learning (TEL), computer-based instruction (CBI),
computer managed instruction,[2] computer-based training (CBT),
computer-assisted instruction or computer-aided instruction (CAI), internet-
based training (IBT), flexible learning, web-based training (WBT), online
education, virtual education, virtual learning environments (VLE) (which are
also called learning platforms), m-learning, and digital education. These
17. alternative names individually emphasize a particular digitization approach,
component or delivery method, but conflate to the broad domain of e-
learning. Forexample, m-learning emphasizes mobility, but is otherwise
indistinguishable in principle from e-learning .
PowerPoint
Why use PowerPoint ?
When used appropriately, PowerPoint can:
Frame a lecture.
Promote student engagement.
Highlight important material.
Incorporate complex graphics and video.
How can you design an effective presentation ?
Consider what information and key concepts you want students to focus on.
Provide visual organizers, such as agendas or chevrons that indicate where
you are within a lecture.
Keep text to a minimum to avoid students merely taking notes and listening
passively, and be conscious ofcolor choices for accessibility.
Use PowerPoint to convey visual ideas, such as graphs, tables, and images
that are not easily represented in other ways.
Incorporate questions directly into the slides and give students time to
discuss and answer them. Also consider also using i>Clickers.
Incorporate a video and frame it with questions students can discuss.
Supportyour PowerPoint presentation with other forms of communication
such as classroom discussion, handouts, and videos.
How can you integrate PowerPoint into a course?
18. Think of your PowerPoint presentation as a tool to supportrather than
replace your classroom instruction.
Pay attention to how students are responding to the information.
Allow time for questions and clarifications at regular intervals.
Turn your screen black (press the "B" button on the keyboard) or white
(press the "W" button) when your class lecture or discussionhas nothing to
do with a slide.
Aim to integrate a couple of interactive activities during each lecture
session.
Have students create their own PowerPoint presentations as part of an
assignment.
Should you provide PowerPointpresentations electronically ?
PowerPoint enables you to provide your students with class presentation
documents electronically.
Providing students with presentations before or after a class can reinforce
ideas and make learning material easily accessible, but may interfere with
class attendance.
Providing students with skeletal presentation slides can supportnote taking,
but may interfere with students’ ability to organize their own thoughts to
take notes.
Providing students with slides, questions, or videos before a sessioncan
introduce students to the class topics.
Providing electronic slides only to students who request them.
E-Portfolios
19. What are ePortfolios ?
ePortfolios are similar to regular portfolios. An ePortfolio is a collection of
materials that documents student accomplishments, and may include
reflections on the learning process and its outcomes.
Since ePortfolios are electronic, they have additional qualities such as:
Requiring students to organize their thoughts and materials using an
electronic interface similar to a personal web page.
Allowing for the presentation and interlinking of various media types.
Being shared easily and continuously edited.
Why use ePortfolios?
The learning purposes ofePortfolios include:
Reflecting upon learning processes and outcomes.
Organizing and presenting learning accomplishments.
Developing self-assessment skills.
Representing learning experiences.
Developing multimedia skills.
Creating electronic text for specific audiences.
Learning how to use technology to supportlifelong learning.
The learning benefits of ePortfolios include:
Personalizing the learning experience.
Allowing students to draw connections between their various learning
experiences over the semester and beyond.
20. Seeing progress over time.
Enhancing critical thinking.
Evaluating and assessing student products and processes.
Assessing course learning outcomes.
Gaining insight into how students experienced a curriculum.
Online discussions
What are online discussions ?
Online discussions are a great toolto extend classroomconversations and
learning by getting students to engage with class material online. Online
discussions are often arranged by discussionboards, forums, and threads.
A discussion board is the tool that hosts the space for online discussions.
Discussion boards can hold multiple forums, which can be organized by
topic.
Discussion threads are conversations within discussion forums and begin
with a leading question or prompt. Users can respond to the original prompt
and can reply to other responses.
Course management systems, such as Blackboard, and other web-based
tools, such as Piazza, host online discussions.
Online discussions can take other forms such as discussions based on a piece
of work, or feedback dialogue on a student’s writing. Forexample, when
creating a wiki collaboratively, students may discuss the process online, or
when posting on a blog, students can discuss the blog postin the comments
section.
Why do online discussions?
21. Flexible, not limited by time or space.
In-class discussions can continue online and vice versa.
Students can prepare for class activities by engaging in online discussions
that are designed to have students get familiar with new topics. For example,
students can discuss their responses to a pre-class reading.
Online discussions can simultaneously reach different types of learners.
Online discussions can enable introverted thinkers time to process a
responseand extraverted thinkers time to reflect on their thoughts before
posting.
Students can use online discussions to give and receive feedback on their
work.
Postings are saved and conversations can be reviewed for assessment and
reflection purposes.
Online discussions can be a spacewhere students answer each other’s
frequently asked questions about the course, helping to alleviate the
instructor workload by dealing with questions that otherwise might be
emailed to her or him.
survey tools
What are survey tools?
Survey tools are online tools that can be used to electronically collect answers or
responses to questions from a target audience. Online survey tools feature a variety
of question types, including multiple-choice, ranking, or open-ended, and many
others.
Why use survey tools in your course?
Easily collect information about your students suchas:
What students know about the coursesubject on the first day.
22. Why they are taking the course.
What their expectations are for doing so.
What students currently think about course topics.
What and how students are learning.
How students are experiencing the course.
Quickly review for themes or common issues with automatically aggregated
survey responses and survey result graphics. Your survey results can inform
your next steps in teaching.
Help students become more reflective and aware of their learning as they
answer surveys about their learning experiences.
Classroom Response System
What is a classroom responsesystem (CRS)?
CRSs are also referred to as ClassroomResponseTechnologies, Classroom
Polling Systems, Clickers or Student/Audience ResponseSystems.
CRS consist of individual remotes that students use to respond to questions
and a handheld transmitter that collects the student responsedata. Other
classroomresponsetechnologies are web-based and students can use cell
phones or other personal mobile devices to answer questions.
With this technology, lecturers can design multiple-choice questions for
students to answer anonymously using a remote device. In addition to
multiple-choice questions, some web-based classroom responsesoftware
like Poll Everywhere allow for questions that require text responses, and
Learning Catalytics allows for even more question types, such as those that
require text, sketch, and mathematical expression responses.
23. Responses are instantly tabulated via the transmitter, or online for web-based
technologies. The instructor can view responses in real time and can also
share this visual with students using a projector.
Blackboard
What is Blackboard?
Blackboard is online course management system. Course Managements Systems
(CMS) are sometimes referred to as Learning Management Systems (LMS).
Why use Blackboard for your course?
Blackboard can be used to:
Create an online forum that can be accessed 24/7.
Facilitate communication with students through announcements and other
tools.
Enhance asynchronous communication (communication that does not
require an immediate response) as a componentof the course experience.
Organize course materials online and make them easily accessible.
Blackboard enables instructors to:
Organize student records.
Establish an online discussioncomponent for the course.
Create learning activities such as quizzes and tests that can be designed to
promote student engagement and check comprehension.
Blackboard enables students to:
Submit and store class notes and other work.
Collaborate with one another.
View their peers’ work and give and receive feedback.
Video
24. Why use video?
When used effectively, videos can:
Stimulate interest, set the tone of a class, or elicit a shared experience
among students (Poonati & Amadia, 2010).
Generate discussion.
Illustrate complex topics or bring otherwise inaccessible material into the
classroom, such as:
a guest lecturer
video-based case studies
an experiment that is difficult or impossible to do in a lecture hall
footage of geographically distant places
Connect complex topics to real-world examples.
Accommodate a variety of learning styles by providing opportunities for
students to engage with material visually and orally.
Develop media literacy and critical thinking.
Promote comprehension and retention of information (Choi & Johnson,
2007).
Engage students with course content outside of the classroom.
Problem-based learning
Problem-based learning (PBL) is a student-centered pedagogy in which students
learn about a subject through the experience of problem solving. Students learn
both thinking strategies and domain knowledge. The PBL format originated from
the medical schoolof thought, and is now used in other schools of thought too. It
25. was developed at McMaster University Schoolof Medicine in Canada in the 1960s
and has since spread around the world. The goals of PBL are to help the students
develop flexible knowledge, effective problem solving skills, self-directed
learning, effective collaboration skills and intrinsic motivation.[1] Problem-based
learning is a style of active learning.
Working in groups, students identify what they
already know, what they need to know, and how and where to access new
information that may lead to resolution of the problem. The role of the instructor
(known as the tutor in PBL) is to facilitate learning by supporting, guiding, and
monitoring the learning process.[2]The tutor must build students' confidence to
take on the problem, and encourage the students, while also stretching their
understanding. PBL represents a paradigm shift from traditional teaching and
learning philosophy,[3] which is more often lecture-based. The constructs for
teaching PBL are very different from traditional classroom/lecture teaching.
Meaning of PBL:
1. Student Centered Learning
2. Learning is done in Small Student Groups, ideally 6-10 people
3. Facilitators or Tutors guide the students rather than teach
4. A Problem forms the basis for the organized focus of the group, and stimulates
learning
5. The problem is a vehicle for the development of problem solving skills. It
stimulates the cognitive process.
6. New knowledge is obtained through Self-Directed Learning (SDL).
26. In order to instill a project based learning
environment into a classroom, the teacher must revolve his or her teaching style
around five main criteria.
1. Centrality
2. Driving Question
3. Constructive Investigations
4. Autonomy
5. Realism .