1. Leveraging technology for music
assessment in a classroom
Dan Massoth
American International School - Kuwait
2. What are the possibilities?
• Online learning and blended classrooms
• Project-based activities
• Game-based learning and assessment
• Learning with mobile and handheld devices
• Instructional tools like interactive whiteboards
• Web-based projects, explorations and research
• Student created media
• Collaborative on-line tools
• Social media
4. How best to leverage technology
• Philosophies
• Overall approach
• TPACK
• SAMR
• Techniques
• Issues related authentic assessment in music
• Internal
• External
• Devices
5. Approach
The teaching of the curriculum as well as
overall instructional goals (including assessment)
should drive the use of technology in the
classroom and not vice-versa.
19. The assessment of student
achievement in a fine arts class has
become essential. Why?
• Educationally, assessment is needed to validate and document student
achievement and to improve teaching and learning.
• Politically, the arts will not be able to thrive without the same level of
measurement and data that is provided by other core subjects
• Funding will be diverted to subjects and programs that can measure
student achievement. “What gets measured, gets funded”
20. Purposes of assessment
• Provide feedback to students about their progress
• Give parents feedback on their child’s performance
• Rate or rank students
• Provide feedback to instructors on their effectiveness
Anything else?
21.
22. Purpose Primary Users Construct
Linkage to
Curriculum Interpretation
Instructional Guidance Teachers,
Students
Narrow Achievement
Targets
Strong CR, Individual
Student Placement and
Selection
School
Administrators
(and Others)
Aptitude, Achievement Varies Varies, Individual
Informing Comparisons
Among Educational
Approaches
School
Administrators,
Researchers
Achievement
(curriculum-specific;
curriculum-neutral)
Varies NR, Group
Educational Management Public, Elected
Officials,
Administrators
Achievement Grows higher NR (may look like
CR), Group
Directing Student Effort Students Aptitude, Achievement Varies (should
be strong)
mostly CR,
Individual
Focusing the System Teachers, School
Administrators
Achievement Grows higher NR (may look like
CR), Group
Shaping Public
Perceptions
Public, Elected
Officials,
Administrators
Achievement Should be
Strong
NR, Group
From: How is Testing Supposed to Improve Schooling?
Presented at the 2012 NCME conference by Dr. Ed Haertel, Stanford University
Seven Broad Purposes of Testing
23. 4 types of arts behaviors: Types of assessment
1.Responding
2.Creating
3.Performing
4.Listening
1. Rubrics
2. Playing tests
3. Written tests
4. Practice sheets
5. Teacher Observation
6. Portfolios
7. Peer and Self-Assessment
Assessing Musical Behaviors: The type of assessment
must match the knowledge or skill
25. Authentic Assessment
Authentic assessment accomplishes each of the following goals:
• Requires students to develop responses rather than select from
predetermined options
• Elicits higher order thinking in addition to basic skills
• Uses samples of student work (portfolios) collected over an
extended time period
• Stems from clear criteria made known to students
• Allows for the possibility of multiple human judgments
• Relates more closely to classroom learning
• Teaches students to evaluate their own work
25
26. 3 Features of a Useful Rubric
• Evaluative Criteria:
– Factors to be used when judging the quality of a student’s response
• Quality Distinctions:
– For each criterion, different levels of quality in a student’s work must
be described
• Application Strategy:
– Users of the rubric are told whether a student’s response is to be
judged using the evaluative criteria or on a criterion-by-criterion basis
(holistic vs. analytic)
• From Popham, W. James (2006). Mastering Assessment: A Self-Service
System for Educators. Routledge: NY
28. Portfolio
A student portfolio can be created from a variety of
sources.
• Performances, Art work,
• Classroom work
• Home work
• Rubrics
• Short answer questions
• Reflections
• Other data
Each of these examples can be collected and then used
for grading and conferences.
28
36. Using “smart” technology for
assesssment
• Turing test
• ELA and the big testing companies
• Issues
37. Issues related to performance
assessment - recognition
• Variations in timbre
• Variations in microphone placement and pickup
• Latency
• Issues with background sound
• Pitch
• Articulation
38. Issues related to data analysis
• Full duplex audio latency
• Quantization
• Difficulty level
• Intonation
• Articulation
• Multiple starts/assessment inconsistency
39. Issues related to data display
• Roadmap/repeat
• Quantity and quality of feedback
• Notation
47. Performance assessment software
comparison
Name Repertoire Assessment Platform Cost
Essential
Elements
Interactive
2 Method books Recording iPad
Mac
PC
Free with
book
iPas 2 Method books Performance - chart Mac
PC
Free with
book
Music
Prodigy
Public domain titles Performance -Red and
green note
iPad
Mac
PC
$24 - 30
SmartMusic Over 10,000 solos,
method books,
exercises, concert titles
Red and green note iPad
Mac
PC
$40 / 140
48. On line tools with some assessment
capabilities
• Music First
• Charanga
51. How did we do?
• Online learning and blended classrooms
• Project-based activities
• Game-based learning and assessment
• Learning with mobile and handheld devices
• Instructional tools like interactive whiteboards
• Web-based projects, explorations and research
• Student created media
• Collaborative on-line tools
• Social media
I’m excited to have the chance to present at the International Symposium for Music Assessment. I attended the first Symposium as an employee of Coda Music Technology. I’ve been a music educator in Minnesota for over 15 years. I spent a total of 10 years in music education software development at MakeMusic. Currently teaching high school music at the American International School of Kuwait.
Over the past 15 years I’ve had the opportunity to help develop the most widely used performance assessment technology on the market today.
One of my roles at MakeMusic was to help develop an online learning management capability that allowed teachers to send assignments for students to complete. I finished my tenure as a VP in the area of product development and though leadership. It was in this role that I had the opportunity to participate on Core Arts Standards Leadership team and was able to co-organize the National Symposium on Music Assessment and Teacher Evaluation.
Today I’m not going to show you 100 apps in 60 seconds or even try to compare all of the softwares available in each of the genres. Instead, we’ll spend half of our time learning about how best to effectively and intelligently chose when and where to use software In the learning process and the other half playing with some of the latest and greatest assessment software being developed today.
http://www.cbc.ca/spark/2011/09/spark-154-september-11-14-2011/ (scroll down for audio segment) technologies what technologies get used in the classroom? why? cc-lisenced image source: http://www.flickr.com/photos/unav/4464311633/ cc-lisenced image source: http://www.flickr.com/photos/av_hire_london/5570763086/ cc-lisenced image source: http://www.flickr.com/photos/osde-info/2224883563/ "Great pedagogy is timeless and shouldn’t be tied to one particular technology" engagement enhancement How can we engage "digital natives"? What would these learning environments look like? Technology in the classroom what does the research say? One study (Kraushaar & Novak, 2010) showed: Kraushaar, J. M., & Novak, D. C. (2010). Examining the Affects of Student Multitasking with Laptops during the Lecture Journal of Information Systems Education, 21(2), 241–251.
Howard-Jones, P. (2011). The impact of digital technologies on human wellbeing (pp. 1–98). Nominet Trust. average student cycled through 65 active windows per lecture
2/3 of applications classified as distractive
only one application ended up having effect on assessments: instant messaging Another study (Sovern, 2011) showed: Sovern, J. (2011). Law Student Laptop Use During Class for Non-Class Purposes: Temptation v. Incentives. St. John's Legal Studies Research Paper Series.
Junco, R., Elavsky, C. M., & Heiberger, G. (2012). Putting twitter to the test: Assessing outcomes for student collaboration, engagement and success. British Journal of Educational Technology, Twitter collaboration & engagement, no–no. doi:10.1111/j.1467-8535.2012.01284.x more than 1/2 of students in lecture used laptops for non-class purposes for more than 1/2 class time
So, using technology in schools is not as easy as it looks. Small and large districts have rolled out iPad, Laptop and tablet programs and have run into serious problems.
Prediction – very soon most schools will be Bring You Own Device (BYOD) institutions. As prices continue to fall and students continue to become more connected, schools will provide less and less hardware and more towards controlling access to data for learning. The role or a teacher shifts from being only a “font of knowledge” to include more of the guide/docent and diagnostician roles.
During this workshop I’ll invite you to be a part of the conversation for how we can most effectively and intelligently use technology to help our students learn given the tools and resources that we have today. Notice that I didn’t just say “assess out students”. This was intentional, for in today’s test-driven and assessment- burdened educational climate, I have noticed that the art of teaching our students sometimes takes a back seat to the lust for a final grade on a report card.
The first part of this presentation will involve a medium depth look at the most frequently used technology integration models. We’ll start with an overall approach that will hopefully keep us heading in a healthy and positive direction. Then we’ll look through three lenses – SAMR, TPaC and Engagement/gaming theory to help develop a balanced and well-thought out approach to using assessment technology in the classroom.
Comment about training in technology – this is all about education and shouldn’t we be excellent at helping each other learn. Training can come in three or more flavors – Just in time, On line, On demand….
Another diagram that uses mostly basic colors…
Content Knowledge (CK) – “Teachers’ knowledge about the subject matter to be learned or taught. The content to be covered in middle school science or history is different from the content to be covered in an undergraduate course on art appreciation or a graduate seminar on astrophysics… As Shulman (1986) noted, this knowledge would include knowledge of concepts, theories, ideas, organizational frameworks, knowledge of evidence and proof, as well as established practices and approaches toward developing such knowledge” (Koehler & Mishra, 2009).
Pedagogical Knowledge (PK) – “Teachers’ deep knowledge about the processes and practices or methods of teaching and learning. They encompass, among other things, overall educational purposes, values, and aims. This generic form of knowledge applies to understanding how students learn, general classroom management skills, lesson planning, and student assessment.” (Koehler & Mishra, 2009).
Technology Knowledge (TK) – Knowledge about certain ways of thinking about, and working with technology, tools and resources. and working with technology can apply to all technology tools and resources. This includes understanding information technology broadly enough to apply it productively at work and in everyday life, being able to recognize when information technology can assist or impede the achievement of a goal, and being able continually adapt to changes in information technology (Koehler & Mishra, 2009).
Pedagogical Content Knowledge (PCK) – “Consistent with and similar to Shulman’s idea of knowledge of pedagogy that is applicable to the teaching of specific content. Central to Shulman’s conceptualization of PCK is the notion of the transformation of the subject matter for teaching. Specifically, according to Shulman (1986), this transformation occurs as the teacher interprets the subject matter, finds multiple ways to represent it, and adapts and tailors the instructional materials to alternative conceptions and students’ prior knowledge. PCK covers the core business of teaching, learning, curriculum, assessment and reporting, such as the conditions that promote learning and the links among curriculum, assessment, and pedagogy” (Koehler & Mishra, 2009).
Technological Content Knowledge (TCK) – “An understanding of the manner in which technology and content influence and constrain one another. Teachers need to master more than the subject matter they teach; they must also have a deep understanding of the manner in which the subject matter (or the kinds of representations that can be constructed) can be changed by the application of particular technologies. Teachers need to understand which specific technologies are best suited for addressing subject-matter learning in their domains and how the content dictates or perhaps even changes the technology—or vice versa” (Koehler & Mishra, 2009).
Technological Pedagogical Knowledge (TPK) – “An understanding of how teaching and learning can change when particular technologies are used in particular ways. This includes knowing the pedagogical affordances and constraints of a range of technological tools as they relate to disciplinarily and developmentally appropriate pedagogical designs and strategies” (Koehler & Mishra, 2009).
Technological Pedagogical Content Knowledge (TPACK) – “Underlying truly meaningful and deeply skilled teaching with technology, TPACK is different from knowledge of all three concepts individually. Instead, TPACK is the basis of effective teaching with technology, requiring an understanding of the representation of concepts using technologies; pedagogical techniques that use technologies in constructive ways to teach content; knowledge of what makes concepts difficult or easy to learn and how technology can help redress some of the problems that students face; knowledge of students’ prior knowledge and theories of epistemology; and knowledge of how technologies can be used to build on existing knowledge to develop new epistemologies or strengthen old ones” (Koehler & Mishra, 2009).
http://www.tpack.org
http://tpack.org
Mishra, P., & Koehler, M. J. (2006). Technological Pedagogical Content Knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017-1054. doi: 10.1111/j.1467-9620.2006.00684.x.
Also see:
http://www.schrockguide.net/samr.html
http://ictevangelist.com/wp-content/uploads/2013/03/SAMR-flow-chart.pdf
8 Minute video
Substitution examples:
Where many get stuck when working with technology is in the initial stages of adoption. In Crossing the Chasm the author argues that there are 4 distict groups of technology adopters – Innovators,Early Adopters, Majority and the Laggards. While most early adopters will easily overcome even large obstacles, the middle majority will not. This is where the Procurement and Basic Skills steps become most important. For without assistance and guidance at these stages, there is little chance of any technology adoption or integration. Which leads us to the exposure step; in today’s world, new technologies are being created every day. More than ever possible to explore and understand. So the initial exposure to meaningful technology becomes an important first step on the road towards adoption. For the middle majority, seeing and understanding how this technology will make their lives, or the lives of their students, betters is essential for them to be willing to make the investment of time and energy to learn how to use it. Of course the early adopters are usually the presenters and the laggards are in the back of the room checking their email.
Special thanks for my colleague, Nyssa Brown, originally from Minnesota who added these Entry Steps.
So here then are all of the steps of the “Enhanced” SAMR Model of Technology Integration. As you can see, technology integration is not simple. The keen understanding of each of these steps and the techniques used to move along these spectrum of integration is not easy and as we have seen many districts large and small have stumbled or worse.
Description: A representative selection of a student’s best products that provides evidence of the degree to which the student has achieved specified learning targets. In SmartMusic this would include scores, feedback, screenshots and recordings of a student performances as well as any response assignment work.
Notes: This approach uses both formative and summative techniques to gather evidence of student achievement. The teacher may choose assignments that contain the learning outcomes or objectives and embed them into the curriculum without mention to the students.
Description: Standardized tests provide summative assessment information and consist of a battery of subtests, each covering a different curricular area. These tests are usually well crafted and supported by research on the validity of the scores. In SmartMusic this would include scores, feedback, screenshots and recordings of a student performances as well as any response assignment work.
Notes: This approach to testing usually relies on a pre and post-test model and requires research to establish validity and reliability. This is the most formal test process and was considered in that past to be uniquely a summative testing experience.
Description: A mastery of work at increasing levels of difficulty In SmartMusic this would include scores, feedback, screenshots and recordings of a student performances as well as any response assignment work. Supports the sequential and spiraling nature of music curriculum taught in most instrumental and some vocal classes.
Notes: This approach uses both formative and summative techniques to gather evidence of student achievement. The teacher may choose assignments that contain the learning outcomes or objectives and embed them into the curriculum without mention to the students.
Where are you from?
https://www.polleverywhere.com/clickable_images/hinpco0FgUDwOL1
2nd year student
Need rubric from state - Texas? Minnesota?
Standard of Excellence Book 2 Kjos publications Bruce Pearson
The Turing test is a test of a machine's ability to exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human. In the original illustrative example, a human judge engages in natural language conversations with a human and a machine designed to generate performance indistinguishable from that of a human being. The conversation is limited to a text-only channel such as a computer keyboard and screen so that the result is not dependent on the machine's ability to render words into audio.[2] All participants are separated from one another. If the judge cannot reliably tell the machine from the human, the machine is said to have passed the test. The test does not check the ability to give the correct answer to questions; it checks how closely each answer resembles the answer a human would give.
The test was introduced by Alan Turing in his 1950 paper "Computing Machinery and Intelligence," which opens with the words: "I propose to consider the question, 'Can machines think?'" Because "thinking" is difficult to define, Turing chooses to "replace the question by another, which is closely related to it and is expressed in relatively unambiguous words."[3] Turing's new question is: "Are there imaginable digital computers which would do well in the imitation game?"[4] This question, Turing believed, is one that can actually be answered. In the remainder of the paper, he argued against all the major objections to the proposition that "machines can think".[5]
In the years since 1950, the test has proven to be both highly influential and widely criticised, and it is an essential concept in the philosophy of artificial intelligence.[1][6] His test has come to be referred to with Turing's name.
A couple of years ago I had the chance to visit Apple’s headquarters in Cupertino, California. The reason for the trip was to show Apple the current versions of the MakeMusic software and to enlist their help with sorting out how to address a significant amount of technical debt. The experience was incredible as we were able to walk through the Apple campus, eat lunch in the cafeteria, and meet some of the people in charge of the design and development of the Apple products. One conversation that made a strong impression on me was from the lead UI designer for Apple at the time. He was demonstrating some functionality on an iPad when he shared that each of the form factors of Apple’s products were designed around a subset of functionalities. For example:
Phone – designed to monitor information
Tablet – designed to consume information
Laptop or desktop – designed to do work/ complete projects
That isn’t to say that you can’t write your next dissertation on your Samsung but moreover that the best user experience would not come from a device that was optimized for monitoring, not creating.
There is an ongoing debate about how music can be displayed on a device that has a smaller screen than a piece of paper.
Image of a tablet
So typing on a tablet is pretty difficult. Most students prefer typing on a full size keyboard. Some apps like Garageband on an iPad have limited funtionality when compared to a laptop.
Image of a laptop
I’ve had the opportunity to study this effect of the implementation of these design principals with my own students. When given the choice of which device to bring to a class that will involve project work, most students will bring a laptop instead of the school-issued laptop. While this seems to hold true in upper grades where more project work that involves higher level thinking skills, there is strong research that demonstrates significant benefits for younger students to be paired up on a table device. To get back to the beginning of this topic, yes it is possible to do many tasks on any of these devices but what we need to focus on is which device will allow students the maximum benefit.
I’ll never forget walking into Dr. Paul Haack’s office at the University of Minnesota back in 2000. He was my advisor for my thesis and we met regularly to go over my progress. This particular day, I brought in my laptop to show him some data I had collected and he made the comment that a laptop would be a common tool for music educators in the future. He was obviously correct and I would say that most learners now have two devices at all times. Using these devices for assessment is a natural extension.
http://www.cbc.ca/spark/2011/09/spark-154-september-11-14-2011/ (scroll down for audio segment) technologies what technologies get used in the classroom? why? cc-lisenced image source: http://www.flickr.com/photos/unav/4464311633/ cc-lisenced image source: http://www.flickr.com/photos/av_hire_london/5570763086/ cc-lisenced image source: http://www.flickr.com/photos/osde-info/2224883563/ "Great pedagogy is timeless and shouldn’t be tied to one particular technology" engagement enhancement How can we engage "digital natives"? What would these learning environments look like? Technology in the classroom what does the research say? One study (Kraushaar & Novak, 2010) showed: Kraushaar, J. M., & Novak, D. C. (2010). Examining the Affects of Student Multitasking with Laptops during the Lecture Journal of Information Systems Education, 21(2), 241–251.
Howard-Jones, P. (2011). The impact of digital technologies on human wellbeing (pp. 1–98). Nominet Trust. average student cycled through 65 active windows per lecture
2/3 of applications classified as distractive
only one application ended up having effect on assessments: instant messaging Another study (Sovern, 2011) showed: Sovern, J. (2011). Law Student Laptop Use During Class for Non-Class Purposes: Temptation v. Incentives. St. John's Legal Studies Research Paper Series.
Junco, R., Elavsky, C. M., & Heiberger, G. (2012). Putting twitter to the test: Assessing outcomes for student collaboration, engagement and success. British Journal of Educational Technology, Twitter collaboration & engagement, no–no. doi:10.1111/j.1467-8535.2012.01284.x more than 1/2 of students in lecture used laptops for non-class purposes for more than 1/2 class time