1. Assessing Scholarly Multimedia by Undergraduates or Faculty Dr. Cheryl E. Ball Editor, Kairos: A Journal of Rhetoric, Technology, and Pedagogy Associate Professor of New Media Studies, Dept. of English Illinois State University, USA cball@ilstu.edu | ceball.com
11. Reasons for Better Performance? Students Scholars Instructions No Formal Instructions 3-day Assignment Deadline 3-4 week volunteer deadline Students’ first experience helping with a publication Rote, albeit invested, role in helping with publications Required; graded Volunteer; no repercussions
For the last decade, the field of digital writing studies has been invested in having students write in multiple media instead of focusing solely on the written word as the main mode of academic communication. This presentation outlines the interconnected nature of editing a scholarly multimedia journal and teaching undergraduates to compose the same kind of texts.
Multimodal theory comes from the New London Group’s (Cope & Kalantzis, 1999) pedagogical work, Multiple Literacies: A Pedagogy for the Design of Social Futures. In this book, this group of interdisciplinary, international education, linguistics, and social-science scholars create a framework for teaching students to design texts across five modes of communication. One of their primary purposes in the book is to make teachers aware that ALL texts are multimodal, so we can teach students to attend to and write within multiple modes.
Since 2000, several major writing programs in the U.S. have begin to incorporate significant multimodal components into their curriculum. These “writing programs” are typically responsible for all of the required writing-intensive courses that new university students have to take. In addition to these programs, many teachers at other universities teach multimodal composition without the support of programs. I am not sure what the rate of teaching multimodal composition (under that or any other title) across the globe is, although I suspect that it is high in Australia and the UK, where much of this theory seemed to originate. I, too, teach a multimodal composition course, which I will talk about momentarily. But first I must turn to how I got interested in teaching multimodal composition, which was through the journal I edit: Kairos.
Kairos is A Journal of Rhetoric, Technology, and Pedagogy. The audience is teachers of writing, primarily for those who teach writing with digital technology at colleges or universities. The mission of this journal is to offer academics a place where they can publish scholarship that can use the affordances of hypertext and multimedia to enact their arguments. The journal started as an experiment by graduate students, 15 years ago, and now has a 10 percent acceptance rate and readers (and we hope, soon, more authors) from more than 180 countries, including Norway. Because Kairos asks authors to enact their argument through both form and content, through linguistic and visual, spatial, even gestural modes of communication—every text submitted to the journal is unique.
Take, for instance, this brief screencast of a recent webtext that uses the metaphor and theory of wunderkammern, or curiosity cabinets, to enact its argument that students should be able to produce texts by arranging and juxtaposing already-existing visual texts. Texts like this one, by Susan Delagrange, exhibits the uniqueness of each text Kairos gets. In this writing situation, form and content become inseparable—an important concept for students to learn and practice in an age when ubiquitous uploads to Facebook and YouTube pervade their lives. But Kairos shows that this kind of work isn’t just for students, and in fact predates students’ consumption and production of digital media texts. Every time Kairos gets a submission that uses a new technology, a new combination of modes or media, a new remix of academic and popular genres, I have to figure out how to READ the text. Going through this relearning requires me to ask myself: What IS scholarship in this field? In some ways, teaching undergraduates to produce this kind of work has helped me to answer that question. And it’s to that class that I turn next.
At Illinois State University, one of the original “normal” or teaching universities for teachers, I teach a class for undergraduate students (from any major), in which they learn to read, analyze, and assess other authors’ scholarly multimedia projects. They also learn to propose, compose, revise, and peer-review their own scholarly multimedia, which they then submit to online journals. Although it may seem like I took this idea directly from my work with Kairos, it was actually a different scholarly multimedia project that prompted my revamp of the syllabus. I don’t have time to talk about that original idea because I want to focus on how the course has turned into a rhetorical genre studies course on scholarly multimedia, where students teach ME about what they value in the kinds of texts I work with authors on every day.
In this particular class, the writing process that students undertake when composing multimodal scholarship includes all of the following types of academic literacy assignments. WALK THROUGH HOW THIS LOOKS WITH JOURNALS. It is the peer-review and the course reflection that I will focus on here.
In order to come up with criteria to evaluate their own webtexts, students first pick favorite or interesting pieces of digital media texts and have to tell the class WHY they thought these pieces were interesting, successful, and other criteria that made them want to share the pieces with the class. They are then given three sets of evaluative criteria mentioned here, all of which are specific to the kinds of scholarly multimedia published in journals like Kairos. (I’m not going to detail these, because they aren’t really the point of this particular talk.) Using the four sets of criteria (their own + the three above), students analyze published webtexts to decide which criteria they find the most useful. They then have to justify their “short list” by publishing it on their class blog and explaining their choices.
In the past, I would make students come up with a class-wide short-list of evaluative criteria to use when workshopping their own scholarly webtextual projects. For instance, the frameworks that students created in my Spring and Fall 2009 Multimodal Composition classes are surprisingly similar to print-based scholarly standards and include the these criteria. However, some previous feedback on this presentation suggested that I should let students use whatever criteria they WANT. That is what I did this semester, and the results were as good as previous semesters.
I decided to do this for two reasons: (1) Because students are constantly teaching me new things about digital media so I should listen to what they value in these pieces (as long as they can justify why they value those criteria), and (2) because the editorial board of Kairos doesn’t have specific evaluation criteria that they use to judge the worth, rigor, and readiness of a webtext for that publication -- instead, they use the criteria they’ve built up in their heads, just as the students have done through my tutelage. So why should I falsely require students to use some concrete set of criteria when that criteria is constantly shifting with each new webtext we receive? I don’t have permission to share any of their peer-review letters with you, but I can tell you that because the students have explicit instructions on HOW to write a peer-review letter (based on how an editorial board does it), AND because the students have spent weeks defining the criteria by which they and I will evaluate each other’s work, their peer-review letters are BETTER than those typically written by the editorial board. And there’s several reasons for that:
Because of their attentive evaluative practice of scholarly multimedia, I want to make sure students can transfer this learning process and outcomes to other courses, so I asked them to annotatate their peer review lettesr this semester, which I have just gotten back, so I do not yet have data on them. In addition to their peer-review letters being better than the editorial board, their final projects are also, in many ways, better than those from semesters when students were not asked to analyze and create evaluation criteria for this mixed genre work.
SHOW EXAMPLE
I will conclude by suggesting a similarity between the level of students’ production values and what authors of Kairos produce when they first submit a webtext for review (many of whom are first-time scholarly multimedia authors). My aim in both cases is to have authors receive (at least) a revise-and-resubmittal letter. As an editor, I cannot expect Kairos authors to produce perfect (i.e., accepted for publication) work the first time around, nor as a teacher should I expect students to produce at that level when they are composing multimedia for the first time.
Students’ portfolio reflections help them articulate what they learned about multimodal composition and rhetoric through the semester (and the group project work). This student, for instance, specifically talked about the peer-review criteria in relation to one of the published webtexts we evaluated and what learning about that process meant to his own academic and personal identity. 1:26 seconds in Remain open to change in answering the question “What is (digital) scholarship?” (EXPLAIN SPECIAL ISSUE AND WATSON CHAPTER & it’s possible impact on teachers!!)
I know this presentation has touched on multiple topics that aren’t usually related: undergraduate writing classrooms, multimodal theory, genre studies, and journal editing. But by combining these different academic identities for myself, I’ve learned a great deal from my undergraduate students’ learning process. I end by suggesting that we need to re-examine our teacherly expectations when working in digital media, although that is not to say that we must lower our standards. When students taking a multimodal class for the first time can produce work that is on par with much of what first-time Kairos authors produce—a bar-raising event for students—my grading of their work must shift to accommodate what that level of work means in the world of scholarly multimedia. It also means that I’ve changed my editorial practices with Kairos: I’ve become more specific about WHAT I’m asking them to review for (GIVE EMAIL EXAMPLE), which has helped them produce better reviews. I’ve also instituted the kind of correspondence feedback loop into the grant work I’m doing for Kairos’ CMS. (EXPLAIN BRIEFLY) I hope as the grant work is integrated into the international journal system that more journals will take on scholarly multlimedia as part of their publications. In turn, I hope to find out that more teachers are asking undergraduate and graduate students to produce this work, and to document the students’ learning outcomes through multimedia using qualitative and quantitative research methods.