uploaded by Hsin-yi and ji

1. How Can Student Logs Inform the Design of Dynamic Visualization for Science Learning?
AERA 2008 Proposal
Chair: Marcia C. Linn, University of California, Berkeley
Discussants: Chris Quintana, University of Michigan
Robert Tinker, The Concord Consortium (to be confirmed)
Objectives
Computer visualization can support student understanding of complex or abstract
concepts in science. Yet students need guidance to effectively interact with and learn from
dynamic visualizations. The purpose of this session is to explore data-driven approaches to the
design of computer environments guiding student learning with interactive-dynamic
visualizations. We present design strategies informed by technology-enhanced research
methodologies such as the analysis of logging data, embedded assessments, or other measures
obtained during the learning process. This session will help researchers establish the theoretical
and empirical foundations of the effectiveness of computer visualizations on student learning in
science.
Background
The value of dynamic visualizations in education is contested (Chandler, 2004).
Cognitive theories that support the use of static visual representations such as the dual coding
model (Pavio, 1971, 1986, 1991) may not fully describe the benefits of dynamic visualization.
Critics argue that benefits of visualizations have not been distinguished from the general impact
of the learning environment (Tversky, Bauer Morrison, & Betrancourt, 2002). Indeed, the
success of a visualization tool in real-world classrooms depends on many factors, including
learners’ prior knowledge, experience, or ability (e.g., Hegarty, Kriz, & Cate, 2003; Rieber,
1989), learners’ strategies, actions and interactions with the visualization (e.g., Lowe, 2004), and
learning processes guided by the instructional practice (Linn & Eylon, 2006). Advances in
technology made it possible to trace students’ responses, actions, and interactions as they learn
with visualizations. Logs of student interactions and embedded assessments can reveal the
quality and trajectory of learning, and cognitive and social processes mediated by the computer
visualization.
This session is organized around a design framework based on a review of research on
dynamic visualization. The framework provides an overview of previous studies and identifies
student difficulties in learning with dynamic visualization and possible theories and strategies to
address these difficulties. In response to the design framework, seven studies in this session
provide empirical findings based on extensive evidence to investigate the effectiveness of
different design approaches to address different areas of student difficulties in learning with
scientific visualizations (see Table 1).
~1~

2. Participants and Structure
The session is planned as an interactive poster session (1.5 hours). The session chair, Dr.
Marcia Linn, will introduce the speakers and the background of the symposium (10 min). Each
presenter will then give a 2-minute introduction to the research (15 min). For the next 45 minutes,
attendees can visit each poster and converse with individual presenters. Presenters will bring
computer-based demonstrations of the technologies used in their research. At the conclusion, Dr.
Robert Tinker and Dr. Chris Quintana will comment on the presentations and moderate a
discussion that allows presenters and attendees to share their insights (20 min).
A Framework for Designing Instructional Practice to Address Student Difficulty in
Learning With Dynamic Visualization in Science
Hsin-Yi Chang
University of California, Berkeley
Computer visualizations show promise for helping students understand complex science
content. However, studies of visualizations have identified at least five types of student
difficulties in learning with visualizations, including attending to the information of the
visualization (e.g., Rieber, 1989), conceiving dynamic processes or abstract relationships (e.g.,
Hegarty, Kriz, & Cate, 2003), connecting visualizations to everyday experiences (e.g., Nakhleh,
Samarapungavan, & Saglam, 2005), transforming between multiple representations (Kozma,
2003), and understanding the purpose of using scientific visualizations (Treagust, Chittleborough,
& Mamiala, 2002).
This paper presents a review of research on the use of dynamic visualization to support
students in learning science. The purposes of this review are (1) to synthesize findings on the
effective design and implementation of dynamic visualizations and (2) to formulate a framework
that presents a rationale and suggests strategies for designing instructional practice to address
student difficulties. Using keywords including dynamic visualization, animation, learning and
science to search the databases of ERIC and PsycINFO 224 citations were obtained. Duplication,
descriptive and position papers lacking empirical data were disregarded, resulting in 68 research
studies included in this review. The results of the review include reframing the definition,
function, and taxonomy, discussing the benefits and limitations, and indicating factors that
influence the effectiveness of dynamic visualization. Finally a framework synthesizing findings
from the literature was proposed to address found students’ difficulties. Future research
directions include the need for methods to capture impacts of visualizations including dynamic
assessments, comparison studies showing how features of visualizations contribute to learning,
and observational studies exploring student interactions with visualizations.
Examining the Role of Self-Monitoring and Explanation Prompts on Students’ Interactions
with Scientific Visualizations
Jennifer L. Chiu
University of California, Berkeley
~2~

3. Computer technology offers powerful visualizations to help students integrate ideas in
science (Dori & Barak, 2000, Pallant and Tinker, 2004; Wu, Krajcik, & Soloway, 2001).
However, research demonstrates that learners have difficulty effectively using dynamic
simulations (Tversky, Morrison, & Betrancourt, 2002). Helping students monitor and evaluate
their understanding while working with these simulations can help students more effectively add
and refine connections among ideas generated from visualizations to their existing knowledge.
This study investigates how triggering learners to assess their understanding after
working with dynamic visualizations can influence students’ interactions with scientific
simulations. Dynamic molecular models of chemical reactions were designed with Molecular
Workbench (Xie & Tinker, 2006), and NetLogo (Wilensky, 1999). These visualizations were
embedded within a week-long computer-based inquiry curriculum unit (Chemical Reactions),
(Linn & Hsi, 2000). This study involves 10 high school chemistry classes taught by three
teachers at an economically and ethnically diverse high school. Half of the students were
prompted to evaluate their understanding immediately after working with a visualization and half
were prompted to assess themselves after generating explanations of the visualization. Student
knowledge was assessed through student responses to prompts embedded within the project, and
pre/posttests. Students’ interactions with the models were captured using data logging
capabilities within the environment. Results suggest that asking students to assess their
understanding helped trigger students to go back and revisit visualizations. These results provide
insight into the design of visualizations and how to help students more effectively monitor their
own knowledge integration.
Scaffolding Students’ Argumentation about Simulations
Douglas Clark, Muhsin Menekse, and Cynthia D’Angelo
Arizona State University
Victor Sampson
Florida State University
Simulations provide rich representations for students exploring science phenomena.
Students often interpret these simulations, however, in non-normative ways. Essentially, novices
have difficulty focusing on the appropriate aspects and the appropriate levels of abstraction that
seem so transparent for experts (e.g., Brewer & Nakamura, 1984; Schank & Abelson, 1977;
Rumelhart & Norman, 1975). Spreading the cognitive load of interpreting visualizations across a
larger social group has been suggested by many theorists (e.g., Andriessen, Baker, & Suthers,
2003; Driver, Newton, & Osborne, 2000; Duschl, 1990, 2000; Koschmann, 2002). The challenge
involves organizing these social interactions to best support students’ investigation of the
richness afforded by the visualizations.
This study investigates 500 students working in groups of two or three in an online
science learning environment. Groups are randomly assigned to experimental condition. In the
first treatment, students first write their interpretations of the phenomena without scaffolding.
Students are then randomly assigned to online discussions where either (a) their own
interpretations of the simulations become the seed comments in the online discussion, or (b)
preselected comments chosen to represent a range of plausible interpretations become the seed
~3~

4. comments in the discussion. In the second treatment, another students use a principle creation
interface constraining the aspects of the visualization upon which they can focus. These groups
of students are then assigned to online discussions of either type (a) or (b). Analysis of the data
from the two phases in terms of students’ incorporation of evidence from the simulations into
their argumentation suggests that students engage in higher percentages of critical thinking about
the simulations in the “high personalization” and “high scaffolding” conditions.
Online Logging of Students’ Performance
Paul Horwitz and Robert Tinker
The Concord Consortium
This presentation is based on several years of research aimed at improving student
performance and the assessment of inquiry skills through the use of interactive models and
logging technology. Today’s classroom computers can run sophisticated simulations of complex
systems and display the results in real time. In parallel with this achievement, data acquisition
and analysis from many kinds of probes is now within reach of any classroom equipped with
standard commercial computers and probeware. These models and tools can greatly extend the
range and depth of inquiry-based learning in K-12 science education through real and simulated
environments. The central challenge to wider use of these resources is that students often lack the
inquiry skills to experiment meaningfully and to interpret the results, and that teachers must be
able to monitor the development of those skills in order to teach them.
Advances in technology and research-based pedagogy have opened up new opportunities
to promote model-based inquiry approaches in the science classroom(Tinker 2003; Xie and
Tinker 2004). Monitoring and logging students’ use of models and probes enables us to guide
their investigations and report on their progress(Horwitz and Tinker 2001). As students use the
technology for inquiry the computer monitors their actions, scaffolding their investigations in
real time, analyzing their inquiry strategies, and formatting reports in the form of formative
assessments for teachers and students(Horwitz, Gobert et al. 2006). We will report on results
obtained in several different NSF-supported projects working in various scientific domains with
middle- and high-school students.
Assessing Spatial Cognition in Visually-Rich Environments
Aaron Price and Hee-Sun Lee
Tufts University
Students find it challenging to understand science concepts that address non-tactile
domains such as those too small (e.g. nuclear fusion) and too large to be seen (e.g., galaxy
clusters). New technologies such as virtual reality and 3-dimensional representations can provide
authentic learning opportunities where students can manipulate and investigate scientific
phenomena at their relevant scales. We developed a prototypical environment that combines two
technologies. One is the Multi-User Virtual Environment (MUVE), an online virtual world
where many users can interact synchronously (Dede, 2004; Linn, in press; Osberg, 1997). The
~4~

5. other is the GeoWall, a 3-dimenaional stereoscopic viewing platform developed by the GeoWall
Consortium (Mir, 2002).
We developed learning tasks to assess middle school students’ understanding of extreme
ranges of scale. These learning tasks were implemented in the Space Visualization Laboratory
(SVL) at the Adler Planetarium in Chicago. For one week, thirty visitors aged 10-14 voluntarily
participated in a one-hour session that was held individually. Each participant took a short survey
consisting of spatial cognition items selected from other standardized sources, received simple
instruction on how to use the environment, and carried out a series of learning tasks. The
learning tasks addressed primarily scale and navigation and tested different aspects of spatial
abilities according to Tversky’s definition (2005) including around the body, of the body, and
external representations. We used assessment results on the written test and the learning tasks as
well as logging data to find whether and how students from different spatial abilities interacted
with the 3d, virtual environment. Preliminary findings show that students’ spatial abilities
assessed with traditional written instruments were positively related to their performance on the
learning tasks.
Use Computer Visualizations to Connect Atomic Models to Observations on Static
Electricity
Ji Shen
University of California, Berkeley
Science educators advocate for a rich learning environment to scaffold students’ learning
(Linn, & Hsi, 2000). Computer visualization provides a powerful means to achieve this goal
(Pallant & Tinker, 2004). This work takes advantage of an online electrostatic module to study
the ideas students use at the observational and atomic levels and reports how computer
visualizations help students connect their observations of electrostatic phenomena to accurate
atomic level explanations.
In electrostatics, a promising solution to help students grasp the particle model is to use
computer simulations (Frederiksen, White, & Gutwill, 1999; Miller, Lehman, and Koedinger,
1999). Manipulative computer simulations will engage students in playing with models, but not
necessarily lead to enhanced understanding (Lowe, 2003). A set of research-based design
principles (Kali, 2006) need to be taken into consideration when designing an online module
employing computer visualizations. Research shows that students bring to science classrooms a
repertoire of ideas on various topics (Linn et. al., 2006) including electrostatics (e.g., Otero, 2004;
Park et al., 2001; Thacker, Ganiel, & Boys, 1999). This paper discusses how these ideas interact
with the learning processes where computer visualizations may help or hinder the development
of scientific concepts based on students’ responses to embedded assessments and notes. The
participants of the study include 36 high school students in VA and 37 high school students in
CA.
~5~

6. Supporting Students’ Experimentation Strategies with Dynamic Visualizations
Keisha Varma
University of California – Berkeley
Even though younger students struggle to design valid experiments, they can learn
effective experimentation strategies (Kuhn, et al, 1992; Schauble, 1996; Lehrer, et al., 2001;
Klahr & Nigam, 2004). However, there is much debate over what should be the focus of
instruction on scientific knowledge and experimentation (Kuhn & Dean, 2005; Klahr, 2005). In
the module, students conduct experiments with the greenhouse visualization by manipulating
levels of solar energy, atmospheric carbon dioxide, Albedo, sunlight, and cloud cover. Activities
prompt students to make predictions and then plan experiments to test their ideas. Following
their investigations, students draw conclusions about the role of the different factors involved in
the greenhouse effect. The guided support also directs students to change only one variable at a
time as they conduct their experiments, to encourage valid investigations leading to normative
scientific ideas (Klahr & Nigam, 2004).
One hundred and thirty seven middle school students worked in pairs to participate in the
module. Each group completed reflection notes embedded throughout the project. The note
prompts helped to guide their experimentation. Students’ responses provide evidence of their
thinking and experimentation strategies. Each individual student also participated in pre/post
assessments of their understanding of the greenhouse effect. Post-test scores about students’
understanding of the greenhouse effect were reliably higher than pretest scores. Following their
participation in the module, students had fewer misconceptions about the factors involved in the
greenhouse effect. Analysis of students’ experimentation plans revealed a wide range of
strategies with very few students understanding that they should use the control of variables
strategy.
Exploring the Impact of A Drawing Activity to Support Learning of Dynamic
Visualizations
Zhihui H. Zhang
Univeristy of California, Berkeley
Learning chemistry involves understanding phenomena at three levels-the microscopic
(molecular), macroscopic and symbolic levels (Johnstone, 1993). Formal instruction often
focuses on the symbolic and macroscopic levels, and assumes that students will automatically
see the relationship of these levels to the microscopic level. However, research shows that
students cannot easily make connections between and within these levels (Kozma, 2000). To
meet such problems, dynamic visualizations of molecular processes are developed to supplement
chemistry instruction.
This controlled study explores whether a drawing activity after working with
visualizations can affect learners’ learning with simulations. The simulations of molecular
processes used in this paper were designed with Molecular Workbench (Xie & Tinker, 2006),
and were embedded within a five-day inquiry-based curriculum unit (Hydrogen Fuel Cell Cars).
183 participants of this study were randomly divided into two groups. The control group learned
~6~

7. by working on the simulations, while the experimental group was required to draw processes of
chemical reactions after working on the same simulations as the control group. Assessments
included five pre/post-test items on atomic structure and chemical bonding and their relationship
to the visible phenomena of chemical reactions. Students in both groups made overall pre/post
gains, demonstrating a highly integrated understanding of the target concepts. Further analysis
revealed the experimental group made significantly better gains than the control group,
indicating that the drawing activity influenced students’ interactions with visualizations by
triggering students to pay attention to crucial features of visualizations.
This work demonstrates the effectiveness of drawing as a supplementary to visualization.
~7~

8. References
Andriessen, J., Baker, M., & Suthers, D. (2003). Argumentation, computer support, and the
educational contexts of confronting cognitions. In J. Andriessen, M. Baker & D. Suthers
(Eds.), Arguing to learn: Confronting cognitions in computer-supported collaborative
learning environments (pp. 1-25). Dordrecht: Kluwer Academic Publishers.
Brewer, W., & Nakamura, G. (1984). The nature and functions of schemas. In R. S. Wyer & T.
K. Skull (Eds.), Handbook of Social Cognition (Vol. 1). Hillsdale, NJ: Lawrence
Erlbaum.
Carey, S., & Smith, C. (1995). On understanding the nature of scientific knowledge. In D. N.
Perkins (Ed.), Software goes to school: Teaching for understanding with new
technologies (pp. 39-55). New York: Oxford University Press.
Chandler, P. (2004). Commentary: The crucial role of cognitive processes in the design of
dynamic visualizations. Learning and Instruction, 14, 353-357.
Dede, C., Nelson, B., Ketelhut, D., Clarke, J., & Bowman, C. (2004). Designbased research
strategies for studying situated learning in a multiuser virtual environment. In Kafai, Y.
B., Sandoval, W. A., Enyedy, N. (Eds.), /Proceedings of the Sixth International
Co//n//ference of Learning Sciences/ (pp. 158-165). Santa Monica, CA: International
Society of the Learning Sciences.
Dori. Y. J., Barak. M. (2001). Virtual and physical modeling: Fostering model perception and
spatial understanding. Educational Technology Society, 4(1) 2001, 61–74.
Duschl, R. (2000). Making the nature of science explicit. In R. Millar, J. Leach J. Osborne
(Eds.), Improving science education: The contribution of research. Philadelphia, PA:
Open University Press.
Duschl, R. A. (1990). Restructuring science education: The importance of theories and their
development. New York: Teachers College Press.
Hegarty, M., Kriz, S., Cate, C. (2003). The roles of mental animations and external animations
in understanding mechanical systems. Cognition and Instruction, 21(4), 325-360.
Horwitz, P., J. Gobert, et al. (2006). Helping Students Learn and Helping Teachers
Understand Student Learning. Concord. 10.
Horwitz, P., R. Tinker (2001). Pedagogica to the rescue: A short history of hypermodels.quot;
Concord 5(1): 1, 12-13.
Johnstone, A. H. (1993). The development of chemistry teaching: A changing response to
changing demand. Journal of Chemical Education, 70(9), 701-704.
Kali, Y., (2006). Collaborative knowledge-building using the Design Principles Database.
International Journal of Computer Support for Collaborative Learning.
Klahr, D. (2005). Early science instruction: Addressing fundamental issues. Psychological
Science, 16), 871-872.
Klahr, D. Nigam, M. (2004). The equivalence of learning paths in early science instruction:
Effects of direct instruction and discovery learning. Psychological Science, 15(10), 661-
667.
Koschmann, T. (2002). Dewey's contribution to the foundations of CSCL research. In G. Stahl
(Ed.), Computer support for collaborative learning, proceedings of CSCL 2002 (pp. 17-
22). Boulder Colorado.
Kozma, R. B. (2003). The material features of multiple representations and their cognitive and
social affordances for science understanding. Learning and Instruction, 13, 205-226.
~8~

9. Kozma, R. B., Chin, E., Russell, J., Marx, N. (2000). The role of representations and tools in
the chemistry laboratory and their implications for chemistry learning. Journal of the
Learning Sciences, 9(3), 105-144.
Kuhn, D., Amsel, E. O’Loughlin, M. (1988). The Development of Scientific Thinking Skills.
New York: Academic Press.
Kuhn, D. Dean, D. (2005). Is developing scientific thinking all about learning to control
variables? Psychological Science,16, 866-870.
Linn, M. C. (in press). Designing for virtual communities in the service of learning (book
review). /The American Journal of Psychology/.
Linn, M. C., Eylon, B.-S. (2006). Science education: Integrating views of learning and
instruction. In P.A. Alexander P.H. Winne (Eds.), Handbook of educational
psychology (2nd Ed., pp.511-544). Mahwah, NJ: Lawrence Erlbaum Associates.
Linn, M. C., Hsi, S. (2000). Computers, Teachers, Peers: Science Learning Partners. Mahwah,
NJ: Lawrence Erlbaum Associates.
Linn, M.C., Lee, H.–S., Tinker, R., Husic, F., Chiu, J.L. (2006). Teaching and assessing
knowledge integration. Science, 313, 1049-1050.
Lowe, R.K. (2003). Animation and learning: selective processing of information in dynamic
graphics. Learning and Instruction, 13, 157-176.
Lowe, R. (2004) Interrogation of a dynamic visualization during learning. Learning and
Instruction, 14, 257-274.
Miller, C.S., Lehman, J.F., and Koedinger, K.R. (1999). Goals and learning in microworlds.
Cognitive Science, 23 (3), 305-336.
Mir, R. (2002). Small Science Museums as a testing ground for virtual reality learning
environment. Paper presented at the VR Workshop, SciTech Hands-On Museum, Aurora
IL.
Nakhleh, M.B., Samarapungavan, A., Saglam, Y. (2005). Middle school students’ belief about
matter. Journal of Research in Science Teaching, 42, 581-612.
Osberg K. (1997). Spatial cognition in the virtual environment. Seattle: Human Interface
Technology Lab. (Technical R-97-18).
Otero, V.K. (2004). Cognitive processes and the learning of physics part 1: the evolution of
knowledge from a Vygotskian perspective. In E.F. Redish M. Vicentini (Eds.)
Proceedings of the international school of physics “Enrico Fermi” course CLVI, Italian
Physical Society. Amsterdam: IOS Press, 409-445.
Pallant, A. and Tinker, R.F. (2004). Reasoning with Atomic-Scale Molecular Dynamic Models.
Journal of Science Education and Technology, 13(1), 51-66.
Park, J., Kim, I., Kim, M., Lee, M. (2001). Analysis of students’ processes of confirmation
and falsification of their prior ideas about electrostatics. International Journal of Science
Education 23(12), 1219-1236.
Pavio, A. (1971). Imagery and verbal process. New York: Holt, Rinehart Winston.
Pavio, A. (1986). Mental representations: A dual-coding approach. New York: Oxford
University Press.
Pavio, A. (1991). Images in minds: The evolution of a theory. New York: Harvester Wheatsheaf.
Rieber, L. P. (1989). The effects of computer animated elaboration strategies and practice on
factual and application learning in an elementary science lesson. Journal of Educational
Computing Research, 5(4), 431-444.
~9~

10. Rumelhart, D., Norman, D. (1975). The active structural network. In D. Norman, D.
Rumelhart L. R. Group (Eds.), Explorations in Cognition. San Francisco, CA: W.H.
Freeman.
Schank, R., Abelson, R. (1977). Scripts, plans, goals, and understanding. Hillsdale, NJ:
Lawrence Erlbaum.
Schauble, L. (1996). The development of scientific reasoning in knowledge-rich contexts.
Developmental Psychology,32(1), 102-119.
Thacker, B., Ganiel, U., Boys, D. (1999). Macroscopic phenomena and microscopic processes:
Student understanding of transients in direct current electric circuits. American Journal of
Physics, 67 (7), 525-531.
Tinker, R. (2003). Experimenting with atoms and molecules. Concord 7(1): 10.
Treagust, D. F., Chittleborough, G., Mamiala, T. L. (2002). Students' understanding of the role
of scientific models in learning science. International Journal of Science Education,
24(4), 357-368.
Tversky, B., Bauer Morrison, J., Betrancourt, M. (2002). Animation: Can it facilitate?
International Journal of Human-Computer Studies, 57, 247-262.
Tversky, B. (2005). Functional significance of visuospatial representations. In P. Shah A.
Miyake (Eds.), /Cambridge// handbook of visuospatial thinking/ (pp. 426-476). New
York, NY: Cambridge University Press.
Xie, Q. and R. Tinker (2004). Molecular Dynamics Simulations of Chemical Reactions for Use
in Education. Journal of Chemical Education.
Xie, Q. Tinker, R. (2006). Molecular Dynamics Simulations of Chemical Reactions for Use in
Education, Journal of Chemical Education, 83, 77-83.
Wilensky, U. (1999). NetLogo [Computer software]. Evanston, IL: Center for Connected
Learning and Computer Based Modeling, Northwestern University.
http://ccl.northwestern.edu/netlogo
~ 10 ~

11. Table 1 Summary of research foci, strategies for designing visualizations, and research outcomes
for each presentation
Presenter Research Focus Design Strategy Research Outcome
Chang A review of 68 studies Proposal of five design A framework for
on dynamic principles to address the synthesizing learning
visualization in learning difficulties from dynamic
supporting science found in the literature visualizations
learning
Chiu Investigation of the use Self-monitoring versus Evidence of the value
of self-monitoring evaluating prompts for self-monitoring
prompts to help students prompts to promote
effectively use dynamic learning from
simulations visualizations
Clark et al. Investigation of social Two levels of Evidence of the value
interactions and scaffolding received and for specific scaffolds on
students’ interpretations personalization allowed learning from
of visualizations visualizations
Horwitz Scaffolding students’ Real-time computer- Evidence that data
Tinker inquiry skills as based scaffolding and mining of student logs
learning with formative assessments can inform visualization
visualizations for teachers and design
students
Price Lee Enhancing students’ 3-D enhanced multi- Evidence that spatial
understanding of user virtual environment cognition influences
extreme ranges of scale interactions with
visualizations
Shen Supporting student Connection with Evidence that students’
connection between everyday experience beliefs influence their
observations of and transformation choice of interaction
scientific phenomena among multiple with visualizations
and atomic level representations
explanations
Varma Promoting students’ Scaffolding students’ Evidence of benefits of
experimentation experiments with visualizations on
strategies and scientific visualizations students’
knowledge experimentation
strategies and content
knowledge
Zhang The use of drawing to Drawing as a way to Evidence of the benefits
help students connect scaffold students’ of the drawing activity
between molecular level interaction with with visualizations
of simulations and visualizations
observable phenomena
~ 11 ~