DISUCSSION 20

CBE—Life Sciences Education • 16:ar28, 1–13, Summer 2017 16:ar28, 1

ARTICLE

ABSTRACT
Drawing by learners can be an effective way to develop memory and generate visual mod-
els for higher-order skills in biology, but students are often reluctant to adopt drawing
as a study method. We designed a nonclassroom intervention that instructed introducto-
ry biology college students in a drawing method, minute sketches in folded lists (MSFL),
and allowed them to self-assess their recall and problem solving, first in a simple recall
task involving non-European alphabets and later using unfamiliar biology content. In two
preliminary ex situ experiments, students had greater recall on the simple learning task,
non-European alphabets with associated phonetic sounds, using MSFL in comparison with
a preferred method, visual review (VR). In the intervention, students studying using MSFL
and VR had ∼50–80% greater recall of content studied with MSFL and, in a subset of trials,
better performance on problem-solving tasks on biology content. Eight months after be-
ginning the intervention, participants had shifted self-reported use of drawing from 2% to
20% of study time. For a small subset of participants, MSFL had become a preferred study
method, and 70% of participants reported continued use of MSFL. This brief, low-cost
intervention resulted in enduring changes in study behavior.

INTRODUCTION
Visual representations are ubiquitous in science as a tool for teaching, understanding,
communicating, and developing ideas (Van Meter and Garner, 2005; Quillin and
Thomas, 2015). Visual representations in primary research publications are used rou-
tinely to present new hypotheses, and many of these visual representations eventually
develop into abstract visual models presented to students as illustrations in textbooks.
Visual representations of concepts drawn by students can help them recall (Wammes
et al., 2016), think, generate hypotheses, develop predictions and experiments, and
communicate results (Van Meter and Garner, 2005; Schwarz et al., 2009; Ainsworth
et al., 2011; Quillin and Thomas, 2015). Drawing may be most useful when it actively
engages students in selecting, organizing, and integrating information to develop a
visual model that represents a mental model (Van Meter and Garner, 2005; Mayer,
2009; Quillin and Thomas, 2015). Learner-generated visual models drawn by students
can aid in their acquisition of knowledge and their communication of ideas to others
(Van Meter and Garner, 2005) while serving as an important tool to aid in problem
solving (Quillin and Thomas, 2015). The literature cited earlier suggests that drawing
may aid students in learning tasks from the simplest, such as developing memory for
core content, to the most complex, including hypothesis generation, prediction, and
analysis.

Instructors in biology commonly draw models in aid of understanding or when
presenting problems to students, but generally not with an explicit goal of developing
drawing skills in students for use in thinking and modeling (Quillin and Thomas,

Paul D. Heideman,* K. Adryan Flores, Lu M. Sevier, and Kelsey E. Trouton
Department of Biology, College of William and Mary, Williamsburg, VA 23187

Effectiveness and Adoption of a Drawing-
to-Learn Study Tool for Recall and
Problem Solving: Minute Sketches with
Folded Lists

Deborah Allen, Monitoring Editor
Submitted June 6, 2016; Revised March 8, 2017;
Accepted March 21, 2017

DOI:10.1187/cbe.16-03-0116

*Address correspondence to: Paul D. Heideman
(

[email protected]

).

© 2017 P. D. Heideman et al. CBE—Life Sciences
Education © 2017 The American Society for Cell
Biology. This article is distributed by The American
Society for Cell Biology under license from the
author(s). It is available to the public under an
Attribution–Noncommercial–Share Alike 3.0
Unported Creative Commons License (http://
creativecommons.org/licenses/by-nc-sa/3.0).

“ASCB®” and “The American Society for Cell
Biology®” are registered trademarks of The
American Society for Cell Biology.

CBE Life Sci Educ June 1, 2017 16:ar28

16:ar28, 2 CBE—Life Sciences Education • 16:ar28, Summer 2017

P. D. Heideman et al.

2015). It may be important to develop interventions that build
student skills (Van Meter and Garner, 2005; Leutner et al.,
2009; Schwarz et al., 2009; Quillin and Thomas, 2015),
because, for some tasks, drawing has not produced gains in
learning (see discussion in Leutner et al., 2009; Quillin and
Thomas, 2015). While drawing-to-learn shows promise as a
learning tool for students, an important challenge is the devel-
opment of interventions that successfully motivate students to
draw and improve skills in drawing and model-based reasoning
(Van Meter and Garner, 2005; Quillin and Thomas, 2015).

It is challenging to motivate college students to change study
methods, even though students commonly use study methods
that are known to be ineffective compared with more active
study methods (Fu and Gray, 2004; Dunlosky et al., 2013). Stu-
dents hesitate to adopt new learning methods, even when they
have evidence for an effective replacement (Pressley et al.,
1989; National Research Council [NRC], 2001; Fu and Gray,
2004). This reluctance to apply a new method is understand-
able when students lack experience and metacognitive skills to
apply new methods and self-monitor the effectiveness of their
learning (Bielaczyc et al., 1995; Bransford, 2000; Tanner, 2012).
Students may require both metacognitive skills to assess new
study methods (Schraw et al., 2006) and personal experience
with an effective new study method. In our experience teaching
drawing to college students for model-based reasoning, the
challenge has been in demonstrating short-term gains in return
for their efforts to produce and practice drawing. Students need
practice and experience with explicit instruction on the value of
investing time and effort on learner-generated drawings. Our
informal interventions with individual students over many
years is consistent with proposals from others: an effective
intervention requires 1) teaching an effective method for draw-
ing (Van Meter and Garner, 2005), 2) practice sessions with
feedback (Schwarz et al., 2009; Quillin and Thomas, 2015),
3) an explanation of drawing for self-regulated learning as
a metacognitive strategy (Bransford, 2000; NRC, 2003), and
4) having each individual self-assess the utility of drawing for
learning gains he or she values.

Here, we describe results of an intervention with three
goals for students: 1) learning a drawing method offering
greater effectiveness of study for recall, understanding, and/
or problem solving than the passive study methods preferred
by most of our students; 2) self-scoring assessments to
self-evaluate effectiveness; and 3) adoption by students of the
method, if effective, for enduring, routine use. This drawing-
to-learn intervention is consistent with research on learning
and applications of drawing methods in teaching (see Van
Meter and Garner, 2005; Forbus et al., 2011; Jee et al., 2014;
Wammes et al., 2016) but was largely developed empirically,
in an iterative process involving many individual students
over many years. The drawing strategy uses a minimalist
approach to drawings (termed “minute sketches,” see Methods
for more details), potentially minimizing the time necessary
for practice and minimizing cognitive load (Sweller, 1988;
Sweller and Chandler, 1994; Schnotz and Kürschner, 2007;
Leutner et al., 2009; Verhoeven et al., 2009). In this approach,
minute sketches are associated with terms using retrieval
practice (Karpicke and Blunt, 2011) with a “foldable” or
“folded list” (see Methods). In combination, minute sketches
with folded lists (MSFL) are intended to be a study method to

develop recall and understanding, producing learning out-
comes that students value sufficiently to adopt drawing as a
long-term study method. Minute sketches are intended for
active-learning applications (Freeman et al., 2014) in high-
er-order cognitive processing, in which students learn to view
each minute sketch as a hypothesis for a structure or a pro-
cess, with the ability to manipulate the visual model to develop
predictions and solve problems. Our ultimate goals for stu-
dents when using minute sketches includes hypothesis gener-
ation, making predictions, and problem solving, but students
in science, technology, engineering, and mathematics (STEM)
also need to have skills at recalling extensive amounts of
discipline-based content. It may be the latter that motivates
students to adopt drawing-to-learn.

The study was both an intervention and a quasi-experimen-
tal test of MSFL as a study tool. In three experiments, we tested
whether students performed better in recall and/or problem
solving with MSFL than with a preferred study method: visual
review (VR; see Results for evidence that, for our students, VR
is a preferred study method). In the third experiment, we also
assessed whether an intervention might result in increases in
drawing as a study method. We used pre–post surveys to assess
the use of drawing for study in the intervention group and a
comparison group. On the basis of the results, we discuss poten-
tial uses of MSFL to help students learn, practice, and self-assess
this addition to their study methods.

METHODS
Minute Sketches
A minute sketch (Figure 1) is a simple diagram, ideally user
generated (Quillin and Thomas, 2015), that uses the minimum
number of lines and symbols necessary to represent a concept
fully to a user, making it reproducible in less than a minute. The
60-second limit prevents wasteful overelaboration, enforces
focus on essentials, and reduces risk of cognitive overload
(Sweller, 1988; and see coherence principles 1 and 3 in Mayer,
2009). All terms needed with the sketch are placed in a list to
one side (an application of multimedia learning; Mayer, 2009),
never on the sketch itself, because humans can find simultane-
ous processing of images and text challenging (Figure 1).
Sketches may include numbers to indicate sequence or letters
that are commonly interpreted as symbols (e.g., Na+ for sodium
ions or H2O for water). The components of the sketch should
always remind the user of any necessary explanation; a written
explanation is not allowed. Explanations in multimedia learn-
ing may cause an unhelpful “redundancy effect” (Sweller and
Chandler, 1994; Mayer, 2009), in which redundant text added
to a diagram increases cognitive load and inhibits learning
(Sweller and Chandler, 1994). If a sketch is sufficient to trigger
recall and understanding, then redundant text wastes time and
energy to process that unnecessary content (Schnotz and
Kürschner, 2007).

Any minute sketch is also a hypothesis about structure,
function, or relationships. Minute sketches typically include
relationships broadly comparable to those in structure–behav-
ior–function (SBF) models that connect structure to behavior
and function in biology (Dauer et al., 2013; Speth et al.,
2014). Changes may be made to the original minute sketch,
similar to alternate pathways in an SBF model (e.g., see
Figure 1 in Speth et al., 2014), allowing the newly manipulated

CBE—Life Sciences Education • 16:ar28, Summer 2017 16:ar28, 3

Drawing-to-Learn Using Minute Sketches

sketch to be used in predicting changes to the system. For
example, in the minute sketch for the water cycle (Figure 1C),
a student might choose to increase atmospheric dust particles.
Using the sketch, a student can identify potential changes
through each succeeding step in the entire water cycle (e.g.,
less sunlight reaching water, causing less evaporation, causing
…). As students use minute sketches for more complex con-
cepts, earlier minute sketches appear in simplified form, con-
sistent with findings that representations may become simpler
with increasing expertise (Hay et al., 2013; Dowd et al., 2015).
For example, in the minute sketch for the water cycle (Figure
1C), symbols exist for evaporation (dots rising from the ocean)
and condensation (dots becoming small droplets), each of
which can be elaborated further in separate minute sketches,

producing an interconnected web. Stu-
dents are capable of misapplying or mis-
understanding symbols such as arrows or
connections within sketches or diagrams
(Novick and Catley, 2007; Wright et al.,
2014), and so an important step in devel-
oping minute sketches is checking under-
standing by referring back to instruc-
tional materials or instructors.

Folded Lists
In addition to understanding, learning
requires fluent recall of essential con-
cepts with associated terms. The tool for
fluent recall of a minute sketch with
associated terms is a folded list or “fold-
able” (Figure 2) for retrieval practice.
Students view only one column at a
time, either minute sketches or terms,
hiding the other column. The hidden
column is rewritten or resketched from
memory, checking a hidden column
when needed (Figure 2). Students are
encouraged to recheck the hidden col-
umn when needed in order to minimize
a potential negative effect of guessing
(Roediger and Marsh, 2005; McDer-
mott, 2006; Chang et al., 2010). When
sketching, students are asked to think
about each term as they sketch, refer-
ring back to instructional materials as
needed to check their understanding.
When writing terms, students are asked
to think about the connection between
each term and the sketch. Students may
vocalize terms and explain sketches
aloud as they draw (see the modality
principle in Mayer, 2009). Students may
save time in later iterations by abbrevi-
ating or removing terms or lines they
recall easily (Figure 2, panels on right
side). When students can readily repro-
duce and explain the sketch from the
terms and the terms from the sketch,
they know they have achieved fluent
recall with their current understanding

on that day. If they cannot, students have an unambiguous
self-diagnosis: they need more practice.

Experiments
Informal feedback from students suggests that after practicing
two complete cycles (sketch to terms and terms to sketch
repeated twice) on three different days, students have fluent
recall with their current understanding for at least 1 day, and
usually for more. Similar feedback suggests that, weeks or
months later, before a final exam or when reviewing content
for advanced courses, practice on as little as one additional
day recovers their recall and understanding. This feedback
was incorporated into the design of the experiments discussed
here.

FIGURE 1. Minute sketches for three concepts, each instructor drawn based on
student examples. (A) Allopatric speciation is hypothesized to occur when individuals
(squares at left) in an original single population (circle at left) are isolated by a
geographic barrier such as an ocean or a mountain range (jagged line at center labeled
“1”). Over time (arrows in lower oval) one population changes (squares becoming
circles in the lower oval), while the other may or may not change (upper oval with
squares). Eventually, even if individuals encounter each other, they have accumulated
too many differences for successful reproduction (circle with X labeled “3”). (B) In a
hypothesis for the costs of territory defense, there are three costs of defending a
territory, indicated by the boundary line, from others of the same species: birds on the
right side facing each other across the territory boundary. Resources used for territory
defense cannot be used for other purposes, such as reproduction (the egg, represent-
ing resource cost). Vigilance and fighting for territory defense increases risks, such as
risk of predation (the predator head, representing risk cost). The time spent defending
the territory is an opportunity cost, time lost from alternative activities, such as
feeding (the bird head pecking at food, representing opportunity cost). In B, note that
the term for each cost has a location that corresponds to its representation in the
sketch. (C) The water cycle. Each of the three minute sketches represents the essen-
tials of the concept or hypothesis, and elements in the sketches may be manipulated
to allow the user to predict outcomes (see the text).

16:ar28, 4 CBE—Life Sciences Education • 16:ar28, Summer 2017

P. D. Heideman et al.

The first two experiments were intended to assess the poten-
tial of MSFL for a simple learning task. Experiment 1 was
intended to assess MSFL in comparison to VR when learning
simple content in the absence of information on learning and
study methods. This experiment compared students using
either MSFL or VR on a simple non–STEM learning task: learn-
ing characters from non-European alphabets with associated
phonetic spellings of sounds. While the goal of the experiment
was not explicitly stated, information to participants was pre-
sented in such a way that the experiment could be interpreted
reasonably as a test of the difficulty of learning different alpha-
bets. A second experiment (experiment 2) gave each partici-
pant the same learning task, but in a crossover design with
matched study time on equivalent content, with one page stud-
ied by MSFL and the other by VR.

The third experiment (experiment 3) was an intervention in
which participants studied using MSFL in comparison with VR
on matched content with self-assessment of their learning with
the two study methods. The first part of experiment 3 allowed
students to assess whether their retention of simple content,
unfamiliar non-European alphabets, was better using MSFL or
VR. Importantly, this part of the experiment also allowed inves-
tigators to correct misapplications of MSFL as a study method.
The second part of experiment 3 allowed students to self-assess
whether their retention and problem solving on unfamiliar
biology content was better after studying using MSFL or VR.

Experiment 1. Between-Subjects Test of MSFL and VR. In an
ex situ experiment with participants recruited from an organic
chemistry course (N = 33; provided with a $15 gift card), we
assessed MSFL in comparison with VR when learning simple
content in the absence of information on learning and study
methods. The non–STEM learning task was to learn characters
from non-European alphabets with associated phonetic spellings
of sounds, a task similar to learning symbols with pronunciations
in STEM courses (such as μ or Å with an associated sound or
term, micron and angstrom, respectively). The use of characters
from an unfamiliar non-European language (factor: language—
Arabic or Korean) allowed us to control for prior knowledge.
Participants were assigned a language at random; participants

already familiar with an assigned alphabet were reassigned to
the alternative. Participants were asked to learn 12 characters
and their phonetic spellings using one of two study methods
(factor: study method—MSFL or VR). Separate sessions were
held for the VR and MSFL treatment groups. Participants were
allowed to infer that the study’s purpose might be to assess
the difficulty of learning different alphabets. After a survey on
demographics and study methods and an introduction to the
character sets, participants were given 5 minutes of instruction
on their study method and 3 minutes of practice studying their
character set. Participants were instructed to practice on their
own for 5 minutes on each of three different days, recording start
and end times on each day (see timeline in Figure 3). Practice
was not allowed on the day of the quiz. The MSFL group was
provided with pages for practice and asked to submit these prac-
tice sheets. In a quiz 7–10 days later, participants reproduced
their character set with phonetic spellings. Quizzes were scored
by the investigators, blind with respect to treatment. Two points
were awarded for a correct pairing of character with phonetic
sound, only one point for correct characters with no phonetic
spelling or a phonetic spelling that was mismatched, and ½
point was removed for minor errors (such as a slightly misdrawn
character) according to a rubric (maximum score = 24).

Experiment 2. Within-Subjects Test of MSFL and VR. In a
crossover design, this ex situ experiment asked each participant
to study one page of content (non-European alphabets) using
MSFL, and a second, equivalent page using VR. Participants
were recruited from college calculus courses (N = 23; provided
with a $20 gift card). Participants were assigned randomly to
the Korean or Arabic alphabet (factor: language); those report-
ing familiarity with an assigned alphabet were reassigned to the
other alphabet. Each participant studied two content pages (fac-
tor: content page) of equivalent difficulty that were assigned at
random for study using MSFL or VR. The session on day 1 fol-
lowed methods of experiment 1, except that all participants
received instruction with both methods, and all practiced one
page with VR and one page with MSFL. Practice before the quiz
7–10 days later was as in experiment 1, except that each partic-
ipant practiced one set of characters with VR and the other set

FIGURE 2. Representation of a minute sketch and list of terms practiced in a folded list in a series of steps (example was drawn by an
investigator based on many student examples). 1) The user writes the list of key terms. 2) While thinking through the terms, the user
redraws the minute sketch from memory. 3) After folding the terms under the paper (or covering the terms), the user rewrites the terms
from memory while thinking through the minute sketch. 4) After hiding the sketch, the user redraws the sketch from memory while
thinking through the terms. As users gain familiarity with the terms and sketch, they are encouraged to skip obvious terms (e.g., lake,
ocean, cloud), abbreviate when the abbreviation is a sufficient reminder (e.g., “w. vap.” and “precip.”), and reduce the sketch to essentials.
In the sketch on the right, there is one cloud instead of two, fewer lines of raindrops, and fewer arrows. Any line that fails to add meaning
can be eliminated. This example shows a typical simplifying progression.

CBE—Life Sciences Education • 16:ar28, Summer 2017 16:ar28, 5

Drawing-to-Learn Using Minute Sketches

with MSFL, each for 5 min, on three different days. There was
no practice on the day of the quiz. The quiz was administered
and scoring was conducted as in experiment 1.

Experiment 3. Within-Subjects Intervention Test for Effec-
tiveness and Adoption of MSFL. This multisession interven-
tion tested whether students who applied the two methods
might change their study methods after self-assessment of
their learning using MSFL. Participants were recruited from a
first-semester biology course (an introduction to ecology, evo-
lution, Mendelian genetics, and biodiversity); participating in
the study was one of several methods through which they
earned ∼1% of the course credit. The intervention began with a
survey and a 25-minute workshop on study methods, including
evidence that passive VR has low effectiveness (summarized
from Dunlosky et al., 2013). Character sets and instructions
were presented as in experiment 2. Two participants who were
familiar with both Arabic and Korean were assigned character
sets from Tamil. The subsequent three sessions included quiz-
zes that were self-scored by participants and later scored by the
investigators, blind with respect to study method (for correla-
tions between self-scores and investigator scores, see Results).
The intent of self-scoring was to make participants aware of any
differences in their learning after studying with the two differ-
ent methods. The pattern of practice sessions and subsequent
quizzes was intended to match studying that might be con-
ducted before an exam (quiz 1), followed by a period of time
without study following an exam (quiz 2), and later a single
study session that might come before a final exam (quiz 3; see
timeline in Figure 4).

In session 4, all continuing participants received a new
20-minute presentation on content that was unrelated to any
course content (thyroid hormone regulation and metabolism),
including two pages of diagrams and accompanying typed text
(see the Supplemental Material). The two sets of diagrams
were approximately matched for difficulty. During the presen-
tation, brief mention was made that minute sketches might be
useful as an aid in solving problems. Next, participants received
instructions on minute sketching and were told to generate
their own minute sketches based on the content provided. As
with the character sets, participants studied one page using VR
and the other page using MSFL, assigned at random. Before
session 5, participants studied each page for 5 minutes, using
VR for one page and MSFL for the other. In session 5, partici-
pants were asked to reproduce the content, with explanations,
along with two problems to solve using the content. Before ses-
sion 6, 2 weeks later, participants were told to do no additional
studying. Before session 7, held 2 weeks after session 6, partic-
ipants were asked to study both pages on the night before the
session, one page as assigned with VR and the other with MSFL.
Again, the pattern of practice sessions was intended to match
studying that might be conducted before an exam (session 5),
time passing after an exam with no additional study (session 6),
and, finally, a single study session that might come before a
final exam (session 7; see timeline in Figure 4). In each of the
three sessions, quizzes were self-scored by students following a
rubric. Quizzes were later scored by the investigators, blind
with respect to study method.

Four months after session 7, a subset of participants were
recruited from the intervention group ($35 gift card) for a brief
intervention on problem solving using minute sketches. After a
presentation on using MS for problem solving, participants
received a 25-minute lecture with diagrams on immune func-
tion. Participants were randomly assigned one of the two pages
to practice for recall and problem solving using VR (10 minutes
on each of 3 days), and the other page to practice content and
problem solving using MSFL (10 minutes on each of 3 days).
One week later, participants solved problems on both pages of
content. In addition to the self-scoring by participants, investi-
gators scored all quizzes, blind with respect to study method.

The survey on study methods (pre) administered to partici-
pants in the intervention group (N = 69) was also issued to a
comparison group drawn from the same class (N = 55). The
same survey was readministered (post) 7 months later (inter-
vention group: N = 44 of the 56 participants who completed at
least two sessions for the intervention, a 75% participation rate;
N = 57 for the comparison group from the same course). Each
survey asked for percentage of study time allocated to the fol-
lowing study methods: 1) rereading notes, 2) rereading presen-
tations (such as PowerPoint), 3) rereading the textbook,
4) rewriting notes, 5) summarizing (from notes, a presentation,
or a textbook), 6) highlighting, 7) flash cards, 8) drawing or
sketching, 9) practicing by redrawing, 10) practice testing,
11) self-testing, 12) writing one’s own exam questions,
13) mnemonics, 14) “chunking” content, 15) retrieval practice,
16) using folded lists/foldables, and 17) minute sketching.
Participants were also asked to indicate the percentage of time
in which they used distributed practice. The full survey is
presented in the Supplemental Materials. Respondents were
given the option to add study methods, but few added any

FIGURE 3. Timeline for experiments 1 and 2: (A, in red) VR of
12 characters and associated phonetic spellings practiced for
5 minutes on each of three different days before the day of a quiz,
and (B, in blue) practice using minute sketches with folded lists
(MSFL) of the same 12 characters and associated phonetic spellings
for 5 minutes on each of three different days before the day of a
quiz. In experiment 1, the only experiment in which participants
used a only a single study method, each participant had only one
set of 12 characters and followed the study method in either A or B.
In experiment 2, each participant was given two pages of alphabet
characters and sounds and studied one page using VR and the
other page using MSFL. See Methods for additional details.

16:ar28, 6 CBE—Life Sciences Education • 16:ar28, Summer 2017

P. D. Heideman et al.

other method. Study methods 1–6 were defined for this exper-
iment as having low effectiveness (based on Dunlosky et al.,
2013) and termed here “passive”; study methods 7–17 often
include retrieval practice and greater physical activity and were
defined for this experiment as having higher effectiveness
and termed here “active” (for more discussion on passive and

active study methods see Ward and Walker,
2008; Barger, 2012; Dunlosky et al., 2013;
Husmann et al., 2016).

Characteristics of Groups across
Experiments
Across the participant groups, demograph-
ics were similar but not identical (Table 1).
The groups in all three experiments were
similar in the perceived amount of study-
ing relative to other students. There were
fewer STEM majors among participants in
experiment 2 than in other experiments
and fewer students of color in the compar-
ison group in experiment 3 than in the
intervention group in experiment 3. With
these sample sizes and the limited data
about participants, we cannot conclude
that the groups were equivalent.

Samples and Statistical Analysis
Data analysis was carried out using RStu-
dio 0.97.449 running R 2.14.0 on a Macin-
tosh computer. In all assessments of recall
and problem solving, participants were
asked to indicate whether they might have
made errors in following the methods, and
if so, to explain those errors. Data from
those known to have made errors follow-
ing the methods were removed from the
data for each quiz before analysis. Sample
sizes provided include only those who
reported following instructions correctly.
However, analyses conducted without
excluding these individuals produced
qualitatively identical results.

In experiment 1 (N = 33), because of
nonnormal distributions and unequal vari-
ance between groups, data were analyzed
using the bootstrap with analysis of vari-
ance (ANOVA; Xu et al., 2013a,b). In
experiment 2 (N = 23), scores were com-
pared using repeated-measures ANOVA;

the factors were language and content page, with study method
as the repeated measure. A G-test assessed whether a greater
proportion of students scored better with VR or MSFL. In exper-
iment 3 (N = 36–51 in the various quizzes), scores were com-
pared as in experiment 2. Self-scores were compared with inves-
tigator scores of recall and problem solving using 1) two-way

TABLE 1. Demographics of participants in surveys and experiments

Experiment Subgroup Source N Age
Sex

(% F)
Students of
color (%)

Freshmen
(%)

STEM
majors (%)

1 Organic Chemistry I 33 18–21 88 36 38 84
2 Introductory Calculus II 23 18–21 61 39 61 57
3 Comparison Pre Introductory Biology I 55 18–21 75 22 73 75
3 Comparison Post Introductory Biology I 57 18–21 82 18 59 72
3 Intervention Pre Introductory Biology I 69 17–21 72 40 74 72
3 Intervention Post Introductory Biology I 44 18–20 77 40 74 84

FIGURE 4. Timeline for experiment 3. In experiment 3, the first two sessions were
identical to experiment 2, with participants instructed to do time-matched study of
alphabet characters and associated phonetic spellings of sounds from non-European
alphabets using VR (solid line in red) and MSFL (dashed line in blue) followed by a quiz
(session 2). Experiment 3 continued with a second quiz after 3 weeks without study
(session 3), and a third quiz after review (session 4). After the quiz in session 4, participants
were given novel biology content in two sets of matched pages, one set for study with VR
and the other with MSFL, followed by a quiz in session 5, another quiz after 2 weeks
without study (session 6), and a final quiz after review (session 7). Sixteen weeks later, a
subset of participants received novel biology content (session 8) and additional instruc-
tions on using minute sketches to solve problems. Following matched study time on
matched content, participants took a quiz that included only problems to solve (session
9). See Methods and the Supplemental Materials for more details.

CBE—Life Sciences Education • 16:ar28, Summer 2017 16:ar28, 7

Drawing-to-Learn Using Minute Sketches

repeated-measures ANOVA, with the factors for each study
method being content page and language (quizzes 1–3) or
biology content (quizzes 4–6), with self-scoring versus investi-
gator-scoring as the repeated measure; and 2) correlation
between self-scores and investigator scores using Spearman’s
correlation coefficient. Survey responses were compared using
t tests, paired for pre–post in the treatment group (in which
the same individuals participated in both surveys) and
unpaired for other comparisons. In experiments 2 and 3, when
preliminary analysis indicated no significant effect of lan-
guage or page of content, these factors were not included in
the final analyses.

In cases in which data did not meet assumptions of normal-
ity or equality of variance, analyses used the bootstrap with
t tests (Konietschke and Pauly, 2014) or with ANOVA (Xu et al.,
2013a,b). In all cases in which multiple statistical tests were
conducted on a single question, the false discovery rate control
(Thissen et al., 2002) was used to adjust significance, with the
experiment-wise likelihood of acceptance of a falsely significant
result set at p < 0.05.

Experiments were approved by the College of William and
Mary Protection of Human Subjects Committee under protocols
PHSC-2014-09-09-9733-pdheid, PHSC-2015-03-23-10285-pd-
heid, and PHSC-2015-01-27-10051-pdheid.

RESULTS
Experiment 1. Between-Subjects Test for Effectiveness
for Recall
The group studying using MSFL had scores for recall that were
20% higher than those for the group studying with VR
(Figure 5, p = 0.046). The difference was significant despite a

ceiling effect, with many perfect or near-perfect scores in
the MSFL group. There was no significant effect of language
(p = 0.96) or interaction between language and study method
(p = 0.61).

Experiment 2. Within-Subjects Test for Effectiveness
for Recall
Unlike the results of experiment 1, in this task with twice the
content of experiment 1, only a few individuals achieved high
scores of greater than 90% (Figure 6). On average, participants
recalled 50% more of the content they studied with MSFL in
comparison with VR (p < 0.01; Figure 6A). Sixteen of 23 par-
ticipants (69%) recalled more of the content studied using MSFL

FIGURE 5. Recall score of characters from an unfamiliar alphabet,
Arabic or Korean, and associated phonetic spellings of pronuncia-
tions by first-year chemistry students after three time-matched
study sessions using VR (N = 16; red bars) or MSFL (N = 17; blue
bars). Sample sizes for each alphabet (N = 7–9) are indicated
within the bars. There was no significant effect of the alphabet
studied (lines above bars for each study method); the difference
between study methods is indicated by the attained level of
significance. A caveat is that the two treatment groups are not
known to be equivalent.

FIGURE 6. Recall by first-year calculus students (N = 23) after three
time-matched study sessions studying two sets of characters from
an unfamiliar alphabet, Arabic or Korean, with associated phonetic
spellings of pronunciations. Each participant studied one set of
characters using VR (red bar) and a second set of characters
using MSFL (blue bar). Differences between alphabets were not
statistically significant and are not shown. (A) Scores for recall with
attained level of significance. (B) The proportion of participants
with higher scores for VR, no difference between methods, or
higher scores for MSFL, with attained level of significance for
independence.

16:ar28, 8 CBE—Life Sciences Education • 16:ar28, Summer 2017

P. D. Heideman et al.

than the content studied using VR (p = 0.03), three were not
different (13%), and four participants recalled more content
using VR (17%) (Figure 6B).

Experiment 3. Within-Subjects Intervention Test for
Effectiveness and Adoption of MSFL
Pre Survey. For both intervention and comparison groups, the
most common study method before the experiment was VR
(50–60% of study time): rereading notes, presentations, or the
textbook. In the pre survey the intervention group reported
proportionately less time using passive study methods than the
comparison group (passive study methods as 63% vs. 80% of
study time, respectively; p < 0.0001; Figure 7A). In the pre
survey, the comparison group reported less willingness than
the treatment group to try different study methods (survey
question: “I am reluctant to change my study habits, because
they have worked very well for me so far”; p < 0.05) and less
matching of study methods to study tasks (survey question: “In
my studying now, I use different study methods and I match my
methods carefully to the material and the skills I need to learn”;
p < 0.05; see survey in the Supplemental Materials). Thus, the
volunteers in experiment 3 were drawn disproportionately
from students in the course 1) spending a greater proportion of

their study time using active study methods, 2) more willing to
change study habits, and 3) more likely to attempt matching
study methods to learning tasks. These a priori differences
between groups limit the conclusions that can be drawn from
comparisons of the two groups.

Quizzes. In self-scored assessment of recall, greater than
75% of participants recalled more with MSFL than with VR in
each assessment (p < 0.01 for each assessment). On the biol-
ogy content, there was a weak but statistically significant
effect on recall of content set A versus B (p < 0.05). In self-
scored assessments based on the ratio of differences, partici-
pants recalled an average of 68% more of the content they
studied with MSFL in comparison with VR (p < 0.01;
Figures 8A and 9A). Self-scores and investigator scores were
highly correlated (characters, R = 0.912, p < 0.0001; biology
content, R = 0.82, p < 0.0001). On the basis of scoring by
investigators, participants recalled an average of 48% more
when using MSFL in comparison with VR (Figures 8B and 9B).
In individual outcomes, 81% of participants recalled more
with MSFL than with VR, 5% had identical recall using the
two methods, and 14% recalled more using VR (Figure 10).
Results were quantitatively similar and qualitatively identical

in analyses broken down by sex, age
(grouped as ages 17–18 and 19–20), eth-
nicity (grouped into underrepresented
minorities, Asian, or white), self-assessed
study time in relation to other students
(grouped into those who felt they study
more than other students or not more
than other students), and willingness to
change study methods (grouped as reluc-
tant to change or willing to change study
methods). In all these groupings, recall
using MSFL was greater than when using
VR, with all p values 0.01 or smaller, with
no significant effect across subgroups.

Self-scores for problem solving were
poorly correlated with investigator scores,
so only investigator scores were used to
assess problem solving. Scores for solving
problems on the biology content studied
using MSFL were significantly better than
VR on quiz 4 (study method: F = 8.33; p =
0.007; content page: F = 1.11; p = 0.30;
interaction: F = 1.07; p = 0.31), while on
quiz 5, after 2 weeks with no studying,
there were no significant factors (study
method: F = 1.25; p = 0.27; content page:
F = 3.39; p = 0.08; interaction: F = 0.70; p =
0.41). On quiz 6, after a single additional
study session, there was a significant inter-
action term, indicating better problem
solving using MSFL in one of two pages of
content (interaction between study method
and content page: F = 4.36; p = 0.04; study
method: F = 1.12; p = 0.30; content
page: F = 1.51; p = 0.23). In the final
problem- solving session, session 9, there
was no significant effect of MSFL on results

FIGURE 7. Self-reported study time devoted to different study methods at the beginning
of the study (September 2014) and the end of the academic year (April 2015) for all individ-
uals in the intervention group (open circles, blue) and those in a comparison survey drawn
from the same biology course (closed diamonds, black) showing the percentage of study
time (A) using active methods, (B) including MSFL, (C) using other methods incorporating
drawing or sketching, and (D) including the combined use of MSFL and drawing or
sketching. In each figure, significant differences between groups are indicated by asterisks
(p < 0.05; adjusted by the false discovery rate control; see Methods).

CBE—Life Sciences Education • 16:ar28, Summer 2017 16:ar28, 9

Drawing-to-Learn Using Minute Sketches

of the problem-solving task (p > 0.10 for all tests). Overall, MSFL
resulted in improved problem solving in time-matched testing in
comparison with the preferred study method (VR) in some
assessments, but not the majority. In no assessment did VR pro-
vide better problem solving than MSFL.

For study using VR, our participants reported two methods:
1) systematically looking at and trying to impress content into
memory and 2) repeatedly looking away or closing eyes to
practice recall, followed by looking back to check accuracy (i.e.,
VR with retrieval practice). These two methods of VR did not
differ in effectiveness: participants who reported method 1 or 2
did not differ in the amount recalled (p > 0.50).

In the post survey, many participants in the intervention
group reported their continued use of minute sketches and/
or folded lists (Figure 11). In the pre-intervention survey,
reported use of minute sketches and folded lists was ∼2% of
study time, and did not differ between the intervention
group and comparison group (Figure 7B; Fisher exact test;
p > 0.1 for both). In the post survey 6 months after introduc-
tion to the method, approximately two-thirds of participants
in the intervention reported use of minute sketches or folded
lists (Figure 11), with a significant increase in comparison
with the pre survey (p < 0.05; Figure 7B). In addition, the
intervention group was significantly more likely to use MSFL

FIGURE 8. Recall by first-year biology students after time-matched
study sessions studying two sets of characters from an unfamiliar
alphabet, Arabic, Korean, or Tamil, with associated phonetic
spellings of pronunciations. Each participant studied one set of
characters using VR (solid squares, red) and a second set of
characters from the same, unfamiliar alphabet using MSFL (open
circles, blue), randomly assigned to a specific study method for
character set A. Participants were quizzed three times (see Figure 4
for timeline). Quiz 1 followed 3 days with practice, quiz 2 occurred
after an additional 3 weeks with no practice, and quiz 3 was given
after an additional 2 weeks with a single practice session the day
before the quiz. The attained level of statistical significance is
indicated for each quiz. Sample sizes for each quiz are indicated
above the x-axis. (A) Self-scores by participants, each scoring his or
her own quiz using a simple rubric supplied by the investigators. (B)
Scoring conducted by investigators, blind with respect to study
method and participant ID. Because there were no statistically
significant effects of the language of the alphabet or content page,
these two factors are collapsed in this figure.

FIGURE 9. Recall by first-year biology students after time-matched
study sessions studying two sets of unfamiliar biology content
(thyroid hormone structure and function and regulation of
metabolism). Each participant studied one set of content using VR
(solid squares, red) and the second set of content using MSFL (open
circles, blue), randomly assigned to a specific study method for
content set A. Participants were quizzed three times (see Figure 4
for timeline): after a week including 3 days with practice (quiz 1),
after an additional 2 weeks with no practice (quiz 2), and after an
additional 2 weeks with a single practice session (quiz 3). The
attained level of statistical significance is indicated for each quiz.
Sample sizes for each quiz are indicated above the x-axis.
(A) Self-scores by participants, each scoring his or her own quiz
using a rubric supplied by the investigators. (B) Scoring by investi-
gators, blind with respect to study method and participant ID.

16:ar28, 10 CBE—Life Sciences Education • 16:ar28, Summer 2017

P. D. Heideman et al.

or other drawing and sketching than the comparison group
(p < 0.05; Figure 7D).

We explored whether continued participation through half
or more of the intervention was related to increases in the use
of drawing or sketching during studying, to the adoption of
MSFL, or to changes in the use of active study methods. In the
pre survey, students in the intervention and comparison
groups made minimal use of drawing or sketching (average:
2% of study time; Figure 7). Individuals participating only in
the first three or four sessions of the intervention did not
change significantly between the pre and post surveys (Figure
12). In contrast, individuals who participated in more than
half of the intervention, at least through the first quiz using
biology content, differed significantly in 1) the percent of time
allocated to active study methods, 2) use of MSFL, 3) drawing
or sketching apart from MSFL, and 4) the combination of
MSFL with other drawing and sketching (Figure 12). Individ-
uals participating for more than half of the intervention were
those most likely to report continued use of MSFL (averaging
8% of study time; Figure 12B) and increased their use of all
forms of drawing and sketching (averaging 24% of study
time) in comparison with the individuals who dropped from
the intervention after the early sessions (Figure 12, C and D)
and the comparison group (compare Figures 7 and 12).

DISCUSSION
This study assessed MSFL as a study method in comparison
with a preferred study method, VR, and taught the method to
students in an intervention and allowed them to self-assess the
effectiveness of the method. Any learning gain that motivates
students to use drawing as a learning strategy will create oppor-
tunities for development of higher-order skills with drawing,
supporting an ultimate goal of students generating drawings
that they use for purposes ranging from recall to complex prob-
lem solving. To overcome student reluctance to adopt a new
study method (Pressley et al., 1989; NRC, 2001; Fu and Gray,
2004; Dunlosky et al., 2013), we focused the attention of

participants on two potential benefits, better recall and better
problem solving. Our results suggest that many participants
perceived gains in learning from MSFL and continued to use
drawing and/or MSFL as a study strategy in subsequent months.

Participants using MSFL retained ∼50% more content in
time-matched comparisons with a preferred study method,
regardless of whether the content was shapes and sounds
(experiment 1, Figure 5; experiment 2, Figure 6A; experiment
3, Figure 8) or new content in biology (experiment 3, Figure
9). The effect was maintained in multiple tests over a period
of 4–5 weeks (experiment 3, Figures 8 and 9). Importantly,
student self-scores for recall were highly correlated with
instructor scores (see Results and Figures 8 and 9), suggesting
that student self-scores were reasonably reliable assessments
of gains. Approximately 80% of participants recalled more
with MSFL than when using VR (Figure 10). Even though
instruction in problem solving using minute sketches was a
minor part of the intervention, there was some evidence for
improved problem solving after study using MSFL. Overall,
when study time was held constant, MSFL enhanced problem
solving in some trials and enhanced recall in all trials. Partici-
pants in the intervention were likely to adopt elements of
MSFL, particularly drawing and sketching, in later studying
(Figures 7, 11, and 12).

Because an observer effect may cause participants to meet
perceived goals of investigators, we conducted one experiment
in which each participant was presented with only one of the
two methods (experiment 1, Figure 5). Participants in experi-
ment 1 were given no information on the potential effectiveness
of study methods and might reasonably have inferred that the
experimental activity compared the difficulty of learning two
different alphabets. There was an unanticipated ceiling effect,
with two-thirds in the MSFL treatment achieving perfect or
near-perfect recall of the single set of 12 characters and 12 asso-
ciated sounds. Nonetheless, studying using MSFL outperformed
VR (Figure 5). In experiment 2, in which each participant
applied both study methods but received no information on
effectiveness of different study methods, recall after study using
MSFL again outperformed VR (Figure 6). While there were
potential differences in group composition in experiment 1 and
small sample sizes in experiments 1 and 2 (see Limitations sec-
tion), these results suggest that greater recall when using MSFL
may be independent of information or expectations given to par-
ticipants regarding effectiveness. The results are consistent with
multiple studies showing benefits to recall from associating sim-
ple drawings with words (see Wammes et al., 2016).

Our second objective was to test whether an intervention
(experiment 3) using MSFL with self-assessment of effective-
ness might be followed by adoption of MSFL in later studying.
The intervention first introduced students to the study method,
allowed them to self-assess gains on a simple memory task, and
then allowed them to self-assess gains on recall and problem
solving with biology content. Six months after the beginning of
the intervention, a majority of survey respondents reported
adopting MSFL (Figure 11). Of those completing more than
half of the intervention in experiment 3, involving 4.5–8.5 hours
of participant time, the proportion of study time using some
form of drawing or sketching increased significantly in the
intervention group (Figure 12, C and D). In contrast, partici-
pants who ended participation before the assessments on

FIGURE 10. Score differentials for study method (data from
experiment 3), showing the distribution of individual outcomes.
Solid circles represent participants with equal or greater recall
using MSFL, and open squares indicate those with greater recall
using VR.

CBE—Life Sciences Education • 16:ar28, Summer 2017 16:ar28, 11

Drawing-to-Learn Using Minute Sketches

gave significantly better problem-solving
outcomes 1) one to several days after
learning and 2) after review. This might be
evidence that minute sketches were being
used for problem solving, but a likely alter-
native is that differences may have been
based on the differences in recall: it is diffi-
cult to solve problems on content that is
not recalled. In the final part of the inter-
vention, in which a small subsample of
participants were given instructions on the
use of minute sketches as visual models to
solve problems, there was no difference in
problem solving on content studied using
MSFL or using VR. If minute sketches are a
format for learner-generated visual models
that may be useful for problem solving, it
may be that most students need more
instruction on problem solving than pro-
vided in this study (see Van Meter and Gar-
ner, 2005; Schwarz et al., 2009; Quillin
and Thomas, 2015). More research is
needed to address this question.

Greater recall using MSFL in compari-
son with VR is unsurprising, because
MSFL involves practice testing and
retrieval practice, approaches to studying
that are more effective than VR with
rereading (Karpicke and Blunt, 2011;
Dunlosky et al., 2013). Drawing or sketch-
ing alone can improve recall and under-
standing and help in solving problems
(Van Meter and Garner, 2005; Ainsworth
et al., 2011; Quillin and Thomas, 2015;
Wammes et al., 2016). Nonetheless, use-
ful study methods misaligned to a partic-
ular learning task might decrease learn-
ing (Leutner et al., 2009; Schwamborn

et al., 2011; Leopold and Leutner, 2012). On the basis of our
results (Figures 4, 6, 8, 9, and 10), MSFL may be a useful study
tool for college students in biology as a replacement for a com-
mon and preferred study method, VR. It may be important that
MSFL be presented as in our intervention, in which students
learn the method and self-assess MSFL in comparison with a
common preferred study method such as VR. In this study, pre-
sentation as an intervention requiring at least 4.5 hours and up
to 8.5 hours of participant time was associated with a substan-
tial increase, 6 months later, in the use of MSFL and other
forms of drawing/sketching, to ∼25% of study time. The results
suggest that this intervention may overcome resistance of stu-
dents to changes in study behavior (Pressley et al., 1989; NRC,
2001; Fu and Gray, 2004). Nonetheless, application of drawing
for more complex tasks may require additional instruction and
practice by students (Van Meter and Garner, 2005; Schwarz
et al., 2009; Quillin and Thomas, 2015).

LIMITATIONS
While results were consistent across experiments, sample sizes
were not large, and we did not assess participant features such
as prior academic ability, course achievement, and proficiency

FIGURE 11. Reported use of MSFL by individuals in the intervention group when surveyed
6 months after initial instruction. (N = 44 of the 56 participants who completed at least
two sessions for the intervention.)

biology content beginning in session 5 were 1) unlikely to
adopt MSFL (Figure 12B) and 2) fairly similar to the compari-
son group in their post survey responses on MSFL, drawing,
and sketching (compare Figure 7, B–D, with Figure 12, B–D).
Continued participation through at least session 5 may have
increased the likelihood of adopting MSFL. It is also possible
that an early decision to adopt MSFL increased the likelihood
of continuing participation in the study. Regardless of the
cause, a high proportion of participants had adopted MSFL in
their studying, with combined use of drawing and sketching
reported at nearly 25% of study time for those participating in
more than half of the intervention.

Minute sketches are intended to be useful for problem solv-
ing, a goal supported by studies on drawing-to-learn (Van Meter
and Garner, 2005; Quillin and Thomas, 2015). After first learn-
ing the biology content, students were better at problem solving
on content they had learned using MSFL. After 2 weeks without
studying, the difference in problem solving was absent (Figure
9), perhaps because content learned with both methods was not
remembered well. Two weeks later, after a single review session,
problem solving was better for one but not both pages of the
content studied using MSFL. Thus, studying content using MSFL

16:ar28, 12 CBE—Life Sciences Education • 16:ar28, Summer 2017

P. D. Heideman et al.

in many content areas in STEM. In fact, if
this intervention was part of a course, the
initial learning task could be used to
begin teaching new symbols. The discov-
ery by students that MSFL can lead to
faster mastery of learning tasks (sessions
2–4), however simple, could motivate
students to continue applying the study
method. The second learning task in the
intervention, unfamiliar discipline-based
content, allows students who have prac-
ticed the method a second chance to
assess their learning, this time as applied
to discipline-based content (sessions
5–7). Again, the discovery that MSFL can
lead to better recall could motivate stu-
dents to apply MSFL, or drawing more
generally, to additional content. While
students showed some gains in problem
solving, it seems likely that, consistent
with other reports, skill at applications of
drawing for higher-order learning might
require continued practice with instruc-
tion (Van Meter and Garner, 2005;
Schwarz et al., 2009; Quillin and Thomas,
2015). Overall, these results suggest that
this intervention, or modifications of this
intervention adapted to specific courses,
could lead to adoption of drawing by stu-
dents at least for lower-order learning
tasks that might set the stage for broader
application of learner-generated visual
models. Even without further practice
and instruction, students applying MSFL
may show improvements in problem
solving. If learner-generated drawing is a
core skill in biology (Quillin and Thomas,
2015), interventions that motivate stu-
dents to apply drawing as a long-term

study strategy will be useful to develop that skill.with languages or biology. Therefore, there may be other con-
tributing variables that explain differences in performance on
our assessments. In addition, the surveys and quizzes have not
been validated to assess how well they gather the information
intended. Nevertheless, our study is a starting point for future
research on learning and problem solving after practice with
quickly reproduced “minute sketches” with associated terms
that capture, for a user, a structure, concept, or series of events.

CONCLUSION
We view this intervention as a practical method to introduce
students to drawing as a study tool, ideally taught along with
information on metacognition and self-regulated studying.
MSFL is designed to have simple instructions that students
learn as part of the intervention, meeting a need for instruc-
tion and practice for effective application of drawing to learn-
ing (Quillin and Thomas, 2015). The initial learning task was
simple—characters with sounds—and analogous to learning
tasks students often face in STEM: learning new symbols (such
as μ or Å) or letters that serve as symbols (such as Na for
sodium or Hg for mercury) and pronunciations, a routine task

FIGURE 12. Comparison of self-reported study time of two subsets of participants
within the intervention group in the pre survey (9/2014) and post survey (4/2015). The two
subgroups comprised those completing at least two quizzes, but less than half of the
study (three to four sessions; N = 10; closed half-circles) and those completing at least half
of the study (five or more sessions; N = 23; open circles). The panels show the percentage
of study time (A) using active methods, (B) using MSFL, (C) using other methods incorpo-
rating drawing or sketching, and (D) for the combined use of MSFL and drawing or
sketching. In each figure, asterisks (*) indicate differences significant at p < 0.05 following
assessment using the false discovery rate control (see Methods).

ACKNOWLEDGMENTS
Funding was provided by National Science Foundation DUE
1339939 (a Robert Noyce Teacher Scholarship Program grant to
the College of William and Mary) and a Howard Hughes Medi-
cal Institute Undergraduate Science Education grant to the Col-
lege of William and Mary. Invaluable assistance in experimental
design was provided by the Biology Scholars Program of the
American Society for Microbiology, particularly by M. A. Kelly,
C. P. Davis, L. Clement, A. Hunter, P. Soneral, K. Wester, and
M. Zwick. P. Soneral provided insightful comments and sugges-
tions on an earlier draft of the manuscript. The William and
Mary Noyce Scholars in Fall 2012 and 2013 suggested the use
of Korean and Arabic characters and helped develop the exper-
imental protocol for assessment when learning characters. For
particularly helpful discussion and feedback in the develop-
ment of the learning method, we thank S. L. Sanderson, M. S.
Saha, R. H. Macdonald, J. J. Matkins, M. M. Mason, K. D. Goff,
and C. Walck. We also thank two anonymous reviewers for
extremely helpful comments and suggestions.

CBE—Life Sciences Education • 16:ar28, Summer 2017 16:ar28, 13

Drawing-to-Learn Using Minute Sketches

REFERENCES
Ainsworth, S., Prain, V., & Tytler, R. (2011). Drawing to learn in science.

Science, 333, 1096–1097.

Barger, J. B. (2012). How do undergraduate students study for anatomy, and
does it matter? FASEB Journal, 26, 528.522.

Bielaczyc, K., Pirolli, P. L., & Brown, A. L. (1995). Training in self-explanation
and self-regulation strategies: investigating the effects of knowledge
acquisition activities on problem solving. Cognition and Instruction, 13,
221–252.

Bransford, J. (2000). How people learn: Brain, mind, experience, and school.
Washington, DC: National Academies Press.

Chang, C. Y., Yeh, T. K., & Barufaldi, J. P. (2010). The positive and negative
effects of science concept tests on student conceptual understanding.
International Journal of Science Education, 32, 265–282.

Dauer, J. T., Momsen, J. L., Speth, E. B., Makohon-Moore, S. C., & Long, T. M.
(2013). Analyzing change in students’ gene-to-evolution models in
college-level introductory biology. Journal of Research in Science
Teaching, 50, 639–659.

Dowd, J. E., Duncan, T., & Reynolds, J. A. (2015). Concept maps for improved
science reasoning and writing: complexity isn’t everything. CBE—Life
Sciences Education, 14, ar39.

Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T.
(2013). Improving students’ learning with effective learning techniques:
Promising directions from cognitive and educational psychology.
Psychological Science in the Public Interest, 14, 4–58.

Forbus, K., Usher, J., Lovett, A., Lockwood, K., & Wetzel, J. (2011). CogSketch:
sketch understanding for cognitive science research and for education.
Topics in Cognitive Science, 3, 648–666.

Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt,
H., & Wenderoth, M. P. (2014). Active learning increases student perfor-
mance in science, engineering, and mathematics. Proceedings of the
National Academy of Sciences USA, 111, 8410–8415.

Fu, W.-T., & Gray, W. D. (2004). Resolving the paradox of the active user:
stable suboptimal performance in interactive tasks. Cognitive Science,
28, 901–935.

Hay, D. B., Williams, D., Stahl, D., & Wingate, R. J. (2013). Using drawings of
the brain cell to exhibit expertise in neuroscience: exploring the bound-
aries of experimental culture. Science Education, 97, 468–491.

Husmann, P. R., Barger, J. B., & Schutte, A. F. (2016). Study skills in anatomy
and physiology: Is there a difference? Anatomical Sciences Education, 9,
18–27.

Jee, B. D., Gentner, D., Uttal, D. H., Sageman, B., Forbus, K., Manduca, C. A., …
Tikoff, B. (2014). Drawing on experience: how domain knowledge is re-
flected in sketches of scientific structures and processes. Research in
Science Education, 44, 859–883.

Karpicke, J. D., & Blunt, J. R. (2011). Retrieval practice produces more learn-
ing than elaborative studying with concept mapping. Science, 331, 772–
775.

Konietschke, F., & Pauly, M. (2014). Bootstrapping and permuting paired
t-test type statistics. Statistics and Computing, 24, 283–296.

Leopold, C., & Leutner, D. (2012). Science text comprehension: Drawing,
main idea selection, and summarizing as learning strategies. Learning
and Instruction, 22, 16–26.

Leutner, D., Leopold, C., & Sumfleth, E. (2009). Cognitive load and science
text comprehension: Effects of drawing and mentally imagining text
content. Computers in Human Behavior, 25, 284–289.

Mayer, R. E. (2009). Multimedia learning. Cambridge, UK: Cambridge
University Press.

McDermott, K. B. (2006). Paradoxical effects of testing: repeated retrieval
attempts enhance the likelihood of later accurate and false recall.
Memory & Cognition, 34, 261–267.

National Research Council (NRC). (2001). Knowing what students know: The
science and design of educational assessment. Washington, DC: Nation-
al Academies Press.

NRC. (2003). BIO2010: Transforming undergraduate education for future
research biologists. Washington, DC: National Academies Press.

Novick, L. R., & Catley, K. M. (2007). Understanding phylogenies in biology:
The influence of a Gestalt perceptual principle. Journal of Experimental
Psychology: Applied, 13, 197.

Pressley, M., Goodchild, F., Fleet, J., Zajchowski, R., & Evans, E. D. (1989). The
challenges of classroom strategy instruction. Elementary School Journal,
89, 301–342.

Quillin, K., & Thomas, S. (2015). Drawing-to-learn: A framework for using
drawings to promote model-based reasoning in biology. CBE—Life
Sciences Education, 14, es2.

Roediger, H. L. III, & Marsh, E. J. (2005). The positive and negative conse-
quences of multiple-choice testing. Journal of Experimental Psychology:
Learning, Memory, and Cognition, 31, 1155–1159.

Schnotz, W., & Kürschner, C. (2007). A reconsideration of cognitive load the-
ory. Educational Psychology Review, 19, 469–508.

Schraw, G., Crippen, K. J., & Hartley, K. (2006). Promoting self-regulation in
science education: Metacognition as part of a broader perspective on
learning. Research in Science Education, 36, 111–139.

Schwamborn, A., Thillmann, H., Opfermann, M., & Leutner, D. (2011). Cognitive
load and instructionally supported learning with provided and learner-
generated visualizations. Computers in Human Behavior, 27, 89–93.

Schwarz, C. V., Reiser, B. J., Davis, E. A., Kenyon, L., Achér, A., Fortus, D., …
Krajcik, J. (2009). Developing a learning progression for scientific model-
ing: Making scientific modeling accessible and meaningful for learners.
Journal of Research in Science Teaching, 46, 632–654.

Speth, E. B., Shaw, N., Momsen, J., Reinagel, A., Le, P., Taqieddin, R., & Long, T.
(2014). Introductory biology students’ conceptual models and explana-
tions of the origin of variation. CBE—Life Sciences Education, 13, 529–539.

Sweller, J. (1988). Cognitive load during problem solving: Effects on learning.
Cognitive Science, 12, 257–285.

Sweller, J., & Chandler, P. (1994). Why some material is difficult to learn. Cog-
nition and Instruction, 12, 185–233.

Tanner, K. D. (2012). Promoting student metacognition. CBE—Life Sciences
Education, 11, 113–120.

Thissen, D., Steinberg, L., & Kuang, D. (2002). Quick and easy implementation
of the Benjamini-Hochberg procedure for controlling the false positive
rate in multiple comparisons. Journal of Educational and Behavioral Sta-
tistics, 27, 77–83.

Van Meter, P., & Garner, J. (2005). The promise and practice of learner-gen-
erated drawing: Literature review and synthesis. Educational Psychology
Review, 17, 285–325.

Verhoeven, L., Schnotz, W., & Paas, F. (2009). Cognitive load in interactive
knowledge construction. Learning and Instruction, 19, 369–375.

Wammes, J. D., Meade, M. E., & Fernandes, M. A. (2016). The drawing effect:
Evidence for reliable and robust memory benefits in free recall. Quarterly
Journal of Experimental Psychology, 69, 1752–1776.

Ward, P. J., & Walker, J. J. (2008). The influence of study methods and knowl-
edge processing on academic success and long-term recall of anatomy
learning by first-year veterinary students. Anatomical Sciences Educa-
tion, 1, 68–74.

Wright, L. K., Fisk, J. N., & Newman, D. L. (2014). DNA→ RNA: What do students
think the arrow means? CBE—Life Sciences Education, 13, 338–348.

Xu, L.-W., Mei, B., Chen, R.-R., Guo, H.-X., & Wang, J-j. (2013a). Parametric
bootstrap tests for unbalanced nested designs under heteroscedasticity.
Journal of Statistical Computation and Simulation, 84, 2059–2070.

Xu, L.-W., Yang, F.-Q., Abula, A., & Qin, S. (2013b). A parametric bootstrap
approach for two-way ANOVA in presence of possible interactions with
unequal variances. Journal of Multivariate Analysis, 115, 172–180.