To help genetics instructors become aware of fundamental concepts that are persistently difficult for students, we have analyzed the evolution of student responses to multiple-choice questions from the Genetics Concept Assessment. In total, we examined pretest (before instruction) and posttest (after instruction) responses from 751 students enrolled in six genetics courses for either majors or nonmajors. Students improved on all 25 questions after instruction, but to varying degrees. Notably, there was a subgroup of nine questions for which a single incorrect answer, called the most common incorrect answer, was chosen by >20% of students on the posttest. To explore response patterns to these nine questions, we tracked individual student answers before and after instruction and found that particular conceptual difficulties about genetics are both more likely to persist and more likely to distract students than other incorrect ideas. Here we present an analysis of the evolution of these incorrect ideas to encourage instructor awareness of these genetics concepts and provide advice on how to address common conceptual difficulties in the classroom.
GENETICS can be a challenging subject for undergraduates, even for those who are biology majors. It has been suggested that many of the incorrect ideas that students have in genetics originate in middle school and high school (AAAS 1993). High school students from different countries appear to share a suite of incorrect ideas, including not understanding the relationship between a gene, a chromosome, and a cell; thinking that different cells contain different genes; not understanding how genetic information is transmitted; and concluding that single genes are responsible for complex traits such as height (Lewis and Wood-Robinson 2000; Chattopadhyay 2005; Mills-Shaw et al. 2008; Boujemma et al. 2010). Some of these incorrect ideas may be perpetuated by K–12 teachers who do not have an adequate understanding of the complexities of genetics, particularly with regard to how concepts relate to each other (Cakir and Crawford 2001). Furthermore, studies with pre-service teachers show that, while they have mastered some basic ideas (e.g., mitosis produces two identical cells and genes are responsible for visible traits), some still retain incorrect ideas about genetics commonly held by younger students (Lewis et al. 2000; Marbach-Ad and Stavy 2000).
There are several terms used to describe student conceptions that are not aligned with an expert view, including “preconceptions” and “misconceptions.” A preconception is a private understanding that a student has before formal science instruction, which can become a misconception when it is in conflict with scientific concepts that a student encounters in a course (Posner et al. 1982; Mestre 1994). However, some authors have suggested that most of the incorrect student ideas expressed about genetics may not represent true misconceptions or alternative conceptions, but instead many levels of confusion on each topic combined with an inability to link the ideas together into a broader conceptual framework (Lewis and Wood-Robinson 2000; Marbach-Ad and Stavy 2000). Because most college students have encountered some genetics concepts during their K–12 biology education, such as dihybrid crosses, but may not have had instruction on other concepts, such as complementation, a typical college course likely consists of students who have some combination of preconceptions and misconceptions, as well as an inability to link such concepts together.
There are a variety of ways to probe common conceptual difficulties among genetics students, including multiple-choice questions with written explanations (e.g., Lewis and Wood-Robinson 2000; Chattopadhyay 2005), short answer questions (e.g., Mills-Shaw et al. 2008; Boujemma et al. 2010), and think-aloud student interviews (e.g., Lewis and Kattman 2004). Written responses to questions and think-aloud interviews can provide valuable input on student thinking and dissecting student understanding, but are not always practical to administer and score in a large enrollment undergraduate genetics course.
An alternative option for faculty who teach genetics is to use response-validated multiple-choice concept assessments such as the Genetics Concept Assessment (GCA) (Smith et al. 2008) and the Genetics Literacy Assessment Instrument (Bowling et al. 2008). These instruments are composed of multiple-choice questions designed to document common conceptual difficulties in genetics. Concept assessments are typically given at the beginning and end of a course to diagnose student difficulties and monitor increased understanding (reviewed in D'Avanzo 2008; Knight 2010). These instruments have been response-validated by student interviews and experts in the field and statistically analyzed for reliability and other measures using responses from hundreds of students at multiple institutions.
In this article, we analyzed responses to the GCA from 751 students enrolled in six genetics courses. These courses are offered to both majors and nonmajors and were taught by four different instructors. We focused on questions that students answer incorrectly even after taking a college-level genetics course. These persistent incorrect answers come from a subset of nine GCA questions where >20% of the students answered a single incorrect answer on the posttest. By tracking the evolution of individual student response patterns to these nine questions, we address two main questions: (1) If students begin a course with one of these incorrect answers, is it harder for them to arrive at the correct answer when the course is over? (2) If students begin a course with one of these incorrect answers, is it more likely that they will stick to their initial incorrect answer? Helping instructors to become aware of these common conceptual difficulties, of how likely they are to persist, and of how to help students understand these concepts can inform instructional practice in genetics courses.
This study was conducted in six undergraduate genetics courses—one for majors (fall semester: 2008, 2009, 2010; spring semester: 2010) and another for non-majors (fall semester: 2008 and 2009) (student demographics shown in Table 1)—taught in the Department of Molecular, Cellular, and Developmental Biology (MCDB) at the University of Colorado, Boulder. The course for majors (taught by one of three tenured professors) is the second in a series of core courses offered by the MCDB; it is preceded by an introductory course in molecular and cell biology. The non-majors course (taught by J.K.K.) is not part of a series and is taken primarily by students fulfilling a science distribution requirement in the College of Arts and Sciences. The majors class includes a weekly laboratory with separate learning goals and grades and minimal overlap in content with the lecture. The non-majors course does not have a laboratory. A detailed description of similarities and differences between the students in the majors and non-majors genetics courses can be found in Knight and Smith (2010).
All instructors in this study participated in the Science Education Initiative at the University of Colorado-Boulder (Wieman 2009) and made changes to align their courses with promising practices in science education. The majors and non-majors genetics courses have in common six broad content-learning goals, daily use of in-class concept questions answered with personal response devices (a.k.a. clickers) and weekly graded homework assignments that addressed commonly difficult topics. All instructors shared course materials, such as lecture slides, clicker, homework, and exam questions. Each instructor modified the materials to fit their needs. In all classes, student attendance rates were, on average, >80%.
An updated version of the GCA (Smith et al. 2008), which consists of 25 multiple choice questions, was administered on the first day of class (pretest) and as part of the final exam (posttest). Students were given 30 min during the first class period to complete the GCA using bubble sheets. Students were asked to put forth their best effort and were informed that their performance would have no bearing on their grade, but that the instructors would use the results of the pretest to help determine which topics in genetics were particularly challenging for this group of students. Students were given a few participation points for taking the pretest. All instructors saw the aggregated pretest results at the beginning of the course and were aware of which concepts were challenging for students; however, the results of the pretest were never discussed with the students. At the end of the course, the identical 25 questions were given as part of the students’ cumulative final exam, composing no more than 20% of the final exam grade. The total time allotted to take the entire final exam was 2.5 hr. Only students who completed both the pretest and posttest were included in the data set, for a total of 630 students in the four majors courses and 121 students in the two non-majors courses.
The change in learning between pretest and posttest was computed for each student using a modified version of the Hake normalized gain formula (Hake 1998) known as normalized change <c> (Marx and Cummings 2007). Normalized change values provide a measure of how much a student's performance increases compared with that individual's maximum possible increase. When calculating the normalized change between pretest and posttest, the following formula was used when an individual's posttest score was equal to or higher than the pretest score (742 of 751 cases): <c> = 100(posttest − pretest)/(100 − pretest). Alternatively, if an individual's pretest score was higher than the posttest score, <c> = 100[(posttest − pretest)/pretest] was used. There were no cases where an individual's posttest score and the pretest score equaled either 100 or 0. Significant differences between mean <c> values between two populations cannot be determined because they are nonlinear computed quantities that are not normally distributed. Instead, the standard error measurements on reported <c> values are used to provide a coarse depiction of the spread of values (Marx and Cummings 2007).
All statistical analyses were performed with SPSS (Chicago, IL) or Excel (Microsoft, Redmond, WA).
Analysis of course materials
To determine how often instructors talked and asked about questions relating to concepts in GCA questions, we examined lecture slides, homework assignments, and exams from all instructors and counted the number of times in which concepts were discussed or tested on for each course.
Student reasoning for incorrect answers was taken from 35 students who were interviewed during the design and validation of the GCA using a “think-aloud” protocol (more details can be found in Smith et al. 2008). At least 10 students were interviewed on any given question. Students were asked to select an answer to each question on the GCA and then to describe to the interviewer their reasoning for the answer, as well as to describe why other answers were incorrect. All students interviewed had completed at least one semester of college-level biology.
Institutional Review Board Statement
Approval to evaluate student pretest and posttest responses (exempt status, protocol no. 0108.9) and to interview students (expedited status, protocol no. 0603.08) was granted by the Institutional Review Board at the University of Colorado, Boulder.
General analysis of performance
GCA pretest and posttest scores and normalized change values (<c>) were calculated for each student. The average scores for each course are shown in Table 2. Students performed similarly on the assessment from semester to semester, on both pre- and posttests. In each course, the average posttest score was significantly higher than the pretest score (dependent t-test, P < 0.05, in all cases). Non-majors do not perform as well on the posttest as do majors and subsequently have a lower average normalized learning change. These results are likely due to differences in preparation, study habits, and motivation (Knight and Smith 2010).
The average percentage correct on each question was calculated using combined data from all 751 students (Figure 1). For each question except question 19, the proportion of students answering a question correctly on the posttest is significantly higher than the proportion answering a question correctly on the pretest (Supporting Information, Table S1, χ2 test, P < 0.05). This difference was likely not observed on question 19 because the question was relatively easy for students on the pretest.
Identifying the most common incorrect answer
An investigation of student posttest answers revealed that student responses to the GCA questions fell into three categories: (1) no obvious difficulty: ≥80% students answered a posttest question correctly (eight questions); (2) no single incorrect idea: <80% of students answered a posttest question correctly, but more than one wrong answer was commonly chosen (eight questions); and (3) incorrect idea: <80% of students answered a posttest question correctly and one answer was chosen by at least 20% of students (nine questions). These nine questions (Figure 1, noted with stars) are not aggregated at the beginning or end of the test, but instead occur throughout. Figure 2 shows an example of the results for all students answering question 4, on the concept of mutations. The text of question 4 is included in Table 3. Table 3 lists all nine questions and the frequency of all posttest answers. For the remainder of the article, we will focus on these nine questions and refer to the single answer chosen by >20% of the students on the posttest as the most common incorrect answer (MCIA).
To ensure that the 20% of students selecting the MCIA on the posttest did not come from a single course or, for example, were only the nonmajor students, we examined the percentage of students who chose incorrect answers for each course separately. In all cases, the MCIA was the most common incorrect posttest answer for all courses. An example of the distribution of posttest answers on question 4 in each course is shown in Figure 3. Because all courses had the MCIA as the most common posttest incorrect answer, subsequent analyses were performed on all 751 students as an aggregate group.
To determine whether the 20% of the students answering the MCIA on the posttest represented the same group of students for each MCIA, or if students tended to answer the MCIA only on a few questions, we examined how often each student selected an MCIA on the posttest. A total of 529 students (69.2%) of the students answered at least one MCIA on the posttest. Of the students who answered at least on the MCIA on the posttest, most chose the MCIA on two or fewer questions, not on all nine (Figure 4).
What are the most common conceptual difficulties?
The nine questions for which students have an MCIA cover three overarching genetics concepts: genetic content and genetic code (questions 1, 15, and 21); the nature of mutations and their effects (questions 4, 11, and 12); and the process and results of meiosis, including probability calculations (questions 3, 10, and 20). Surprisingly, several of the questions address what most instructors would consider to be very basic concepts: for example, question 1 (DNA content is the same in every cell) and question 4 (Mutations do not have to affect proteins to be called mutations). Table 3 outlines each of the most common conceptual difficulties represented by the MCIA answers and also shows sample quotes from students explaining their thinking when selecting the MCIAs. The student interview data suggest that, when students select an MCIA, they have conceptual difficulties; they do not make an incorrect choice because they are misreading the question or honing in on noncontent cues (a deeper discussion of this issue appears in Smith et al. 2008). For example, students tend to think that different cells in individuals contain different genes and gave responses such as: “only genes in the eye contribute to eye things.” In addition, when thinking about the question regarding mutations, students responded with statements such as “mutations only count if they alter the amino acid sequence,” suggesting that they tend to think a DNA change is only a mutation if it changes a protein.
Instructors addressed the concepts in these nine GCA questions in their courses through a combination of lecture, formative questions (clicker and homework questions), and summative questions (exam questions). The only exception was for question 1 (DNA content is the same in all cells), which was not addressed in any way in two of the four major courses. An analysis of the instructors’ course materials revealed considerable variability in the number of formative and summative questions used to assess student understanding of the concepts in the nine questions (Figure 5). Some concepts were asked about frequently (e.g., question 3 on X-linkage, question 4 on mutations, and question 20 on nondisjunction), while others were covered in formative questions but were infrequently tested (e.g., question 1 (all cells have the same DNA), question 15 (DNA and amino acid sequence variability), and question 21 (the relationship between protein function and amino acid sequence variability). An example of a series of formative and summative questions on the topic of mutations, which is asked about in GCA question 4, is shown in Figure S2. Despite the variable number of questions asked of students, each concept was typically addressed in one lecture, one homework assignment, and one exam in all classes.
If a student answers the MCIA on the pretest, is it harder to move to the correct answer on the posttest?
By tracking the evolution of student answers from the pretest to posttest, we find that, if students answer the MCIA on the pretest, it is not difficult, on average, for them to move to the correct answer on the posttest. To perform this analysis, we calculated the proportion of students who answered the MCIA on the pretest and moved to the correct answer on the posttest. We compared this proportion to the proportion of students who answered another incorrect answer on the pretest and answered the correct answer on the posttest. An example of how we set up this comparison for question 4 is in Figure 6. In all cases, there was no significant difference between the proportions (Table 4, χ2 test, P > 0.05).
If a student answers correctly on the pretest, will he or she continue to answer correctly on the posttest?
We find that the majority of students who answered the nine questions correctly on the pretest continued to answer correctly on the posttest. Table 5 shows that many of the students who selected the correct pretest answer went on to select the correct posttest answer; staying correct is the most likely outcome for eight of the nine questions. However, the next most likely outcome or, in the case of question 20, the most likely outcome, is that students move from the correct pretest answer to the MCIA on the posttest.
Of the students who answer the pre- and posttest incorrectly, is a student who answers the MCIA on the pretest more likely to respond in the same way for the posttest?
If a student selects the MCIA on the pretest and incorrectly answers the same question on the posttest, he or she will most likely select the MCIA again. To perform this analysis, for each of the nine questions, we examined the subset of students who answered incorrectly on both the pretest and the posttest and asked what proportion of students selected identical answers. Figure 7 shows an example of how this comparison was made for question 4. The proportion of students who persisted in the identical incorrect answer is significantly higher for students who began with the MCIA on the pretest than for students who selected another incorrect answer on the pretest (Table 6, χ2 test, P < 0.05). In all cases, students who pick the MCIA on the pretest and continue to answer incorrectly on the posttest predominantly select that MCIA again on the posttest.
For students who chose non-MCIA incorrect answers on the pretest, the majority of those students did not continue to choose the same incorrect answer, but rather chose the MCIA on the posttest (Table 7).
Our work supports other studies showing that biology concept assessments can be used to document common student conceptual difficulties (D'Avanzo 2008; Garvin-Doxas and Klymkowsky 2008). Specifically, by investigating the evolution of individual student answers on the GCA, we have characterized the persistency of such conceptual difficulties in genetics. For nine GCA questions (Figure 1), representing three overarching themes of genetics, many students answer a single incorrect idea on the posttest, which we called the MCIA. The MCIAs are prevalent among all students tested, regardless of course or instructor, and even when all instructors have taught these concepts and taken steps to transform their courses (Figure 5).
Our evaluation of the evolution of student answers provides some encouragement; namely, if students answer the MCIA on the pretest, this does not preclude learning this concept or overcoming this incorrect idea (Table 4). For many students, the MCIA is simply an incorrect idea that can be overcome with instruction. However, our analysis also shows that the conceptual difficulties described in the MCIAs might be particularly resistant to change, an hypothesis that should be further tested in settings with different pedagogical approaches. For the subset of students who responded incorrectly on both the pre- and the posttest, those choosing the MCIA on the pretest were more likely to continue selecting the MCIA on the posttest (Table 6). This outcome was not true for students who selected non-MCIA incorrect answers on the pretest; they were more likely to select a different incorrect answer on the posttest, with a preference toward the MCIA (Tables 6 and 7). In addition, the fact that some students who answered correctly on the pretest changed their answer to the MCIA on the posttest (Table 5) suggests that the MCIA remains a more attractive posttest answer than other incorrect answers.
As described in the Introduction, there are several terms used to characterize persistent student conceptions that are not aligned with the expert view of a geneticist. The term “misconception” is generally reserved for times when students exhibit stable and coherent false beliefs. Our results suggest that, for students who answer an MCIA on both pre- and posttests (Table 6), the incorrect answers may represent actual misconceptions that are resistant to change. Furthermore, such misconceptions can prevent students from learning a concept in a way that is consistent with what scientific data suggest to be true (Smith et al. 1993; Chi et al. 1994; Macbeth 2000; Modell et al. 2005).
Learning that occurs when students must relinquish alternative conceptions in favor of scientific ones is referred to as “conceptual change.” Within the extensive conceptual change literature, both the nature of student conceptions and how change occurs are contested (diSessa 2006). However, researchers generally agree on two points. First, learners tend not to reject or discard alternative conceptions, even after they are shown to be incorrect (Strike and Posner 1992). Rather, existing conceptions remain prominent in a learner's mind and compete with newly introduced ones. As a result, adherence to new conceptions is often weak (Mazur 1997), and thinking can shift from one concept to another depending on contextual factors (diSessa 1993). Second, new concepts are generally not learned within the short time frame of a single lecture or homework assignment that instructors typically use. Instead, conceptual understanding develops slowly over many experiences (Smith et al. 1993; Chi et al. 1994; Carey 2009; Ohlsson 2009).
Genetics instructors can help students by addressing conceptual difficulties throughout the course
Because the population of students studied here is large and composed of both majors and nonmajors, we hypothesize that students at other institutions are also likely to have some of the same conceptual difficulties identified in the MCIAs. What can be done to improve student learning of these fundamental genetics concepts? Asking additional questions on concepts for which there are few formative and summative questions (Figure 5) may have value. However, students also struggled with concepts for which many formative and summative questions were asked, such as nondisjunction, addressed in GCA question 20. Therefore, only increasing the number of questions is unlikely to be sufficient in helping students overcome their common conceptual difficulties.
Interestingly, in all classes studied, formative and summative questions on concepts relating to the MCIAs were typically compartmentalized into one lecture, homework assignment, and exam. Addressing conceptual difficulties in this way is incommensurate with what it takes to develop new understandings, as discussed above. Furthermore, this limited exposure to concepts does not allow students to engage in the beneficial process of “deliberate practice,” in which they solve problems and monitor their own learning (Ericsson et al. 1993). For genetics instructors, who often feel pressed for time to cover content, these goals could be achieved by a number of pedagogical approaches such as emphasizing these concepts in laboratory experiments, designing in-class activities, and placing questions that address common conceptual difficulties on multiple homework assignments. For example, instead of asking several formative questions about different types of mutations on one homework assignment, questions on mutations could be asked about on several homework assignments over the duration of the semester. This approach would also follow the recommendations of several cognitive psychology studies that show that long-term retention is better achieved by (1) asking students to “test” themselves with questions, rather than simply having them study information (Roediger and Karpicke 2006), and (2) interleaving different concepts together rather than massing the learning of similar concepts (Kornell and Bjork 2008).
Genetics instructors can help students by handling misconceptions in a supportive way
The traditional way of helping students overcome conceptual difficulties is to create assignments that present students with evidence contradicting their pre-instruction ideas to help students reinterpret what they are thinking (e.g., Strike and Posner 1985; Chinn and Brewer 1993). However, more recent approaches focus on supporting student thinking in addition to having them confront conceptual difficulties. For example, instructors can help students overcome misconceptions by designing lessons that reflect on current knowledge, find new contexts for existing knowledge, and refine knowledge for specific scientific purposes (Smith et al. 1993). Genetics instructors teaching about mutations could follow the steps listed below:
Reflect on current knowledge: Ask students a question such as “Randomly pick a base in a coding sequence of your favorite gene and change it to any other base. If this change is copied during replication, does it create a new allele?” Follow this with a class-wide discussion of what constitutes a mutation and what the word mutation means genetically vs. colloquially.
Find contexts for existing knowledge: Ask students to find examples of known mutations that fall into different categories: silent, missense, frameshift, etc. Share the example of single nucleotide polymorphisms as mutations that have proved useful as an experimental tool, even though they are typically silent mutations.
Refine knowledge for a specific scientific purpose: for students to combine new and old ideas, test students, preferably in a low-stakes environment, with application-type questions exposing students to additional contexts in which mutations do not affect proteins as well as contexts in which they do. For example, students could be asked the following question: “Mutations in the FGFR3 gene have been linked to achondroplasia, an autosomal dominant type of dwarfism. Chris acquired a mutation in the FGFR3 gene in a primary spermatocyte. If a sperm with this mutation fertilizes an egg, will his child be short in stature?”
This instructional approach could also help clarify other problems that students have with the concept of mutations, such as GCA question 12, in which students fail to see that a frameshift mutation could lead to an early stop codon.
Will students demonstrate similar conceptual difficulties in answering free response questions?
Multiple-choice questions that address conceptual difficulties are helpful in large enrollment classes where reading and grading short-answer responses is not practical. Furthermore, the incorrect answers on concept assessments like the GCA were generated from student responses and have been response-validated with student interviews. In addition, the authors did not know which answers would be the MCIAs when the assessment was created (Smith et al. 2008). Nonetheless, some students might pick the MCIA because it is written in front of them rather than having thought of the idea on their own. Therefore, we are currently developing question stems that integrate the ideas of these nine difficult questions and will use an online format to collect student open-ended responses. Such responses can be categorized with the help of computer programs to pick out patterns of student answers from large groups of students (Haudek et al. 2011; Nehm et al. 2011). Related studies have revealed unexpected trends in student responses that would not be captured by answers to multiple-choice questions. For example, students often combine incorrect and correct ideas and answer questions differently depending on their surface features [e.g., a question using bacteria vs. an animal (Nehm et al. 2012)]. We will be analyzing student responses to versions of the GCA questions discussed here by assessing the frequency with which students use these persistently incorrect ideas and evaluating their explanations to help us further explore why students have these misunderstandings.
Conclusion and a call to action
For instructors not currently using pre and post concept-assessment testing in their classes, the work described here can help them become aware of conceptual difficulties that may be preventing students from understanding genetics at a deeper level. We hope that studies like these will encourage instructors to discuss these concepts with their students and collectively identify common conceptual difficulties among their students. Furthermore, we encourage genetics instructors to develop and refine, in a systematic manner, instructional materials grounded in research findings from the education field to help students overcome prevalent conceptual difficulties. Such materials exist in other fields such as physics (examples of research-based instructional physics materials: McDermott and Redish 1999; description of research-based curriculum development: McDermott 2001) and can be used as examples of how the genetics community could move in a similar direction.
If you would like to find out whether your students have similar incorrect ideas about genetics to those presented here, we encourage you to use the GCA as a pre- and posttest. Please contact the authors for a full list of questions and advice on how to analyze the data.
We thank Josh Akey, Jennifer Kaplan, and Jeremy Smith for their help with data and statistical analysis. Also, we thank Ken Krauter, Tin Tin Su, and Mark Winey for administering the GCA in their courses. Finally, we thank Shirly Avagril, Daniel Capps, Jonathan Shemwell, and MacKenzie Stetzer for their comments on the manuscript. We are grateful to the Science Education Initiative at the University of Colorado, Boulder, for partial support of J.K.K.
Supporting information is available online at http://www.genetics.org/content/suppl/2012/02/23/genetics.111.137810.DC1.
Communicating editor: M. D. Rose
- Received December 14, 2011.
- Accepted January 29, 2012.
- Copyright © 2012 by the Genetics Society of America