refutation research paper

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Springer Nature - PMC COVID-19 Collection

Refutation Text Facilitates Learning: a Meta-Analysis of Between-Subjects Experiments

Noah l. schroeder.

1 Department of Leadership Studies in Education and Organizations, Wright State University, 3640 Colonel Glenn Highway, Dayton, OH 45435 USA

Aurelia C. Kucera

2 Lake Erie College of Osteopathic Medicine at Seton Hill, 20 Seton Hill Dr, Greensburg, PA 15601 USA

Associated Data

Scientific misconceptions are ubiquitous, and in our era of near-instant information exchange, this can be problematic for both public health and the public understanding of scientific topics. Refutation text is one instructional tool for addressing misconceptions and is simple to implement at little cost. We conducted a random-effects meta-analysis to examine the effectiveness of the refutation text structure on learning. Analysis of 44 independent comparisons ( n  = 3,869) showed that refutation text is associated with a positive, moderate effect ( g  = 0.41, p  < .001) compared to other learning conditions. This effect was consistent and robust across a wide variety of contexts. Our results support the implementation of refutation text to help facilitate scientific understanding in many fields.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10648-021-09656-z.

Misconceptions are rampant in daily life. This is particularly true in the age of the internet and smart devices, as individuals have rapid access to a wide variety of resources of varying degrees of reputability. Individuals may engage with inaccurate information and then build inaccurate conceptions based on this inaccurate information. For the purposes of this paper, we will refer to inaccurate conceptions as misconceptions. In some cases, the implications of misconceptions may be minimal. For example, not having a scientifically accepted understanding of why tides occur may not tangibility influence an inland city dweller’s daily life. However, in other cases, there can be tangible costs to failing to hold scientifically accepted understandings of some topics. For example, the world is currently fighting the coronavirus disease 2019 (COVID-19) and its causative agent, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Misconceptions about this disease may have disastrous consequences, particularly if they are held and promoted by people involved in public health policy. Reputable sources of health information are attempting to combat myths about SARS-CoV-2 through websites with information presented in an expository format (Mayo Clinic Staff, 2020 ; World Health Organization, n.d. ). Unfortunately, expository text may not be the most effective approach to provide these scientific facts, as it does not always promote learning effectively or efficiently (Sinatra et al., 2011 ). We then face a critical question: how should we address misconceptions and help students obtain scientifically accepted understandings?

Misconceptions are inaccurate ideas and beliefs (Kendeou & Van Den Broek, 2007 ; Tippett, 2010 ; Will et al., 2019 ) which can influence an individual’s conceptual change and learning (Dole & Niederhauser, 1990 ). Conceptual change is thought to occur spontaneously or intentionally (Sinatra et al., 2011 ), and the extent to which it occurs is context-dependent (Posner et al., 1982 ). However, the process by which conceptual change occurs is incompletely understood (Rusanen, 2014 ). A classic model of the conceptual change process requires the learner to realize the information they hold is incorrect ( dissatisfaction ) and then be presented with understandable and plausible alternative information that offers some benefit for explaining the phenomenon ( fruitful ) (Pintrich et al., 1993 ; Posner et al., 1982 ). It is also likely that social, emotional, and motivational variables can influence the conceptual change process, and the interaction between these factors is complex (Pintrich et al., 1993 ; Sinatra, 2005 ). As such, it is important to identify instructional strategies that can facilitate the conceptual change process.

A variety of different text formats have been used to facilitate conceptual change and learning more broadly. For instance, expository texts are based on scientific facts, as one may often see in a textbook. Conceptual change texts are slightly different, beginning with a question and following it with explanations of why common misconceptions are incorrect and providing the scientifically accepted explanation (Çalik et al., 2007 ; Roth, 1985 ). Similar to conceptual change text is another promising text format, refutation text, which is the focus of this research. A refutation text states a common misconception about a subject, refutes it, and provides an explanation of the correct conception (Hynd, 2001 ; Mason et al., 2019 ; Tippett, 2010 ). Importantly, there is occasionally overlap between the format of a conceptual change text and a refutation text, with the difference being that conceptual change texts ask learners to make predictions by posing a question, whereas a refutation text does not require this (Chambers & Andre, 1997 ). An example of a refutation text is as follows:

Many people believe that the peregrine falcon is on the United States Fish and Wildlife Service’s (USFWS) endangered and threatened species list. However, this is not true. The last subspecies of peregrine falcon were removed from the USFWS endangered and threatened species list in 1999.

Refutation text has been investigated in relation to the learning of general, non-politicized, and well-accepted science concepts such as tides (Ariasi & Mason, 2014 ) and energy (Diakidoy et al., 2016 ). Recently however, researchers have begun investigating more politicized subjects (Zengilowski et al., 2021 ) and medical topics such as influenza (Vaughn & Johnson, 2018 ). In this meta-analysis, we broadly analyze the work around refutation text to build an understanding of under what conditions and for whom it is an effective instructional strategy.

Literature Review

The conceptual change process.

There are a variety of theoretical perspectives as to how conceptual change may occur, ranging from strictly cognition-oriented theories through those that incorporate aspects of affect or other aspects of their persons and varying from epistemological views to ontological views (for a history and review, see Treagust & Duit, 2008 ). Since the purpose of this paper is not to validate one theory as opposed to another, we refer readers to existing works dedicated to these theories (see, for example, Pintrich et al., 1993 ; Posner et al., 1982 ; Sinatra, 2005 ; Sinatra & Broughton, 2011 ; Treagust & Duit, 2008 ; Vosniadou, 2008 ).

For the purposes of this paper, it is important to broadly understand the theorized cognitive mechanisms behind how learners’ mental models can change over time. This includes transitions from inaccurate preconceptions through scientifically accepted understandings. As such, rather than explore the many different views of conceptual change present in the literature (e.g., Pintrich et al., 1993 ; Posner et al., 1982 ; Sinatra, 2005 ; Sinatra & Broughton, 2011 ; Treagust & Duit, 2008 ; Vosniadou, 2008 ), we focus on one cognition-oriented framework that explains why refutation text may be effective for supporting conceptual change and learning and is largely complementary to the aforementioned explanations of conceptual change that incorporate social, emotional, and motivational variables.

The Knowledge Revision Components Framework (KReC) (Kendeou et al., 2014 , 2019 ; Will et al., 2019 ) provides a cognition-oriented model of knowledge revision. The KReC framework is based on two assumptions. First, information within the long-term memory cannot be eradicated; it is always present. Second, this information held in the long-term memory can be activated by a learner as they read a text. These are known as the encoding and passive activation principles, respectively (Kendeou et al., 2014 , 2019 ; Will et al., 2019 ). For knowledge revision to occur, the KReC framework suggests that novel information gained from text must interact with existing schema in the working memory. This activation of long-term memories can promote integration of the novel information with the learner’s prior knowledge, thus creating an integrated representation in the learner’s mental model (Kendeou et al., 2014 , 2019 ; Will et al., 2019 ). KReC posits that the new information and prior knowledge then compete for activation, and knowledge revision occurs when the new information wins this competition for activation (Kendeou et al., 2014 , 2019 ; Will et al., 2019 ).

With this cognition-oriented understanding of how knowledge revision may occur, a critical question remains: how does refutation text facilitate knowledge revision? Recall that refutation text states a common misconception about a subject, refutes it, and provides an explanation of the correct conception (Hynd, 2001 ; Mason et al., 2019 ; Tippett, 2010 ). If one were to align this with the KReC framework, presumably the statement of the misconception directly activates the existing schema, and an alternative conception is immediately offered and explained. Butterfuss and Kendeou ( 2020 ) suggested that this format “draws activation towards the correct idea and away from the misconception” (p. 790). Thus, conceptual change can result.

Learning with Refutation Text

What do we know about learning with refutation texts? Examining the research syntheses that have occurred, we can gain insights from both meta-analyses and qualitative reviews. For instance, Guzzetti et al. ( 1993 ) meta-analyzed the refutation text literature and concluded that “text can be used effectively to eradicate misconceptions either when text is refutational or when text is used in combination with other strategies that cause cognitive conflict” (p. 130). These results were promising, and a more recent meta-analysis of studies published between 1995 and 2010 examined conceptual change texts and found a very strong overall effect (Armağan et al., 2017 ). However, as noted, conceptual change texts are not the same as refutation texts, and this meta-analysis was not very clearly reported (e.g., the abstract and results report different overall effect sizes).

When we examine qualitative review efforts rather than meta-analyses, we gain additional understandings as to when and to what extent refutation texts may facilitate learning. For instance, Tippett’s ( 2010 ) review highlighted differential effects of refutation text depending on the age group reading the texts, and Zengilowski et al. ( 2021 ) noted that there are a number of variables that may influence the effectiveness of refutation texts.

While research synthesis efforts have supported the idea that refutation text can facilitate learning in some situations, it seems as though there are variables that may moderate this effect. Throughout this paper, we categorize these potentially moderating variables as features of the text and features of the research.

Features of the Text that May Influence Learning

We first examine the features of the text itself that may influence learning outcomes. These variables include the domain of the learning materials, the use of additional materials, the format of the instructional media, the length of the text, and the reading level of the text.

The Domain of the Learning Materials

Zengilowski et al. ( 2021 ) noted that the refutation text literature often examines only one particular refutation text compared to a comparable expository text rather than testing the effects of refutation texts across multiple texts. They further highlighted how, in many cases, the same few topics were being examined. Through meta-analysis, we can broadly examine the effects of refutation texts across various subject domains, as we will in this study. However, we note, as Zengilowski et al. did, that this is only a partial solution to the problem of many studies only examining one text in relation to another comparable text. But alas, a moderation analysis examining the effects of refutation texts across various knowledge domains would be a starting point which may provide insights into if there are varied outcomes depending on the knowledge domain.

The Use of Additional Learning Materials

In their review of the refutation text literature, Tippett ( 2010 ) noted that it is important for researchers to examine the effects of refutation texts when they are accompanied by images. This suggestion is supported by the multimedia principle, which suggests that people learn more when words and images are present compared to only text or narration (Mayer, 2017 , 2021 ). The multimedia principle has been found to be robust in the multimedia learning literature; thus, it is an open question as to whether providing images will add benefits to refutation texts.

The Format of the Instructional Media

Recent meta-analyses have found that reading from screens was not as effective as reading from paper (Clinton, 2019 ; Kong et al., 2018 ). However, Clinton ( 2019 ) found that this effect was moderated by the type of text. More specifically, expository texts were associated with lower (i.e., worse) effects than narrative texts when reading on an electronic device. Given that refutation texts are in some ways more similar to expository texts than narrative texts, in this meta-analysis, we examine if there are differential effects of refutation text depending on whether it is presented on paper or electronically.

The Length of the Text

One design limitation of refutation text is that it is often longer than expository texts. One potential view on this phenomenon is that a longer text may encourage more time engaged with the content, which may aid learning. Alternatively, refutation texts may provide additional information (e.g., a misconception and direct refutation) not present in the expository text, in addition to all the text in the expository text. It seems plausible that this extra amount of information may benefit learning, as opposed to the actual misconception and refutation itself. An open question then is whether refutation texts which are of similar length to the control condition texts vary in effectiveness compared to those that are longer. If so, it would imply that it is the text structure that facilitates learning rather than the simple addition of more information. Thus, we sought to investigate how a difference in length between the refutation text and the control condition moderated the effectiveness of refutation text.

The Reading Level of the Text

A final feature of the text is the reading level of the text. It seems plausible that texts which are lower in reading level may be easier to understand, whereas those that are higher in reading level are more complex. We question whether refutation text works differentially across these textual complexities.

Features of the Research Design that May Influence Learning

Researchers have also used a variety of research designs when investigating the use of refutation text. Thus, we sought to examine how features of the research design moderate learning with refutation text by investigating the following variables: the comparison condition, the age of the learner, the learners’ prior knowledge, how participants were assigned to condition, the type of outcome test, the timing of the outcome test, the type of outcome test, and the location of the study.

The Comparison Condition

While dated meta-analytic results have shown that refutation text can facilitate learning (Guzzetti et al., 1993 ), one outstanding question is how the effects of refutation text vary compared to other specific interventions such narrative or expository texts. Tippett ( 2010 ) found that learners preferred refutation text to expository text, but it is unclear whether this carries over to learning outcomes. Further, it is not clear to what extent refutation text may benefit, or hinder, learning compared to other interventions, such as narrative texts.

The Age of the Learner

Given that refutation texts are simple to create, a critical question is for what age groups are they effective? Tippett’s ( 2010 ) review of 22 studies provides some initial insights into this question. Tippet found that only one study investigated the use of refutation text in grades K-2, and it did not find any significant benefit to reading a refutation text. Similarly, 11 studies investigated refutation texts with students in grades 11 or higher, and they also found no significant benefit. However, 10 studies investigated the effects of refutation texts with students in grades 3–10, and refutation text was found to be more likely to result in conceptual change then expository texts (Tippett, 2010 ). Thus, a critical question remains: a decade later, do we still see differential patterns of effectiveness from refutation text depending on the learners’ age?

The Learner’s Prior Knowledge

As noted above, a learner’s existing misconceptions can influence an individual’s conceptual change and learning (Dole & Niederhauser, 1990 ). These misconceptions represent a learners’ prior knowledge and beliefs about a subject. Thus, it is reasonable, and perhaps expected, that a learner’s prior knowledge may significantly moderate the effects of refutation text on learning.

How Participants Were Assigned to Conditions

It is plausible that the experimental design could have some influence on the outcomes found in an experiment. This is important contextual information to understand when interpreting a meta-analysis. For instance, if studies that used non-random and non-stratified assignment are associated with statistically higher effects than those that used random assignment, it may indicate that methodological choices in study design may be attributable to the observed effects rather than the intervention. Accordingly, in this meta-analysis, we examine how the type of participant assignment influenced learning from refutation texts.

The Type of Outcome Test

Various types of tests have been used to measure learning in the refutation text literature. It seems plausible that different types of tests may illuminate differential benefits of refutation text. For instance, if multiple choice retention tests were associated with small effects, while open-ended transfer tasks were associated with larger effects, this may imply that while refutation text may benefit retention to some degree, its benefits can be more strongly seen in tests of learning transfer. Given that many researchers may focus on how well students transfer knowledge to new domains whereas others are more interested in an immediate change in understanding, in this meta-analysis, we examine if the type of test moderates the effectiveness of refutation text.

The Timing of the Outcome Test

A critical question around the efficacy of any educational intervention is how well the benefits of the intervention persist over time. Tippett ( 2010 ) found in their review that refutation text maintained its effectiveness over time. However, in their critical review of the refutation text literature, Zengilowski et al. ( 2021 ) found that the evidence was not so clear cut, having found evidence that both supported and undermined the learning benefits of refutation text over time. Thus, it seems pertinent to examine whether the effects of refutation text are maintained over time, and if so, how well they are maintained.

The Type of Publication

Publication bias is a consistent issue in meta-analysis, as researchers have argued that statistically significant results are more likely to be published than non-significant results (Rosenthal, 1979 ). As such, in this study, we examine whether the type of publication, such as a journal article, conference paper, or dissertation, is associated with statistically different effects on learning.

The Location of the Study

Different cultures often have varied norms and expectations as well as different written languages. It seems plausible that these may carry over into learning environments. In an effort to broadly analyze whether locale influences learning with refutation text, we categorize studies by continent in which the study took place. We acknowledge that this is a coarse measurement, but it provides a starting point to look for trends, which, if found, could lead to a variety of potentially interesting research questions.

As shown, there are many factors that could potentially influence the efficacy of refutation text. The purpose of this meta-analysis is to help understand the effects of refutation text across these various potentially moderating variables.

The Present Meta-analysis

In this era of near-instant information exchange, misinformation and misconceptions can spread rapidly. Refutation text is a simple and low-cost intervention to help correct misconceptions which could easily be implemented across print and electronic media, but research on the effects of conceptual change texts has not been meta-analyzed in more than 10 years (Armağan et al., 2017 , whose literature search was conducted in 2010); the effects of refutation texts specifically have not been quantitatively synthesized in nearly 20 years (Guzzetti et al., 1992 , 1993 ), nor qualitatively synthesized in ten (Tippett, 2010 ). In fact, the only recent review of the refutation text literature is Zengilowski et al.’s ( 2021 ) critical review, which highlighted a number of the field’s limitations. In addition to the lack of recent synthesis efforts, it is still unclear what factors can influence the effects of refutation text. As such, we performed a random-effects meta-analysis to address the following questions about refutation text:

• RQ1: How effective is refutation text in promoting learning, and is there any evidence of publication bias in the analysis?
• RQ2: To what extent do features of the refutation text moderate its effectiveness?
• RQ3: To what extent do methodological features of the research moderate the effectiveness of refutation text?

Literature Search

Databases were searched on March 28, 2020, for the key term refutation* text*. We did not apply any starting time limits on the searches, meaning the databases were searched for articles as old as they contained. The databases produced the following number of research items (number per database in parenthesis): Academic Search Complete (182), APA PsychINFO (137), CINAHL Plus with Full Text (11), Education Research Complete (101), ERIC (97), MEDLINE (29), and Web of Science Core Collection (266). In addition, we searched the reference list of Tippet’s ( 2010 ) review, which provided another 32 research items for consideration. In total, our search revealed 855 research items, which after the removal of duplicates yielded 473 individual research items.

Inclusion Criteria

In order to be considered in this meta-analysis, studies had to include a between-subjects comparison of a refutation text condition and a separate non-refutation text condition. In addition, the non-refutation text condition had to contain the same general instructional information as the refutation text, meaning studies with no instructional materials (Heddy et al., 2017 ) or filler texts (Kendeou et al., 2014 ) as the non-refutation text conditions were excluded. Additional inclusion criteria were that the study had to quantitatively measure learning outcomes of factual information rather than beliefs or opinions; report enough data to accurately calculate an effect size; report the number of participants in the experimental and control groups, as this allowed for a meta-analysis examining clear, transparent data and the ability to report the refutation text and comparison group numbers for each comparison; use a research design with few or minimal confounding variables (e.g., Zielinski, 2017 was excluded as it seemed one could not plausibly conclude that the refutation text alone caused any difference that may have occurred); be published in English; and be publicly available.

Study Screening and Coding

Phase i, initial study review.

During the first phase of study screening, we examined the titles and abstracts of the research items to see if they met the inclusion criteria. After excluding the studies which did not meet the inclusion criteria, we were left with 129 research items.

Phase II, Full-Text Review

The 129 research items from phase I were retrieved in their full-text form. After reviewing the full text of the items which could be retrieved, 33 research items met the inclusion criteria.

A flow chart of the coding process appears in Fig.  1 .

PRISMA (Moher et al., 2009 ) inspired flow chart of coding process

Phase III, Data Extraction

The final phase of our study screening and coding process was extracting the relevant information from each study (Supp. Data 1 ). Below, we delineate our data extraction paradigm.

Selection of Relevant Comparisons

We took steps to ensure the comparisons examined had as few confounding variables as possible. For example, if a study contained three groups, such as an expository text group, a refutation text group, and a refutation text with image group, then we analyzed the refutation text group compared to the expository text group. In this case, we would exclude the refutation text with an image group because the image represents a confounding variable compared to the expository text condition it would be compared to.

In practical terms, we preferred comparisons within studies that compared refutation texts to other types of texts (often expository) while ignoring other treatment conditions present in the study. For example, some studies contained multiple comparisons, such as Adesope et al. ( 2017 ), who examined the use of a concept map, refutation text, and scientific text. In this case, we coded the comparison of the refutation text and scientific text, ignoring the concept map condition.

Reconciling Multiple Publications of the Same Data

If a conference proceeding was located that appeared to have the same data as a journal article, the journal article was coded, unless it did not contain enough information to meet the inclusion criteria. For example, Diakidoy et al. ( 2002 ) had the same title and much of the same text as Diakidoy et al. ( 2003 ), but Diakidoy et al. ( 2003 ) did not specify the number of individuals in groups on post-testing; hence, Diakidoy et al. ( 2002 ) was coded.

Outcome Tests Coded

In order to abide by the principle of statistical independence, we analyzed one outcome test from each independent comparison. We coded learning outcomes preferentially as follows: conceptual change tests regardless of format were preferred, followed by transfer tests, comprehension or retention tests, and finally unspecified learning tests. If more than one test was present, we coded the most delayed test to capture learning maintenance over time. For example, Aguilar et al. ( 2019 ) reported tests at the time of instruction and 1 week later. For our analysis, we examined the results for test administered at 1 week after the intervention. If more than one test took place at relatively the same time and the types of test could not be preferentially coded based on the basis outlined above, we coded open-ended transfer tests rather than Likert or multiple-choice tests. For example, Liu and Nesbit ( 2018 ) used measures of free recall, short answer transfer, and a multiple-choice measure. We coded the short answer transfer measure.

Moderating Variables

We categorized potential moderator variables as either features of the experimental text or as methodological features of the research. Note that regardless of the potentially moderating variable, if the details needed to code for the specific moderator variable were not reported in the primary study, we coded that specific variable as not reported for that study. Table ​ Table1 1 describes the individual variables and coding schemes used for each variable.

The list of variables coded as potential moderators. Note that all categories had a “not reported” option if the specific variable was not reported in the primary study

Inter-rater Reliability

One author coded all of the studies. The second author coded eight randomly selected studies from those that met the inclusion criteria (24.24% of the sample). IBM SPSS 26 was used to calculate Cohen’s Kappa and the percent agreement. The Kappa statistic showed strong inter-rater reliability ( k  = 0.89, p  < 0.001), and inter-rater agreement was strong, with raters agreeing on 90.4% of the coded variables. Disagreements were reconciled through discussion.

Meta-analyzing and Interpreting the Results

We used Comprehensive Meta-Analysis (version 2.2.064) to run the meta-analysis, moderator analyses, publication bias tests, and sensitivity analysis. We used the random-effects model for the meta-analysis due to the nature of our sample (Borenstein et al., 2010 ). Borenstein et al. ( 2011 , p. 83–84) suggest that random-effects analyses are appropriate when the studies within the analysis have been conducted by independent researchers, resulting in studies that are not functionally identical, and the goal is to generalize the results across a wide variety of scenarios outside of a narrow population. We report the effect size Hedges g , the standardized mean difference effect size with an adjustment for small sample size bias. Positive effects indicate a benefit of refutation text compared to non-refutation text conditions. Effect sizes were interpreted as follows: g  = 0.20 being a small effect, g  = 0.40 being a moderate effect, and g  = 0.60 being a large effect (Hattie, 2015 ).

RQ1: How Effective Is Refutation Text in Promoting Learning, and Is There Any Evidence of Publication Bias in the Analysis?

After broadly searching the medical science, social science, psychological science, and science education literature, 33 studies that included 44 independent comparisons met the inclusion criteria. A random-effects meta-analysis of the 44 independent comparisons found a moderate overall effect size ( g  = 0.41, p  < 0.001, 95% CI [0.30, 0.51], n  = 3,869) with significant heterogeneity ( Q (43) = 109.59, p  < 0.001, I 2  = 60.76). The forest plot is in Fig.  2 .

Forest plot showing the random-effects meta-analysis results

Publication Bias

Publication bias is a consistent concern with meta-analyses, so we took steps to examine whether this was an issue within our data set. We examined the funnel plot for asymmetry and calculated a number of indicators of publication bias. The funnel plot appeared relatively symmetrical (Fig.  3 ). Egger’s regression intercept (Egger et al., 1997 ) showed no significant evidence of funnel plot asymmetry ( t (42) = 0.55, one-tailed p  = 0.29).

Funnel plot of the studies included in the meta-analysis

Despite the result of the Egger’s regression intercept test, we also utilized Duval and Tweedie’s ( 2000 ) trim and fill analysis. The analysis found no missing studies to the right of the mean. However, 10 studies were adjusted to the left of the mean (Fig.  4 ), which would make the overall effect of refutation text compared to other conditions g  = 0.28, 95% CI [0.16, 0.39]. Borenstein et al. ( 2011 ) noted that when evaluating the impact of bias, Duval and Tweedie’s ( 2000 ) trim and fill may indicate either a trivial change, a modest change that does not change the overall major finding, or a substantial change that would change the interpretation of the study. In this case, the adjusted effect size is still considered a small to moderate effect showing that refutation text benefits learning and that the effect size found from the trim and fill analysis is within two hundredths of being within the 95% confidence interval of the meta-analytic result. Moreover, the 95% confidence interval of the adjusted effect size overlaps with the 95% confidence interval of the overall meta-analytic result. Accordingly, we see indications of a modest change that does not change the overall interpretation of the key conclusion.

Funnel plot of the studies adjusted to the left of mean in the trim and fill analysis. The solid marks indicate adjusted studies and overall effect

Finally, we used the classic fail-safe n test (Rosenthal, 1979 ) to examine how many studies would be needed to bring p  > 0.05. The test showed that 1,625 studies would be needed to bring p  > 0.05.

Since the majority of test results indicated that publication bias is not a notable issue in this sample, and the trim and fill analysis found an adjusted effect size that does not change the key finding of the study, publication bias was determined not to be a substantial concern in this sample.

Sensitivity Analysis

We used Comprehensive Meta-Analysis to run the random-effects meta-analyses with each individual study omitted. The results were very similar to the results of the overall meta-analysis (Fig.  5 ), with the overall effect sizes ranging from g  = 0.39 to g  = 0.43 depending on the individual study omitted. From this analysis, it appears that no individual study had noticeably strong influence on the results of the meta-analysis.

Forest plot of the leave one out meta-analysis. Note that there is no dramatic effect on the overall effect size when any individual study is omitted

RQ2: To What Extent Do Features of the Refutation Text Moderate Its Effectiveness?

To examine this research question, we examined potentially moderating variables in relation to the features of the text (Table ​ (Table2). 2 ). Each potentially moderating variable is addressed below.

Potentially moderating variables related to text features

* p  < .05

We first examined whether refutation text was moderated by the field in which it was conducted, categorizing studies as either science, mathematics, or social sciences. We found that the effects of refutation text were robust across content domains and did not significantly vary based on the domain of the materials ( Q b (2) = 1.09, p  = 0.58).

Next, we examined how having additional materials with the refutation text, such as images, influenced the effectiveness of the refutation text. The effects of refutation text did not significantly vary based on the inclusion of additional instructional materials ( Q b (2) = 5.23, p  = 0.07). However, the majority of studies were text-only interventions that did not include these features.

Due to the vast amount of media being consumed in electronic format, it was important to examine whether the effectiveness of refutation text was moderated by media format (e.g., print, electronic). We found that the effectiveness of refutation text was robust across media formats, as the effects did not significantly vary based on the format of the media ( Q b (3) = 7.03, p  = 0.07).

One common feature of refutation text is that it tends to be longer than an expository text. Accordingly, we examined how the effectiveness of refutation text was moderated by length differences between the refutation text and the comparison condition. We found that the effectiveness of refutation text did not significantly vary by text length differences ( Q b (4) = 5.23, p  = 0.26). Therefore, the effect of refutation text is robust even when the texts are of similar length to the control text.

It was also important to examine whether the effectiveness of refutation text varied with the reading level of the text. While we found that reading level did not significantly influence the effectiveness of the refutation text ( Q b (3) = 1.12, p  = 0.77), most study authors did not report the reading level of the text. Therefore, these results should be interpreted with caution.

RQ3: To What Extent Do Methodological Features of the Research Moderate the Effectiveness of Refutation Text?

To examine this research question, we examined potentially moderating variables in relation to the methodological features of the research (Table ​ (Table3 3 ).

Potentially moderating variables related to methodological features

We sought to examine how the effects of refutation text vary when compared to other conditions such as narrative or expository texts. Importantly, this moderator analysis showed that refutation text was significantly more effective than all comparison conditions, and these positive effects did not vary significantly depending on the comparison condition ( Q b (3) = 2.56, p  = 0.47).

We also questioned whether the age of the learner moderated the effectiveness of refutation text. We found that the effects of refutation text were robust and consistent regardless of the age of the learners ( Q b (5) = 6.60, p  = 0.25). While most of the comparisons in our sample used post-secondary samples, the results revealed that refutation text was associated with positive effects across all other age groups examined.

The learners’ prior knowledge could also potentially moderate the effectiveness of refutation text. However, we found that most studies did not categorize the prior knowledge of participants ( k  = 37), and the moderator analysis revealed that prior knowledge did not have a significant moderating effect ( Q (2) = 2.18, p  = 0.34).

We next examined whether the method used to assign participants to conditions moderated the effectiveness of refutation text. The results showed that the type of participant assignment did not significantly influence the effectiveness of refutation text ( Q b (2) = 4.33, p  = 0.11). However, most comparisons assigned participants to conditions randomly, and those that did not were associated with notably larger effect sizes.

Another key element to investigate was potential moderation depending on type of outcome test. We found that the type of outcome test did not significantly influence the effectiveness of refutation text ( Q b (5) = 3.76, p  = 0.58).

Does the effect of refutation text persist over time? This question prompted our inclusion of the timing of the outcome test as a potential moderator. We found that the effectiveness of refutation text did not significantly differ depending on when the outcome test occurred ( Q b (4) = 0.48, p  = 0.98). While most studies used a test on the same day as the refutation text was read, two studies found that the positive effects of refutation text persist more than a month after reading the refutation text.

We did find that the effectiveness of refutation text significantly varied by the type of publication the comparison appeared in ( Q b (2) = 9.01, p  = 0.01). Conference proceedings were associated with the highest effect size ( g  = 0.73, p  = 0.001), followed by journal articles ( g  = 0.45, p  < 0.001) and dissertations or theses ( g  = 0.11, p  = 0.38).

Finally, we examined whether the location the study took place in had any significant role in moderating the effects of refutation text, and it did not ( Q b (2) = 0.63, p  = 0.73).

Inspired by a desire to help move learners towards scientifically accepted understandings, we conducted this meta-analysis to assess the effectiveness of refutation text on learning outcomes. In this section, we discuss the results in relation to each research question.

Previous research has indicated that refutation text can be an effective intervention for facilitating learning (Guzzetti et al., 1993 ). However, that work was completed nearly 2 decades ago, and there has been a plethora of research around refutation text since. Examining research around a similar construct, conceptual change texts, Armağan et al. ( 2017 ) found very strong effects. Our findings are largely consistent with these prior syntheses of the literature. Across 44 independent comparisons spanning 3,869 participants, our results revealed that refutation text was broadly effective for all learners, irrespective of the content domain, test type, or timing of test administration. Across all of our moderator analyses, we found only one negative effect size, and it was not statistically significant. Of the individual comparisons included in the meta-analysis, only five of the 44 showed an advantage for the non-refutation text condition. We looked for publication bias using four different techniques or analyses, and no notable evidence of publication bias was found that would dramatically change the interpretation of the results. Our sensitivity analysis also revealed that no individual study greatly influenced the overall meta-analytic result. Together, the results presented here provide compelling evidence that refutation text can be a broadly effective strategy for facilitating learning.

Through moderator analyses, we sought to examine how various design features of the text may moderate the effectiveness of the refutation text. We first examined the domain of the learning materials. We found the effects of refutation text did not differ across high-level domain classifications (science, social science, mathematics). However, Zengilowski et al. ( 2021 ) observed that many refutation texts covered a relatively small knowledge domain, and our findings largely confirm this as most studies were about scientific concepts. We therefore reiterate their call for refutation text research around new topics to see if the effects vary in different knowledge domains. This is potentially important because the learners’ prior knowledge and beliefs can influence their learning (Van Loon et al., 2015 ; Zengilowski et al., 2021 ). While many subjects, like photosynthesis, may not have strong personal beliefs involved, other topics, like vaccination, may have strong personal beliefs involved. Thus, it seems plausible that the effects of refutation texts may vary in these different subject areas.

Based on the multimedia learning literature (Mayer, 2017 , 2021 ), we had expected that refutation texts containing images would be more effective than refutation texts without images. Although the multimedia principle has been found to be robust, we did not find any evidence of it in this meta-analysis. In fact, the effect size found for studies that used images was very similar to the effect size found for text-only interventions. This is an interesting finding that is difficult to explain; however, we question if the images used simply did not add value compared to the text and were therefore ignored by the learners. Since relatively few studies used images in our sample ( k  = 9), more research is needed to better understand why graphics may, or may not, aid learning with refutation texts.

We also examined whether the refutation text being presented on a screen or on paper moderated the effect. Based on recent meta-analyses (Clinton, 2019 ; Kong et al., 2018 ), we expected that reading on paper would be superior. However, this was not replicated in our results. Clinton ( 2019 ) found that expository texts were associated with lower (i.e., worse) effects than narrative texts when reading on an electronic device. We wonder if refutation texts would lead to similar results if they had been included in their analysis. Primary research is warranted to investigate how well refutation texts maintain their efficacy when presented on a computer screen as compared to paper.

The length of refutation texts is also a concern, as in many cases the nature of a refutation text demands additional text as compared to an expository text on the same subject. Our analysis indicated that the length of the refutation text did not make a significant difference on learning. In other words, it did not matter whether the refutation texts were 25% longer than expository texts or < 10% longer than the expository texts; there were no statistically significant differences between levels of the moderator. Recall that a refutation text states a common misconception about a subject, refutes it, and provides an explanation of the correct conception (Hynd, 2001 ; Mason et al., 2019 ; Tippett, 2010 ). In order for a refutation text to be the same length as or only marginally longer than a comparable expository text, it is likely that some text must be removed from the explanation of the accepted understanding. In other words, it appears that the refutation text structure potentially allows for a shorter explanation of the subject and yet produces superior learning outcomes than expository texts.

We also sought to understand whether the reading level of the refutation text moderated the effect based on the hypothesis that texts at lower reading levels may be easier to understand, and texts at higher reading levels may be more complex or use more advanced terminology. However, we did not find any evidence to support this hypothesis in the moderator analysis as the learning effect did not significantly vary with experimental text reading level.

In summary, we found no evidence that any of the refutational text design features we analyzed significantly moderated the effects of refutation text. Rather, the effect was robust across a variety of different design components. However, research is certainly warranted around the domain of the learning materials, and we echo Zengilowski et al.’s ( 2021 ) call for research in new topic areas.

Our third research question was focused on examining whether specific aspects of the primary research significantly moderated the effectiveness of refutation text. Based on prior work, we knew that learners preferred refutation texts (Tippett, 2010 ), but we questioned whether they would be more effective compared to some conditions as opposed to others. For example, we questioned whether a refutation text had differential effects when compared to expository texts or narrative texts. Interestingly, we did not find significant differences by control condition, implying that refutation texts consistently facilitated learning. The question, then, is why this occurred? Presumably there are cognitive mechanisms that can explain these benefits. Through the lens of KReC, it seems plausible that, as Butterfuss and Kendeou ( 2020 ) argued, the format of the refutation text helps the learner move towards activating the correct conception rather than the incorrect one. Researchers have been investigating why refutation text may be effective through alternative measurements such as think-aloud protocols (Kendeou et al., 2019 ) and eye tracking (Ariasi & Mason, 2011 ; Ariasi et al., 2017 ), which could be a productive way forward for understanding how learners process refutation texts compared to other conditions.

We also questioned whether the learners’ age played any role in moderating the effects of refutation text. Based on Tippett’s ( 2010 ) review, we expected to see greater benefits for the middle grades rather than elementary, secondary, or post-secondary education. We did not find any significant differences depending on the age of the learner. However, we speculate that this difference between our work and Tippet’s may be due to a few factors. First, we categorized a learner’s age by the grade range in schooling (i.e., primary, secondary), whereas Tippet examined by specific grade level. Second, there are comparably few studies in our sample that investigated the effects of refutation texts with students not in post-secondary education. In addition, our inclusion criteria were different than Tippet’s. While we did not find statistically significant differences between levels of the moderator for learner age, we note that while there were fewer studies investigating students in K-12 settings, these were associated with larger effect sizes even if they were not statistically different than the effect derived from post-secondary learners. Therefore, future work may be warranted to investigate this phenomenon. Perhaps younger learners are more open to changing their conceptions about things they do not understand well, or perhaps the novelty of refutation texts has more influence on their learning. It also seems plausible that the topics of study investigated with these younger learners have fewer personal beliefs attached, which may explain the strong effect sizes. For instance, Mason et al. ( 2008 ) investigated refutation texts with fifth grade students learning about light. This is opposed to some studies with older learners that may have investigated topics that the learners felt more invested in or had stronger beliefs around, such as influenza (Vaughn & Johnson, 2018 ).

When we examined the learners’ prior knowledge as a potentially moderating variable, we had hoped to be able to decipher whether refutation text was more or less effective for learners of differing levels of preparation. This could be important for two reasons. First, we know that learners misconceptions can influence their conceptual change and learning (Dole & Niederhauser, 1990 ). Second, there has been evidence for expertise-reversal effects in other areas of research, where learners with high prior knowledge do not find the same benefits as learners with low prior knowledge (Chen et al., 2017 ; Kalyuga & Renkl, 2010 ). However, the reporting of the primary studies largely inhibited our ability to examine how refutation text differed depending on learners’ prior knowledge, in that 37 comparisons did not categorize their learners’ level of prior knowledge. This could be due in part to our coding scheme that set to classify learners as low, high, or mixed prior knowledge. We note that in many studies, authors did report the scores of the learning outcome pre-test. However, we feel it is important for future researchers to not only provide the scores, but also classify whether they consider the learners to have low or high prior knowledge so phenomenon such as the expertise-reversal effect can be addressed in this area.

As a methodological note in relation to meta-analysis more generally, our results show the importance of broadly searching the literature. We found that the largest effects were located in conference proceedings, whereas non-significant small effects were found in dissertations. Meanwhile, moderate effects were extracted from journal articles. Together, these results show the benefits of a comprehensive literature search that is not restricted to only peer-reviewed journal articles.

Finally, we note that specific aspects of the research design, such as how participants were assigned to condition and the type of outcome test used, can also have significant implications for the interpretation of meta-analyses. For instance, if significant effects of the intervention were found in studies that used convenience sampling but not for studies that used random assignment, one may question the efficacy of the intervention. In addition, if studies found benefits for multiple-choice tests but not for applied tasks, transfer tasks, or written essay questions, one may question whether the information was learned in a generalizable and transferable way or only through rote memorization. In this case, we did not find any significant moderation by the way participants were assigned to condition, nor due to the type of outcome test. This adds strength and some level of trustworthiness to the overall meta-analytic result that refutation text aids learning.

These results are further strengthened by the finding that the effects of refutation text did not vary depending on when the learning test occurred. By coding the most delayed test, we were able to examine the effects of refutation text across a variety of time periods, from the same day of the intervention through more than a month later. We found that the effects of refutation text persisted over time, although we note that the number of studies that examined the effects more than a month later was minimal ( k  = 2). The findings of the meta-analysis are also strengthened by the finding that the effects of refutation text did not vary by the continent on which the study took place. Overall, we found relatively robust evidence that refutation text can be an effective learning intervention across multiple contexts.

Implications for Theory

Recall that the purpose of this paper is not to validate one theory of conceptual change as opposed to another. Therefore, we explain our results in relation to the cognition-oriented KReC framework for similar reasons as explained in the literature review.

As noted, the KReC framework suggests that knowledge can be revised through activating relevant prior knowledge in relation to the new information, which facilitates the development of an integrated mental model which can then be used for activation in place of previous incorrect or incomplete mental models (Kendeou et al., 2019 ; Lassonde et al., 2017 ; Trevors et al., 2017 ; Will et al., 2019 ). In line with this theoretical perspective, it would appear that refutation texts can facilitate these knowledge revision processes, resulting in the increased activation needed for the new mental models to outcompete the older, incorrect, or incomplete mental models. However, research has shown that these incorrect or incomplete models do not just disappear; rather, they are maintained but can be suppressed (Mason & Zaccoletti, 2021 ; Treagust & Duit, 2008 ). This supports the idea that conceptual change is an ongoing process, rather than an instantaneous and permanent shift in conceptions (Treagust & Duit, 2008 ). Researchers have begun examining the role of inhibition in this process and have theorized it to be important for conceptual change, but more research in the area is needed (for a review, see Mason & Zaccoletti, 2021 ).

It should be noted that while KReC is a cognition-oriented theory, it can potentially account for the affective and non-cognitive factors (e.g., message characteristics) posited to underlie conceptual change by other researchers (e.g., Pintrich et al., 1993 ; Sinatra, 2005 ). For example, Trevors et al. ( 2017 ) investigated that the role emotion can play in knowledge revision through the lens of KReC. Moreover, it seems challenging to argue that the multitude of factors discussed by scholars (e.g., Pintrich et al., 1993 ; Sinatra, 2005 ; Treagust & Duit, 2008 ), such as learners’ prior conceptions, motivation, and characteristics of the learning materials, would not influence learners’ activation of knowledge. Moving forward, it seems plausible that these views on knowledge revision or conceptual change could be generally complimentary.

Due to the reporting of the primary studies, one theoretical issue that this study could not meaningfully address is the question of whether learners’ prior knowledge or prior misconceptions (or preconceptions) influences the effectiveness of refutation text. However, prior work has indicated that these beliefs, and the strength of these beliefs, can be an important factor in moderating the effects of refutation text (Van Loon et al., 2015 ; Zengilowski et al., 2021 ). In our sample, few studies reported the learners’ level of prior knowledge as low or high. Furthermore, we had planned to examine how learners’ prior misconceptions (or inaccurate preconceptions) moderated the effectiveness of refutation text. Unfortunately, very few studies specified the level of misconceptions held by learners and those that did took varied approaches. For example, Mason et al. ( 2017 ) stated that all their participants held a common misconception, Ariasi and Mason ( 2011 ) used a cutoff score on the pre-test as evidence of what they termed alternative conceptions, and Liu and Nesbit ( 2018 ) stated that participants demonstrated misconceptions at the pre-test. As of a result of these varied reporting methods and the considerable number of studies which did not mention inaccurate preconceptions, we did not examine this variable in the analysis. Thus, how and to what degree learners’ preconceptions or misconceptions impact learning outcomes when using refutation text is still an open question. In addition, Chi ( 2008 ) differentiated between beliefs and mental models. Beliefs make up mental models and therefore require a finer grain of measurement than we were able to pursue with a meta-analysis such as this. This may become a particularly important question into the future as refutation text is used with concepts that do not have a clear scientifically correct answer or are more tied to political or social issues (Zengilowski et al., 2021 ).

Implications for Practice

While the implications of this meta-analysis encourage reconciliation among theoretical perspectives, the practical implications are quite clear. Our results show that refutation text is effective for facilitating learning in all situations that were examined through the moderator analyses. Moreover, we did not find differential effects based on learner age, content domain, or test timing, which shows that refutation texts have been consistently effective across many contexts. In short, our results show that refutation texts can be used when there are commonly held misconceptions in the subject area (but recall that a common misconception must be known to write a refutation text), at least for one-time interventions involving shorter texts.

One particularly notable finding of this meta-analysis is that the benefits of refutation text were maintained over time. Zengilowski et al. ( 2021 ) recently qualitatively reviewed and critiqued the refutation text literature. Their analysis revealed that the persistent benefits of refutation text over time tended to occur among studies that dealt with scientific topics rather than those addressing political or social issues. This is consistent with our findings. In our analysis, we had two studies that examined learning more than a month after the intervention, one around light (Mason et al., 2008 ), and one around heat and temperature (Yürük, & Eroğlu, 2016 ). The question of how the effects of refutation text vary over time depending on the knowledge domain highlights an important question worthy of future research. Similarly, we question whether learners broadly generalize what they learned from refutation text outside of the original context, as conceptual change is viewed as domain specific (Treagust & Duit, 2008 ).

It is important to note that while we did examine some features of refutation text design as potentially moderating variables, this study was not intended to examine how to specifically design refutation texts. To do so in depth would require comparisons of design elements between multiple refutation texts, and this type of study would not have met our inclusion criteria. We note that not all forms of refutation text that were included in this study followed an identical format. For example, some used explicit refutation (Rodrigues & Thacker, 2019 ), examined the credibility of the explanation (Van Boekel et al., 2017 ), or used more implicit refutation (Adesope et al., 2017 ). In addition, some studies used questions rather than statements of misconceptions (Ariasi & Mason, 2011 , 2014 ), and some aligned more with the conceptual change text format but still contained the components of refutation text (Wang & Andre, 1991 ). These different designs could potentially influence the social, emotional, or motivational beliefs of an individual learner and thereby impact the conceptual change process (Pintrich et al., 1993 ; Sinatra, 2005 ). As such, research synthesis is needed to disentangle these design elements to better understand which types of refutation text are more effective, if any difference exists at all.

Limitations

Like all meta-analyses, this study is limited by the primary studies it is based upon. Studies meeting the inclusion criteria generally focused on scientific or mathematical concepts rather than social sciences, public policy, or public health issues, and we were unable to address how prior knowledge and prior misconception (and preconception) differences influenced the effectiveness of refutation text due a lack of explicit reporting in primary studies. In addition, the studies we evaluated were primarily one-time interventions with relatively short text passages (the longest passage reported was 1,542 words). The effectiveness of refutation text in these short passages is supportive of its use in common communication tools, such as news media and websites specifically designed to counter myths or misconceptions. However, it is unclear whether the effect would persist in longer text readings, such as textbooks, or other contexts such as repeated viewings over time. This type of research is needed to better understand if refutation text, as a mainstream, commonly used teaching strategy, maintains its effectiveness.

Another limitation was the reporting of primary studies. We had a substantial number of studies that were excluded from the analysis because they did not provide enough information for effect size extraction ( k  = 15) or did not provide very basic information about their experiment, such as the number of participants in each group ( k  = 15). Had this basic information been provided, our sample of studies analyzed would have nearly doubled in size. We encourage researchers in future studies to clearly and fully report their experimental methods and results.

We broadly and comprehensively systematically reviewed the literature and conducted a meta-analysis examining the extent to which learning is facilitated by refutation text compared to other learning conditions. Across 44 independent comparisons, we found compelling evidence to support the use of refutation text as a low-to-zero cost strategy to facilitate learning. Transitioning textual materials, particularly short pre-existing texts, to refutation text format would likely require little effort and minimal time investment. Therefore, we suggest that when knowledge revision is critical, particularly about scientific topics, refutation text may be an effective way to promote scientific understanding. However, we concur with Zengilowski et al. ( 2021 ) that refutation text should likely not be viewed as the panacea to combating all misinformation and misconceptions. Working from the foundational understanding that refutation text can aid learning (as shown in this meta-analysis), but may need to be designed differently depending on the context (see Zengilowski et al., 2021 ), researchers and practitioners alike can use refutation text as a tool for combating misconceptions.

Below is the link to the electronic supplementary material.

Acknowledgements

We are grateful to Dr. William Romine for his comments on a previous version of this manuscript.

Declarations

The authors declare no competing interests.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

* indicates study included in the meta-analysis

• Adesope OO, Cavagnetto A, Hunsu NJ, Anguiano C, Lloyd J. Comparative effects of computer-based concept maps, refutational texts, and expository texts on science learning. Journal of Educational Computing Research. 2017; 55 (1):46–69. doi: 10.1177/0735633116654163. [ CrossRef ] [ Google Scholar ]
• Aguilar SJ, Polikoff MS, Sinatra GM. Refutation texts: A new approach to changing public misconceptions about education policy. Educational Researcher. 2019; 48 (5):263–272. doi: 10.3102/0013189X19849416. [ CrossRef ] [ Google Scholar ]
• Ariasi N, Mason L. Uncovering the effect of text structure in learning from a science text: An eye-tracking study. Instructional Science. 2011; 39 (5):581–601. doi: 10.1007/s11251-010-9142-5. [ CrossRef ] [ Google Scholar ]
• Ariasi N, Mason L. From covert processes to overt outcomes of refutation text reading: The interplay of science text structure and working memory capacity through eye fixations. International Journal of Science & Mathematics Education. 2014; 12 (3):493–523. doi: 10.1007/s10763-013-9494-9. [ CrossRef ] [ Google Scholar ]
• Ariasi N, Hyönä J, Kaakinen JK, Mason L. An eye-movement analysis of the refutation effect in reading science text. Journal of Computer Assisted Learning. 2017; 33 (3):202–221. doi: 10.1111/jcal.12151. [ CrossRef ] [ Google Scholar ]
• Armağan FÖ, Keskin MÖ, Akin BS. Effectiveness of conceptual change texts: A meta analysis. European Journal of Science and Mathematics Education. 2017; 5 (3):343–354. doi: 10.30935/scimath/9515. [ CrossRef ] [ Google Scholar ]
• Borenstein M, Hedges LV, Higgins JP, Rothstein HR. A basic introduction to fixed-effect and random-effects models for meta-analysis. Research Synthesis Methods. 2010; 1 (2):97–111. doi: 10.1002/jrsm.12. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Borenstein M, Hedges LV, Higgins JP, Rothstein HR. Introduction to Meta-Analysis. John Wiley & Sons; 2011. [ Google Scholar ]
• Butterfuss R, Kendeou P. Reducing interference from misconceptions: The role of inhibition in knowledge revision. Journal of Educational Psychology. 2020; 112 (4):782–794. doi: 10.1037/edu0000385. [ CrossRef ] [ Google Scholar ]
• Çalik M, Ayas A, Coll RK. Enhancing pre-service elementary teachers' conceptual understanding of solution chemistry with conceptual change text. International Journal of Science and Mathematics Education. 2007; 5 (1):1–28. doi: 10.1007/s10763-005-9016-5. [ CrossRef ] [ Google Scholar ]
• Chambers SK, Andre T. Gender, prior knowledge, interest, and experience in electricity and conceptual change text manipulations in learning about direct current. Journal of Research in Science Teaching: The Official Journal of the National Association for Research in Science Teaching. 1997; 34 (2):107–123. doi: 10.1002/(SICI)1098-2736(199702)34:2<107::AID-TEA2>3.0.CO;2-X. [ CrossRef ] [ Google Scholar ]
• Chen O, Kalyuga S, Sweller J. The expertise reversal effect is a variant of the more general element interactivity effect. Educational Psychology Review. 2017; 29 (2):393–405. doi: 10.1007/s10648-016-9359-1. [ CrossRef ] [ Google Scholar ]
• Chi MTH. Three types of conceptual change: Belief revision, mental model transformation, and categorical shift. In: Vosniadou S, editor. Handbook of Research on Conceptual Change. Erlbaum; 2008. pp. 61–82. [ Google Scholar ]
• Clinton V. Reading from paper compared to screens: A systematic review and meta-analysis. Journal of Research in Reading. 2019; 42 (2):288–325. doi: 10.1111/1467-9817.12269. [ CrossRef ] [ Google Scholar ]
• *Diakidoy, I.-A. N., Kendeou, P., & Ioannides, C. (2002). Reading about energy: The effects of text structure in science learning and conceptual change. Paper presented at the Annual Meeting of the American Educational Research Association.
• Diakidoy I-AN, Kendeou P, Ioannides C. Reading about energy: The effects of text structure in science learning and conceptual change. Contemporary Educational Psychology. 2003; 28 (3):335. doi: 10.1016/S0361-476X(02)00039-5. [ CrossRef ] [ Google Scholar ]
• Diakidoy I-AN, Mouskounti T, Ioannides C. Comprehension and learning from refutation and expository texts. Reading Research Quarterly. 2011; 46 (1):22–38. doi: 10.1598/RRQ.46.1.2. [ CrossRef ] [ Google Scholar ]
• Diakidoy I-AN, Mouskounti T, Fella A, Ioannides C. Comprehension processes and outcomes with refutation and expository texts and their contribution to learning. Learning & Instruction. 2016; 41 :60–69. doi: 10.1016/j.learninstruc.2015.10.002. [ CrossRef ] [ Google Scholar ]
• Dole JA, Niederhauser DS. Students’ level of commitment to their naive conceptions and their conceptual change learning from texts. National Reading Conference Yearbook. 1990; 39 :303–310. [ Google Scholar ]
• Duval S, Tweedie R. Trim and fill: A simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics. 2000; 56 (2):455–463. doi: 10.1111/j.0006-341X.2000.00455.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Egger M, Smith GD, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997; 315 (7109):629–634. doi: 10.1136/bmj.315.7109.629. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Frède V. Teaching astronomy for pre-service elementary teachers: A comparison of methods. Advances in Space Research. 2008; 42 (11):1819–1830. doi: 10.1016/j.asr.2007.12.001. [ CrossRef ] [ Google Scholar ]
• Guzzetti BJ, Snyder T, Glass G. Promoting conceptual change in science: Can texts be used effectively? Journal of Reading. 1992; 35 (8):642–649. [ Google Scholar ]
• Guzzetti, B. J., Snyder, T. E., Glass, G. V., & Gamas, W. S. (1993). Promoting conceptual change in science: A comparative meta-analysis of instructional interventions from reading education and science education. Reading Research Quarterly , 117–159.
• Hattie J. The applicability of visible learning to higher education. Scholarship of Teaching and Learning in Psychology. 2015; 1 (1):79. doi: 10.1037/stl0000021. [ CrossRef ] [ Google Scholar ]
• Heddy BC, Danielson RW, Sinatra GM, Graham J. Modifying knowledge, emotions, and attitudes regarding genetically modified foods. Journal of Experimental Education. 2017; 85 (3):513–533. doi: 10.1080/00220973.2016.1260523. [ CrossRef ] [ Google Scholar ]
• Hynd CR. Refutational texts and the change process. International Journal of Educational Research. 2001; 35 (7–8):699–714. doi: 10.1016/S0883-0355(02)00010-1. [ CrossRef ] [ Google Scholar ]
• Kalyuga S, Renkl A. Expertise reversal effect and its instructional implications: Introduction to the special issue. Instructional Science. 2010; 38 (3):209–215. doi: 10.1007/s11251-009-9102-0. [ CrossRef ] [ Google Scholar ]
• Kendeou P, Van Den Broek P. The effects of prior knowledge and text structure on comprehension processes during reading of scientific texts. Memory & Cognition. 2007; 35 (7):1567–1577. doi: 10.3758/BF03193491. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kendeou P, Walsh EK, Smith ER, O’Brien EJ. Knowledge revision processes in refutation texts. Discourse Processes. 2014; 51 (5/6):374–397. doi: 10.1080/0163853X.2014.913961. [ CrossRef ] [ Google Scholar ]
• Kendeou P, Butterfuss R, Kim J, Van Boekel M. Knowledge revision through the lenses of the three-pronged approach. Memory & Cognition. 2019; 47 (1):33–46. doi: 10.3758/s13421-018-0848-y. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kong Y, Seo YS, Zhai L. Comparison of reading performance on screen and on paper: A meta-analysis. Computers & Education. 2018; 123 :138–149. doi: 10.1016/j.compedu.2018.05.005. [ CrossRef ] [ Google Scholar ]
• Korur F, Enil G, Göçer G. Effects of two combined methods on the teaching of basic astronomy concepts. Journal of Educational Research. 2016; 109 (2):205–217. doi: 10.1080/00220671.2014.946121. [ CrossRef ] [ Google Scholar ]
• *Kutza, D. M. (2000). The role of prior beliefs, refutational text, intrinsic and extrinsic goals, and extrinsic reward structure, in the conceptual change of preservice teachers (2001–95003–130; Issues 8-A) [ProQuest Information & Learning].
• Lassonde KA, Kolquist M, Vergin M. Revising psychology misconceptions by integrating a refutation-style text framework into poster presentations. Teaching of Psychology. 2017; 44 (3):255–262. doi: 10.1177/0098628317712754. [ CrossRef ] [ Google Scholar ]
• Lem S, Kempen G, Ceulemans E, Onghena P, Verschaffel L, Dooren W. Combining multiple external representations and refutational text: An intervention on learning to interpret box plots. International Journal of Science & Mathematics Education. 2015; 13 (4):909–926. doi: 10.1007/s10763-014-9604-3. [ CrossRef ] [ Google Scholar ]
• Liu Q, Nesbit JC. Conceptual change with refutational maps. International Journal of Science Education. 2018; 40 (16):1980–1998. doi: 10.1080/09500693.2018.1515513. [ CrossRef ] [ Google Scholar ]
• Lombardi D, Danielson RW, Young N. A plausible connection: Models examining the relations between evaluation, plausibility, and the refutation text effect. Learning & Instruction. 2016; 44 :74–86. doi: 10.1016/j.learninstruc.2016.03.003. [ CrossRef ] [ Google Scholar ]
• Mason L, Zaccoletti S. Inhibition and conceptual learning in science: A review of studies. Educational Psychology Review. 2021; 33 :181–212. doi: 10.1007/s10648-020-09529-x. [ CrossRef ] [ Google Scholar ]
• Mason L, Gava M, Boldrin A. On warm conceptual change: The interplay of text, epistemological beliefs, and topic interest. Journal of Educational Psychology. 2008; 100 (2):291–309. doi: 10.1037/0022-0663.100.2.291. [ CrossRef ] [ Google Scholar ]
• Mason L, Baldi R, Di Ronco S, Scrimin S, Danielson RW, Sinatra GM. Textual and graphical refutations: Effects on conceptual change learning. Contemporary Educational Psychology. 2017; 49 :275–288. doi: 10.1016/j.cedpsych.2017.03.007. [ CrossRef ] [ Google Scholar ]
• Mason L, Zaccoletti S, Carretti B, Scrimin S, Diakidoy I-AN. The role of inhibition in conceptual learning from refutation and standard expository texts. International Journal of Science & Mathematics Education. 2019; 17 (3):483–501. doi: 10.1007/s10763-017-9874-7. [ CrossRef ] [ Google Scholar ]
• Mayer RE. Using multimedia for e-learning. Journal of Computer Assisted Learning. 2017; 33 (5):403–423. doi: 10.1111/jcal.12197. [ CrossRef ] [ Google Scholar ]
• Mayer RE. Evidence-based principles for how to design effective instructional videos. Journal of Applied Research in Memory and Cognition. 2021; 10 (2):229–240. doi: 10.1016/j.jarmac.2021.03.007. [ CrossRef ] [ Google Scholar ]
• Mayo Clinic Staff. (2020). Debunking COVID-19 (coronavirus) myths . Mayo Clinic. Retrieved June 10, 2020, from https://www.mayoclinic.org/diseases-conditions/coronavirus/in-depth/coronavirus-myths/art-20485720
• Moher D, Liberati A, Tetzlaff J, Altman DG, Prisma Group Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med. 2009; 6 (7):e1000097. doi: 10.1371/journal.pmed.1000097. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Muis KR, Sinatra GM, Pekrun R, Winne PH, Trevors G, Losenno KM, Munzar B. Main and moderator effects of refutation on task value, epistemic emotions, and learning strategies during conceptual change☆ Contemporary Educational Psychology. 2018; 55 :155–165. doi: 10.1016/j.cedpsych.2018.10.001. [ CrossRef ] [ Google Scholar ]
• *Nasim, A. M. (2018). Using Refutation Texts to Change Attitudes and Knowledge Concerning Auditory Verbal Hallucinations
• *Peltier, T. K., Heddy, B. C., & Peltier, C. (2020). Using conceptual change theory to help preservice teachers understand dyslexia. Annals of Dyslexia . MEDLINE. 10.1007/s11881-020-00192-z [ PubMed ]
• Pintrich PR, Marx RW, Boyle RA. Beyond cold conceptual change: The role of motivational beliefs and classroom contextual factors in the process of conceptual change. Review of Educational Research. 1993; 63 (2):167–199. doi: 10.3102/00346543063002167. [ CrossRef ] [ Google Scholar ]
• Posner GJ, Strike KA, Hewson PW, Gertzog WA. Accommodation of a scientific conception: Toward a theory of conceptual change. Science Education. 1982; 66 (2):211–227. doi: 10.1002/sce.3730660207. [ CrossRef ] [ Google Scholar ]
• Prinz A, Golke S, Wittwer J. Refutation texts compensate for detrimental effects of misconceptions on comprehension and metacomprehension accuracy and support transfer. Journal of Educational Psychology. 2019; 111 (6):957–981. doi: 10.1037/edu0000329. [ CrossRef ] [ Google Scholar ]
• *Ramsburg, J. T. (2016). The lesson and the learner: The effect of individual differences and task scaffolding on category learning (2017–19723–005; Issues 7-B(E)) [ProQuest Information & Learning].
• *Rodrigues, J., & Thacker, I. (2019). Refuting a fraction misconception: A brief intervention promotes teachers' conceptual change. Conference Papers -- Psychology of Mathematics & Education of North America , 781–785.
• Rosenthal R. The file drawer problem and tolerance for null results. Psychological Bulletin. 1979; 86 (3):638. doi: 10.1037/0033-2909.86.3.638. [ CrossRef ] [ Google Scholar ]
• Roth, K. J. (1985). Conceptual change learning and student processing of science texts. Paper presented at the Annual Meeting of the American Educational Research Association, Chicago, IL. Retrieved from https://eric.ed.gov/?id=ED267980
• Rusanen A-M. Towards to an explanation for conceptual change: A mechanistic alternative. Science & Education. 2014; 23 (7):1413–1425. doi: 10.1007/s11191-013-9656-8. [ CrossRef ] [ Google Scholar ]
• *Safadi, R., Safadi, E., & Meidav, M. (2017). Reflection on solutions in the form of refutation texts versus problem solving: The case of 8th graders studying simple electric circuits. Physics Education , 52 (1).
• Salisbury-Glennon JD, Stevens RJ. Addressing Preservice Teachers’ Conceptions of Motivation. Teaching and Teacher Education. 1999; 15 (7):741–752. doi: 10.1016/S0742-051X(99)00023-2. [ CrossRef ] [ Google Scholar ]
• *Sanchez, E., & Garcia-Rodicio, H. (2013). Using online measures to determine how learners process instructional explanations. In LEARNING AND INSTRUCTION (Vol. 26, pp. 1–11). PERGAMON-ELSEVIER SCIENCE LTD. 10.1016/j.learninstruc.2012.12.003
• Sinatra GM. The" warming trend" in conceptual change research: The legacy of Paul R. Pintrich. Educational Psychologist. 2005; 40 (2):107–115. doi: 10.1207/s15326985ep4002_5. [ CrossRef ] [ Google Scholar ]
• Sinatra GM, Broughton SH. Bridging reading comprehension and conceptual change in science education: The promise of refutation text. Reading Research Quarterly. 2011; 46 (4):374–393. doi: 10.1002/RRQ.005. [ CrossRef ] [ Google Scholar ]
• Sinatra GM, Broughton SH, Diakidoy I-AN, Kendeou P, van den Broek P. Bridging reading comprehension and conceptual change in science education: The promise of refutation text. Reading Research Quarterly. 2011; 46 (4):374–393. doi: 10.1002/RRQ.005. [ CrossRef ] [ Google Scholar ]
• Södervik I, Mikkilä-Erdmann M, Vilppu H. Promoting the understanding of photosynthesis among elementary school student teachers through text design. Journal of Science Teacher Education. 2014; 25 (5):581–600. doi: 10.1007/s10972-013-9373-9. [ CrossRef ] [ Google Scholar ]
• Södervik I, Virtanen V, Mikkilä-Erdmann M. Challenges in understanding photosynthesis in a university introductory biosciences class. International Journal of Science & Mathematics Education. 2015; 13 (4):733–750. doi: 10.1007/s10763-014-9571-8. [ CrossRef ] [ Google Scholar ]
• Tippett CD. Refutation text in science education: A review of two decades of research. International Journal of Science & Mathematics Education. 2010; 8 (6):951–970. doi: 10.1007/s10763-010-9203-x. [ CrossRef ] [ Google Scholar ]
• Treagust DF, Duit R. Conceptual change: A discussion of theoretical, methodological and practical challenges for science education. Cultural Studies of Science Education. 2008; 3 (2):297–328. doi: 10.1007/s11422-008-9090-4. [ CrossRef ] [ Google Scholar ]
• Trevors GJ, Kendeou P, Butterfuss R. Emotion processes in knowledge revision. Discourse Processes. 2017; 54 (5/6):406–426. doi: 10.1080/0163853X.2017.1312201. [ CrossRef ] [ Google Scholar ]
• Van Boekel M, Lassonde K, O’Brien E, Kendeou P. Source credibility and the processing of refutation texts. Memory & Cognition. 2017; 45 (1):168–181. doi: 10.3758/s13421-016-0649-0. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• van Loon MH, Dunlosky J, van Gog T, van Merriënboer JJG, de Bruin ABH. Refutations in science texts lead to hypercorrection of misconceptions held with high confidence. Contemporary Educational Psychology. 2015; 42 :39–48. doi: 10.1016/j.cedpsych.2015.04.003. [ CrossRef ] [ Google Scholar ]
• Vaughn AR, Johnson ML. Communicating and enhancing teachers’ attitudes and understanding of influenza using refutational text. Vaccine. 2018; 36 (48):7306–7315. doi: 10.1016/j.vaccine.2018.10.040. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Vilppu H, Mikkilä-Erdmann M, Ahopelto I. The role of regulation and processing strategies in understanding science text among university students. Scandinavian Journal of Educational Research. 2013; 57 (3):246–262. doi: 10.1080/00313831.2011.637229. [ CrossRef ] [ Google Scholar ]
• Vosniadou S, editor. International handbook of research on conceptual change. Routledge; 2008. [ Google Scholar ]
• Wang T, Andre T. Conceptual change text versus traditional text and application questions versus no questions in learning about electricity. Contemporary Educational Psychology. 1991; 16 (2):103–116. doi: 10.1016/0361-476X(91)90031-F. [ CrossRef ] [ Google Scholar ]
• Will KK, Masad A, Vlach HA, Kendeou P. The effects of refutation texts on generating explanations. Learning & Individual Differences. 2019; 69 :108–115. doi: 10.1016/j.lindif.2018.12.002. [ CrossRef ] [ Google Scholar ]
• World Health Organization. (n.d.). COVID-19 Mythbusters – World Health Organization . World Health Organization. Retrieved July 26, 2020, from https://www.who.int/emergencies/diseases/novel-coronavirus-2019/advice-for-public/myth-busters
• Yürük N, Eroğlu P. The effect of conceptual change texts enriched with metaconceptual processes on pre-service science teachers' conceptual understandings of heat and temperature. Journal of Baltic Science Education. 2016; 15 (6):693–705. doi: 10.33225/jbse/16.15.693. [ CrossRef ] [ Google Scholar ]
• Zengilowski A, Schuetze BA, Nash BL, Schallert DL. A critical review of the refutation text literature: Methodological confounds, theoretical problems, and possible solutions. Educational Psychologist. 2021 doi: 10.1080/00461520.2020.1861948. [ CrossRef ] [ Google Scholar ]
• *Zeruth, J. A. (2008). Promoting probabilistic reasoning: The interplay of learner, task, and text (2009–99090–593; Issues 11-A) [ProQuest Information & Learning].
• Zielinski, S. F. (2017). From No to Yes: The Impact of an Intervention on The Persistence of Algebraic Misconceptions among Secondary School Algebra Students

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

21 Argument, Counterargument, & Refutation

In academic writing, we often use an Argument essay structure. Argument essays have these familiar components, just like other types of essays:

• Introduction
• Body Paragraphs

But Argument essays also contain these particular elements:

• Debatable thesis statement in the Introduction
• Argument – paragraphs which show support for the author’s thesis (for example: reasons, evidence, data, statistics)
• Counterargument – at least one paragraph which explains the opposite point of view
• Concession – a sentence or two acknowledging that there could be some truth to the Counterargument
• Refutation (also called Rebuttal) – sentences which explain why the Counterargument is not as strong as the original Argument

Consult  Introductions & Titles for more on writing debatable thesis statements and  Paragraphs ~ Developing Support for more about developing your Argument.

Imagine that you are writing about vaping. After reading several articles and talking with friends about vaping, you decide that you are strongly opposed to it.

Which working thesis statement would be better?

• Vaping should be illegal because it can lead to serious health problems.

Many students do not like vaping.

Because the first option provides a debatable position, it is a better starting point for an Argument essay.

Next, you would need to draft several paragraphs to explain your position. These paragraphs could include facts that you learned in your research, such as statistics about vapers’ health problems, the cost of vaping, its effects on youth, its harmful effects on people nearby, and so on, as an appeal to logos . If you have a personal story about the effects of vaping, you might include that as well, either in a Body Paragraph or in your Introduction, as an appeal to pathos .

A strong Argument essay would not be complete with only your reasons in support of your position. You should also include a Counterargument, which will show your readers that you have carefully researched and considered both sides of your topic. This shows that you are taking a measured, scholarly approach to the topic – not an overly-emotional approach, or an approach which considers only one side. This helps to establish your ethos as the author. It shows your readers that you are thinking clearly and deeply about the topic, and your Concession (“this may be true”) acknowledges that you understand other opinions are possible.

Here are some ways to introduce a Counterargument:

• Some people believe that vaping is not as harmful as smoking cigarettes.
• Critics argue that vaping is safer than conventional cigarettes.
• On the other hand, one study has shown that vaping can help people quit smoking cigarettes.

Your paragraph would then go on to explain more about this position; you would give evidence here from your research about the point of view that opposes your own opinion.

Here are some ways to begin a Concession and Refutation:

• While this may be true for some adults, the risks of vaping for adolescents outweigh its benefits.
• Although these critics may have been correct before, new evidence shows that vaping is, in some cases, even more harmful than smoking.
• This may have been accurate for adults wishing to quit smoking; however, there are other methods available to help people stop using cigarettes.

Your paragraph would then continue your Refutation by explaining more reasons why the Counterargument is weak. This also serves to explain why your original Argument is strong. This is a good opportunity to prove to your readers that your original Argument is the most worthy, and to persuade them to agree with you.

Activity ~ Practice with Counterarguments, Concessions, and Refutations

A. Examine the following thesis statements with a partner. Is each one debatable?

B. Write  your own Counterargument, Concession, and Refutation for each thesis statement.

Thesis Statements:

• Online classes are a better option than face-to-face classes for college students who have full-time jobs.
• Students who engage in cyberbullying should be expelled from school.
• Unvaccinated children pose risks to those around them.
• Governments should be allowed to regulate internet access within their countries.

Is this chapter:

…too easy, or you would like more detail? Read “ Further Your Understanding: Refutation and Rebuttal ” from Lumen’s Writing Skills Lab.

Note: links open in new tabs.

reasoning, logic

emotion, feeling, beliefs

moral character, credibility, trust, authority

goes against; believes the opposite of something

ENGLISH 087: Academic Advanced Writing Copyright © 2020 by Nancy Hutchison is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

[stmool | Adobe Stock}

Refutation Text and Critical Thinking

The Rationality of Science

Critical thinking is important in both education and everyday thinking. It has been shown to have positive associations with a range of important outcomes (Stanovich et al. 2016), and a key component of critical thinking is holding evidence-based beliefs (epistemic rationality). 

Misconceptions—beliefs that are contradicted by evidence—can be a roadblock to critical thinking. When these misconceptions are strong they are highly resistant to change; overcoming this resistance may require strategies that go beyond what is used in standard textbooks and teaching models. Presenting students with factual knowledge, alone, and expecting them to critically evaluate and make conceptual changes appear to be inadequate, especially when attempting to refute inaccurate or incomplete beliefs.

This type of information, often referred to as expository text , is intended to explain material without mention of associated misconceptions. The scientific literature shows there is better method for combating misconceptions. Refutation (or refutational) text has been used in various disciplines and has been shown to be particularly useful in combating misconceptions about science. Although misconceptions exist in virtually every subject area, they appear to be highly prevalent in science (Maria 2000). 

Refutational Teaching

Refutation, as it used in textbooks and lectures, has three key components: the misconception is explicitly stated, the misconception is then identified as being false, then scientific information is given that refutes the misconception (Kowalski and Taylor 2011). An example of the components of a refutation text: misconception : some people believe humans only use 10 percent of their brains; refutation cue : this belief is incorrect; refutation with scientific explanation : Our brains are constantly bombarded with sensory information, and widely distributed brain areas are involved in communication and processing information virtually all the time. Humans use much more than 10 percent of their brains. 

It is essential that the refutation text explicitly states a misconception and then refutes it with evidence. Refutation text includes elements of argumentation that are sometimes referred to as being some of the most effective strategies for changing people’s misconceptions. The conditions for changing misconceptions include being highly motivated, the information must be understandable (subjectively meaningful), the alternative concept must be plausible, and the information needs to be perceived as being useful in helping solve problems (Hughes et al. 2013). 

Kowalski and Taylor (2009) conducted a study to determine whether refutational lecture and text are effective in changing student misconceptions. The researchers tested students’ knowledge of forty-five common misconceptions about the brain, and then taught the course with refutational lecture and readings or taught the course using a standard format (simply presenting accurate facts). When tested at the end of the semester students showed significant changes in their beliefs when refutational approaches were used. 

In a study conducted by Broughton and colleagues (2010), undergraduates read either a refutation or standard text on seasonal change. Participant reading times were recorded and their conceptions were measured at pre- and posttest. Results showed readers spent less time reading the refutation paragraph compared to the standard text paragraph, and the refutation text group showed fewer misconceptions at posttest. The results showed intentional resources were allocated differently between the two types of text. The researchers asserted this difference in attention likely contributed to the reduction in misconceptions shown by those in the refutation text group. It may be that readers perceive the refutation text information as important, interesting, and easier to process as reflected by the brief amount of time spent reading the information. The finding of less time spent reading when readers are exposed to information that contradicts belief is supported by research examining people’s interests. Other research on interest shows that readers process material fast when they find it interesting.

Aguilar and colleagues (2019) examined refutation texts in the context of controversial educational policies (including Common Core State Standards and charter schools). Results of two experiments showed that refutation texts reduced participants’ misconceptions and led to correct conceptions about both policy issues. The researchers concluded that their findings hold promise for policymakers, implementers, and researchers seeking support for policies through evidence-based refutation texts.

Kowalski and Taylor (2011) conducted a study examining the effectiveness of refutational readings and lecture on decreasing psychological misconceptions for students of high versus low levels of achievement. High achieving students had fewer misconceptions after completing the refutational readings or after they heard the refutational lecture. However, low achieving students had fewer misconceptions only after hearing the refutational lecture. The refutational readings did not lead to conceptual change in low achieving students. This study indicates that different forms of refutational teaching may produce different outcomes, and that in some cases refutational readings are not enough to promote conceptual change. 

Tippett (2010) reviewed two decades of refutation text reasearch in the fields of science and reading education, and found than reading refutation text rather than expository text is more likely to result in conceptual changes. The review examined studies across age groups and grade levels. “[T]he results of 20 years of research conducted in both reading and science education are clear: reading refutation text can effectively promote conceputal change when readers possess science misconceptions, regardless of age or grade level” (Tippett 2010). 

Developing and implementing tools for critical thinking has widespread implications. Educators and other knowledge workers may benefit society by promoting evidence based information, while at the same time combating misconceptions. A relatively large body of research suggests that refutational teaching, in various forms, can assist in forming evidence-based beliefs. Refutational text and reading should be used in various domains, particularly in domains where misconceptions are most prevalent.   

References 

Aguilar, et al. 2019. Refutation Texts: A New Approach to Changing Public Misconceptions about Education Policy. Educational Researcher 48, 263–272.

Broughton, et al. 2010. The Nature of the Refutation Text Effect: An Investigation of Attention Allocation. The Journal of Educational Research  103(6), 407–423.

Hughes, S., et al. 2013. Misconceptions about Psychological Science: a review. Psychology Learning and Teaching 12(1), 20–31.

Kowalski, P., and Taylor, A.K. 2009. The effect of refuting misconceptions in the introductory psychology class. Teaching of Psychology 36(3), 153–159.    

Kowalski, P., and Taylor, A.K. 2011. Effectiveness of refutational teaching for high- and low- achieving students. Journal of the Scholarship of Teaching and Learning 11(1), 79–90.

Maria, K. 2000. Conceptual change instruction: a social constructivist perspective. Reading and Writing Quarterly 16, 5–22. 

Stanovich, et al. 2016. The Rationality Quotient: Toward A Test Of Rational Thinking . Cambridge, MA: The MIT Press. 

Tippett, C. 2010. Refutation Text in Science Education: A Review of Two Decades of Research. International Journal of Science and Mathematics Education 8(6), 951–970. 

Hill Street Studios/Getty Images

• An Introduction to Punctuation
• Ph.D., Rhetoric and English, University of Georgia
• M.A., Modern English and American Literature, University of Leicester
• B.A., English, State University of New York

In rhetoric, refutation is the part of an argument in which a speaker or writer counters opposing points of view. Also called  confutation .

Refutation is "the key element in debate," say the authors of The Debater's Guide  (2011). Refutation "makes the whole process exciting by relating ideas and arguments from one team to those of the other" ( The Debater's Guide , 2011).

In speeches, refutation and confirmation are often presented "conjointly with one another" (in the words of the unknown author of Ad Herrenium ): support for a claim ( confirmation ) can be enhanced by a challenge to the validity of an opposing claim ( refutation ).

In classical rhetoric , refutation was one of the rhetorical exercises known as the  progymnasmata .

Examples and Observations

"Refutation is the part of an essay that disproves the opposing arguments. It is always necessary in a persuasive paper to refute or answer those arguments. A good method for formulating your refutation is to put yourself in the place of your readers, imagining what their objections might be. In the exploration of the issues connected with your subject, you may have encountered possible opposing viewpoints in discussions with classmates or friends. In the refutation, you refute those arguments by proving the opposing basic proposition untrue or showing the reasons to be invalid...In general, there is a question about whether the refutation should come before or after the proof . The arrangement will differ according to the particular subject and the number and strength of the opposing arguments. If the opposing arguments are strong and widely held, they should be answered at the beginning. In this case, the refutation becomes a large part of the proof . . .. At other times when the opposing arguments are weak, the refutation will play only a minor part in the overall proof." -Winifred Bryan Horner, Rhetoric in the Classical Tradition . St. Martin's, 1988

Indirect and Direct Refutation

• "Debaters refute through an indirect means when they use counter-argument to attack the case of an opponent. Counter-argument is the demonstration of such a high degree of probability for your conclusions that the opposing view loses its probability and is rejected... Direct refutation attacks the arguments of the opponent with no reference to the constructive development of an opposing view...The most effective refutation, as you can probably guess, is a combination of the two methods so that the strengths of the attack come from both the destruction of the opponents' views and the construction of an opposing view." -Jon M. Ericson, James J. Murphy, and Raymond Bud Zeuschner,  The Debater's Guide , 4th ed. Southern Illinois University Press, 2011
• "An effective refutation must speak directly to an opposing argument. Often writers or speakers will claim to be refuting the opposition, but rather than doing so directly, will simply make another argument supporting their own side. This is a form of the fallacy of irrelevance through evading the issue." -Donald Lazere,  Reading and Writing for Civic Literacy: The Critical Citizen's Guide to Argumentative Rhetoric . Taylor & Francis, 2009

Cicero on Confirmation and Refutation

"[T]he statement of the case . . . must clearly point out the question at issue. Then must be conjointly built up the great bulwarks of your cause, by fortifying your own position, and weakening that of your opponent; for there is only one effectual method of vindicating your own cause, and that includes both the confirmation and refutation. You cannot refute the opposite statements without establishing your own; nor can you, on the other hand, establish your own statements without refuting the opposite; their union is demanded by their nature, their object, and their mode of treatment. The whole speech is, in most cases, brought to a conclusion by some amplification of the different points, or by exciting or mollifying the judges; and every aid must be gathered from the preceding, but more especially from the concluding parts of the address, to act as powerfully as possible upon their minds, and make them zealous converts to your cause." -Cicero, De Oratore , 55 BC

Richard Whately on Refutation

"Refutation of Objections should generally be placed in the midst of the Argument; but nearer the beginning than the end. If indeed very strong objections have obtained much currency, or have been just stated by an opponent, so that what is asserted is likely to be regarded as paradoxical , it may be advisable to begin with a Refutation." -Richard Whately, Elements of Rhetoric , 1846)​

FCC Chairman William Kennard's Refutation

"There will be those who say 'Go slow. Don't upset the status quo.' No doubt we will hear this from competitors who perceive that they have an advantage today and want regulation to protect their advantage. Or we will hear from those who are behind in the race to compete and want to slow down deployment for their own self-interest. Or we will hear from those that just want to resist changing the status quo for no other reason than change brings less certainty than the status quo. They will resist change for that reason alone. So we may well hear from a whole chorus of naysayers. And to all of them, I have only one response: we cannot afford to wait. We cannot afford to let the homes and schools and businesses throughout America wait. Not when we have seen the future. We have seen what high capacity broadband can do for education and for our economy. We must act today to create an environment where all competitors have a fair shot at bringing high capacity bandwidth to consumers—especially residential consumers. And especially residential consumers in rural and underserved areas." -William Kennard, Chairman of the FCC, July 27, 1998

Etymology: From the Old English, "beat"

Pronunciation: REF-yoo-TAY-shun

• Proof in Rhetoric
• Rogerian Argument: Definition and Examples
• AP English Exam: 101 Key Terms
• 5 Steps to Writing a Position Paper
• Reductio Ad Absurdum in Argument
• Anthypophora and Rhetoric
• Definition and Examples of Anti-Rhetoric
• How to Write a Persuasive Essay
• Paradox in English Grammar
• Argument (Rhetoric and Composition)
• Preparing an Argument Essay: Exploring Both Sides of an Issue
• Definition and Examples of Evidence in Argument
• Socratic Dialogue (Argumentation)
• Logos (Rhetoric)
• Tips on How to Write an Argumentative Essay
• Definitions and Examples of Debates

• school Campus Bookshelves
• menu_book Bookshelves
• perm_media Learning Objects
• login Login
• how_to_reg Request Instructor Account
• hub Instructor Commons
• Download Page (PDF)
• Download Full Book (PDF)
• Periodic Table
• Physics Constants
• Scientific Calculator
• Reference & Cite
• Tools expand_more
• Readability

selected template will load here

This action is not available.

9.18: Further Your Understanding- Refutation and Rebuttal

• Last updated
• Save as PDF
• Page ID 58393
• Lumen Learning

Learning Objectives

• Evaluate strategies for rebuttal and refutation of counterargument

Now let’s take a look at examples of rebuttal and refutation and consider how students follow these guidelines to approach counterarguments to their viewpoints:

• They accurately represent opposing viewpoints
• They use a respectful, non-incendiary tone
• They use reliable information
• They use qualifying words

Felix is writing his argument paper on why his university should not have cut funding to the school’s library. His arguable thesis reads as follows: Because Northern State University has a mission statement that includes becoming a Research 1 (R1) institution, full funding should be restored to the library to ensure faculty and students have adequate resources to enhance their research agendas.

Felix has done his research, and he knows that a couple of the main counterarguments are that the school needs funds to renovate the student union and to construct a new building for the Engineering Department. Thus, he can anticipate counterarguments and include them in his paper. While Felix cannot prove beyond doubt that the school should use more funding for the library instead of using it to address other needs, he can try to make the case.

Read over Felix’s passage below to see how he strengthens his case, and note the annotations to help you see parts of the formula in action.

Download the PDF of these examples

Now let’s take a look at another example:

Janae is also writing her argument paper on why NSU should not have cut funding to the library. During her research, though, she found evidence that some people on campus feel that the library has been careless with previous funding by mismanaging a $200,000 direct donation. Janae looked closer into the library budget, however, and found that the $200,000 donation was used to establish an emergency account for future years when funding did not meet their anticipated needs. Janae included as a source an editorial from the school newspaper written by a non-library faculty member who argued that since the library squandered $200,000, it should lose funding in favor of the student union and new Engineering building.

See Janae’s example in the passage below, and again, read over the annotations to see how she uses parts of the formula:

Notice how each student has a different goal and approach, yet they both still use parts of the formula to help them accomplish their rhetorical aims.

https://assessments.lumenlearning.co...essments/20287

Contributors and Attributions

• Rebuttal and Refutation. Provided by : University of Mississippi. Project : PLATO Project. License : CC BY-SA: Attribution-ShareAlike

Unit 6: Argumentative Essay Writing

41 Counterargument and Refutation Development

In an argumentative essay, you need to convince your audience that your opinion is the most valid opinion. To do so, your essay needs to be balanced—it needs an opposing (opposite) viewpoint, known as a counter-argument . Even though you are arguing one side of an issue, you must include what someone from the other side would say. After your opponent’s view, include a refutation to demonstrate why the other point of view is wrong.

Identifying Counterarguments

There are many ways to identify alternative perspectives.

• Have an imaginary dialogue with a "devil's advocate."
• Discuss your topic with a classmate or group of classmates.
• Interview someone who holds the opposite opinion.
• Read about the topic to learn more about different perspectives.

Example Argument

In the conversation below the writer talks to someone with the opposite opinion. Roberto thinks professors should incorporate Facebook into their teaching. Fatima argues the opposing side. This discussion helps the writer identify a counterargument.

Roberto: I think professors should incorporate Facebook into their teaching . Students could connect with each other in and out of the classroom. ( Position and pro-argument )

Fatima : Hmmm… that could work, but I don’t think it’s a very good idea . Not all students are on Facebook. Some students don’t want to create accounts and share their private information. ( Counterargument )

Roberto: Well…. students could create an account that’s just for the course.

Fatima : Maybe, but some students won’t want to use their personal accounts and would find it troublesome to create an additional “temporary class account.” Plus, I think more young people prefer Instagram. 

Example Counterargument paragraph

Roberto used information from the conversation and evidence from sources to write the counterargument paragraph. This paragraph concludes with a concession of validity and is followed by the refutation.

Example Refutation paragraph

Counterargument and refutation stems.

Below are the stems organized in a table.

Watch this video

The video refers to counterarguments as “counterclaims” and refutations as “rebuttals.

From: Karen Baxley

someone who presents a counterargument; someone who pretends to be against the issue for the sake of discussing the issue

Academic Writing I Copyright © by UW-Madison ESL Program is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License , except where otherwise noted.

Share This Book

• Link to facebook
• Link to linkedin
• Link to twitter
• Link to youtube
• Writing Tips

A Guide to Rebuttals in Argumentative Essays

4-minute read

• 27th May 2023

Rebuttals are an essential part of a strong argument. But what are they, exactly, and how can you use them effectively? Read on to find out.

What Is a Rebuttal?

When writing an argumentative essay , there’s always an opposing point of view. You can’t present an argument without the possibility of someone disagreeing.

Sure, you could just focus on your argument and ignore the other perspective, but that weakens your essay. Coming up with possible alternative points of view, or counterarguments, and being prepared to address them, gives you an edge. A rebuttal is your response to these opposing viewpoints.

How Do Rebuttals Work?

With a rebuttal, you can take the fighting power away from any opposition to your idea before they have a chance to attack. For a rebuttal to work, it needs to follow the same formula as the other key points in your essay: it should be researched, developed, and presented with evidence.

Rebuttals in Action

Suppose you’re writing an essay arguing that strawberries are the best fruit. A potential counterargument could be that strawberries don’t work as well in baked goods as other berries do, as they can get soggy and lose some of their flavor. Your rebuttal would state this point and then explain why it’s not valid:

Read on for a few simple steps to formulating an effective rebuttal.

Step 1. Come up with a Counterargument

A strong rebuttal is only possible when there’s a strong counterargument. You may be convinced of your idea but try to place yourself on the other side. Rather than addressing weak opposing views that are easy to fend off, try to come up with the strongest claims that could be made.

In your essay, explain the counterargument and agree with it. That’s right, agree with it – to an extent. State why there’s some truth to it and validate the concerns it presents.

Step 2. Point Out Its Flaws

Now that you’ve presented a counterargument, poke holes in it . To do so, analyze the argument carefully and notice if there are any biases or caveats that weaken it. Looking at the claim that strawberries don’t work well in baked goods, a weakness could be that this argument only applies when strawberries are baked in a pie.

Find this useful?

Subscribe to our newsletter and get writing tips from our editors straight to your inbox.

Step 3. Present New Points

Once you reveal the counterargument’s weakness, present a new perspective, and provide supporting evidence to show that your argument is still the correct one. This means providing new points that the opposer may not have considered when presenting their claim.

Offering new ideas that weaken a counterargument makes you come off as authoritative and informed, which will make your readers more likely to agree with you.

Summary: Rebuttals

Rebuttals are essential when presenting an argument. Even if a counterargument is stronger than your point, you can construct an effective rebuttal that stands a chance against it.

We hope this guide helps you to structure and format your argumentative essay . And once you’ve finished writing, send a copy to our expert editors. We’ll ensure perfect grammar, spelling, punctuation, referencing, and more. Try it out for free today!

Frequently Asked Questions

What is a rebuttal in an essay.

A rebuttal is a response to a counterargument. It presents the potential counterclaim, discusses why it could be valid, and then explains why the original argument is still correct.

How do you form an effective rebuttal?

To use rebuttals effectively, come up with a strong counterclaim and respectfully point out its weaknesses. Then present new ideas that fill those gaps and strengthen your point.

Share this article:

Post A New Comment

Got content that needs a quick turnaround? Let us polish your work. Explore our editorial business services.

The benefits of using an online proofreading service.

Proofreading is important to ensure your writing is clear and concise for your readers. Whether...

2-minute read

6 Online AI Presentation Maker Tools

Creating presentations can be time-consuming and frustrating. Trying to construct a visually appealing and informative...

What Is Market Research?

No matter your industry, conducting market research helps you keep up to date with shifting...

8 Press Release Distribution Services for Your Business

In a world where you need to stand out, press releases are key to being...

3-minute read

How to Get a Patent

In the United States, the US Patent and Trademarks Office issues patents. In the United...

The 5 Best Ecommerce Website Design Tools 

A visually appealing and user-friendly website is essential for success in today’s competitive ecommerce landscape....

Make sure your writing is the best it can be with our expert English proofreading and editing.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Rebuttals and Refutations

 introduction to rebuttal and refutation of counterargument.

An integral part of composing a strong argument is including counterargument. This can be difficult, especially if a writer is arguing for a position s/he already agrees with. In such cases, writers can sometimes make good points to support their stances; however, their arguments are vulnerable unless they anticipate and address counterarguments. When a writer does this, it is often referred to as rebuttal or refutation. Some scholars of rhetoric differentiate the two words in terms of if you can actually disprove a claim or just argue against it; however, in this section, we will use the terms as basically interchangeable to help get you more used to their function in argument. When writers are able to skillfully rebut or refute a view that runs counter to their claims, it strengthens their work. Rebuttal and refutation are common in all types of argument, including academic argument. As you complete more advanced work in college, you will be expected to address counterargument often. And while you might not always need to or be able to prove that other points of view are wrong, you may at least need to try to argue against them.

Formula for Refutation and Rebuttal

Though writers may handle rebuttal and refutation in different ways, there is a formula for success in academic argument. Here are the key parts of that formula:

Accurately represent opposing viewpoints

If you don’t accurately and thoroughly represent opposing viewpoints in your own writing, some of your potential audience will automatically be turned off. Good rebuttal and refutation begins with a solid understanding of all possible points of view on your topic. That may mean you even need to acknowledge and accommodate opposing points of view. Acknowledging other views shows you are aware of ideas that run counter to your claims. You will almost always be expected to at least acknowledge such views in your work. You may also, though, need to accommodate opposing views, especially if many people see them as reasonable. If, for example, you were writing a piece arguing that students should take a gap year between high school and college, it would benefit your work to acknowledge that a gap year isn’t realistic for or even desired by all students. You may further accommodate this other view by explaining how some students may thrive in the structure that school provides and would gain by going directly from high school to college. Remember that even if you cannot prove positions that counter your own are wrong, you can still use rebuttal and refutation to show why they might be problematic, flawed, or just not as good as another possible position for some people.

Use a respectful, non-incendiary tone

It doesn’t help the writer’s cause to offend, upset, or alienate potential readers, even those who hold differing views. Treating all potential readers with respect and avoiding words or phrases that belittle people and/or their views will help you get your points across more effectively. For example, if you are writing a paper on why America would benefit from a third viable major political party, it will not help your cause to write that “Republicans are dumb, and Democrats are whiny.” First, those claims are too general. But even if they weren’t, they won’t help your cause. If you choose to break down the perceived problems with members of political parties, you must do so in a way that is as respectful as possible. Calling someone a name or insulting them (directly or indirectly) is very rarely a successful strategy in argument.

Use reliable information in your rebuttal/refutation

Always be sure to carefully check the ideas or claims you make in rebutting a counterargument. The brain is not an infallible computer, and there are instances when we think we know information is accurate but it isn’t. Sometimes we know a lot about a particular subject but we get information confused or time has changed things a bit. Additionally, we may be tempted to use a source that backs up our ideas perfectly, but it might not be the most reputable, credible, or up-to-date place for information. Don’t assume you just have all of the information to shoot down counterarguments. Use your knowledge, but also do thorough research, double- and triple-check information, and look for sources that are likely to carry weight with readers. For example, it is widely assumed that bulls are attracted to the color red; however, in reality, bulls are colorblind, so what many people assume as fact is incorrect. Be thorough so you have confidence in your claims when you are rebutting/refuting and likewise when you are attempting to prevent yourself from being open to rebuttal/refutation.

Use qualifying words when applicable

Qualifying words are terms such as “many,” “most,” “some,” “might,” “rarely,” “doubtful,” “often,” etc. You get the point. These are words that don’t lock you into a claim that could be easily refuted and that can help you more easily rebut counterarguments. For example, if someone says “Nobody dies of tuberculosis anymore” we might get the point that it isn’t as common as it used to be. Still, it isn’t an accurate statement, and a more precise way to phrase such a claim would be to qualify it: “Not many people die each year in America from tuberculosis.” You might not always need to use qualifying terms. If you are making a point that is absolute, feel free to make it strongly; however, if there is a need to give your claim more flexibility, use qualifying words to help you.

Now let’s take a look at examples of rebuttal and refutation to further your understanding:

Download the PDF of these examples .

Felix is writing his argument paper on why his university should not have cut funding to the school’s library. His arguable thesis reads as follows: Because Northern State University has a mission statement that includes becoming a Research 1 (R1) institution, full funding should be restored to the library to ensure faculty and students have adequate resources to enhance their research agendas.

Felix has done his research, and he knows that a couple of the main counterarguments are that the school needs funds to renovate the student union and to construct a new building for the Engineering Department. Thus, he can anticipate counterarguments and include them in his paper. While Felix cannot prove beyond doubt that the school should use more funding for the library instead of to address other needs, he can try to make the case.

Read over Felix’s passage below to see how he strengthens his case, and note the annotations to help you see parts of the formula in action:

Notice how each student has a different goal and approach, yet they both still use parts of the formula to help them accomplish their rhetorical aims.

UNM Core Writing OER Collection Copyright © 2023 by University of New Mexico is licensed under a Creative Commons Attribution 4.0 International License , except where otherwise noted.

Share This Book

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Classical Argument

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

This resource describes the fundamental qualities of argument developed by Aristotle in the vital rhetorical text  On Rhetoric.  

A (Very) Brief History of Rhetoric

The study of rhetoric has existed for thousands of years, predating even Socrates, Plato and the other ancient Greek philosophers that we often credit as the founders of Western philosophy. Although ancient rhetoric is most commonly associated with the ancient Greeks and Romans, early examples of rhetoric date all the way back to ancient Akkadian writings in Mesopotamia.

In ancient Greece and Rome, rhetoric was most often considered to be the art of persuasion and was primarily described as a spoken skill. In these societies, discourse occurred almost exclusively in the public sphere, so learning the art of effective, convincing speaking was essential for public orators, legal experts, politicians, philosophers, generals, and educators. To prepare for the speeches they would need to make in these roles, students engaged in written exercises called  progymnasmata . Today, rhetorical scholars still use strategies from the classical era to conceptualize argument. However, whereas oral discourse was the main focus of the classical rhetoricians, modern scholars also study the peculiarities of written argument.

Aristotle provides a crucial point of reference for ancient and modern scholars alike. Over 2000 years ago, Aristotle literally wrote the book on rhetoric. His text  Rhētorikḗ ( On Rhetoric ) explores the techniques and purposes of persuasion in ancient Greece, laying the foundation for the study and implementation of rhetoric in future generations. Though the ways we communicate and conceptualize rhetoric have changed, many of the principles in this book are still used today. And this is for good reason: Aristotle’s strategies can provide a great guide for organizing your thoughts as well as writing effective arguments, essays, and speeches.

Below, you will find a brief guide to some of the most fundamental concepts in classical rhetoric, most of which originate in  On Rhetoric.

The Rhetorical Appeals

To understand how argument works in  On Rhetoric , you must first understand the major appeals associated with rhetoric. Aristotle identifies four major rhetorical appeals: ethos (credibility), logos (logic), pathos (emotion), and Kairos(time). 

• Ethos –  persuasion through the author's character or credibility. This is the way a speaker (or writer) presents herself to the audience. You can build credibility by citing professional sources, using content-specific language, and by showing evidence of your ethical, knowledgeable background.
• Logos –  persuasion through logic. This is the way a speaker appeals to the audience through practicality and hard evidence. You can develop logos by presenting data,  statistics, or facts by  crafting a clear claim with a logically-sequenced argument.  ( See enthymeme and syllogism )
• Pathos –  persuasion through emotion or disposition . This is the way a speaker appeals to the audience through emotion, pity, passions, or dispositions. The idea is usually to evoke and strengthen feelings already present within the audience. This can be achieved through story-telling, vivid imagery, and an impassioned voice.  Academic arguments in particular ​benefit from understanding pathos as appealing to an audience's academic disposition on a given topic, subject, or argument.
• Kairos – an appeal made through the adept use of time. This is the way a speaker appeals to the audience through notions of time. It is also considered to be the appropriate or opportune time for a speaker to insert herself into a conversation or discourse, using the three appeals listed above. A Kairotic appeal can be made through calls to immediate action, presenting an opportunity as temporary, and by describing a specific moment as propitious or ideal.

​*Note:  When using these terms in a Rhetorical Analysis, make sure your syntax is correct. One does not appeal to ethos, logos, or pathos directly. Rather, one appeals to an audience's emotion/disposition, reason/logic, or sense of the author's character/credibility within the text. Ethos, pathos, and logos are themselves the appeals an author uses to persuade an audience. 

An easy way to conceptualize the rhetorical appeals is through advertisements, particularly infomercials or commercials. We are constantly being exposed to the types of rhetoric above, whether it be while watching television or movies, browsing the internet, or watching videos on YouTube.

Imagine a commercial for a new car. The commercial opens with images of a family driving a brand-new car through rugged, forested terrain, over large rocks, past waterfalls, and finally to a serene camping spot near a tranquil lake surrounded by giant redwood trees. The scene cuts to shots of the interior of the car, showing off its technological capacities and its impressive spaciousness. A voiceover announces that not only has this car won numerous awards over its competitors but that it is also priced considerably lower than comparable models, while getting better gas mileage. “But don’t wait,” the voiceover says excitedly, “current lessees pay 0% APR financing for 12 months.”

In just a few moments, this commercial has shown masterful use of all four appeals. The commercial utilizes pathos by appealing to our romantic notions of family, escape, and the great outdoors. The commercial develops ethos by listing its awards, and it appeals to our logical tendencies by pointing out we will save money immediately because the car is priced lower than its competitors, as well as in the long run because of its higher MPG rate. Finally, the commercial provides an opportune and propitious moment for its targeted audience to purchase a car immediately. 

Depending on the nature of the text, argument, or conversation, one appeal will likely become most dominant, but rhetoric is generally most effective when the speaker or writer draws on multiple appeals to work in conjunction with one another. To learn more about Aristotle's rhetorical appeals, click here.

Components and Structure

The classical argument is made up of five components, which are most commonly composed in the following order:

• Exordium –  The introduction, opening, or hook.
• Narratio –  The context or background of the topic.
• Proposito and Partitio –  The claim/stance and the argument.
• Confirmatio and/or Refutatio –  positive proofs and negative proofs of support.
• Peroratio –  The conclusion and call to action.

Think of the exordium as your introduction or “hook.” In your exordium, you have an opportunity to gain the interest of your reader, but you also have the responsibility of situating the argument and setting the tone of your writing. That is, you should find a way to appeal to the audience’s interest while also introducing the topic and its importance in a professional and considerate manner. Something to include in this section is the significance of discussing the topic in this given moment (Kairos). This provides the issue a sense of urgency that can validate your argument.

This is also a good opportunity to consider who your intended audience is and to address their concerns within the context of the argument. For example, if you were writing an argument on the importance of technology in the English classroom and your intended audience was the board of a local high school, you might consider the following:

• New learning possibilities for students (General Audience Concerns)
• The necessity of modern technology in finding new, up-to-date information (Hook/Kairos)
• Detailed narrative of how technology in one school vastly improved student literacy (Hook/Pathos) 
• Statistics showing a link between exposure to technology and rising trends in literacy (Hook/Logos)
• Quotes from education and technology professors expressing an urgency for technology in English classrooms (Hook/Ethos)

Of course, you probably should not include all of these types of appeals in the opening section of your argument—if you do, you may end up with a boring, overlong introduction that doesn’t function well as a hook. Instead, consider using some of these points as evidence later on. Ask yourself:  What will be most important to my audience? What information will most likely result in the action I want to bring about?  Think about which appeal will work best to gain the attention of your intended audience and start there.

The narratio provides relevant foundational information and describes the social context in which your topic exists. This might include information on the historical background, including recent changes or updates to the topic, social perception, important events, and other academic research. This helps to establish the rhetorical situation for the argument: that is, the situation the argument is currently in, as impacted by events, people, opinion, and urgency of some kind. For your argument on technology in the English classroom, you might include:

• Advances in education-related technology over the centuries
• Recent trends in education technology
• A description of the importance of digital literacy
• Statistics documenting the lack of home technology for many students
• A selection of expert opinions on the usefulness of technology in all classrooms

Providing this type of information creates the setting for your argument. In other words, it provides the place and purpose for the argument to take place. By situating your argument within in a viable context, you create an opportunity to assert yourself into the discussion, as well as to give your reader a genuine understanding of your topic’s importance.

Propositio and Partitio

These two concepts function together to help set up your argument. You can think of them functioning together to form a single thesis. The propositio informs your audience of your stance, and the partitio lays out your argument. In other words, the propositio tells your audience what you think about a topic, and the partitio briefly explains why you think that way and how you will prove your point. 

Because this section helps to set up the rest of your argument, you should place it near the beginning of your paper. Keep in mind, however, that you should not give away all of your information or evidence in your partitio. This section should be fairly short: perhaps 3-4 sentences at most for most academic essays. You can think of this section of your argument like the trailer for a new film: it should be concise, should entice the audience, and should give them a good example of what they are going to experience, but it shouldn’t include every detail. Just as a filmgoer must see an entire film to gain an understanding of its significance or quality, so too must your audience read the rest of your argument to truly understand its depth and scope. 

In the case of your argument on implementing technology in the English classroom, it’s important to think not only of your own motivations for pursuing this technology in the classroom, but also of what will motivate or persuade your respective audience(s). Some writing contexts call for an audience of one. Some require consideration of multiple audiences, in which case you must find ways to craft an argument which appeals to each member of your audience. For example, if your audience included a school board as well as parents andteachers, your propositio might look something like this:

“The introduction of newer digital technology in the English classroom would be beneficial for all parties involved. Students are already engaged in all kinds of technological spaces, and it is important to implement teaching practices that invest students’ interests and prior knowledge. Not only would the marriage of English studies and technology extend pedagogical opportunities, it would also create an ease of instruction for teachers, engage students in creative learning environments, and familiarize students with the creation and sharing technologies that they will be expected to use at their future colleges and careers. Plus, recent studies suggest a correlation between exposure to technology and higher literacy rates, a trend many education professionals say isn’t going to change.”

Note how the above paragraph considers the concerns and motivations of all three audience members, takes a stance, and provides support for the stance in a way that allows for the rest of the argument to grow from its ideas. Keep in mind that whatever you promise in your propositio and partitio (in this case the new teaching practices, literacy statistics, and professional opinion) must appear in the body of your argument. Don’t make any claims here that you cannot prove later in your argument.

Confirmatio and Refutatio  

These two represent different types of proofs that you will need to consider when crafting your argument. The confirmatio and refutatio work in opposite ways, but are both very effective in strengthening your claims. Luckily, both words are cognates—words that sound/look in similar in multiple languages—and are therefore are easy to keep straight. Confirmatio is a way to confirm your claims and is considered a positive proof; refutatio is a way to acknowledge and refute a counterclaim and is considered a negative proof.

The confirmatio is your argument’s support: the evidence that helps to support your claims. For your argument on technology in the English classroom, you might include the following:

• Students grades drastically increase when technology is inserted into academics
• Teachers widely agree that students are more engaged in classroom activities that involve technology
• Students who accepted to elite colleges generally possess strong technological skills

The refutatio provides negative proofs. This is an opportunity for you to acknowledge that other opinions exist and have merit, while also showing why those claims do not warrant rejecting your argument. 

If you feel strange including information that seems to undermine or weaken your own claims, ask yourself this: have you ever been in a debate with someone who entirely disregarded every point you tried to make without considering the credibility of what you said? Did this make their argument less convincing? That’s what your paper can look like if you don’t acknowledge that other opinions indeed exist and warrant attention. 

After acknowledging an opposing viewpoint, you have two options. You can either concede the point (that is, admit that the point is valid and you can find no fault with their reasoning), or you can refute their claim by pointing out the flaws in your opponent’s argument. For example, if your opponent were to argue that technology is likely to distract students more than help them (an argument you’d be sure to include in your argument so as not to seem ignorant of opposing views) you’d have two options:

• Concession: You might concede this point by saying “Despite all of the potential for positive learning provided by technology, proponents of more traditional classroom materials point out the distractive possibilities that such technology would introduce into the classroom. They argue that distractions such as computer games, social media, and music-streaming services would only get in the way of learning.” 

In your concession of the argument, you acknowledge the merit of the opposing argument, but you should still try to flip the evidence in a positive way. Note how before conceding we include “despite all of the potential for positive learning.” This reminds your reader that, although you are conceding a single point, there are still many reasons to side with you.

• Refutation: To refute this same point you might say something like, “While proponents of more traditional English classrooms express concerns about student distraction, it’s important to realize that in modern times, students are already distracted by the technology they carry around in their pockets. By redirecting student attention to the technology administered by the school, this distraction is shifted to class content. Plus, with website and app blocking resources available to schools, it is simple for an institution to simply decide which websites and apps to ban and block, thereby ensuring students are on task.”

Note how we acknowledged the opposing argument, but immediately pointed out its flaws using straightforward logic and a counterexample. In so doing, we effectively strengthen our argument and move forward with our proposal.

Your peroratio is your conclusion. This is your final opportunity to make an impact in your essay and leave an impression on your audience. In this section, you are expected to summarize and re-evaluate everything you have proven throughout your argument. However, there are multiple ways of doing this. Depending on the topic of your essay, you might employ one or more of the following in your closing:

• Call to action (encourage your audience to do something that will change the situation or topic you have been discussing).
• Discuss the implications for the future. What might happen if things continue the way they are going? Is this good or bad? Try to be impactful without being overly dramatic.
• Discuss other related topics that warrant further research and discussion.
• Make a historical parallel regarding a similar issue that can help to strengthen your argument.
• Urge a continued conversation of the topic for the future.

Remember that your peroratio is the last impression your audience will have of your argument. Be sure to consider carefully which rhetorical appeals to employ to gain a desirable effect. Make sure also to summarize your findings, including the most effective and emphatic pieces of evidence from your argument, reassert your major claim, and end on a compelling, memorable note. Good luck and happy arguing!

Refutations and Reasoning in Undergraduate Mathematics

• Open access
• Published: 10 July 2023

Cite this article

You have full access to this open access article

• Lara Alcock   ORCID: orcid.org/0000-0003-2504-5953 1 &
• Nina Attridge   ORCID: orcid.org/0000-0001-9586-372X 2  

847 Accesses

Explore all metrics

This paper concerns undergraduate mathematics students’ understandings of refutation and their related performance in abstract conditional inference. It reports on 173 responses to a refutation instrument that asked participants to: 1) state ‘true’ or ‘false’ for three statements, providing counterexamples or reasons if they thought these false (all three were false); 2) evaluate possible counterexamples and reasons, where reasons were ‘corrected’ versions of the statements but not valid refutations; and 3) choose which of the counterexamples and the corrected statements were better answers, explaining why. The data show that students reliably understood the logic of counterexamples but did not respond normatively according to the broader logic of refutations. Many endorsed the corrected statements as valid and chose these as better responses; we analyse their explanations using Toulmin’s model of argumentation. The data further show that participants with better abstract conditional inference scores were more likely to respond normatively by giving, endorsing, and choosing counterexamples as refutations; conditional inference scores also predicted performance in a proof-based course.

Similar content being viewed by others

Non-deductive Justification in Mathematics

“This cannot be”—refutation feedback and its potential affordances for proof comprehension

Alon Pinto & Jason Cooper

How Persuaded Are You? A Typology of Responses

Avoid common mistakes on your manuscript.

Introduction

Refutation is an important part of mathematical reasoning. In mathematics education, it is often discussed in terms drawn from Lakatos’s seminal Proofs and Refutations (Lakatos, 1976 ). Lakatos characterises mathematical development as a process in which conjectures, proofs and refutations inform one another in multiple ways: mathematicians find global counterexamples to conjectures and clarify definitions to refine concepts; they analyse proofs to identify implicit lemmas, identify local counterexamples to suspect lemmas, and improve conjectures by incorporating unfalsified lemmas as conditions; they use insights so developed to extend concepts, bringing counterexamples into the domains of deeper, deductively generated theorems.

Research in mathematics education that builds on Lakatos ( 1976 ) has developed in several directions. Some researchers have studied characteristics of counterexamples that support conceptual change by convincingly refuting naïve conceptions (Balacheff, 1991 ; Peled & Zaslavsky, 1997 ; Zazkis & Chernoff, 2008 ). Some have designed task structures or teacher input to encourage students to generate counterexamples and modify conjectures and/or proofs (Buchbinder & Zaslavsky, 2011 ; Koichu, 2008 ; Komatsu, 2016 , 2017 ; Lin, 2005 ; Reid, 2002 ; Stylianides & Ball, 2008 ; Yang, 2012 ; Yopp, 2013 ). Some have sought to engage students in the full range of authentic mathematical activity as characterised by Lakatos, designing collaborative inquiry-based learning in which students identify and use local and global counterexamples to clarify conceptual meanings and to test and revise conjectures and proofs (Komatsu & Jones, 2022 ; Larsen & Zandieh, 2008 ; Stylianides & Stylianides, 2009 ; Yim et al., 2008 ).

In most cases, the focus of this work has been epistemological: researchers want to improve mathematical knowledge, and they seek to elicit and address refutations in service of that goal. Where the focus has been logical, it has usually been on helping students to understand the status of counterexamples in relation to general statements (Peled & Zaslavsky, 1997 ; Stylianides & Stylianides, 2009 ; Yopp et al., 2020 ; Zazkis & Chernoff, 2008 ) or on the importance of checking for counterexamples to steps in deductive arguments (Alcock & Weber, 2005 ; De Villiers, 2004 ; Ko & Knuth, 2013 ; Weber, 2010 ). It has less commonly addressed the logic of proposed refutations that are not counterexamples, although students and teachers have been observed to produce such alternatives, typically of two types. First, they might claim that a statement is false because no known theorem proves it. This has been reported in cases involving geometry (Potari et al., 2009 ) and inequalities and absolute values in calculus (Giannakoulias et al., 2010 ). Second, they might offer amended conjectures, in effect ‘correcting’ original, false conjectures. Creager ( 2022 ), for instance, reported on interviews in which pre-service teachers refuted geometry conjectures; of 32 arguments brought forth, 12 involved providing a conjecture with an alternative conclusion. Lin ( 2005 ) reported on a survey in which over 2000 7th and 8th graders considered the conjecture ‘a quadrilateral, in which one pair of opposite angles are right angles, is a rectangle’; 32% of 7th graders and 16% of 8th graders disagreed with the conjecture and explained by providing an alternative conclusion.

We believe that this second phenomenon merits deeper investigation, and were initially brought to this view by responses to test items in the first author’s real analysis course. Students were asked to examine various statements, to state ‘true’ or ‘false’ for each one and, if they stated ‘false’, to give a counterexample or a brief reason. Three items from the course appear below, with students’ purported refutations in the form of corrected statements.

For all \(y\in {\mathbb{R}}\) , the sequence \(\left({y}^{n}\right)\to \infty\) .

FALSE. Reason: It should be ‘For all \(y>1\) , the sequence \(\left({y}^{n}\right)\to \infty\) ’.

Every sequence has an increasing subsequence.

FALSE. Reason: It should be ‘Every sequence has a monotonic subsequence’.

\(\sum_{n=1}^{\infty }{a}_{n}\) converges if and only if \(\left({a}_{n}\right)\to 0\) .

FALSE. Reason: It should be ‘ \(\sum_{n=1}^{\infty }{a}_{n}\) converges if and only if its sequence of partial sums converges’.

The statements are indeed all false, and all three reasons are true theorems or correct definitions. They thus have epistemological value: the first two might result from successful heuristic refutation in Lakatos’s ( 1976 ) sense, and they all show that students have paid attention in the course. However, as refutations per se, they are logically inadequate because none rules out the possibility that the original statement is also true. In the analysis course, such responses were not occasional idiosyncratic curiosities – they were common across these items and others of similar structures.

We believe that this phenomenon deserves research attention because it is not obvious to what extent students providing corrected statements believe them to be logically adequate refutations. The above responses are spontaneous and short, so they give little information on the underlying reasoning. They might come to mind for students who then think no further. They might arise when students fail to find counterexamples and consequently give statements that they at least know are correct. They might arise from successful heuristic refutation in which students identify counterexamples, use these to inform corrections to the statements, and write down the correct versions as outcomes of that process. In the last two cases, we do not know to what extent students engage with the relevant logic. Maybe they do not think about it at all. Maybe they think about it and believe that their answers do logically contradict the original statements. Maybe they think about it and believe that their answers contradict the originals via looser conversational norms (Grice, 1989 ), assuming that a reader would infer that by correcting to ‘ \(y>1\) ’ they intend to exclude counterexamples among the cases where \(y\le 1\) .

This is important because, whether we intend to enforce logical precision or to allow more ‘natural’ communication, we do want undergraduate mathematics students to know that mathematicians notice cases in which an alternative claim does not logically refute a conjecture and that they might value corrected statements without deeming them valid refutations. For educational purposes, it would therefore be useful to know how accessible this view is based on students’ current understandings. This information is not inferable from spontaneous student responses as cited in the above research and anecdotal observations, because these provide no information on understandings of the relative value of different potential refutations: if a student provides a corrected statement, we learn nothing about whether they would also endorse a counterexample or vice versa, and nothing about which they think is the better refutation and why. We also learn nothing about whether their responses are related to their broader logical reasoning skills and mathematical performance.

We therefore designed a novel refutation instrument to investigate whether the logical issue was relatively easy to address – perhaps students would immediately realise that counterexamples were preferable – or whether corrected statements were endorsed as valid refutations, even in the face of alternatives. We used this, along with a standard abstract conditional inference task as a measure of logical reasoning, to address the following research questions.

How prevalent are counterexamples and corrected statements when students attempt to refute a statement?

When presented with both counterexamples and corrected statements, what proportions of students evaluate these as valid?

When asked to choose between counterexamples and corrected statements, what proportions of students deem each better?

How do students explain their reasons for deeming counterexamples or corrected statements better?

Are student responses to refutation tasks systematically related to their performance in abstract conditional inference?

Do refutation responses and abstract conditional inference performance predict course grades?

Below, we expand on the relevant research and set up a frame for analysis using Toulmin’s ( 1958 ) model of argumentation.

Theoretical Background

Refutations and counterexamples.

As noted above, research on the logic of refutations has usually focused on counterexamples. There has been concern that students might be reluctant to reject statements based on one or two counterexamples: Galbraith ( 1981 ) reported that some students aged 12–17 preferred to classify statements as partially right; Zeybek ( 2017 ) observed that pre-service primary teachers with less well-developed deductive reasoning skills tended to believe that more counterexamples would strengthen refutations; and Ko and Knuth ( 2013 ) found that two of 16 mathematics majors accepted a proof and a counterexample for the same statement. Stylianides and Al-Murani ( 2010 ), in contrast, found that although some high-attaining year 10 students expressed agreement with or uncertainty about arguments for and against the same proposition, when followed up in interviews they recognised the inconsistency.

Overall, research is somewhat mixed but broadly consistent with the idea that in simple enough cases and when not fooled by salient examples in which a statement is true, students can apply the basic logic of counterexamples. Roh and Lee ( 2017 ), for instance, reported that undergraduates introduced and used counterexamples when considering possible definitions of sequence convergence. Yopp et al. ( 2020 ) reported that eighth graders could be trained to use ‘eliminating counterexamples’ as a framework for constructing or critiquing arguments for general claims. Hamami et al. ( 2021 ) reported that adults without specialist mathematics training produced counterexamples when rejecting inferences about topological relations, though performance was impaired when counterexamples were ‘further from’ starting configurations. At a larger scale, Hoyles and Küchemann ( 2002 ) reported that of 2600 year 8 students presented with a simple false conjecture in number theory, 36% gave valid counterexamples. Küchemann and Hoyles ( 2006 ) reported that of 1500 students presented with a simple false conjecture in geometry, 42% of year 8 students and 56% of year 10 students gave valid counterexamples. Lin ( 2005 ) reported that of 2200 students presented with false geometry conjectures, between 4 and 34% of seventh and eighth graders gave counterexamples. In teacher education, Peled and Zaslavsky ( 1997 ) found that experienced teachers were almost universally able to give valid counterexamples. However, counterexamples might not be frequent in teaching: Zodik and Zaslavsky ( 2008 ) reported that of 604 teacher-produced examples across 54 observed lessons, only 18 were counterexamples.

We might therefore expect mathematics undergraduates to have limited experience in working with counterexamples, but to be able to handle the logic in simple cases. We might, however, expect them to have trouble generating counterexamples in more complex cases. Researchers have noted that mathematics undergraduates might fail to identify local counterexamples when validating purported proofs, and thus fail to recognise flaws in those arguments (Alcock & Weber, 2005 ; Ko & Knuth, 2013 ). They have also reported that it can be challenging for advanced students or for teachers to coordinate all conditions in a problem in order to produce valid counterexamples, for example in geometry problems with multiple components (Buchbinder & Zaslavsky, 2011 ; Komatsu et al., 2017 ; Potari et al., 2009 ), in problems involving continuous or quadratic functions (Ko & Knuth, 2009 ; Lee, 2017 ), and in purported proofs in calculus involving inequalities and absolute values (Giannakoulias et al., 2010 ).

Arguments and Warrants

To consider counterexamples alongside alternative refutations in a theoretically coherent way, we follow other researchers in employing Toulmin’s ( 1958 ) model of argumentation. Toulmin’s full model appears in Fig.  1 , which represents data put forth in support of a conclusion , perhaps with a suitable qualifier ; the data and conclusion are linked by a warrant , possibly with backing , and there might be a rebuttal capturing conditions under which the warrant would not adequately justify the conclusion based on the data.

Toulmin’s model of argumentation

In mathematics education, this model has been used in various contexts, including those involving refutation. Hoyles and Küchemann ( 2002 ), for instance, used it to model student arguments about whether an implication and its converse are equivalent. Giannakoulias et al. ( 2010 ) modelled arguments teachers formulated to convince hypothetical students that there were errors in written proofs. Inglis et al. ( 2007 ) pointed out that although various researchers have modelled mathematical arguments using only data, warrants, and conclusions, mathematical practice is better understood using the full model because both novice and expert mathematicians do use non-absolute arguments that admit rebuttals, so we should avoid assuming on limited evidence that qualifiers are absolute. We respect this point and in theoretical discussions we consider all six components; in analysing arguments that students put forth when evaluating and choosing between refutations, we simplify our diagrams by representing only those components that students invoke and those that are listed in the task.

In theoretical discussion, we also use the distinction Toulmin ( 2003 ) draws between warrants and backing, characterising warrants as ‘hypothetical, bridgelike statements’, whereas backing ‘can be expressed in the form of categorical statements of fact’ (p.98). Toulmin ( 1958 ) notes (pp.102–103) that an argument can be in the form with ‘data, warrant, so conclusion’, as in ‘(D) Petersen is a Swede, (W) A Swede is almost certainly not a Roman Catholic, so (C) Petersen is almost certainly not a Roman Catholic’, or in the form ‘data, backing, so conclusion’, as in ‘(D) Petersen is a Swede, (B) The proportion of Roman Catholic Swedes is minute, so (C) Petersen is almost certainly not a Roman Catholic’. This distinction is not straightforward – there are different possible relationships between warrants and backing (e.g. Castaneda, 1960 ; Simpson, 2015 ) – but in this form it is useful when we consider explanations students give to justify their choices between counterexamples and corrected statements.

We apply Toulmin’s model now to consider arguments about mathematical statements of the form \(\forall x S(x)\) . Refuting such a statement means arguing for the conclusion that \(\forall x S(x)\) is false. When a student offers a counterexample \({x}_{0}\) or a corrected statement \(\forall x T(x)\) , these function as data in support of that conclusion, and this is all we see of the argument; see Fig.  2 .

Arguments that \(\forall x S(x)\) is false, where data is a counterexample or a corrected statement

When the data is a counterexample \({x}_{0}\) , a mathematically valid argument can be completed. The implicit warrant is the principle that a valid counterexample \({x}_{0}\) refutes a general statement \(\forall x S(x)\) , with factual backing that \(S({x}_{0})\) is, indeed, false. Such an argument admits no rebuttal so is a valid refutation (the statement \(\forall x S(x)\) is refuted because the argument that it is false admits no rebuttal); see Fig.  3 . When the data is a corrected statement \(\forall x T(x)\) , the rest of the argument is less obvious. The student presumably intends to rely upon the fact that \(\forall x T(x)\) is true, which is factual so in Toulmin’s terms is understood as backing. The warrant is implicit, and several different warrants might be intended. Students might intend to say that \(\forall x S(x)\) and \(\forall x T\left(x\right)\) cannot both be true, so that the truth of \(\forall x T\left(x\right)\) implies the falsity of \(\forall x S(x)\) in a logical sense. Or they might intend to align with Grice’s ( 1989 ) maxims for communication, relying on the assumption that if \(\forall x T\left(x\right)\) is true, it is uncooperative instead to say \(\forall x S(x)\) , so that the truth of \(\forall x T\left(x\right)\) implies the falsity of \(\forall x S(x)\) in a conversational sense. Or they might rely on an assumed didactic contract (Brousseau, 1997 ), asserting that, because \(\forall x T\left(x\right)\) is correct, it should be viewed favourably as a replacement for \(\forall x S(x)\) , without much notion of implication. In any of these cases, the argument does admit a rebuttal, because a true mathematical statement, even if closely related to an original, proves that original false only if the two are contradictory (the statement is not refuted because the argument that it is false does admit a rebuttal); again, see Fig.  3 .

Warrants, backing and rebuttals for arguments that \(\forall x S(x)\) is false, where data is a counterexample or a corrected statement

This breakdown makes clear the complex interactions between mathematical and everyday arguments. If a student refutes a statement by correcting it, this might indicate a failure of logical reasoning. But it might indicate that the student did not attempt logical reasoning due to the salience of an argument with an alternative warrant or backing. Or it might indicate that they did attempt logical reasoning and perhaps get it right, but did not express this conventionally. We think that these distinctions are important because learning in academic disciplines involves learning about the weight given to different types of justification. In mathematics, we care about clear communication, and we certainly care about true statements. But valid logic – in the mathematical rather than the everyday sense – trumps everything.

Conditional Inference

Logical reasoning, then, is crucial in making theoretical sense of refutations. But does it have a statistically significant effect on students’ refutation attempts? Are students with better logical reasoning skills more likely to understand refutations in a mathematically normative way?

We investigated this by relating students’ refutation responses to their performance on a standard conditional inference task, for which sample items appear in Fig.  4 . Each item presents two premises concerning an imaginary letter-number pair, where the major premise is a conditional. The instructions read ‘If you think the conclusion necessarily follows, please tick YES, otherwise tick NO’. The inferences are modus ponens (MP, top left), denial of the antecedent (DA, top right), affirmation of the consequent (AC, bottom left), and modus tollens (MT, bottom right).

Four items from the conditional inference task used in this study (Evans et al., 1995 )

Such tasks are very common in research on the psychology of reasoning, though they vary in content and their handling of negation (Evans et al., 1996 ). Our task had abstract content (as opposed to everyday real-world content) and implicit negation (an explicit negation of ‘The letter is B’ would be ‘The letter is not B’ rather than ‘The letter is H’ as appears in Fig.  4 ). Across such tasks, educated adults almost always correctly endorse MP inferences. But, beyond that, their responses vary systematically with content (E.g., Evans et al., 2015 ) and they do not reason in ways that align with the material conditional interpretation accepted as normatively correct (e.g., Oaksford & Chater, 2020 ). For instance, Evans et al. ( 2007 ) used an abstract task like ours and found that (non-mathematics) undergraduates endorsed only 50% of valid MT inferences, and endorsed 47% of invalid DA and 74% of invalid AC inferences.

We used this specific task for both theoretical and practical reasons. Theoretically, many mathematical theorems and conjectures can be expressed in the form of a universally quantified conditional ‘ \(\forall x,\) if \(P(x)\) then \(Q(x)\) ’. The quantifier is often omitted (Solow, 2005 ), which can cause miscommunication when students interpret the conditional predicate ‘if \(P(x)\) then \(Q(x)\) ’ as lacking a truth value (Durand-Guerrier, 2003 ; Hub & Dawkins, 2018 ). Nevertheless, a valid counterexample can be understood as an \({x}_{0}\) for which \({P(x}_{0})\) but not \({Q(x}_{0})\) , and rejecting invalid inferences is closely related to recognising that ‘ \(\forall x,\) if \(P(x)\) then \(Q(x)\) ’ is not necessarily refuted by ‘ \(\forall x,\) if \(U(x)\) then \(Q(x)\) ’ or by ‘ \(\forall x,\) if \(P(x)\) then \(V(x)\) ’. We might therefore predict that students with better abstract conditional inference skills would work in more mathematically valid ways with refutations. Our research tests this prediction.

Practically, it has been established that performance in abstract conditional inference is linked to performance in mathematics. Research in the UK and Cyprus (Attridge & Inglis, 2013 ; Attridge et al., 2015 ) has shown that studying mathematics intensively at age 16–18 leads to better rejection of DA and AC inferences, but equivalent or poorer endorsement of MT inferences. Research at the undergraduate level has found that better rejection of invalid inferences predicted better performance on a proof comprehension test and in a proof-based course (Alcock et al., 2014 ). So, mathematics students can be expected to have better than average but imperfect conditional inference skills, and differences in these are known to have measurable effects on their performance in educationally relevant tasks. We extend this research by testing the relationship between abstract conditional inference skill and interpreting refutations.

Participants and Administration

Our research took place in a real analysis course at a UK university. Approximately two thirds of the class of over 200 were first-year students spending 75–100% of their study time on mathematics; the remainder were second-year students on degree programmes with about 50% mathematics. All of the programmes, like many UK mathematics-focused programmes, required high prior mathematical attainment. The majority of participants therefore had an A or A* grade in A level Mathematics and A or B grades in their two Footnote 1 other A level subjects (or the equivalent, if they were from overseas); some had taken an extra A level and/or an A level in Further Mathematics, but neither this nor special entry examinations were required. These students were all in one class because the UK higher education system operates a cohort model, meaning that students on a specific programme all attend core courses together.

Our study used two paper instruments, administered at different times so that participants would be less likely to perceive them as linked. Both instruments provided informed consent information and asked for participants’ university ID numbers; both ended with tick-boxes for agreement to data being used and to our acquiring their grades from the university’s system once the course finished. Participants completed the refutation instrument in 15 min in week 3 of the course; 173 agreed that their data could be used. They completed the abstract conditional inference instrument in 10 min in week 1 of the course; 157 of the 173 agreed that their data could be used, and 151 of these gave permission to access their course scores.

Refutation Instrument

We designed a refutation instrument with items structured similarly to those for which we had previously observed invalid refutations. Because we wanted to provide early collective feedback, our items used content from the first two weeks of the course. We refer to them as the reciprocal item , absolute value item , and sequence item respectively:

If \(x<3\) then \(1/x>1/3\) .

\(\forall a,b\in {\mathbb{R}},\left|a+b\right|<\left|a\right|+|b|\) .

A sequence \(\left({a}_{n}\right)\) is increasing if and only if \(\forall n\in {\mathbb{N}}\) , \({a}_{n+2}\ge {a}_{n}\) .

All three items are false (assuming universal quantification – we provide a note on this in the Results).

The refutation instrument contained four main pages. The initial response page showed all three items, spread out with space for responses. Participants were asked to ‘Answer TRUE or FALSE to each question. For those that are FALSE, give a counterexample or a brief reason.’ This instruction matched that used in the course on weekly non-assessed retrieval practice quizzes, so we used it here to avoid overlooking of or confusion over unfamiliar instructions. We acknowledge, however, that it might be read as implicitly condoning responses of both types. We consider implications of this in our interpretations.

After the initial response page, the refutation instrument had three evaluation and forced choice pages. Each showed one item again, together with two possible responses: FALSE with a counterexample and FALSE with a corrected statement. The counterexamples and corrected statements appear below

Reciprocal item

FALSE. Counterexample: If \(x=-2\) then \(1/x=1/-2<1/3\) .

FALSE. Reason: It should be ‘If \(0<x<3\) then \(1/x>1/3\) ’.

Absolute value item

FALSE. Counterexample: If \(a=1\) and \(b=6\) then \(\left|a+b\right|=7\) and \(\left|a\right|+\left|b\right|=7\) .

FALSE. Reason: It should be ‘ \(\forall a,b\in {\mathbb{R}},\left|a+b\right|\le \left|a\right|+|b|\) ’.

Sequence item

FALSE. Counterexample:

The sequence \(\mathrm{1,3},\mathrm{2,4},\mathrm{3,5},\mathrm{4,6},\dots\) satisfies the condition but is not increasing.

FALSE. Reason:

It should be ‘A sequence \(\left({a}_{n}\right)\) is increasing if and only if \(\forall n\in {\mathbb{N}}\) , \({a}_{n+1}\ge {a}_{n}\) ’.

The layout for the evaluation and forced choice pages is illustrated in the Appendix (Fig. 9 ). As shown there, participants were asked, separately for each counterexample and reason, to select one option from:

The answer is correct and the counterexample [reason] is valid;

The answer is correct but the counterexample [reason] is invalid;

The answer is incorrect.

They were then asked to make a forced choice, stating which response was better and why.

We intended the three evaluation options to make clear that correct/incorrect applied to the answer ‘FALSE’ and valid/invalid applied to the refutation provided by the counterexample or corrected statement; as this is arguably the only obvious interpretation for the counterexample, we expected that offering parallel options for the corrected statement would reinforce it. We intended that the forced choice would be interpreted as asking which of the counterexample and corrected statement constituted the better refutation. Under these interpretations, evaluating the corrected statement as valid would indicate reliance upon logically inappropriate warrants – as discussed under " Arguments and Warrants " above – and the explanations provided in the forced choice would provide insight into which warrants were invoked. However, as with many surveys, we cannot guarantee that every participant interpreted the questions as intended. Because participants were not explicitly asked whether the corrected statement proved the original false, it could be that some interpreted ‘valid’ as asking whether it was mathematically correct in isolation, and ‘better’ as asking about general mathematical quality rather than value as a refutation. We consider implications of these possibilities across our results sections, and draw the relevant issues together when discussing relationships between refutation responses and conditional inference task scores.

We constructed six versions of the refutation instrument, each with a different permutation of the three items. For each evaluation and forced choice page in each version, copies were created with the order of the counterexample and corrected statement randomised. To discourage participants from going back to change initial ‘TRUE’ responses, each evaluation and forced choice page ended with a space for participants to state that they would change their initial answers.

Conditional Inference Instrument

To measure conditional inference, we used a short version of the Abstract Conditional Inference Task (Evans et al., 1995 ), as used previously by Inglis and Simpson ( 2008 ) and Attridge and Inglis ( 2013 ). This comprised 16 items including those in Fig.  4 , four each for MP, AC, DA, and MT inferences. The 16 items form the more difficult half of the original task because negations in the minor premise are implicit. The order of the items was randomised at the participant level.

We present the results in four stages corresponding to our research questions. The section " Initial, Evaluation and Forced Choice Responses " presents quantitative data from the initial response, evaluation and forced choice stages of the refutation instrument. The section " Explanations for Forced Choices " presents qualitative detail on the explanations students gave for their forced choices. The section " Conditional Reasoning and Refutation " presents statistical analyses relating refutation responses to performance on the conditional inference instrument. Section " Conditional Inference, Refutation and Course Scores " presents statistical analyses relating both refutation responses and performance on the conditional inference instrument to course performance.

Initial, Evaluation and Forced Choice Responses

Initial responses.

The 173 initial responses are summarised in Table 1 . The two columns on the right show that about one fifth of participants incorrectly answered ‘true’ for each item (these were not the same participants each time – 19 answered ‘true’ twice and none answered ‘true’ three times); smaller numbers gave responses not classifiable according to our main distinctions. The three columns to the left show counts of participants who correctly answered ‘false’, split according to whether they provided counterexamples (including single counterexamples or correctly specified classes), corrected statements, or both.

Table 1 shows that participants were most likely to give normatively valid refutations for the reciprocal item. Of the 132 who gave a classifiable answer while avoiding the incorrect ‘true’ response, 127 (96%) included valid counterexamples (no participant wrote that the statement was neither true nor false, so it appears that these students did assume universal quantification). Only 3% gave a corrected statement alone, and only 5% gave both one or more counterexamples and a reason. This demonstrates understanding of the logic of counterexamples: most participants provided them in this simple case and most did not feel the need to elaborate by providing a corrected statement.

In more complex cases, however, the picture was different. For the absolute value item, 25% gave a corrected statement alone and, of the 130 who gave a classifiable answer while avoiding the ‘true’ response, 87 (67%) included valid counterexamples. This difference could be because a valid counterexample requires two numbers and is thus harder to generate, or because this item is more obviously related to a theorem from the course so that a corrected statement is more accessible. For the sequence item, 42% gave a corrected statement alone and, of the 109 who gave a classifiable answer while avoiding the ‘true’ response, only 37 (34%) included valid counterexamples. Again, this could be because a valid counterexample requires a sequence that is considerably harder to generate, or because this item is obviously related to a definition from the course. It is also worth noting that this item attracted more responses not classifiable according to this simple scheme.

Overall, initial responses were in line with our expectation that mathematics undergraduates would understand the logic of counterexamples but would be less likely to provide them when they are harder to construct or when corrected statements are likely to come to mind. Our interpretation is cautious because, as noted above, the task instruction might have discouraged critical evaluation of corrected statements. The next stage of our instrument addressed that in one way by asking explicitly for evaluations.

Evaluation Responses

The evaluation stage of the refutation instrument confronted every participant with both valid counterexamples and corrected statements that were not valid refutations.

Table 2 summarises the evaluation data, with normatively correct responses shaded. Footnote 2 For all three items, almost all participants agreed that the answer FALSE was correct, at least for the simpler cases; numbers were slightly lower for the mathematically most complex item. However, most participants did not respond in a normatively correct way regarding validity. For instance, for the reciprocal item, 44 participants (26%) gave the normatively correct response, evaluating the counterexample as valid and the corrected statement as invalid. Far more, 119 (69%), evaluated both the counterexample and the corrected statement as valid. This pattern was repeated across all three items.

At this evaluation stage, participants were not giving spontaneous responses; the design was intended to encourage critical reflection on both possibilities. These data therefore present stronger evidence that participants did not reliably recognise the logical inadequacy of corrected statements as refutations. We remain cautious, because the initial task instruction might have nudged participants toward positive views of both counterexamples and reasons and because some might have applied valid/invalid to the corrected statement in isolation rather than as a refutation. If participants did interpret the questions as asking about validity as refutations, we believe that the scale on which they endorsed both provides educationally useful information and makes it interesting to see whether, when forced to choose, they would come down in favour of counterexamples (see the section " Forced Choice Responses ") and how they would explain their choices (see the section " Explanations for Forced Choices "). If participants did not interpret the questions as asking about validity as refutations, this indicates that they were less sensitive to the question of mathematical refutation than we might hope, despite the parallel evaluation options for the counterexample and the corrected statement. This raises a question about whether logical reasoning performance – measured here with our abstract conditional inference task – influences sensitivity to the issue of refutation (see the section " Conditional Reasoning and Refutation ").

Forced Choice Responses

The forced choice stage of the refutation instrument asked participants to compare counterexamples and corrected statements directly and decide which was better. Table 3 summarises the responses.

For each item, when forced to choose, a majority judged the counterexample better. But these majorities were not overwhelming: a substantial minority in each case judged the corrected statement better (a small number refused to choose). This held even for the reciprocal item, for which only 3% had initially given a corrected statement alone: 31% nevertheless judged the corrected statement better. This provides yet stronger evidence that some students either do not reason well enough to recognise the logical inadequacy of corrected statements as refutations – perhaps relying upon warrants that should carry less weight than mathematical logic – or are not sensitive to the mathematical issue of refutation (or both). To gain insight into these possibilities, we turn to a qualitative analysis of the explanations students gave for their forced choices.

Explanations for Forced Choices

The initial, evaluation and forced-choice data show the prevalence of corrected statements in spontaneous responses and the value participants attached to counterexamples and corrected statements individually and comparatively. For information on their underlying reasoning, we turn to the explanations for their forced choices. In the section " Response Types ", we qualitatively categorise these explanations, providing illustrations. In the section " Proportions of Explanation Types and Toulmin Analysis ", we document the prevalence of explanations of different types and consider common types in relation to Toulmin’s model.

Response Types

The forced-choice explanations were typically single sentences and straightforward to classify qualitatively. We identified 11 explanation types, and below we present a descriptive analysis showing illustrations for each type for each item (reciprocal, absolute value, and sequence) where these are pertinent to our analyses. We group the illustrations under explanations from those who judged the counterexample better, explanations from those who judged the corrected statement better, and (smaller in number) explanations exhibiting more serious misunderstandings, or refusals to choose, or nothing beyond the evaluation responses. Because randomisation affected participants’ use of referents – the ‘first answer’ for some was the second for others – we write CEX where they referred to the counterexample and CS where they referred to the corrected statement.

Of the participants who judged a counterexample better, some explained only in relation to the counterexample (Type 1). Others commented additionally on the logical inadequacy of the corrected statement (Type 2).

Type 1: CEX disproves the statement

‘CEX, because it gives a specific counterexample to disprove the statement.’

‘CEX because it uses a clear example to show it’s false.’

‘CEX – valid counter example.’

Type 2: CEX disproves the statement + CS does not

‘CEX is better. It proves that the statement is false, whereas CS just provides an alternative true statement.’

‘CS states something with no reasoning or evidence whereas CEX gives evidence in the form of an example proving the statement to be wrong.’

‘CEX as it provides a reason for why the original statement is wrong. CS just states the actual definition, it does not prove the statement is wrong.’

Participants who judged the corrected statement better gave a wider variety of explanations. Some, as anticipated, cited its status as correct (Type 3). Others focused on generality, judging the corrected statement more general (Type 4) or critiquing the counterexample as insufficiently general (Type 5). Some focused on expression, judging the corrected statement to have better terminology or notation (Type 6) or considering the counterexample ‘unfinished’ because it was not explicitly related to the statement (Type 7).

Type 3: CS is correct

‘CS because it amends the false statement to make it true.’

‘CS because it gives the true definition of the statement.’

‘CS as it is the correct definition.’

Type 4: CS is more general

‘CS answer is better because it generalises the explanation rather than giving only one example.’

‘CS as it is more general and applies in all cases.’

‘The CS, as it is more general and “mathematicians like to generalise”.’

Type 5: Single counterexample is unsatisfactory

‘CS because it shows a range of values for which x satisfies the equation and not just one value.’

‘CS, since it actually means that the inequality is satisfied \(\forall a,b\in {\mathbb{R}}\) , whereas the second just takes two random integers from infinite amount of numbers.’

[N/A for sequence item]

Type 6: CS uses better terminology or notation

[N/A for reciprocal item]

‘CS as it shows the student fully understands the “formula”.’

‘CS: better terminology.’

Type 7: CEX is unfinished

‘CS gives more detail whereas CEX just stops.’

‘CS, as CEX is unfinished, in CEX they haven’t compared the values with the initial statement.’

‘CS is better as, although the counterexample is correct, it doesn’t explain why it disproves it, the actual definition for increasing must be included also.’

Alternative interpretations were evident in other responses. Small numbers of participants explicitly misinterpreted some aspect of logic in either the counterexample or the reason (Type 8); these are less pertinent to our analyses but are illustrated below for interest. Small numbers responded in terms of personal preferences, or refused to choose, or gave explanations that did not go beyond their evaluations; we refer to these later as types 9, 10 and 11 but omit illustrations.

Type 8: Logic misunderstood

‘CS because it gives a restricted domain so out of this range it must not be valid.’

‘The CS is better because the CEX implies that |a  +  b|  = |a|  + |b| and not that can also be |a  +  b|  < |a|  + |b|.’

‘The CS isn't true as you could have a sequence \({a}_{n}=\mathrm{1,1},\mathrm{2,2},\mathrm{3,3},\dots\) which is increasing but does not have \({a}_{n+1}\ge {a}_{n}\) .’

Overall, some responses showed good understanding of the logic of refutations. Some did not, but are reasonable in relation to Toulmin’s model of argumentation – see below – or indicate a lack of sensitivity to the issue of mathematical refutation – see the section " Conditional Reasoning and Refutation ".

Proportions of Explanation Types and Toulmin Analysis

The proportions of explanations of each type were broadly predictable based on participants’ evaluation and forced choice responses. They are summarised in Table 4 .

If we assume that participants’ explanations were about which was the better refutation then, as in the section " Arguments and Warrants ", they can be represented using Toulmin’s ( 1958 ) model with the counterexample and/or corrected statement as data and ‘ \(\forall x S\left(x\right)\) is false' as conclusion. We make this assumption in Figs.  5 and 6 , representing components explicitly addressed in the explanations in black and those offered in the task but not explicitly addressed in grey; all other components we omit.

Toulmin diagrams for explanations provided by participants who judged counterexamples better

Toulmin diagrams for explanations provided by participants who judged corrected statements better

Figure  5 represents explanations from participants who judged counterexamples better. Most fleshed out the mathematically normative argument discussed in the section " Arguments and Warrants ". Encouragingly, up to about 40% of these participants provided not only a warrant for the counterexample-based argument (Type 1) but also an explicit rebuttal of the corrected statement-based argument (Type 2).

Explanations from participants who judged corrected statements better are represented in Fig.  6 . The largest proportion focused on correctness of the corrected statement (Type 3). As discussed in the section " Arguments and Warrants ", this amounts to providing an argument of the form ‘D, B, so C’, where backing is provided but the warrant is implicit. A warrant could be the relatively naïve notion that correctness is valued in mathematics, or a more sophisticated communicative principle that correcting something about the statement implies that there were counterexamples in the scope of the original statement but outside that of the corrected version. Of participants who focused on generality, most explained that the corrected statement is more general (Type 4), again providing an argument of the form ‘D, B, so C’ – the student who noted that “mathematicians like to generalise” provided a warrant too. Smaller numbers explained instead that a single counterexample is unsatisfactory (Type 5), thereby providing a logically invalid rebuttal to the counterexample-based argument; this doubt over single counterexamples has been noted elsewhere and represents a non-normative understanding of the relevant logic, but it was not prevalent in our data. Of participants who focused on mathematical expression, some explained that the corrected statement uses better mathematical terminology or notation (Type 6), again providing an argument of the form ‘D, B, so C’, presumably with the intended warrant that good terminology and notation are mathematically valued. Others explained that the counterexample was unfinished or inadequate (Type 7), which amounts to a didactic contract-based rebuttal to the counterexample-based argument, addressing presentation rather than logical validity.

Seen in this way, the explanation types – especially those most prevalent – are predictable based on the earlier research and the theoretical analysis presented in the section " Arguments and Warrants ". The fact that they all appeared confirms the potential complexity that students face in reasoning about refutations.

This analysis, however, assumes that students were indeed reasoning about refutations, so that their explanations referred to arguments with the conclusions of the form ‘ \(\forall x S\left(x\right)\) is false’. We think this assumption sensible for Type 1 and Type 2 explanations, which explicitly stated that a counterexample proved its original statement false (Type 1) and, in some cases (Type 2), that the corresponding reason did not. It is less sensible for Type 3 – Type 7 explanations. If some participants applied ‘better’ to qualities of the corrected statements in isolation rather than as refutations, then their explanations might be better understood as arguments in which the conclusion is ‘the corrected statement is better mathematics’ rather than ‘ \(\forall x S\left(x\right)\) is false’. This raises the question of what drives sensitivity to refutation as an issue in mathematics, which we address next.

Conditional Reasoning and Refutation

All undergraduate students have been exposed to mathematical education in which correctness, generality and good presentation have been valued. So all might reasonably cite such issues when judging one mathematical response better than another, thereby failing to show sensitivity to logically valid refutation as an issue that should take mathematical precedence. Our remaining data provide information on whether this happens at random, or whether better attention to mathematically acceptable refutation is systematically associated with better logical reasoning as measured by performance in abstract conditional inference. In the section " Conditional Inference Performance ", we summarise participants’ performance on the abstract conditional inference task; in the section " Refutation Scores ", we relate this to their refutation responses.

Conditional Inference Performance

Conditional inference scores for the 157 participants who allowed use of their data are summarised in Fig.  7 and Table 5 . Figure  7 shows a broad distribution of total scores, with mean 10.22 out of 16 (SD = 2.25). Table 5 counts participants responding in a normatively correct way to 0, 1, 2, 3 or 4 items for modus ponens, denial of the antecedent, affirmation of the consequent, and modus tollens inferences. Both overall scores and the breakdown by inference type are approximately as expected given earlier research (e.g., Attridge & Inglis, 2013 ); performance was somewhat better than has been observed in non-mathematics-student samples, but far from normatively perfect.

Counts of total scores out of 16 for normatively correct responses on the conditional reasoning instrument

Refutation Scores

To link conditional reasoning to refutation responses, we constructed three refutation scores, each in the range 0–3. Initial score counts the number of items for which participants included a single counterexample or a correctly specified class of counterexamples. Evaluation score counts the number of items for which participants endorsed only the counterexample – not the corrected statement – as a valid refutation. Choice score counts the number of items for which participants judged the counterexample better. Score distributions are summarised in Table 6 (which takes a by-participant perspective in contrast with the by-item perspective in earlier sections).

The distribution of initial scores shows that most participants included at least one counterexample, confirming that most could apply the logic of counterexamples in some cases; only 13% did not, which is partly attributable to erroneous ‘true’ responses. However, the distribution of evaluation scores shows that most participants did not make normatively correct evaluations: 60% never accepted the counterexample and rejected the corrected statement (unsurprisingly, given that the most common response on all items was that both counterexample and corrected statement were valid). The distribution of choice scores shows that 81% of participants judged the counterexample better at least once, but only 35% chose it for all three items. In fact, only 9 participants (5%) scored 3 out of 3 for all three stages. Overall, this confirms that most participants responded according to the mathematical logic of refutations sometimes but not reliably.

Conditional Inference and Refutation Scores

To investigate whether better logical reasoning predicts more normatively correct refutation responses, we conducted Spearman correlations between abstract conditional inference score and each of the refutation scores, correcting for multiple comparisons by using \(\propto\) = 0.05/3 = 0.0167. Abstract conditional inference score was significantly related to initial score ( r s (157) = .273, p  = .001), evaluation score ( r s (157) = .192, p  = 0.016) and choice score ( r s (157) = .224, p  = .005); students with higher conditional inference scores were more likely to give counterexamples, evaluate only counterexamples as valid, and choose counterexamples as better refutations. The correlations are fairly small: in Fig.  8 , the line of best fit in the Initial Score plot shows that participants who gave three counterexamples on average answered two more conditional inference items correctly than those who gave none, and the bubble plots show the wide spread of conditional inference scores for all four possible refutation scores. However, this shows that participants with better abstract conditional inference scores were more likely to answer across all three tasks in ways in line with mathematically valid refutations. This means that they were more likely to interpret our questions as intended and answer them in normative ways.

Bubble plots showing conditional reasoning score against initial, evaluation and choice scores. Note . Bubble area represents the number of participants with each pair of scores

Conditional Inference, Refutation and Course Scores

We use this short final Results section to take a step back, relating both conditional reasoning and refutation scores to a standard measure of learning: performance in the analysis course. For the 151 participants who gave permission to access their course scores, we first ran a Pearson correlation between course grade (scored 0–100) and abstract conditional inference score (0–16). This revealed a significant positive correlation with a medium effect size, r (141) = .355, p  < .001; participants with better conditional inference performance tended to achieve higher grades. In line with the results in Alcock et al. ( 2014 ), this shows that conditional reasoning predicts grades in a proof-based course. We then ran Spearman correlations between course grade (scored 0–100) and initial score (0–3), evaluation score (0–3) and choice score (0–3). We found small positive correlations between course grade and initial score, r s (151) = .150, p  = .067, between course grade and evaluation score, r s (151) = .125, p  = .127, and between course grade and choice score, r s (151) = .192, p  = .018. These were all in the expected direction but not significant with alpha level corrected to 0.0167.

This paper addresses an issue that we believe is important but under-examined: that of whether students understand the logic of refutations. We designed a novel three-stage instrument to assess not only spontaneous refutations, but also how students evaluate counterexamples and corrected statements, and which they think better and why. We related students’ explanations for their choices to Toulmin’s model of argumentation, and their refutation responses to performance on an abstract conditional inference task.

On the one hand, we found some results indicating mathematically appropriate understandings of refutation. In the initial stage of our refutation instrument, many students gave valid counterexamples. Some gave both valid counterexamples and corrected statements, which might be considered the perfect response (cf. Lakatos, 1976 ). Where students gave only corrected statements, almost all were true, showing that students could incorporate lemmas appropriately and/or had learned course material. Large majorities of students evaluated counterexamples as valid, and more than half chose them as the better refutations, explaining that they proved the original statements false and in some cases supplying rebuttals to arguments based on corrected statements. This last finding especially is encouraging for classroom practice: if similar tasks were used in discussion-based activities, we would expect a focus on mathematically valid refutation to be common enough to support student-generated normative arguments.

On the other hand, our results provide a sense of where students respond to refutations in mathematically normative ways and where they do not. Students were less likely to provide counterexamples where requirements were more complex and statements were more obviously related to a theorem or definition from the course. Large majorities evaluated corrected statements as valid, and substantial minorities judged them better than the counterexamples. On this evidence, it seems that students might benefit from input encouraging them to recognise that a statement that is correct, general and expressed using good terminology and notation might nevertheless fail to refute a related statement. We consider it important to provide this input. Mathematics instructors might not always want to focus on logical warrants – in classroom communities, norms regarding correctness, generality and expression must be developed too. And they almost certainly will not want to devalue corrected statements – building mathematical theory is an important part of the job. But they will want students to develop a sense of priority in mathematical warrants, recognising that an alternative statement is a valid refutation only if it logically contradicts an original. And they will want students to develop the skills to work out whether and when this is the case.

In research terms, our data leave questions open because they do not allow us to disaggregate the effects of some variables. First, our more complex statements were also more closely related to course material; in future studies, this could be controlled by avoiding statements close to course material or by matching course-based statements and more neutral-content statements for complexity. Second, our task instructions might have implicitly condoned acceptance of both counterexamples and reasons, leading more students to accept invalid refutations than would otherwise have been the case; future studies might use a more neutral request to ‘justify your answer’, might explicitly ask whether a corrected statement proves the original false, or might use interviews to explore student views in more depth. Third, now that we have a pool of student explanations, a follow-up study might ask participants to evaluate, rank or compare these. Again, interview studies could explore student views in more depth.

Interview studies do not, however, provide information at scale (usually). We consider scale a strength of our study: while our participant numbers are not in the thousands like some school-based studies of proof and refutation (Hoyles & Küchemann, 2002 ; Küchemann & Hoyles, 2006 ; Lin, 2005 ), we believe that having 173 participants provides a sense of the range and prevalence of arguments that might be produced elsewhere.

More importantly, this scale enabled us not only to study participants’ refutation responses but also to relate these systematically to performance on a standard abstract conditional inference task. We established statistically that conditional inference skill predicts refutation responses: students with higher conditional inference scores were significantly more likely to give counterexamples, evaluate only counterexamples as valid, and judge counterexamples to be better. This is theoretically pertinent to the issue of learning about the relative mathematical value of different justifications. As noted earlier, corrected statements have epistemological value: teachers and researchers might recognise their role in mathematical theory-building (De Villiers, 2004 ; Yang, 2012 ) and perhaps therefore consider single counterexamples unsatisfactory stopping points (Creager, 2022 ; Lee, 2017 ; Peled & Zaslavsky, 1997 ; Zeybek, 2017 ). Indeed, researchers have designed tasks with the explicit goal of having students construct amended conjectures (Koichu, 2008 ; Yang, 2012 ; Yopp, 2013 ). Here, the fact that participants with better conditional inference scores were more likely to value counterexamples – despite possible ambiguity in our task instructions – indicates that better logical reasoning makes people more likely to override didactic messages they might have received about correctness, generality and expression, and expect a logically warranted refutation.

This does not mean, of course, that each individual would always respond in the same way, which returns us to considerations central in Toulmin’s ( 1958 ) work. Toulmin wanted to understand arguments made by real people about situations with varied substantive content. We believe that this might be pertinent in understanding how our results contrast with other extant work. Specifically, our participants produced counterexamples at fairly high rates, where teachers in some studies did not (e.g., Giannakoulias et al., 2010 ). This could be related to the wider argumentative context. In studies of refutation, teachers have often been asked to refute hypothetical student-produced arguments. While this can be done with counterexamples, it makes sense that teachers might attempt instead to infer students’ reasoning and explain why it does not apply, focusing on inferred warrants rather than rebuttals of the main statement. In such contexts and in ours, it might therefore be illuminating to follow other research on proof (e.g., Healy & Hoyles, 2000 ) by manipulating who students and teachers are asked to address: do they give and endorse different refutations if answering for a mathematician, a fellow student, or a younger student, for instance?

In all, our study addressed understanding of refutations. It used an original three-stage instrument to collect not only spontaneous refutation attempts but also separate and comparative evaluations of counterexamples and corrected statements. By using Toulmin’s model of argumentation, we offered a theoretically coherent picture of where students focus when evaluating refutations and how non-logical warrants, backing and rebuttals might draw attention away from mathematically valid logic. We also used the scale of our study to establish that logical reasoning as measured using a standard abstract conditional inference task significantly predicts normatively valid responses to possible refutations. Our research suggests a number of possible avenues for productive future enquiry, as outlined in this discussion. We suggest that research might profitably elucidate what it takes to ensure that students see past non-logical considerations and engage with mathematical logic, and what it takes to help them understand that logic well enough to recognise when two statements are and are not contradictory.

UK students specialise early. Because most students take only three A levels at ages 16–18, they arrive at university having already studied much of the material that would appear in the first 1–2 years of a US calculus sequence. Degree programmes often begin with courses covering the later parts of such a sequence, together with linear algebra and other proof-based courses.

This table appeared in an early conference presentation of parts of this work, together with qualitative detail on the response types that is not presented here; see Alcock and Attridge ( 2022 ).

Alcock, L., & Attridge, N. (2022). Counterexamples and refutations in undergraduate mathematics. In Proceedings of the 2022 Conference on Research in Undergraduate Mathematics Education . Boston, MA, USA: SIGMAA on RUME.

Alcock, L., Bailey, T., Inglis, M., & Docherty, P. (2014). The ability to reject invalid logical inferences predicts proof comprehension and mathematics performance. In Proceedings of the 17th Conference on Research in Undergraduate Mathematics Education . Denver, CO: SIGMAA on RUME.

Alcock, L., & Weber, K. (2005). Proof validation in real analysis: Inferring and checking warrants. Journal of Mathematical Behavior, 24 , 125–134.

Article   Google Scholar  

Attridge, N., Doritou, M., & Inglis, M. (2015). The development of reasoning skills during compulsory 16 to 18 mathematics education. Research in Mathematics Education, 17 (1), 20–37.

Attridge, N., & Inglis, M. (2013). Advanced mathematical study and the development of conditional reasoning skills. PLoS One, 8 , e69399.

Balacheff, N. (1991). Treatment of refutations: Aspects of the complexity of a constructivist approach to mathematics learning. In E. von Glasersfeld (Ed.), Radical constructivism in mathematics education (pp. 89–110). Kluwer Academic Publishers.

Chapter   Google Scholar  

Brousseau, G. (1997). Theory of didactical situations in mathematics . Kluwer.

Google Scholar  

Buchbinder, O., & Zaslavsky, O. (2011). Is this a coincidence? The role of examples in fostering a need for proof. ZDM: The International Journal on Mathematics Education, 43 , 269–281.

Castaneda, H. N. (1960). On a proposed revolution in logic. Philosophy of Science, 27 , 279–292.

Creager, M. A. (2022). Geometric refutations of prospective secondary mathematics teachers. International Journal of Education in Mathematics, Science and Technology, 10 , 74–99.

De Villiers, M. (2004). The role and function of quasi-empirical methods in mathematics. Canadian Journal of Science, Mathematics and Technology Education, 4 , 397–418.

Durand-Guerrier, V. (2003). Which notion of implication is the right one? From logical considerations to a didactic perspective. Educational Studies in Mathematics, 53 (1), 5–34.

Evans, J. S. B., Clibbens, J., & Rood, B. (1996). The role of implicit and explicit negation in conditional reasoning bias. Journal of Memory and Language, 35 (3), 392–409.

Evans, J. S. B., Handley, S. J., Neilens, H., & Over, D. E. (2007). Thinking about conditionals: A study of individual differences. Memory & Cognition, 35 (7), 1772–1784.

Evans, J. S. B. T., Clibbens, J., & Rood, B. (1995). Bias in conditional inference: Implications for mental models and mental logic. Quarterly Journal of Experimental Psychology, 48A , 644–670.

Evans, J. S. B. T., Thompson, V. A., & Over, D. E. (2015). Uncertain deduction and conditional reasoning. Frontiers in Psychology, 6 , 398.

Galbraith, P. L. (1981). Aspects of proving: A clinical investigation of process. Educational Studies in Mathematics, 12 , 1–28.

Giannakoulias, E., Mastorides, E., Potari, D., & Zachariades, T. (2010). Studying teachers’ mathematical argumentation in the context of refuting students’ invalid claims. Journal of Mathematical Behavior, 29 , 160–168.

Grice, P. (1989). Studies in the way of words . Harvard University Press.

Hamami, Y., Mumma, J., & Amalric, M. (2021). Counterexample search in diagram-based geometric reasoning. Cognitive Science, 45 , e12959.

Healy, L., & Hoyles, C. (2000). A study of proof conceptions in algebra. Journal for Research in Mathematics Education, 31 , 396–428.

Hoyles, C., & Küchemann, D. (2002). Students’ understanding of logical implication. Educational Studies in Mathematics, 51 , 193–223.

Hub, A., & Dawkins, P. C. (2018). On the construction of set-based meanings for the truth of mathematical conditionals. Journal of Mathematical Behavior, 50 , 90–102.

Inglis, M., Mejía-Ramos, J. P., & Simpson, A. (2007). Modelling mathematical argumentation: The importance of qualification. Educational Studies in Mathematics, 66 , 3–21.

Inglis, M., & Simpson, A. (2008). Conditional inference and advanced mathematical study. Educational Studies in Mathematics, 67 (3), 187–204.

Ko, Y.-Y., & Knuth, E. (2009). Undergraduate mathematics majors’ writing performance producing proofs and counterexamples about continuous functions. Journal of Mathematical Behavior, 28 , 68–77.

Ko, Y.-Y., & Knuth, E. J. (2013). Validating proofs and counterexamples across content domains: Practices of importance for mathematics majors. Journal of Mathematical Behavior, 32 , 20–35.

Koichu, B. (2008). If not, what yes? International Journal of Mathematical Education in Science and Technology, 39 , 443–454.

Komatsu, K. (2017). Fostering empirical examination after proof construction in secondary school geometry. Educational Studies in Mathematics, 96 , 129–144.

Komatsu, K. (2016). A framework for proofs and refutations in school mathematics: In- creasing content by deductive guessing. Educational Studies in Mathematics, 92 , 147–162.

Komatsu, K., & Jones, K. (2022). Generating mathematical knowledge in the classroom through proof, refutation, and abductive reasoning. Educational Studies in Mathematics, 109 , 567–591.

Komatsu, K., Jones, K., Ikeda, T., & Narazaki, A. (2017). Proof validation and modification in secondary school geometry. Journal of Mathematical Behavior, 47 , 1–15.

Küchemann, D., & Hoyles, C. (2006). Influences on students’ mathematical reasoning and patterns in its development: Insights from a longitudinal study with particular reference to geometry. International Journal of Science and Mathematics Education, 4 , 581–608.

Lakatos, I. (1976). Proofs and refutations . Cambridge University Press.

Book   Google Scholar  

Larsen, S., & Zandieh, M. (2008). Proofs and refutations in the undergraduate mathematics classroom. Educational Studies in Mathematics, 67 , 185–198.

Lee, K. (2017). Students’ proof schemes for mathematical proving and disproving of propositions. Journal of Mathematical Behavior, 41 , 26–44.

Lin, F.-L. (2005). Modeling students’ learning on mathematical proof and refutation. In H.L. Chick & J.L. Vincent (Eds.), Proceedings of the 29th Conference of the International Group for the Psychology of Mathematics Education , Vol.1, pp. 3–18. Melbourne, Australia: PME.

Oaksford, M., & Chater, N. (2020). New paradigms in the psychology of reasoning. Annual Review of Psychology, 71 , 305–330.

Peled, I., & Zaslavsky, O. (1997). Counter-examples that (only) prove and counter-examples that (also) explain. Focus on Learning Problems in Mathematics, 19 (3), 49–61.

Potari, D., Zachariades, T., & Zaslavsky, O. (2009). Mathematics teachers reasoning for refuting students’ invalid claims. In V. Durand-Guerrier, S. Soury-Lavergne & F. Arzarello, (Eds.), Proceedings of the Sixth Congress of the European Society for Research in Mathematics Education . Lyon, France: ERME.

Reid, D. (2002). Conjectures and refutations in Grade 5 mathematics. Journal for Research in Mathematics Education, 33 , 5–29.

Roh, K. H., & Lee, Y. H. (2017). Designing tasks of introductory real analysis to bridge a gap between students’ intuition and mathematical rigor: The case of the convergence of a sequence. International Journal of Research in Undergraduate Mathematics Education, 3 , 34–68.

Simpson, A. (2015). The anatomy of a mathematical proof: Implications for analyses with Toulmin’s scheme. Educational Studies in Mathematics, 90 , 1–17.

Solow, D. (2005). How to read and do proofs . Hoboken, NJ: John Wiley & Sons Inc.

Stylianides, A. J., & Al-Murani, T. (2010). Can a proof and a counterexample coexist? students’ conceptions about the relationship between proof and refutation. Research in Mathematics Education, 12 , 21–36.

Stylianides, A. J., & Ball, D. L. (2008). Understanding and describing mathematical knowledge for teaching: Knowledge about proof for engaging students in the activity of proving. Journal of Mathematics Teacher Education, 11 , 307–332.

Stylianides, G. J., & Stylianides, A. J. (2009). Facilitating the transition from empirical arguments to proof. Journal for Research in Mathematics Education, 40 , 314–352.

Toulmin, S. (2003). The uses of argument (Updated). Cambridge University Press.

Toulmin, S. (1958). The uses of argument . Cambridge University Press.

Weber, K. (2010). Mathematics majors’ perceptions of conviction, validity and proof. Mathematical Thinking and Learning, 12 , 306–336.

Yang, K.-L. (2012). Providing opportunities for students to create mathematics. Procedia – Social and Behavioral Sciences, 46 , 3905–3909.

Yim, J., Song, S., & Kim, J. (2008). Mathematically gifted elementary students’ revisiting of Euler’s polyhedron theorem. The Mathematics Enthusiast, 5 , 125–142.

Yopp, D. A. (2013). Counterexamples as starting points for reasoning and sense making. The Mathematics Teacher, 106 , 674–679.

Yopp, D. A., Ely, R., Adams, A. E., Neilsen, A. W., & Corwine, E. C. (2020). Eliminating counterexamples: A case study intervention for improving adolescents’ ability to critique direct arguments. Journal of Mathematical Behavior, 57 , 100751.

Zazkis, R., & Chernoff, E. J. (2008). What makes a counterexample exemplary? Educational Studies in Mathematics, 68 , 195–208.

Zeybek, Z. (2017). Pre-service elementary teachers’ conceptions of counterexamples. Inter- National Journal of Education in Mathematics, Science and Technology, 5 , 295–316.

Zodik, I., & Zaslavsky, O. (2008). Characteristics of teachers’ choice of examples in and for the mathematics classroom. Educational Studies in Mathematics, 69 , 165–182.

Download references

Acknowledgements

This research was unfunded. Thank you to the three anonymous reviewers whose comments improved this manuscript.

Author information

Authors and affiliations.

Department of Mathematics Education, Loughborough University, Loughborough, Leicestershire, LE11 3TU, UK

Lara Alcock

Department of Psychology, University of Portsmouth, Portsmouth, Hampshire, UK

Nina Attridge

You can also search for this author in PubMed   Google Scholar

Contributions

Lara Alcock conceived and designed the study, collected, analysed, and interpreted the data, and wrote and revised the manuscript. Nina Attridge contributed to the design of the study, the collection, analysis and interpretation of the data, and provided feedback on the manuscript.

Corresponding author

Correspondence to Lara Alcock .

Ethics declarations

Conflict of interest.

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Sample evaluation and forced choice page from the refutation instrument

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Alcock, L., Attridge, N. Refutations and Reasoning in Undergraduate Mathematics. Int. J. Res. Undergrad. Math. Ed. (2023). https://doi.org/10.1007/s40753-023-00220-4

Download citation

Accepted : 06 May 2023

Published : 10 July 2023

DOI : https://doi.org/10.1007/s40753-023-00220-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Counterexample
• Conditional inference
• Find a journal
• Publish with us
• Track your research

Help | Advanced Search

Computer Science > Operating Systems

Title: aios: llm agent operating system.

Abstract: The integration and deployment of large language model (LLM)-based intelligent agents have been fraught with challenges that compromise their efficiency and efficacy. Among these issues are sub-optimal scheduling and resource allocation of agent requests over the LLM, the difficulties in maintaining context during interactions between agent and LLM, and the complexities inherent in integrating heterogeneous agents with different capabilities and specializations. The rapid increase of agent quantity and complexity further exacerbates these issues, often leading to bottlenecks and sub-optimal utilization of resources. Inspired by these challenges, this paper presents AIOS, an LLM agent operating system, which embeds large language model into operating systems (OS) as the brain of the OS, enabling an operating system "with soul" -- an important step towards AGI. Specifically, AIOS is designed to optimize resource allocation, facilitate context switch across agents, enable concurrent execution of agents, provide tool service for agents, and maintain access control for agents. We present the architecture of such an operating system, outline the core challenges it aims to resolve, and provide the basic design and implementation of the AIOS. Our experiments on concurrent execution of multiple agents demonstrate the reliability and efficiency of our AIOS modules. Through this, we aim to not only improve the performance and efficiency of LLM agents but also to pioneer for better development and deployment of the AIOS ecosystem in the future. The project is open-source at this https URL .

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

Internet Explorer lacks support for the features of this website. For the best experience, please use a modern browser such as Chrome, Firefox, or Edge.

New Research Reveals Full Diversity of Killer Whales as Two Species Come into View on Pacific Coast

March 27, 2024

Long viewed as one worldwide species, killer whale diversity now merits more. Southern Resident Connections - Post 35

Scientists have resolved one of the outstanding questions about one of the world’s most recognizable creatures, identifying two well-known killer whales in the North Pacific Ocean as separate species.

Killer whales are one of the most widespread animals on Earth. They have long been considered one worldwide species known scientifically as Orcinus orca , with different forms in various regions known as “ecotypes.”

However, biologists have increasingly recognized the differences between resident and Bigg’s killer whales. Resident killer whales maintain tight-knit family pods and prey on salmon and other marine fish. Bigg’s killer whales roam in smaller groups, preying on other marine mammals such as seals and whales. (Killer whales actually belong to the dolphin family.) Bigg’s killer whales, sometimes called transients, are named for Canadian scientist Michael Bigg, the first to describe telltale differences between the two types.

He noted in the 1970s that the two animals did not mix with each other even when they occupied many of the same coastal waters. This is often a sign of different species.

The finding recognizes the accuracy of the listing of Southern Resident killer whales as a Distinct Population Segment warranting protection under the Endangered Species Act in 2005. At the time, NOAA described the distinct population segment as part of an unnamed subspecies of resident killer whales in the North Pacific.

Now a team of scientists from NOAA Fisheries and universities have assembled genetic, physical, and behavioral evidence. The data distinguish two of the killer whale ecotypes of the North Pacific Coast—residents and Bigg’s—as separate species.

“We started to ask this question 20 years ago, but we didn’t have much data, and we did not have the tools that we do now,” said Phil Morin, an evolutionary geneticist at NOAA Fisheries’ Southwest Fisheries Science Center and lead author of the new paper . “Now we have more of both, and the weight of the evidence says these are different species.”

Genetic data from previous studies revealed that the two species likely diverged more than 300,000 years ago and come from opposite ends of the killer whale family tree. That makes them about as genetically different as any killer whale ecotypes around the globe. Subsequent studies of genomic data confirm that they have evolved as genetically and culturally distinct groups, which occupy different niches in the same Northwest marine ecosystem.

“They’re the most different killer whales in the world, and they live right next to each other and see each other all the time,” said Barbara Taylor, a former NOAA Fisheries marine mammal biologist who was part of the science panel that assessed the status of Southern Residents. “They just do not mix.”

Recognizing New Species

The Taxonomy Committee of the Society of Marine Mammalogy will determine whether to recognize the new species in its official list of marine mammal species . The committee will likely determine whether to accept the new designations at its next annual review this summer.

The scientists proposed scientific names for the new species based on their earliest published descriptions in the 1800s. Neither will keep the ubiquitous worldwide moniker, orca . The team proposed to call resident killer whales Orcinus ater , a Latin reference to their dominant black coloring. Bigg’s killer whales would be called Orcinus rectipinnus , a combination of Latin words for erect wing, probably referring to their tall, sharp dorsal fin.

Both species names were originally published in 1869 by Edward Drinker Cope, a Pennsylvania scientist known more for unearthing dinosaurs than studying marine mammals. He was working from a manuscript that California whaling captain Charles Melville Scammon had sent to the Smithsonian Institution describing West Coast marine mammals, including the two killer whales. While Cope credited Scammon for the descriptions, Scammon took issue with Cope for editing and publishing Scammon’s work without telling him. (See accompanying story .)

The Smithsonian Institution had shared Scammon’s work with Cope, and a Smithsonian official later apologized to Scammon for what he called “Cope’s absurd blunder.”

Species Reflect Ecosystem

The contested question of whether Southern Residents were distinct enough to merit endangered species protections initially drove much of the research that helped differentiate the two species, said Eric Archer, who leads the Marine Mammal Genetics Program at the Southwest Fisheries Science Center and is a coauthor of the new research paper. The increasing processing power of computers has made it possible to examine killer whale DNA in ever finer detail. He said the findings not only validate protection for the animals themselves, but also help reveal different components of the marine ecosystems the whales depend on.

“As we better understand what makes these species special, we learn more about how they use the ecosystems they inhabit and what makes those environments special, too,” he said.

The new research synthesizes the earliest accounts of killer whales on the Pacific Coast with modern data on physical characteristics. The team also use aerial imaging (called photogrammetry ), and measurement and genetic testing of museum specimens at the Smithsonian and elsewhere. While the two species look similar to the untrained eye, the evidence demonstrates they are very different species. The two species use different ecological niches, such as specializing in different prey, said Kim Parsons, a geneticist at the NOAA Fisheries Northwest Fisheries Science Center in Seattle and coauthor of the new research.

Recent research with drones that collect precise aerial photos has helped differentiate Bigg’s killer whales as longer and larger. This might better equip them to go after large marine mammal prey. The smaller size of residents is likely better suited to deep dives after their salmon prey, said John Durban, an associate professor at Oregon State University’s Marine Mammal Institute. His killer whale drone research is done collaboratively with Holly Fearnbach, a researcher at SR³.

The different prey of the two species may also help explain their different trajectories. Southern Residents are listed as endangered in part because of the scarcity of their salmon prey. Bigg’s killer whales, by contrast, have multiplied while feeding on plentiful marine mammals, including California sea lions.

While killer whales represent some of the most efficient predators the world has ever seen, Durban said science is still unraveling the diversity among them. The identification of additional killer whale species is likely to follow. One leading candidate may be “Type D” killer whales identified in the Southern Ocean around Antarctica.

Other killer whales in Antarctic waters also look very different from the best-known black and white killer whales. This reflects a wider diversity within the species, said Durban, who has used drones to study killer whales around the world. “The more we learn,” he said, “the clearer it becomes to me that at least some of these types will be recognized as different species in due course.”

Southern Resident Connections

Southern Resident killer whales are icons of a vibrant but struggling marine ecosystem that is important to us all. Join us in exploring the ecological connections that tie this system together, and the ways we are protecting and working to recover the whales we all care so much about.

Read more entries

More Information

• New Research Reveals Two Species of Killer Whale
• How Scientists Chose Names for Newly Identified Killer Whale Species
• Two Species of Killer Whale Infographic
• Marine Mammal Genetics Research
• 2004 Status Review of Southern Resident Killer Whales
• Saving the Southern Resident Killer Whales
• Listing of Southern Resident Killer Whale Under the ESA
• Killer Whale Ecotypes Poster

Recent News

Lost skulls and latin: how scientists chose names for newly identified killer whale species.

Pioneering Project to Restore Bull Kelp Forests in Greater Farallones National Marine Sanctuary in California

Closure of 2019–2023 Eastern North Pacific Gray Whale Unusual Mortality Event

Last updated by Southwest Fisheries Science Center on March 28, 2024

More On This Topic

Documenting the Elusive North Pacific Right Whale

Scientists Explore how Sardine Populations and Fisheries Harvest Control Rules May Respond to Climate Change

New Research Asks, “Can Pacific Salmon Keep Pace with Climate Change?”

Inbreeding: A Conservation Challenge for Iconic Killer Whales

We use cookies to provide you with the best possible experience on our website

Visit our cookies policy for more information about cookies and privacy. Manage your cookie settings here … Read more

On May 2018, we published revised versions of the OMRON Privacy Policy and Cookie Policy. Please read these updated terms by clicking here . Your use of our products and services is subject to these revised terms

• Global - English
• Global - Deutsch
• Global - Español
• Global - Français
• Asia Pacific - English
• Brasil - Português
• Japan - 日本語
• Korea - 한국어

• Global News

OMRON SINIC X Research Paper Accepted for ICASSP 2024, the World’s Largest and Most Comprehensive Technical Conference focused on Acoustics, Speech, and Signal Processing

• April 02, 2024

OMRON SINIC X Corporation (Headquarters: Bunkyo-ku, Tokyo; President and CEO: Masaki Suwa; hereinafter referred to as "OSX") is pleased to announce that its research paper by Yoshitaka Ushiku, Vice President for Research at OSX, and Atsushi Hashimoto, Principal Investigato at OSX, and others, has been accepted for the 2024　IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP 2024).

ICASSP is the world's largest international conference in the field of acoustics, speech, and signal processing, which will be held in Seoul, South Korea, from April 14 to 19, 2024. A paper accepted for ICASSP 2024 presents a formula-driven trajectory dataset called Polar Database (PolarDB) for pre-training trajectory encoders. The presentation will take place in a session starting at 1:10 PM local time on April 16.

■Paper accepted for ICASSP2024

For a list of OSX's publications, please visit https://www.omron.com/jp/ja/technology/publications/?affiliation=%5B%22OSX%22%5D

About OMRON SINIC X Corporation OMRON SINIC X Corporation is a strategic subsidiary seeking to realize the "near future designs" that OMRON forecasts. It is comprised of researchers with cutting-edge knowledge and experience across a wide range of technological domains, including AI, Robotics, IoT, and sensing. With the aim of addressing social issues, OSX integrates innovative technologies with business models and strategies in technology and IP to create near-future designs. Additionally, the company accelerates the creation of these designs through collaborative research with universities and external research institutions. For more details, please refer to https://www.omron.com/sinicx/en/

Adobe Acrobat Reader is free software that lets you view and print Adobe Portable Document Format (PDF) files.

Winners of national penmanship contest crowned as handwriting is 'having a moment'

It’s regarded, hands down, as the Super Bowl of penmanship tournaments.

The Zaner-Bloser National Handwriting Contest, now in its 33rd year, crowned its 2024 grand champions on Monday, rewarding nine students from six states for their picture-perfect letters.

Ten-year old Zita Miller of White Bear Lake, Minnesota, took top honors in the fifth grade category. Her winning submission was one of the contest’s 80,000 entries.

“I like handwriting because it’s like art, drawing swirls and vines and curls,” Miller said, adding that she enjoys penning original mystery stories by hand.

Namuun Baasanbold, from Carmel, Indiana, was named grand champion in the first grade category, and said she likes to give handwritten “love notes” to family and friends.

“Writing by hand makes me feel special,” she said.

The contest celebrates a centuries-old practice, but the victories come as handwriting is experiencing a kind of renaissance in the U.S. In January, California became the 22nd state to require cursive to be taught in schools — a significant jump from 2016, when just 12 states mandated it.

At the same time, various studies published over the past decade have detailed how writing with pencil and paper can benefit memory, cognitive development, reading comprehension and fine motor skills.

“Handwriting is definitely having a moment,” said Sharon Quirk-Silva, a member of the California state Assembly who sponsored the bill. She said she heard from people from all over the country who penned “beautifully handwritten notes” of support for the new law.

“We live in a very polarized nation. So many issues are contentious. But with this handwriting bill, we had full bipartisan support and goodwill. The importance of handwriting is something people seem to agree on,” she said.

Quirk-Silva said she backed the bill, in large part, because of her own experience — before becoming a lawmaker, she taught elementary school for 25 years.

“For years, technology has been taking over the curriculum in schools, with many kids being dormant in front of the screen, using two or three screens a day. Now, there’s a feeling of, ‘Let’s get pens and pencils back in kids’ hands,’” she said.

Although the California law mandates that first through sixth graders in the state receive cursive instruction, Quirk-Silva said she believes that writing by hand — in print or cursive — is an important language arts tool.

“It’s a way of slowing down a little bit, taking your thoughts from your brain to your hand and physically doing the writing,” she said.

Sophia Vinci-Booher, an assistant professor of psychology and human development at Vanderbilt University, said her research found that writing by hand enabled preschool students to form connections in the brain that likely support early letter recognition.

For that study , published in 2016, 20 children were asked to practice certain letters by writing them over and over, and practice others by pressing a button.

“Then we asked the children to go into an MRI scanner and look at those letters they’d been practicing,” Vinci-Booher explained. Her team analyzed the children’s brain activity to assess the functional connectivity between different areas of their brains.

“We found that the connection was stronger with letters they wrote by hand than those they tapped,” she said.

The research underscores the importance of the physical act of forming symbols, Vinci-Booher added.

“Writing by hand is a good thing for kids because it supports early reading development and it engages the fine motor system, which is developmentally important,” she said.

A 2021 study measured people’s brain activity during a memory task, this time finding that University of Tokyo students exhibited stronger activity and better recall after they had written information down on paper than when they did on a smartphone or even with a stylus on a tablet. The researchers suggested that the physical act of writing on paper provides the brain more details that trigger memory, and concluded that using paper notebooks can help students retain information in part because of their “tangible permanence.”

A similar study published in January compared the brain activity of students at the Norwegian University of Science and Technology who took notes by hand to the activity of those who typed their notes. The findings suggested that the students who wrote by hand had higher levels of electrical activity across a wide range of brain regions responsible for sensory processing and memory.

The results come as little surprise to many educators.

“I’ve seen firsthand that the kids learn more when they write by hand,” said Geeta Kadakia, who teaches second through fifth grade at the DAV Montessori School in Houston. “The lightbulb goes off through those achievements in handwriting, and handwriting leads to achievements in other areas, even math. When students make their numbers more neatly, their math scores improve.”

Laura Gajderowicz taught elementary school for 33 years in Indiana before retiring in 2022. She said she worried as she watched handwriting take a back seat to technology in U.S. classrooms in the early 2000s.

“Writing by hand does so much to help with the development of a student’s eye-hand coordination,” Gajderowicz said, adding: “I’m not against technology — I just think there’s a place at the table for both technology and handwriting when it comes to learning.”

This year, Gajderowicz served as a regional judge in the Zaner-Bloser contest.

“I was pleasantly surprised to see how many entries we had, especially from children in the upper grades,” she said.

Gajderowicz selected winners using criteria that analyzed the mechanics and precision of the letters students wrote, including their shapes, sizes, slant and spacing.

Contestants were asked to write the sentence, “The quick brown fox jumps over the lazy dog,” because it includes the entire alphabet, as well as a sentence explaining why handwriting makes them a better reader and writer.

Baasanbold said she was “over the moon” to find out she won: “I screamed and celebrated with friends at a restaurant with pizza and an appetizer and a sundae for dessert,” she said.

Her prizes include a trophy and $500 — plus bragging rights.

“I like to use my handwriting to impress people,” she said.

Mary Pflum is a national field producer for NBC News, based in New York.

Doctors can do more to help prevent gun violence, USF paper says

• Sam Ogozalek Times staff

Doctors can do more to help prevent gun violence and offer counseling on firearms safety, according to a review by University of South Florida researchers, including a medical student who survived the 2018 mass shooting at Marjory Stoneman Douglas High School in Parkland.

The review, published in February in the journal Advances in Pediatrics, noted that many doctors believe they should talk to patients about firearms, but often don’t because of time constraints, a lack of training and discomfort with the topic.

In Florida, a law passed in 2011 limited what questions doctors could ask patients about guns. But a federal appeals court found the restrictions to be unconstitutional and struck them down six years later.

The law “had a very chilling effect I think for a lot of providers,” said Cameron Nereim, one of the review’s authors and an assistant professor of pediatrics at USF who focuses on patients ages 12 to 25 in Tampa.

The review noted that the American Medical Association calls gun violence a public health crisis and firearm-related injuries were the leading cause of death for U.S. children in 2020, surpassing motor vehicle crashes.

Doctors can ask patients about whether guns are kept at home and if they are locked, unloaded and separated from ammunition, according to the review.

USF medical student Nikhita Nookala, who survived the shooting at Marjory Stoneman Douglas High School and covered it as a reporter at The Eagle Eye, the school’s student newspaper, contributed to the review.

The Tampa Bay Times spoke with Nereim and Nookala about their work, which was published with two other authors from North Carolina and Texas medical centers.

The interviews have been edited for clarity and length.

What are the major takeaways you hope physicians get from this paper?

Nookala: It’s important to assess patients’ risks and to see if you can do some harm reduction in terms of encouraging safe storage or just telling kids to be careful around guns.

It doesn’t have to be super political. It’s just about children’s safety at the end of the day.

The paper noted some of the major barriers to physicians counseling on firearms safety are the lack of formal training, lack of confidence that patients will follow their recommendations and low perceived self-efficacy. Are there any ways to address that?

Nereim: I think there are. ... One of the things is how do we do a better job of incorporating this into our training, whether that’s while you’re going through your residency program, while you’re going through your medical school, making sure these things are actually being plugged into our curricula.

Keep up with Tampa Bay’s top headlines

Subscribe to our free DayStarter newsletter

You’re all signed up!

Want more of our free, weekly newsletters in your inbox? Let’s get started.

Is there a particular stage of education where you think this would be best emphasized?

Nereim: I think medical school, before students have even differentiated in terms of what specialty or subspecialty they’ll be choosing, that’s a great opportunity to make sure we’re relaying this information.

In school so far, have people talked about gun violence as a public health issue?

Nookala: Everyone acknowledges that it’s become a problem and it’s getting worse. I think a lot of people feel very helpless because it’s something that is hard to bring up.

You’re not accusing someone of “You don’t store your gun safely, you’re putting your child in danger.” ... You want to (tread) the line of “Oh, you have guns in the home, that’s fine, are you storing them safely, are (you) using gun locks?”

It’s just small things that you can remind people, nicely, without discouraging them from coming back.

During routine visits, do you ask patients or patients’ parents whether there are guns in the home, if they are locked, securely unloaded (and) separated from ammunition?

Nereim: Yes, we do.

With the ongoing mental health crisis that’s involving young people ... I think it becomes even more critical that we do our due diligence and we ask those questions about things like firearm ownership ... how ammunition is being stored, what level of training the family members or the kids themselves have in terms of actually using these objects.

How do those conversations normally go for you? Are they difficult?

Nereim: When you’re able to successfully create that nonjudgemental space ... the vast majority of patients and families are really receptive to the questions we’re asking.

Do you ask patients more often than not? Or do you ask their parents?

Nereim: I tend to do both.

Do you ask firearm-related questions in every visit with a patient, or do you only ask them with someone who would be considered high risk?

Nereim: I wouldn’t say we ask it at every visit. I think we’re getting better at making this more a universal thing that we’re screening for, in the same way that 15, 20 years ago we would make sure at every visit we’re asking “Are you wearing your safety helmet if you’re riding your bike? Are you buckling your seat belt if you’re riding in a car?” We’re moving in that direction.

It happens a lot more consistently in these higher-risk situations, so if you have a patient who’s experiencing mood symptoms, depression or maybe even anger, irritability, impulsive behaviors.

(The paper suggested) that physicians can link those at risk of gun violence to other programs that offer support in the community. Do you do that regularly? How common is that?

Nereim: It’s extremely common. ... In the space where I work, probably anywhere from 50 to 70% of patients may have some significant social need. ... A lot of young people confide in me that they’re fearful just to be outside and to be walking in the place where they live because there have been times where they’ve heard gunshots. Or they know there was a shooting that occurred in that same square, that same block.

As you can imagine, when you have these kinds of social needs that manifest over time, there can be pretty major health consequences.

Why is this research important to you?

Nookala: In the aftermath of the (Marjory Stoneman Douglas High School) shooting, the focus was on Parkland, and Parkland became this beacon of gun violence prevention. But the reality is that outside of that one incident, which was horrible, gun violence doesn’t really occur in cities like Parkland every day. It occurs in cities like Tampa every day in marginalized communities. … The focus needs to be shifted back to that.

Training to be a doctor, I wanted to know more about ways you could make changes on an interpersonal level with your patients without being involved in these really political movements that oftentimes (are) just kind of doing nothing or (do) a little something and it gets reversed a few years later. It’s frustrating to watch that.

Nereim: Violence leads to more violence. The only really, truly effective way for us to move forward as a society is we just have to realize that prevention is incredibly important. … I just don’t think we can ignore the role this is playing in the lives of our patients and their families.

Sam Ogozalek is a reporter covering the healthcare system and mental health. He can be reached at [email protected].

MORE FOR YOU

• Advertisement

ONLY AVAILABLE FOR SUBSCRIBERS

The Tampa Bay Times e-Newspaper is a digital replica of the printed paper seven days a week that is available to read on desktop, mobile, and our app for subscribers only. To enjoy the e-Newspaper every day, please subscribe.

We've detected unusual activity from your computer network

To continue, please click the box below to let us know you're not a robot.

Why did this happen?

Please make sure your browser supports JavaScript and cookies and that you are not blocking them from loading. For more information you can review our Terms of Service and Cookie Policy .

For inquiries related to this message please contact our support team and provide the reference ID below.