Sometimes in education, certain pieces of research reach an almost totemic status.
I normally stay away from areas I am less interested in and know less about, such as reading in primary school, but because I do understand education research quite well, I was recently drawn to revisit one of those totemic pieces of work: Recht and Leslie’s baseball study.
This is an interesting study from 1988 in which the researchers looked at how prior knowledge influences the amount of short-term, non-verbal and verbal memory and long-term retention in students of high and low ability in reading comprehension.
Primary reading: the baseball study
Sixty-four lower secondary school students were assessed on reading ability (classified as high and low) and prior knowledge of baseball (high and low) and were asked to read an account of part of a baseball game. Having four groups of 16 students for a research study is not a lot, although fairly typical in the 80s. The students were asked to complete a variety of tasks, showing that prior knowledge of baseball had a main effect on all the outcome measures.
But the practical implications of the study aren’t clear cut. Many people who cite the baseball study are making a general point about background knowledge, but, obviously, if the claim is that you do better on a test on a certain topic if you know more about that topic, that seems true in a trivial sense.
More from Christian Bokhove:
One frequent response to criticism of a research study is that the finding has been replicated, which means that other researchers did the same study and found the same result. It is useful to actually follow up on claims like that, because it matters to what extent a study really was the same and confirms an original finding.
In the literature, we sometimes make the distinction between a “direct replication”, where you try to keep as much as possible the same as the original study, and “conceptual replications”, where it is more the concept that is under scrutiny. Obviously, in the case of the latter, you very quickly come into the weeds of whether a replication has been similar enough.
In a systematic review of these baseball studies, Reynolds et al (2024) showed that 19 such studies were published from 1978 to 2018, with only a few using similar measures of baseball knowledge.
About the baseball study, they note quite large differences in the gender compositions of the different groups; something that might be problematic with gendered topics. When it came to replicating the study, only one study used the same measures. That study, by Gaultney (1995), only involved 45 boys in a dropout prevention programme. Overall, I found that the evidence base of the baseball study was quite meagre.
So what can we learn?
This does not mean that knowledge in reading comprehension is not important. Smith et al (2021) conducted a broader critical review of background knowledge in reading comprehension and found that the impact of higher levels of background knowledge is influenced by a range of factors, including the nature of the text and reader misconceptions of the text.
They found that readers with lower background knowledge seem to benefit more from text with high cohesion, while weaker readers can somewhat compensate for their weaker reading skills through more background knowledge. But the key is in the “somewhat”: the researchers concluded that such compensation is “partial” and not as strong as the original baseball study states.
It shows that we need to be careful of loading too much weight on to one specific, totemic work. Although interesting, the baseball study is not particularly strong and I don’t think it has undergone real, direct replication.
Simply alleging that all studies somewhat in the same area are replications is not warranted. And if we do broaden the pool, reality is almost always more complex than some think.
Christian Bokhove is professor of mathematics education at the University of Southampton and a specialist in research methodologies
