“I’ve discovered a key paper on a topic I know relatively little about. How do I see where it sits in the broader literature?”
For many researchers, this is a common task. When finding a key paper - one that has received hundreds or thousands of citations, been the subject of debate, or made a controversial claim - understanding how it has influenced a scientific field or been received by others can be a daunting task.
In this blog post, we will use scite to examine an important paper and answer a number of key questions about it, such as:
Here we use as an example Graham, Haidt, and Nosek’s seminal work, Liberals and conservatives rely on different sets of moral foundations - a paper that charted a new direction for moral psychology. According to the scite database, it has been cited by roughly 2,400 publications in over 3,000 citation statements since its publication in 2009. In short, the authors claim that there are five basic subdomains of moral cognition (how people reason about right and wrong), and that political liberals and conservatives differ in how much importance they place on each subdomain.
The scite report reveals a few interesting facts at a glance in the “Filters” panel on the left: first, it has received nearly 200 supporting citations and a handful of contrasting citations (for more information on what these categories mean and how citations are classified, see our help article).
Moreover, its influence appears to be at least consistent - if not increasing - over the past decade, given the sheer number of citations it has received across time (the y-axis represents the number of citations).
Let’s start by filtering by paper section to see whether the paper was cited as a theoretical work (and thus cited in the Introduction) or in an attempt to replicate and/or extend its results. We can also limit our results to peer-reviewed papers (excluding, for example, preprints and other non- peer-reviewed manuscripts).
Finally, we can limit our results to those appearing in a select set of journals - ones that directly address moral psychology.
This combination of filters reduces the citation count from over 3,000 statements to a mere eight. In these filtered results, we can see various instances in which the key theoretical contribution of the paper in question – Moral Foundations Theory – has been applied to a variety of topics, such as political ideology, political activism, and specific issues, such as Scottish independence from the United Kingdom.
We can also use scite to find major challenges to Graham, Haidt and Nosek. To do so, let’s limit our findings to citation statements that seem to be in contrast with their central empirical claims.
A number of results are definitely relevant. For example, Igarashi and Laurence (2021) find that the original findings do not necessarily hold with respect to attitudes about immigration.
Similarly, Hubert Lyall, Järvikivi (2021) were unable to replicate the original paper’s central claims about disgust and political ideology.
However, perhaps the most interesting results are Schein, Gray’s (2015) paper entitled The myth of harmless wrongs in moral cognition: Automatic dyadic completion from sin to suffering, in which the authors challenge Moral Foundations Theory and propose an alternative (dyadic morality), as well as provide evidence for it.
The scite report page for this paper also contains a link to another paper by the same authors which outlines dyadic morality in greater detail. As it turns out, this is the most well-developed challenge to Moral Foundations Theory. By examining this paper’s report, we can further see how this alternate theory of moral reasoning has been developed and tested. This process is sometimes called “forward chaining” (for more information on how to use this technique with scite, see our help page).
As mentioned earlier, this paper has a fair number of supporting citations (198). Filtering by citation type gives us a report of only those statements. By further filtering by section - limiting to Results and Discussion sections, where empirical evidence is presented and discussed - we can further narrow our report to 88 articles.
If we sort by engagement, we can see that the theory presented in the paper has been applied by highly-cited articles to many different domains (e.g., immigration, climate change, COVID-19) to produce results that are roughly consistent with Moral Foundations Theory. At this point, it might be helpful to browse through the first few pages of results, looking for topics that are most relevant to our own research.
We also might be interested in how Graham, Haidt, and Nosek has been supported in the most recent literature. This can be accomplished by moving the “Year Published” slider to limit results to the past two years. This further limits our results to citations from 15 papers, all from 2020 on.
A final question we can ask about the paper in question is how its authors continue to cite it. That is, have Graham, Haidt and Nosek modified their theory? Found conflicting results? Abandoned it altogether? This can be accomplished by resetting all the other fields and limiting results to self-citations, which shows 64 papers that have one or more of the paper’s original authors citing themselves.
Sorting by engagement, we see Iyer, Koleva, Graham, Ditto, and Haidt (2012) listed on the first page of results. In this paper, two of our original authors, along with three additional researchers, explore the possibility that the original theory should be modified to account for moral reasoning in a group heretofore unexamined (political libertarians).
Using the same approach we used for other citing papers, we can examine the scite report for Iyer et al. and see how its findings have replicated (or not) and extended. For example, by limiting results to those citations which appear in Method sections, we can see a number of subsequent papers that used the survey instrument Iyer et al. developed for their 2012 paper. Given that these citations continue through 2022, it seems that this modification of Graham, Haidt and Nosek’s (2009) original theory continues to be an active area of research.
To wrap up, let’s review what we have learned about Graham, Haidt and Nosek’s (2009) seminal paper thanks to a quick review of citing research: