The Signpost

File:Red flag warning banner at Cal Fire Green Springs Station, May 2022.JPG
Pi.1415926535
CC BY-SA 4.0
50
400
Special report

RetractionBot is back to life!

A bit of history, context, and what you can expect to see in articles

RetractionBot, from pre-history to v2

Back in 2012, Doc James made a query over at WikiProject Medicine about what sort of work could be done by a bot to find retracted papers. At the time, there was no centralized way of finding retracted papers, so Rich Farmbrough queried the PubMed database looking for retraction-related keywords (like 'retraction of publication' in the metadata). Of the roughly 4000 retractions, he found 138 that matched papers cited on Wikipedia. The template {{retracted}} was created to flag those papers, and was manually and semi-automatically added to articles.

Then in 2018, JenOttawa noticed the then newly-launched RetractionDatabase.org, a database of retractions maintained by Retraction Watch. This led Samwalton9 to code the first iteration of RetractionBot. The bot was then doing automatically what people did manually, saving everyone a lot of hassle. However, the bot only ran for a few months, and hit a snag: several Cochrane Reviews were flagged as retracted for technical reasons, while they were never retracted in actuality. The bot was put on hiatus, and Samwalton9 never got to fixing the issue.

Five years later, motivated by the slew of retractions hitting major publishers from Elsevier, Hindawi, SAGE, and many others, as well as the opening up of RetractionDatabase.org (now with nearly 40,000 retractions), I thought it would do us some good to kick the hornet's nest and see if I could interest someone in revisiting this project.

Turns out I could. Not even a week after probbing the volunteers at WP:BOTREQ, mdann52 graciously took over maintenance of RetractionBot, and the bot is now back alive, with many improvements. In particular, it now covers expressions of concerns, not only retractions, which are early warning signs that a paper might be dubious and could be retracted/in need of a major revision. This led to the creation of {{expression of concern}}, which works very similarly to {{retracted}} (see below).

What the bot does

Related articles
citations

RetractionBot is back to life!
8 June 2024

Tens of thousands of freely available sources flagged
4 December 2023

Citation tools for dummies!
1 August 2023

Journals cited by Wikipedia
1 August 2023

The unexpected rabbit hole of typo fixing in citations...
31 August 2022


More articles

Cleaning up awful citations with Citation bot
1 August 2022

Yes, the sky is blue
27 March 2022

Detecting spam, and pages to protect; non-anonymous editors signal their intelligence with high-quality articles
30 August 2020

The Wikipedia SourceWatch
31 March 2019

Reliable Sources Noticeboard editors discuss deprecating sources
24 December 2018

New guideline for technical collaboration
4 November 2016

Citations are needed
14 January 2015

Gender gap and skills gap; academic citations on the rise; European food cultures
26 November 2014

Citations, non-free content, and a MediaWiki meeting
19 June 2013

WebCite proposal; request for adminship reform
11 February 2013

Barnstars work; Wiktionary assessed; cleanup tags counted; finding expert admins; discussion peaks; Wikipedia citations in academic publications; and more
30 April 2012

Journalist regrets not checking citation, PR firms issue advice on how to "survive" Wikipedia (but U.S. Congressman caught red-handed)
22 August 2011

Oral citations; the state of global development; a gentler Huggle; brief news
25 July 2011

Second Wikipedian in Residence, citation needed for sanity
8 November 2010

Good faith vs. bad faith, climate change, court citations, weirdest medieval fact, brief news
25 October 2010

Wikipedia in British schools, Hitler's Downfall meme, and more
11 January 2010

WMF Elections, Annual Financial Plan, Google Image Search, and more
13 July 2009

Lessons for Brits, patent citations
23 February 2009

Tutorial: Adding citations
4 February 2008

Efforts to reform Requests for Adminship spark animated discussion
23 April 2007

News and notes: More legal citations, milestones
5 February 2007

Court decisions citing Wikipedia proliferate
29 January 2007

Wikipedia in the news
14 August 2006

Wikipedia cited by the High Court of England and Wales
3 July 2006

Wikipedia in the news
19 June 2006

Wikipedia in the news
20 February 2006

Wikipedia in the news
6 February 2006

Wikipedia in the news
16 January 2006

Wikipedia in the news
26 December 2005

Wikipedia in the news
19 December 2005

Bugs, Repairs, and Internal Operational News
28 November 2005

Wikipedia in the news
28 November 2005

Wikipedia in the news
21 November 2005

Wikipedia in the news
14 November 2005

Wikipedia in the news
7 November 2005

Wikipedia in the news
31 October 2005

Wikipedia in the news
24 October 2005

Wikipedia in the news
17 October 2005

Wikipedia in the news
10 October 2005

Wikipedia in the news
3 October 2005

Wikipedia in the news
26 September 2005

Wikipedia in the news
19 September 2005

Wikipedia in the news
12 September 2005

Wikipedia in the news
5 September 2005

Wikipedia in the news
29 August 2005

Wikipedia in the news
22 August 2005

Wikipedia citations abound; Wikimania article published
15 August 2005

Wikipedia praised in media, including by competitor of sorts
25 July 2005

Even without London events, Wikipedia draws media coverage, award
11 July 2005

Media focus on collaboration includes Wikipedia
20 June 2005

Wikipedia inspires fight against disease, newspaper "wikitorials"
13 June 2005

Press coverage this week
6 June 2005

Wikipedia featured in Time Magazine
30 May 2005

Featured article citation rules discussed, featured lists invented
30 May 2005

In the news: Wikipedia serves as supplement to science, BBC
23 May 2005

Press coverage this week
16 May 2005

Press coverage this week
9 May 2005

Press coverage this week
2 May 2005

Encarta pseudo-wiki debate continues
25 April 2005

Press covers Wikipedia after being beaten to the punch
18 April 2005

Media covers German Wikipedia DVD, plans for English
11 April 2005

News outlets note Wikipedia's rapid updates, jokes
4 April 2005

Outside sites build more indexes and links to Wikipedia
28 March 2005

Wikipedia described as best or worst of the Web, depending on the source
21 March 2005

In the media: Prolific editors featured, German magazine plagiarizes
14 March 2005

Student use of Wikipedia citations debated
7 March 2005

The bot first downloads a .csv file containing all the information in the RetractionDatabase (a 50MB download available here). Then it crosschecks retracted DOIs and PMIDs in the database against those found on Wikipedia.

If a match is found, the bot will, for example, change

  • ...Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.[1]
1. ^ Restrepo-Arango, Marcos; Gutiérrez-Builes, Lina Andrea; Ríos-Osorio, Leonardo Alberto (April 2018). "Seguridad alimentaria en poblaciones indígenas y campesinas: una revisión sistemática". Ciência & Saúde Coletiva. 23 (4): 1169–1181. doi:10.1590/1413-81232018234.13882016. PMID 29694594.

to

  • ...Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.[1]
1. ^ Restrepo-Arango, Marcos; Gutiérrez-Builes, Lina Andrea; Ríos-Osorio, Leonardo Alberto (April 2018). "Seguridad alimentaria en poblaciones indígenas y campesinas: una revisión sistemática". Ciência & Saúde Coletiva. 23 (4): 1169–1181. doi:10.1590/1413-81232018234.13882016. PMID 29694594. (Retracted, see doi:10.1590/1413-81232018241.32242011, PMID 30698268,  Retraction Watch. If this is an intentional citation to a retracted paper, please replace {{retracted|...}} with {{retracted|...|intentional=yes}}.)

It is now up to humans like you to review if this is problematic for the article. If the citation is no longer reliable, then the article needs to be updated, which could be as minor as the removal/replacement of the citation with a reliable one, to rewriting an entire section that was based on flawed premises. If the citation to a retracted paper was intentional, like in the context of a controversy noting that a paper was later retracted, you can replace {{retracted|...}} with {{retracted|...|intentional=yes}}, suppressing the red notice

  • ...Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.[1]
1. ^ Restrepo-Arango, Marcos; Gutiérrez-Builes, Lina Andrea; Ríos-Osorio, Leonardo Alberto (April 2018). "Seguridad alimentaria en poblaciones indígenas y campesinas: una revisión sistemática". Ciência & Saúde Coletiva. 23 (4): 1169–1181. doi:10.1590/1413-81232018234.13882016. PMID 29694594. (Retracted, see doi:10.1590/1413-81232018241.32242011, PMID 30698268,  Retraction Watch )

What you can do

If you are interested in doing systematic work involving Wikipedia articles citing retractions, the category Category:Articles intentionally citing retracted publications will be automatically populated by {{retracted}}. The retractions that haven't yet been reviewed by a human can be found in the sub-category Category:Articles citing retracted publications instead.

Otherwise? Well... carry on as usual. But if you see one of those big red notices, don't panic. Treat it like any other unreliable source, and update the article accordingly. If a retraction paper (or one with an expression of concern) is intentionally cited, then simply follow the instructions and replace {{retracted|...}} with {{retracted|...|intentional=yes}} (or {{expression of concern|...}} with {{expression of concern|...|intentional=yes}}) to suppress the red notice.

And while these are not currently handled by RetractionBot, {{erratum}} and Category:Articles intentionally citing publications with errata / Category:Articles citing publications with errata work very similarly to the above categories for papers with errata and might also be of interest to you.

Happy editing!

+ Add a comment

Discuss this story

These comments are automatically transcluded from this article's talk page. To follow comments, add the page to your watchlist. If your comment has not appeared here, you can try purging the cache.
  • Good idea for a bot - never heard about it or seen it in action, but I'm glad it's back up and running - kudos to those responsible! —Ganesha811 (talk) 12:41, 8 June 2024 (UTC)[reply]
    See Special:Contributions/RetractionBot if you want to see some of its edits. Though the categories mentioned in the article are the best way to check what currently needs attention. Headbomb {t · c · p · b} 12:44, 8 June 2024 (UTC)[reply]
  • Aren't retracted articles below unreliable sources in terms of quality? With an unreliable source, we don't know if the information is good or not; with a retracted article, we know that somebody has found a major problem. No? Compassionate727 (T·C) 00:48, 9 June 2024 (UTC)[reply]
    That's why we're flagging them. This way we don't cite them thinking they are reliable. Headbomb {t · c · p · b} 01:22, 9 June 2024 (UTC)[reply]
    Generally, but not always. Upon reading this article, I took a look at Category:Articles citing retracted publications, and clicked through to Autism therapies, a topic I don't know very much about. The relevant text in the article reads:

    In the early 1990s, it was hypothesized that autism could be caused or aggravated by opioid peptides like casomorphine that are metabolic products of gluten and casein. Based on that hypothesis, diets that eliminate foods containing either gluten or casein, or both, are widely promoted, and many testimonials can be found describing benefits in autism-related symptoms, notably social engagement and verbal skills. Studies supporting those claims had significant flaws, so those data were inadequate to guide treatment recommendations.

    The claim in the first sentence—that there was a hypothesis of a link between autism and metabolic products of gluten or casein—is supported by a 1991 paper. That paper isn't online, but the abstract says it is based on a study of 30 children. It doesn't really matter whether it is accurate, because it does establish what it is being cited for—that a hypothesis of the sort existed, not that it is true.
    The second sentence is uncited—but it does not seem particularly unbelievable that people jumped on a hypothetical link between gluten/casein metabolisation and autism and promoted diets around it of dubious credibility. Wikipedia has an entire category on autism-related pseudoscience, after all.
    The final sentence—saying that there isn't really any truth to the hypothesis—cites a 2006 review article and a now retracted Cochrane review originally published in 2008. Why was the Cochrane review retracted?

    This review was withdrawn from the Cochrane Library in Issue 4, 2019, as it has not been updated since its last revision in 2008. The editorial group responsible for this previously published document have withdrawn it from publication.

    They're not saying "this is debunked nonsense", they're saying "it's old and unless the article gets updated, our policy is to retract it". If an article is retracted because it's trash, then yes, that's a good reason to not rely on it. If an article is retracted because a diligent body like the Cochrane Collaboration are concerned that it might be in need of an update... that doesn't immediately make it useless. Editors have brains, and it is fine to use them. Carefully considering how Wikipedia uses it as a source and the reasons behind the article's retraction would be strongly advised in deciding whether to continue using it. If the article was retracted because it is a load of made up junk, that's a real problem. But that's not always going to be the case. —Tom Morris (talk) 16:33, 9 June 2024 (UTC)[reply]
    The Cochrane review being flagged here is what I was referring to by "several Cochrane Reviews were flagged as retracted for technical reasons". Ideally, RetractionBot would leave those alone, and let User:Pi bot deal with Cochrane Reviews (see BRFA and Wikipedia:WikiProject Medicine/Cochrane update). Headbomb {t · c · p · b} 16:50, 9 June 2024 (UTC)[reply]
    It's incredibly bad luck that I happened to randomly choose this as an example then. I'm glad people who actually know something about these topics are thinking hard about it. Between the rapid rise in conspiracy theories, the replication crisis and the preprint-to-tweet pipeline that flourished during the peak years of COVID, knowing stuff seems to have had a tough time recently—any effort to try and fix that that is commendable. —Tom Morris (talk) 14:17, 10 June 2024 (UTC)[reply]
    I'd argue it was good luck. I didn't have room in the piece to got into the nitty gritty, and didn't really know how to summarize the issue concisely and intelligibly. But here, with more space, you summarized it better than I ever could. Headbomb {t · c · p · b} 23:10, 10 June 2024 (UTC)[reply]
    (edit conflict)Tom Morris well, you've just cited the exact article which is partially why the bot stopped running in the first place and has been causing me struggles since - see User talk:RetractionBot#How to avoid an edit warring bot and (more recently) Wikipedia talk:WikiProject Medicine#Dealing with Cochrane Review retractions. The consensus over there, seemed/seems to be that the 2008 paper had not been withdrawn, but I'm still trying to get my head around that and other papers and exactly how they've handled it before I actually process Cochrane Review stuff! Mdann52 (talk) 16:52, 9 June 2024 (UTC)[reply]
    I'm personally just tempted to remove all retracted notices from Cochrane Reviews and let Pi bot deal with it. But we should have that discussion at WT:MED. Headbomb {t · c · p · b} 17:04, 9 June 2024 (UTC)[reply]
  • Currently resisting the temptation to wikilink humans like you to Viewers like you. → FeRDNYC (talk) 04:59, 9 June 2024 (UTC)[reply]
    Instead, I diddled the formatting of the example references; see edit summary for details/justification. FeRDNYC (talk) 05:11, 9 June 2024 (UTC)[reply]
  • Wow this is a real public service, thank you to all involved. Innisfree987 (talk) 07:56, 9 June 2024 (UTC)[reply]
  • Great article! Frostly (talk) 17:37, 20 June 2024 (UTC)[reply]

















Wikipedia:Wikipedia Signpost/2024-06-08/Special_report