The Wikimedia Foundation has long promised that the Wikimedia Endowment – held by the Tides Foundation and managed as an opaque, organisationally completely separate entity by a board led by Jimmy Wales – would soon be transferred to a financially transparent 501(c)(3) organisation. These promises date back to 2017 (see Signpost coverage this year and last year).

In April 2021, Endowment Director Amy Parker and Director of Development Caitlin Virtue again said on Meta:

"We are in the process of transitioning the Endowment to a new US 501c3 charity, after which it will begin making grants and will publish its own Form 990. ... As we approach the $100 million funding milestone, we are in the process of establishing the Endowment as a separate 501c3. ..."

"We are in the process of establishing a new home for the endowment in a stand-alone 501(c)(3) public charity. We will move the endowment in its entirety to this new entity once the new charity receives its IRS 501(c)(3) determination letter."

A full year has now passed since that 501(c)(3) determination letter (pictured) was received in June 2022. Yet the money has still not been transferred. This means that another year will have passed without public reporting on the Endowment's revenue and expenses, as it is organisationally separate from the Wikimedia Foundation (its revenue and assets are not included in WMF revenue and assets) and the Tides Foundation does not provide any such reporting either.

In response to an inquiry on the Wikimedia mailing list, WMF Chief Financial Officer Jaime Villagomez recently posted the following update on Meta:

Work is underway to move the Endowment assets out of Tides to its own charity. The transition is complex, due to the nature of banking activities and donor commitments so we cannot instantaneously move from one entity to the other. We anticipate that it will take a few more weeks to transfer most of our transactional and banking activity away from Tides. We will maintain the old endowment accounts to process residual income (such as dividend payments) for some months before we close those accounts. More importantly though, we will also be sharing an update on the Endowment's activities in FY 22-23 in the annual fundraising report to be published in the next quarter.
— User:JVillagomez (WMF)

In addition, WMF CEO Maryana Iskander and WMF board member Nataliia Tymkiv said on the Wikimedia mailing list:

The Board of Trustees will meet next on August 15 in Singapore. Following this meeting, there will also be an open session with the Wikimedia Foundation and Endowment Boards during Wikimania to answer questions on these topics or others you may have.

The Wikimedia Endowment holds a significant proportion of all the funds the public has ever donated to the Wikimedia cause. Yet it does not follow the same standards of transparency that apply to other parts of the movement. For example, it would be unimaginable for any WMF affiliate to ingest over $100 million over the best part of a decade without ever publishing audited accounts detailing revenue and expenses.

Why should the Wikimedia Endowment be different? – AK

The Wikimania Scholarship outcomes were recently published on the Wikimania site. According to the summary provided there,

We had a total of 3800 applications started, after removing spam and incomplete applications we had 1209 actual applications for review. These application were then grouped according to regions.

This means that less than one in six scholarship applications was approved. To one editor, at least, this seemed a "paltry" amount of funding:

I find it rather disgraceful that the Wikimedia Foundation accepted only 197 of the 1209 completed scholarship applications for this year's Wikimania conference, or 16%. While I recognize that travel scholarships aren't cheap, I presume that a sizable portion of the applicants are heavily involved in Wikimedia projects, devoting many hours a week to volunteer work. Wikimania scholarships are one of the few ways the WMF can use its ample financial resources to show tangible appreciation to volunteers and aid participation in the movement. You could have afforded to assist more than 16% of applicants, and it's disappointing that you deemed the expense not worthwhile when you put together your budget.
— User:Sdkb

A WMF spokesperson responded by saying:

The Foundation sponsors the whole event—not just the scholarships—and in this year's Annual Plan, despite reducing expenses across the Foundation, funding for Wikimania increased. Not to undermine any disappointment that any applicant may feel for not having been selected, of course that’s completely valid and understandable, but it does feel relevant to mention that this year there are ~200 scholarships, around 66% more than the ~120 from the last in-person Wikimania in 2019. Together with each year’s Core Organizing Team, the Foundation always thinks about how to spend the funds to reach the most Wikimedians possible, because we completely agree with you that recognizing people for their contributions is critical. This year, that meant increasing the number of scholarships that could be awarded by the volunteer subcommittee, working to keep virtual registration free despite the costs of the virtual event, and working to keep the in-person ticket subsidized. I know it's of course still disappointing for anyone who wanted to attend in person and didn’t get selected. I really do hope those people will consider applying again for future Wikimanias.
— User:ELappen (WMF)

User:Sdkb seemed unimpressed. – AK

In a rare reversal, User:Gitz6666 had his global lock overturned after lodging an appeal with the stewards. Gitz6666 had been indefinitely blocked on the Italian Wikipedia in May, along with another user, and then had his account globally locked by an Italian steward.

The underlying dispute concerned a sociologist's Italian Wikipedia biography that had attracted press attention for its alleged unfairness (see previous Signpost coverage). – AK

• New administrators: The Signpost welcomes the English Wikipedia's newest administrator, Firefangledfeathers.
• Articles for Improvement: This week's Article for Improvement (beginning 17 July) is Religious philosophy. It will be followed the week after by Act (drama). Please be bold in helping improve these articles!

An investigative article in The New Yorker, titled "The Dirty Secrets of a Smear Campaign", describes how "Sheikh Mohammed bin Zayed, the ruler of the United Arab Emirates, paid a Swiss private intelligence firm millions of dollars to taint perceived enemies". Most of the lengthy article (whose audio version runs 1 hour and 13 minutes) isn't about Wikipedia, but there are several paragraphs about how the firm ("Alp Services", founded by an investigator named Mario Brero) used it for their purposes alongside many other interesting tools (such as illegitimately obtaining phone call records or tax records of their targets, and planting stories in various news outlets).

The first part is about an American oil trader named Hazim Nada, founder of a company called Lord Energy:

On January 5, 2018, Sylvain Besson, a journalist who had written a book purporting to tie [Hazim Nada's father] Youssef Nada to a supposed Islamist conspiracy, published an article, in the Geneva newspaper Le Temps, claiming that Lord Energy was a cover for a Muslim Brotherhood cell. “The children of the historical leaders of the organization have recycled themselves in oil and gas,” Besson wrote. A new item in Africa Intelligence hinted darkly that Lord Energy employees had “been active in the political-religious sphere.” Headlines sprang up on Web sites, such as Medium, that had little editorial oversight: “Lord Energy: The Mysterious Company Linking Al-Qaeda and the Muslim Brotherhood”; “Compliance: Muslim Brotherhood Trading Company Lord Energy Linked to Crédit Suisse.” A Wikipedia entry for Lord Energy [probably fr:Lord Energy, created by a single-purpose account (SPA) in June 2018] suddenly included descriptions of alleged ties to terrorism.

This outside view from the victim's perspective is later matched to what the reporter learned from leaked/hacked internal emails of "Alp Services":

In February, 2018, [Brero] asked for more money to expand his operation against Nada, and proposed “to alert compliance databases and watchdogs, which are used by banks and multinationals, for example about Lord Energy’s real activities and links to terrorism.” His “objective,” he explained, was to block the company’s “bank accounts and business.” [...]

Alp quickly put the Emiratis’ money to work. An Alp employee named Raihane Hassaine e-mailed drafts of damning Wikipedia entries. On an invoice dated May 31, 2018, the company paid Nina May, a freelance writer in London, six hundred and twenty-five pounds for five online articles, published under pseudonyms and based on notes supplied by Alp, that attacked Lord Energy for links to terrorism and extremism. (Hassaine did not respond to requests for comment. May told me that she had worked for Alp in the past but had signed a nondisclosure agreement.)

And:

Alp operatives bragged to the Emiratis that they had successfully thwarted Nada’s efforts to correct the disparaging Lord Energy entry on Wikipedia. “We requested the assistance of friendly moderators who countered the repeated attacks,” Brero wrote in an “urgent update” to the Emiratis in June, 2018. “The objective remains to paralyze the company.” To pressure others to shun Lord Energy, Alp added dubious allegations about the company to the Wikipedia entries for Credit Suisse [presumably corresponding to this July 2019 edit by a SPA, subsequently removed in January 2021] and for an Algerian oil monopoly [possibly Sonatrach, referring to these edits - which were removed on English Wikipedia after the publication of the New Yorker article].

And regarding another target:

Brero’s campaign sometimes involved secret retaliation. In a 2018 report, a U.N. panel of human-rights experts concluded that the U.A.E. may have committed war crimes in its military intervention in Yemen. The Emiratis commissioned Brero to investigate the panel’s members, especially its chairman, Kamel Jendoubi, a widely admired French Tunisian human-rights advocate. [...] “Today, in both Google French and Google English, the reputation of Kamel Jendoubi is excellent,” Brero noted in a November, 2018, pitch to the Emiratis. “On both first pages, there is not a single critical article.” Within six months, Brero promised, Jendoubi’s image could be “reshaped” with “negative elements.” The cost: a hundred and fifty thousand euros.

Rumors spread through Arab news outlets and European Web publications that Jendoubi was a tool of Qatar, a failed businessman, and tied to extremists. A French-language article posted on Medium suggested that he might be “an opportunist disguised as a human-rights hero.” An article in English asked, “Is UN-expert Kamel Jendoubi too close to Qatar?” Alp created or altered Wikipedia entries about Jendoubi, in various languages, by citing claims from unreliable, reactionary, or pro-government news outlets in Egypt and Tunisia.
Jendoubi told me that he’d been perplexed by the flurry of slander that followed the war-crimes report. “Wikipedia is a monster!” he told me. He had managed to clean up the French entry, but the English-language page still stymied him. He said, “You speak English—can you help?”

On fr:Kamel Jendoubi, a "Controverses" section was added by a SPA in January 2019, and expanded by another SPA in August 2019. Most of it was deleted in April/May 2021, by an account with only one earlier edit, and then by an experienced editor. Around the same time, the English Wikipedia's article Kamel Jendoubi likewise saw an attempt by an IP editor to remove similar information, which was reverted as "Likely censorship of content"; although a July 2022 edit that provided a more detailed rationale for a more limited removal was successful.

The New Yorker article was published in April. Its findings were put into a much wider context earlier this month when various European news media collaborating in the European Investigative Collaborations (EIC) network reported on the results of an investigation dubbed "Abu Dhabi Secrets", revealing that

[...] Alp Services has been contracted by the UAE government to spy on citizens of 18 countries in Europe and beyond. Alp Services has sent to the UAE intelligence services the names of more than 1000 individuals and 400 organizations in 18 European countries, labelling them as part of the Muslim Brotherhood network in Europe.

This investigation was based on a stash of "78,000 confidential documents obtained by the French online newspaper Mediapart", according to Middle East Eye, which summarized the modus operandi of the campaign as follows:

Alp Services - and Brero - were paid tens of thousands of euros per individual targeted, according to Le Soir. The Swiss group then produced reports on the identified individuals.

[...]
Once the information was sent over to Emirati intelligence services, agents were able to target the individuals further through press campaigns, forums published about them, the creation of fake profiles and the modification of Wikipedia pages.

Many or most of the news reports emanating from the collective EIC investigation don't seem to have focused on the Wikipedia angle. Still, the Spanish publication Infolibre reveals some further details, quoting from messages where Alp's paid Wikipedia editors report about their efforts to their Emirati clients, in particular edits (presumably including these) on the English and Spanish Wikipedia to connect Mohammed Zouaydi (known as the "Al Qaeda's financier") to the Muslim Brotherhood. On the Spanish Wikipedia, they claim to have entered "an intense battle with pro-Muslim Brotherhood elements who wanted to censor information about the Brotherhood and its links to Al Qaeda" (translated back from Spanish).

At the French Wikipedia's "Projet Antipub", editors are currently looking into various other articles and accounts that may be connected to the campaign. – H

The Telegraph reports (non-paywalled) on the founding of Ruwiki (see previous Signpost coverage):

Wikipedia's top editor in Russia has quit the online encyclopaedia to launch a rival service sympathetic to Vladimir Putin.

Vladimir Medeyko, the long-serving leader of Wikipedia editors in the country, has copied the website's existing 1.9 million Russian articles into a new Kremlin-approved version.

The creation of the new service, called Ruwiki, was announced by a State Duma deputy from Putin's political party.

It comes as the Russian leader steps up efforts to censor coverage of the war in Ukraine, amid growing signs of discontent at home. [...]

According to The Telegraph, Ruwiki lacks specific content compared to the Wikipedia version:

The Ruwiki entry for Ukraine makes no mention of Russia's invasion or the international support for Kyiv's resistance.

The service has also chosen not to replicate articles on Yevgeny Prigozhin’s attempted coup against Putin.

In related news, on July 5 between around 2 am and 4 am Moscow time, access to Wikipedia and other "Western internet services" including Google was temporarily disrupted as Russian authorities tested the country's "Sovereign Internet system", as reported in a Twitter thread by Access Now staff member Natalia Krapiva. – AK, H

• Wiki wars: A BBC radio programme presented by Lara Lewington tries to explain how the Wikipedia sausage is made, touching on hoaxes and Things Gone Wrong like the Croatian and Scots Wikipedias. The programme contains two gross errors – it claims that IP editors lost the ability to edit biographies of living people after the Seigenthaler incident (ahem) and that paid editors are forbidden from editing articles directly (they are only "very strongly discouraged" from doing so).
• TEDx talk: Annie Rauwerda (of Depths of Wikipedia) gave a TEDx talk titled "Why an encyclopedia is my favorite place on the Internet". Watch the recording to find out which encyclopedia it is.
• Jimmy Wales interview: The "co-founder of Wikipedia" was interviewed by Lex Fridman on his popular podcast (video, transcript), for three hours and 15 minutes. One user compiled some excerpts they found especially interesting from a Wikipedian perspective, and journalist Stephen Harrison (known for his Wikipedia column in Slate) summarized several points from the interview in a Twitter thread, e.g. about Wales "push[ing] back on Wikipedia’s alleged left wing bias."

David died on November 25, 2022, at the age of 83.^[1] He was a prolific editor and a self-declared gnome who added to articles on Philadelphia, United States, and created many new articles.^[2] David lived in Fairmount, Philadelphia. After earning a degree at Lafayette College and serving for two years with the US Army in West Germany, he worked for 27 years as a computer programmer for Sunoco.^[2][3]

David began his post-retirement Wikipedia career on March 13, 2008; five years later, he had made 100,000 edits, mostly through wikignoming. As of this issue, he is the 52nd-most-active editor in Wikipedia. David was a member of the Guild of Copy Editors, and took part in its monthly drives and blitzes. He was also part of the Article Rescue Squadron, saving uncited-but-promising articles from deletion.^[2] David was openly proud of his Wikipedia-editing activities, and he even designed his own Wikipedia-themed caps, which he wore to Philadelphia wiki-meetups and gave away to his fellow Wikipedians.^[2] Outside Wikipedia, David was a member of the Historical Society of Pennsylvania.^[3]

Dthomsen8 has left an indelible legacy at Wikipedia, and the encyclopedia and editing community are poorer for his loss. Messages of condolence can be left on his talk page.

Wikimedia Tanzania and Jenga Wikipedia ya Kiswahili have published a tribute to User:Kipala, a much-loved German Wikimedian who – as a fluent Swahili speaker – was particularly active in the African Wikimedia community (see also Wikimedia-l thread).

Ingo was an important stakeholder of Swahili/Kiswahili Wikipedia whose immeasurable contribution will not be forgotten. We will always remember our beloved Ingo Koll for many things, especially his significant contribution to editing, building, and maintaining Wikipedia Kiswahili, Wikimedia Tanzania and Jenga Wikipedia Kiswahili. He was a selfless person who gave his all whenever needed. His sudden death is a big blow to the Swahili/Kiswahili Wikipedia community and Wikimedia Tanzania.

He was a wonderful teacher who wanted to see Kiswahili grow and be increasingly used in education. He was committed to spreading free knowledge through Wikimedia projects and programs. His exemplary work is seen in radio programs such as “Macho Angani” and his remarkable production of Astronomy topics into Swahili language “Astronomia kwa Kiswahili”. Ingo loved everyone who fondly called him Babu Ingo or Kipala. We will honor him by keeping his values of hard work, selfless spirit, truthfulness and going all the way always. May he rest in eternal peace. Amen.

– AK

A monthly overview of recent academic research about Wikipedia and other Wikimedia projects, also published as the Wikimedia Research Newsletter.

Reviewed by Nicolas Jullien

From the abstract:^[1]:

"we analyze a large dataset of citations from Wikipedia and model the role of open access in Wikipedia's citation patterns. We find that open-access articles are extensively and increasingly more cited in Wikipedia. What is more, they show a 15% higher likelihood of being cited in Wikipedia when compared to closed-access articles, after controlling for confounding factors. This open-access citation effect is particularly strong for articles with low citation counts, including recently published ones. Our results show that open access plays a key role in the dissemination of scientific knowledge, including by providing Wikipedia editors timely access to novel results."

This article is a first draft of an analysis of the relationship between the availability of a scientific journal as open access and the fact that it is cited in the English Wikipedia (note: although it speaks of "Wikipedia", the article looks only at the English pages). It is a preprint and has not been peer-reviewed, so its results should be read with caution, especially since I am not sure about the robustness of the model and the results derived from it (see below). It is of course a very important issue, as access to scientific sources is key to the diffusion of scientific knowledge, but also, as the authors mention, because Wikipedia is seen as central to the diffusion of scientific facts (and is sometimes used by scientists to push their ideas).

The results presented in the article (and its abstract) highlight two important issues for Wikipedia that will likely be addressed in a more complete version of the paper:

• The question of the reliability of the sources used by Wikipedians

→ The regressions seem to indicate that the reputation of the journal is not important to be cited in Wikipedia.

→ Predatory journals are known to be more often open access than classical journals, which means that this result potentially indicates that the phenomenon of open access reduces the seriousness of Wikipedia sources.

The authors say on p. 4 that they provided "each journal with an SJR score, H-index, and other relevant information." Why did they not use this as a control variable? (this echoes a debate on the role of Wikipedia: is it to disseminate verified knowledge, or to serve as a platform for the dissemination of new theories? The authors seem to lean towards the second view: p. 2: "With the rapid development of the Internet, traditional peer review and journal publication can no longer meet the need for the development of new ideas".)

• The solidity of the paper's conclusions

The authors said: "STEM fields, especially biology and medicine, comprise the most prominent scientific topics in Wikipedia [17]." "General science, technology, and biomedical research have relatively higher OA rates."

→ So, it is obvious that, on average, there are more citations of Open Access articles in Wikipedia (than in the entire available research corpus), and explain that open access articles are cited more.

→ Why not control for academic discipline in the models?

More problematic (and acknowledged by the authors, so probably in the process of being addressed), the authors said, on p.7, that they built their model with the assumption that the age of a research article and the number of citations it has both influence the probability of an article being cited in Wikipedia. Of course, for this causal effect to hold, the age and the number of citations must be taken into account at the moment the article is cited in Wikipedia. For example, if some of the citations are made after the citation in Wikipedia, one could argue that the causal effect could be in the other direction. Also, many articles are open access after an embargo period, and are therefore considered open access in the analysis, whereas they may have been cited in Wikipedia when they were under embargo. The authors did not check for this, as acknowledged in the last sentence of the article. Would their result hold if they do their model taking the first citation in the English Wikipedia, for example, and the age of the article, its open access status, etc. at that moment?

Although this first draft is probably not solid enough to be cited in Wikipedia, it signals important research in progress, and I am sure that the richness of the data and the quality of the team will quickly lead to very interesting insights for the Wikipedia community.

• "Quantifying Engagement with Citations on Wikipedia" (about a 2020 paper that among other results found that "open access sources [...] are particularly popular" with readers)
• "English Wikipedia lacking in open access references" (2022)

Reviewed by Andreas Kolbe

From the abstract:^[2]

This study investigates the development of historical revisionism on Wikipedia. The edit history of Wikipedia pages allows us to trace the dynamics of individuals and coordinated groups surrounding controversial topics. This project focuses on Japan, where there has been a recent increase in right-wing discourse and dissemination of different interpretations of historical events.

This brief study, one of the extended abstracts accepted at the Wiki Workshop (10th edition), follows up on reports that some historical pages on the Japanese Wikipedia, particularly those related to World War II and war crimes, have been edited in ways that reflect radical right-wing ideas (see previous Signpost coverage). It sets out to answer three questions:

1. What types of historical topics are most susceptible to historical revisionism?
2. What are the common factors for the historical topics that are subject to revisionism?
3. Are there groups of editors who are seeking to disseminate revisionist narratives?

The study focuses on the level of controversy of historical articles, based on the notion that the introduction of revisionism is likely to lead to edit wars. The authors found that the most controversial historical articles in the Japanese Wikipedia were indeed focused on areas that are of particular interest to revisionists. From the findings:

Articles related to WWII exhibited significantly greater controversy than general historical articles. Among the top 20 most controversial articles, eleven were largely related to Japanese war crimes and right-wing ideology. Over time, the number of contributing editors and the level of controversy increased. Furthermore, editors involved in edit wars were more likely to contribute to a higher number of controversial articles, particularly those related to right-wing ideology. These findings suggest the possible presence of groups of editors seeking to disseminate revisionist narratives.

The paper establishes that articles covering these topic areas in the Japanese Wikipedia are contested and subject to edit wars. However, it does not measure to what extent article content has been compromised. Edit wars could be a sign of mainstream editors pushing back against revisionists, while conversely an absence of edit wars could indicate that a project has been captured (cf. the Croatian Wikipedia). While this little paper is a useful start, further research on the Japanese Wikipedia seems warranted.

See also our earlier coverage of a related paper: "Wikimedia Foundation builds 'Knowledge Integrity Risk Observatory' to enable communities to monitor at-risk Wikipedias"

Reviewed by Tilman Bayer

This preprint^[3] (by three graduate students at Stanford University's computer science department and Monica S. Lam as fourth author) discusses the construction of a Wikipedia-based chatbot:

"We design WikiChat [...] to ground LLMs using Wikipedia to achieve the following objectives. While LLMs tend to hallucinate, our chatbot should be factual. While introducing facts to the conversation, we need to maintain the qualities of LLMs in being relevant, conversational, and engaging."

The paper sets out from the observation that

"LLMs cannot speak accurately about events that occurred after their training, which are often topics of great interest to users, and [...] are highly prone to hallucination when talking about less popular (tail) topics. [...] Through many iterations of experimentation, we have crafted a pipeline based on information retrieval that (1) uses LLMs to suggest interesting and relevant facts that are individually verified against Wikipedia, (2) retrieves additional up-to-date information, and (3) composes coherent and engaging time-aware responses. [...] We focus on evaluating important but previously neglected issues such as conversing about recent and tail topics. We find that WikiChat outperforms all baselines in terms of the factual accuracy of its claims, by up to 12.1%, 28.3% and 32.7% on head, recent and tail topics, while matching GPT-3.5 in terms of providing natural, relevant, non-repetitive and informational responses."

The researchers argue that "most chatbots are evaluated only on static crowdsourced benchmarks like Wizard of Wikipedia (Dinan et al., 2019) and Wizard of Internet (Komeili et al., 2022). Even when human evaluation is used, evaluation is conducted only on familiar discussion topics. This leads to an overestimation of the capabilities of chatbots." They call such topics "head topics" ("Examples include Albert Einstein or FC Barcelona"). In contrast, the lesser known "tail topics [are] likely to be present in the pre-training data of LLMs at low frequency. Examples include Thomas Percy Hilditch or Hell's Kitchen Suomi". As a third category, they consider "recent topics" ("topics that happened in 2023, and therefore are absent from the pre-training corpus of LLMs, even though some background information about them could be present. Examples include Spare (memoir) or 2023 Australian Open"). The latter are obtained from a list of most edited Wikipedia articles in early 2023.

Regarding the "core verification problem [...] whether a claim is backed up by the retrieved paragraphs [the researchers] found that there is a significant gap between LLMs (even GPT-4) and human performance [...]. Therefore, we conduct human evaluation via crowdsourcing, to classify each claim as supported, refuted, or [not having] enough information." (This observation may be of interest regarding efforts to use LLMs as a tools for Wikipedians to check the integrity of citations on Wikipedia. See also the "WiCE" paper below.)

In contrast, the evalution for "conversationality" is conducted "with simulated users using LLMs. LLMs are good at simulating users: they have the general familiarity with world knowledge and know how users behave socially. They are free to occasionally hallucinate, make mistakes, and repeat or even contradict themselves, as human users sometimes do."

In the paper's evaluation, WikiChat impressively outperforms the two comparison baselines in all three topic areas (even the well-known "head" topics). It may be worth noting though that the comparison did not include widely used chatbots such as ChatGPT or Bing AI. Instead, the authors chose to compare their chatbot with Atlas (describing it as based on a retrieval-augmented language model that is "state-of-the-art [...] on the KILT benchmark") and GPT-3.5 (while ChatGPT is or has been based on GPT-3.5 too, it involved extensive additional finetuning by humans).

Compiled by Tilman Bayer

As part of an effort "to understand how Wikimedia can become the essential infrastructure of free knowledge in a possible future state where AI transforms knowledge search", on July 13 the Wikimedia Foundation announced a new Wikipedia-based plugin for ChatGPT. (Such third-party plugins are currently available to all subscribers of ChatGPT Plus, OpenAI's paid variant of their chatbot; the Wikipedia plugin's code itself is available as open source.) The Foundation describes it as an experiment designed answer research questions such as "whether users of AI assistants like ChatGPT are interested in getting summaries of verifiable knowledge from Wikipedia".

The plugin works by first performing a Google site search on Wikipedia to find articles matching the user's query, and then passing the first few paragraphs of each article's text to ChatGPT, together with additional (hidden) instruction prompts on how the assistant should use them to generate an answer for the user (e.g. "In ALL responses, Assistant MUST always link to the Wikipedia articles used").

The Wikimedia Foundation's Research department has published its biannual activity report, covering the work of the department's 10 staff members as well as its contractors and formal collaborators during the first half of 2023.

The Wikimedia Foundation announced the public release of "almost 8 years of pageview data, partitioned by country, project, and page", sanitized using differential privacy to protect sensitive information. See documentation

See the page of the monthly Wikimedia Research Showcase for videos and slides of past presentations.

Other recent publications that could not be covered in time for this issue include the items listed below. Contributions, whether reviewing or summarizing newly published research, are always welcome.

Compiled by Tilman Bayer

From the abstract:^[4]

"Large Language Models (LLMs) may hallucinate and generate fake information, despite pre-training on factual data. Inspired by the journalistic device of 'according to sources', we propose according-to prompting: directing LLMs to ground responses against previously observed text. To quantify this grounding, we propose a novel evaluation metric (QUIP-Score) that measures the extent to which model-produced answers are directly found in underlying text corpora. We illustrate with experiments on Wikipedia that these prompts improve grounding under our metrics, with the additional benefit of often improving end-task performance."

The authors tested various variations of such "grounding prompts" (e.g. "As an expert editor for Wikipedia, I am confident in the following answer." or "I found some results for that on Wikipedia. Here’s a direct quote:"). The best performing prompt was "Respond to this question using only information that can be attributed to Wikipedia".

From the abstract:^[5]

"This paper explores new methods for locating the sources used to write a text, by fine-tuning a variety of language models to rerank candidate sources. [...] We conduct experiments on two datasets, English Wikipedia and medieval Arabic historical writing, and employ a variety of retrieval and generation based reranking models. [...] We find that semisupervised methods can be nearly as effective as fully supervised methods while avoiding potentially costly span-level annotation of the target and source documents."

From the abstract:^[6]

"We propose WiCE, a new textual entailment dataset centered around verifying claims in text, built on real-world claims and evidence in Wikipedia with fine-grained annotations. We collect sentences in Wikipedia that cite one or more webpages and annotate whether the content on those pages entails those sentences. Negative examples arise naturally, from slight misinterpretation of text to minor aspects of the sentence that are not attested in the evidence. Our annotations are over sub-sentence units of the hypothesis, decomposed automatically by GPT-3, each of which is labeled with a subset of evidence sentences from the source document. We show that real claims in our dataset involve challenging verification problems, and we benchmark existing approaches on this dataset. In addition, we show that reducing the complexity of claims by decomposing them by GPT-3 can improve entailment models' performance on various domains."

The preprint gives the following examples of such an automatic decomposition performed by GPT-3 (using the prompt "Segment the following sentence into individual facts:" accompanied by several instructional examples):

Original Sentence:

The main altar houses a 17th-century fresco of figures interacting with the framed 13th century icon of the Madonna (1638), painted by Mario Balassi.

[Sub-claims predicted by GPT-3:]

• The main altar houses a 17th-century fresco.
• The fresco is of figures interacting with the framed 13th-century icon of the Madonna.
• The icon of the Madonna was painted by Mario Balassi in 1638.

From the abstract:^[7]

"[...] we introduce the SWiPE dataset, which reconstructs the document-level editing process from English Wikipedia (EW) articles to paired Simple Wikipedia (SEW) articles. In contrast to prior work, SWiPE leverages the entire revision history when pairing pages in order to better identify simplification edits. We work with Wikipedia editors to annotate 5,000 EW-SEW document pairs, labeling more than 40,000 edits with proposed 19 categories. To scale our efforts, we propose several models to automatically label edits, achieving an F-1 score of up to 70.6, indicating that this is a tractable but challenging NLU [Natural-language understanding] task."

From the abstract:^[8]

"we introduce the novel task of automatically generating short descriptions for Wikipedia articles and propose Descartes, a multilingual model for tackling it. Descartes integrates three sources of information to generate an article description in a target language: the text of the article in all its language versions, the already-existing descriptions (if any) of the article in other languages, and semantic type information obtained from a knowledge graph. We evaluate a Descartes model trained for handling 25 languages simultaneously, showing that it beats baselines (including a strong translation-based baseline) and performs on par with monolingual models tailored for specific languages. A human evaluation on three languages further shows that the quality of Descartes’s descriptions is largely indistinguishable from that of human-written descriptions; e.g., 91.3% of our English descriptions (vs. 92.1% of human-written descriptions) pass the bar for inclusion in Wikipedia, suggesting that Descartes is ready for production, with the potential to support human editors in filling a major gap in today’s Wikipedia across languages."

From the abstract:^[9]

"In this paper, we introduce WikiDes, a novel dataset to generate short descriptions of Wikipedia articles for the problem of text summarization. The dataset consists of over 80k English samples on 6987 topics. [...] [The autogenerated descriptions are preferred in] human evaluation in over 45.33% [cases] against the gold descriptions. [...] The automatic generation of new descriptions reduces the human efforts in creating them and enriches Wikidata-based knowledge graphs. Our paper shows a practical impact on Wikipedia and Wikidata since there are thousands of missing descriptions."

From the introduction:

"With the rapid development of Wikipedia and Wikidata in recent years, the editor community has been overloaded with contributing new information adapting to user requirements, and patrolling the massive content daily. Hence, the application of NLP and deep learning is key to solving these problems effectively. In this paper, we propose a summarization approach trained on WikiDes that generates missing descriptions in thousands of Wikidata items, which reduces human efforts and boosts content development faster. The summarizer is responsible for creating descriptions while humans toward a role in patrolling the text quality instead of starting everything from the beginning. Our work can be scalable to multilingualism, which takes a more positive impact on user experiences in searching for articles by short descriptions in many Wikimedia projects."

See also the "Descartes" paper (above).

From the abstract:^[10]

"we investigate using GPT-2, a neural language model, to identify poorly written text in Wikipedia by ranking documents by their perplexity. We evaluated the properties of this ranking using human assessments of text quality, including readability, narrativity and language use. We demonstrate that GPT-2 perplexity scores correlate moderately to strongly with narrativity, but only weakly with reading comprehension scores. Importantly, the model reflects even small improvements to text as would be seen in Wikipedia edits. We conclude by highlighting that Wikipedia's featured articles counter-intuitively contain text with the highest perplexity scores."

From the abstract:^[11]

"In this paper we [are] presenting a new corpus annotated for biographical event detection. The corpus, which includes 20 Wikipedia biographies, was compared with five existing corpora to train a model for the biographical event detection task. The model was able to detect all mentions of the target-entity in a biography with an F-score of 0.808 and the entity-related events with an F-score of 0.859. Finally, the model was used for performing an analysis of biases about women and non-Western people in Wikipedia biographies."

From the abstract:^[12]

"The proposed approach employs Latent Dirichlet Allocation (LDA) to analyze linked entities in a cross-lingual knowledge graph in order to determine topic distributions for Wikipedia articles in 28 languages. The distance between paired articles across language editions is then calculated. The potential applications of the proposed algorithm to detecting sources of information disparity in Wikipedia are discussed [...]"

From the paper:

"In this PhD project, leveraging the Wikidata Knowledge base, we aim to provide empirical evidence as well as theoretical grounding to address the following questions:

RQ1) How can we measure the information gap between different language editions of Wikipedia?

RQ2) What are the sources of the cross-lingual information gap in Wikipedia?

[...]
The results revealed a correlation between stronger similarities [...] and languages spoken in countries with established historical or geographical connections, such as Russian/Ukrainian, Czech/Polish, and Spanish/Catalan."

From the abstract:^[13]

"In this paper, we try to recount [Wikidata's] remarkable journey, and we review what has been accomplished, what has been given up on, and what is yet left to do for the future."

From the abstract:^[14]

"Priority conflicts and the attribution of contributions to important scientific breakthroughs to individuals and groups play an important role in science, its governance, and evaluation.[....] Our objective is to transform Wikipedia into an accessible, traceable primary source for analyzing such debates. In this paper, we introduce Webis-WikiSciTech-23, a new corpus consisting of science and technology Wikipedia articles, focusing on the identification of their history sections. [...] The identification of passages covering the historical development of innovations is achieved by combining heuristics for section heading analysis and classifiers trained on a ground truth of articles with designated history sections."

On Wednesday, the English Wikipedia's Arbitration Committee took some sorely needed action on the long-standing subject of fringe theories and WP:PSEUDOSCIENCE, issues where tense disagreements and POV-pushing have been causing trouble for decades.

Drafting arbitrator Hubert Glockenspiel, in an interview with the Signpost, said that the Committee was introducing a set of brand-new fringe opinions, conspiracy theories, and pseudoscientific claims, which will be available for free to anyone interested in arguing on Wikipedia.

The new topics span a broad range of subjects, academic disciplines and national concerns. "We tried to get a little bit of everything", said Glockenspiel. "Because, after all, Wikipedia was meant to be the sum of all human arguments about politics. And we're committed to belonging, inclusion, and equity; we need to amplify diverse voices."

A full list (along with suggested arguments for and against each theory) is available at WP:NEWFRINGE, but here is a summary of each one:

• English is actually a dialect of Basque

While it's true that virtually no grammar, syntax or etymology are shared between the two languages, it is obvious that they share a recent common origin; what else could account for the fact that the two languages use the same words for "Code of Conduct", "cookie", and "Wikipedia"?

• Oroville, California was founded by Alexander the Great in 317 BC

"Oro" is a Greek prefix. How did a city in California (inhabited by Native Americans, then Spanish-speakers, then English-speakers) get a Greek name? The answer is obvious. The Macedonian Menace and his troops didn't stop at India, like we have been told: instead he and his armies kept going east through China, crossed the Pacific Ocean, and founded a city in the Golden State, getting the drop on other Europeans by several thousand years. Note that San Francisco, Los Angeles, Sacramento, et cetera, were indeed founded by the Spanish or the Americans: Oroville was the only Hellenic city in the state.

Evidence of this historic achievement has been suppressed by the powers that be, since they are afraid people will realize how based Alexander really was, and return to his projects: uniting Macedon, Greece and Persia (thereby angering all three countries) and worshipping Zeus and Apis as the true progenitors of humanity (thereby angering the Church of the SubGenius).

• Lake Superior goes down to the center of the Earth

Evidence can be found by trying to swim in it — even on the hottest day in the hottest month of the year, it will always be cold as hell. If you "wait for it to warm up in the afternoon", it will be even colder. How is that possible? It doesn't add up.

• The center of the Earth is cold instead of hot

Why else would Lake Superior always be cold as hell?

• Hell is cold instead of hot

Why else would we compare Lake Superior to it?

• Barack Obama never existed

In reality, the United States simply didn't have a president from 2009 to 2017. The works attributed to "Barack Obama" were written by a variety of authors, orators and politicians; alleged videos of his public "appearances" were simply CGI. This one is fairly easy to figure out: he was allegedly from "Hawaii", an obviously fictional location (the United States somehow contains a tropical island with volcanoes on it?), and started his political career in "Chicago", another prima facie farcical city (a wacky noir setting filled with gangsters and tommy guns?)

• Joseph Stalin was a CIA plant

Come on. They expect us to believe that a good old boy named Joey Ashville — from the sweet, sweet state of Georgia, no less — just happened to wander into the Russian Revolution halfway across the globe, somehow ended up in charge of the whole thing, and then coincidentally spent his entire career making Communism look terrible?

• Various artifacts on the Moon were planted by beings from outer space

The native inhabitants of the Moon could never have developed such advanced technology — it had to have been put there by aliens. In fact, careful analysis of the so-called "Moon missions" reveals several entities bearing a distinctive resemblance to the animals of Earth.

• The media and banking system is controlled by the Church of the SubGenius

This is the only explanation for the sheer scale of the suppression campaign regarding the reality of Hellenic Oroville.

• 2001 Bomis pee tape

I don't care what anybody says: it's real to me.

While many of the new fringe theories have already been associated with one of the two American political parties, others remain undecided. The Hellenic Oroville theory, in particular, is currently the subject of ardent debate as to what political affiliation its supporters have: some have said that it's an obvious left-wing dogwhistle and critique of American imperialism, whereas some argue that it's an obvious right-wing dogwhistle and fantasy of Macedonian imperialism. There is also a secondary, less-important argument about whether it is correct or not.

One thing's for certain, though: we will have a bunch of AN/I threads about it.

Long-time tendentious editor (and WMF-banned troll) Snowpisser said, through a spokesman sockpuppet, that he welcomed the challenge of the new theories. "I can't wait to start a big clusterfuck over these. Nobody even knows what side they're supposed to be on yet! I will probably be able to catch a few dozen people off guard, and get them to freak out and get themselves banned."

Meanwhile, controversial administrator DarkAngelBlademaster666 said in a talk page comment that she was looking forward to figuring out what the right opinion was to have on them, and then immediately INVOLVED-blocking everyone who she disagreed with. "It's perfect, because none of my existing topic bans apply to this stuff yet. By the time they're expanded to cover these, I will have already gotten to fire off like thirty indefs".

Note: My friends Hazzard and Skutz came up with two of these (the moon one and the Stalin one, respectively).

One featured topic, by MaranoFan, a.k.a. NØ, was promoted this period.

view | edit | discuss

Takin' It Back

4 articles

"Made You Look"

"Bad for Me"

"Mother"

• These lists exclude the Wikipedia main page, non-article pages (such as redlinks), and anomalous entries (such as DDoS attacks or likely automated views). Since mobile view data became available to the Report in October 2014, we exclude articles that have almost no mobile views (5–6% or less) or almost all mobile views (94–95% or more) because they are very likely to be automated views based on our experience and research of the issue. Please feel free to discuss any removal on the Top 25 Report talk page if you wish.