Drug articles accurate and largely complete; women "slightly overrepresented"; talking like an admin

A monthly overview of recent academic research about Wikipedia and other Wikimedia projects, also published as the Wikimedia Research Newsletter.

Review by William Skaggs

Recently when my 83-year-old father was undergoing medical treatment, the doctor wanted to change one of his blood pressure drugs, and in order to let us know what the effects would be, she printed out the Wikipedia article on the drug and handed it to us. This accords with the overall impression I have developed: Wikipedia's articles on drugs are pretty good – good enough to impress even doctors. A new research study^[1] adds some substance to that impression.

A team of German pharmacologists picked a set of 100 drugs described in pharmacology textbooks, and compared the textbook descriptions with Wikipedia articles about the drugs, for accuracy (meaning that the Wikipedia article matched the information in the textbook) and comprehensiveness. They found that 99.7% of the facts in the Wikipedia articles were accurate, and 83.8% of the facts from the textbooks made it into the Wikipedia articles. These numbers were derived from the German Wikipedia, but the authors state that similar results were obtained for the English language version. They conclude that "our results suggest that Wikipedia is an accurate and informative source of drug information for undergraduate medical students." They also revisited the drug articles examined in 2010 by an earlier study which came to less positive conclusions (see coverage in this newsletter: "Quality of drug information in Wikipedia"), and "found the quality of pharmacological information significantly improved". Upon reviewing several other empirical studies which evaluated the quality of medical information on Wikipedia, the authors observe that "despite different methodologies, the main conclusion of these studies was that Wikipedia articles on health topics contain few errors and are well referenced, while the information provided often lacks depth."

Obviously this is something we should be proud of, but let me note a caveat. Articles about specific drugs are a prime example of the sort of thing Wikipedia is best at: articles about topics that can be handled in a systematic way, without requiring mastery of a large body of literature. As a rule, the more comprehensive a topic, the lower the quality of the Wikpedia article. Thus our article on the drug chlordiazepoxide (commonly known as Librium) is better than our benzodiazepine article, which covers the class of drugs to which Librium belongs. The latter article contains a lot of good information but is poorly organized. Our article pharmaceutical drug shows this flaw to an even greater degree. The general take-home message, supported by the German study, is that our medical articles can be very useful to people who are looking for specific facts, but tend to be less useful to people who are trying to understand broad principles.

Review by Maximilianklein

"It's a man's Wikipedia? Assessing gender inequality in an online encyclopedia",^[2] presented at the Ninth International AAAI Conference on Web and Social Media (ICWSM) this week, is an investigation into the gender of biography articles of six different Wikipedias. Four different biases that are investigated are coverage bias (who makes it into the encyclopedia), structural bias (which articles link to which), lexical bias (the type of words used in the articles), and visibility bias (who is featured on the Main Page).

Coverage bias is analysed by seeing who from the reference databases of notable humans of Freebase, MIT's Pantheon, and Human Accomplishment are in Wikipedia. A surprising result here is that women are not proportionally underrepresented as hypothesised, but even "slightly overrepresented". (The researchers acknowledge that the first two of these three are at least partly based on Wikipedia themselves, but try to address this issue by "seeking patterns that exist across all three datasets".)

The structural bias is a graph theoretical measure of how men and women's articles link to each other. Here it is shown that across all six languages, articles about women tend to link more to articles about men than vice versa. The Smurfette Principle, that women are less central in the link graph, is also tested. The in-degree of the two gendered article categories is compared, and it is found that men are indeed significantly more central in all language editions, except in the Spanish Wikipedia, where men and women are equally central.

The lexical bias notion stems from the idea of the Finkbeiner test, that a female scientist will often be noted as a woman as much as a scientist. It is indeed found that articles about women place linguistic emphasis on relationship, gender, and family. Whereas top terms in men's articles focus on their professions. The authors mention that this ties into the concept of male as the null gender. For instance the word "divorced" is 4.4 times more frequent in a woman's article than a man's on English Wikipedia. For German and Russian, that multiplier increases to 4.7 and 4.8 times, respectively.

Lastly visibility bias, the propensity of gendered articles to appear on the English Wikipedia Main Page is tested. Yet no significant difference is found in the propensity of the two genders to appear on the Main Page.

Unfortunately this paper suffers from its Euro-focus. The six languages in question are English, German, French, Italian, Spanish and Russian, but the width of the methods used still show wide-scale issues. The authors conclude that Wikipedia does show some signs of addressing systemic bias, like equal visibility on the main page, and coverage bias equality; but still there are stark differences in their portrayal. Whether this is due to biases in the real world, or the way that Wikipedians write about the real world, they say, is still an unknown mixed bag.

Review by Piotr Konieczny

An article^[3] in the Journal of the Association for Information Science and Technology (JASIST) examines Wikipedia editors' public communication using social network analysis theory. This research suggests that Wikipedia editors who engage in communication with others using user talk pages "are more experienced in editing high quality articles and are more integrated in the community". The author distinguishes quantitative and qualitative contributions, noting that the use of communication tools is more directly related to contributing not just to many articles, but to high quality articles, as well as larger number of namespaces. The use of such tools is centered on "coordinating and mentoring editors who edit lower quality articles", or in other words, the author observes that editors who edit high quality articles and use communication tools a lot seem to be more likely to reach out to less experienced editors than the other way around. The author concludes that online collaboration systems are improved through features that allow creation of what the author calls "personal" communication network. Through the study excluded bots, it does not seem to have investigated the details of communication (ex. templates, warnings, awards, others), and so its conclusions on the nature of communications (rather than who engages in it) are more tentative.

Review by Piotr Konieczny

This paper,^[4] likewise published in the JASIST, looks at the Talk:Vietnam War page (and its archives) and analyses it in the context of theories dealing with the concept of collective memory (cultural memory, memory space, and the "floating gap" concept introduced by Pentzold (2009) in his paper on Wikipedia.^{[supp 1]} As such, this paper is one of several works that argues that Wikipedia is a place where modern world's memories are being recorded and, to some extent, shaped for posterity. The paper finds that the Wikipedia's article is affected by two major debates ("(a) whether the US actually lost the war and (b) whether the voice of the American Vietnam veteran should be privileged.") It reviews major, recurring arguments presented by the talk page participants, and concludes that Wikipedia allows us to study how collective memory is shaped. The author also argues that it is the very fact that such debates can be observed on Wikipedia that may distance some educators, primarily librarians, who are used to works that conceal their knowledge production processes. The author ends with a call for librarians to edit Wikipedia, and help their patrons do the same, in order to participate in the 21st century curation of collective memories.

In a separate paper, published earlier in the Journal of Documentation,^[5] the author examined the debate about reliable sources on the same talk page and concluded (according to the abstract) that while much of it "is conducted without acrimony, the level of analysis one finds in the talk pages is rather shallow while the attention of individual contributors is not overly concentrated."

Review by Gamaliel

Three researchers have conducted a survey^[6] of the use and perceptions of Wikipedia among secondary school teachers and librarians in the United States. Twenty-two teachers and librarians responded to the survey. The vast majority (91%) reported that "Wikipedia had some effect on student research". Responses were mixed about how positive or negative that effect was, however. Positive comments included responses that Wikipedia is "easily understood...thorough, up-to-date, and easily edited" and "students use it to get the basic ideas for their research, then go to other websites to verify it." Negative comments largely centered on the fact that many students did not go beyond Wikipedia in their research, such as the responses that "students rely on it too heavily and do not expand their research to prove or disprove their findings" and "Students don’t want to check sources when they can just get their work done in one stop." Most (91%) reported that their schools had no policy regarding the use of Wikipedia, but responses were roughly split regarding the need for one. Teachers and those responding that Wikipedia had a negative effect were more likely to respond there was a need for such a policy, as opposed to librarians and those responding it had a positive effect. Based on the results, the authors concluded that any policy should not restrict Wikipedia use. They write "instead of banning and fighting against the usage, students need to be taught the skills to utilize it an effective way, such as how to use Wikipedia as a jumping off point to other potentially more trustworthy resources and how to evaluate the reliability of articles." Given the very small sample size of the survey, this article is more useful for its excellent literature review.

• "User engagement on Wikipedia, a review of studies of readers and editors": Another ICWSM conference paper^[7] frames itself as a literature review of topics that are of key interest to Wikipedia community: editor motivations, engagement, and retention. Unfortunately, it lacks a proper methodology (how did the author select papers to review?), which makes it difficult to discuss how its comprehensiveness. It nonetheless provides a good summary of many other key work in this field, and creates an interesting framework for recognizing some patterns in this subfield of Wikipedia studies. Unsurprisingly, the authors conclude that the Wikipedia community needs to improve its communication with newbies in order to increase their retention (fewer templates, stark warnings; more friendly personal outreach). (Review by Piotr Konieczny)

  

• Freedom of panorama in Europe: This paper^[8] presents an advocacy towards adopting freedom of panorama laws in the context of the European Union law harmonization. It is enriched with case studies from Wikipedia community's history, and has been supported by the Wikimedia Foundation (though the paper does not make it clear how, nor is it released under a free license itself). While suffering from a few minor issues (such as not clearly recognizing that Wikimedia Commons does not accept non-commercial images, and a law that would grant freedom of panorama to non-commercial uses would be of little value to Wikipedia), and heavily geared towards European legislation framework, it is a valuable addition to the discussion of the freedom of panorama concept. (Review by Piotr Konieczny)
• Talking like an admin: linguistic mimicry and network centrality on Wikipedia. A new conference paper^[9] in the field of sociolinguistics examines whether Wikipedia editors are more likely to linguistically coordinate with (use the same words as) their interlocutors when those others are more centrally located within the social network of Wikipedia, or when speaking to admins. The study draws on an annotated corpus of talkpage discussions^{[supp 2]} in which the admin status of each participant is known, and uses several measures of network centrality (Betweenness and Eigenvector) to calculate the distance between all editors in terms of the number of times they have directly replied to others in a talkpage thread. The authors determine that while editors align their vocabularies more when speaking to admins than non-admins, highly central editors (those who have engaged in a lot of discussions with a lot of different editors) tend to be aligned with whether or not they are admins. Their results suggest that admin status follows high centrality, not the other way around.

A list of other recent publications that could not be covered in time for this issue – contributions are always welcome for reviewing or summarizing newly published research.

• From March's CSCW conference (see also m:Research:CSCW 2015):
  • "Functional roles and career paths in Wikipedia"^[10]
  • "'Is' to 'was': coordination and commemoration in posthumous activity on Wikipedia biographies"^[11]
  • "The virtuous circle of Wikipedia: recursive measures of collaboration structures"^[12]
  • "Effects of a Wikipedia orientation game on new user edits"^[13] (about The Wikipedia Adventure)
• "Wikipedia and the politics of openness"^[14] (book, see also 2011 Signpost interview with the author)
• "Wikipédia, objet scientifique non identifié"^[15] ("Wikipedia, unidentified scientific object", book in French)
• "Improving disease surveillance: sentinel surveillance network design and novel uses of Wikipedia"^[16]
• "Disaster monitoring with Wikipedia and online social networking sites: structured data and linked data fragments to the rescue?"^[17]
• "Barriers to the localness of volunteered geographic information"^[18]
• "Amateur encyclopedia editors as nonprofessional journalists: Wikipedia as a gateway for breaking news"^[19] (German, with extended abstract in English)
• "How to extract seasonal features of sightseeing spots from Twitter and Wikipedia"^[20]
• "Analysing the use and perception of Wikipedia in the professional context of translation"^[21]
• "Cross-language Wikipedia editing of Okinawa, Japan"^[22]
• "Property type distribution in Wordnet, corpora and Wikipedia"^[23]
• "Quality assessment of Wikipedia articles using h-index"^[24] From the abstract: "In this paper, we propose a method for assessing quality values of Wikipedia articles from edit history using h-index. One of the major methods for assessing Wikipedia article quality is a peer-review based method. In this method, we assume that if an editor's texts are left by the other editors, the texts are approved by the editors, then the editor is decided as a good editor [ see m:Research:Content persistence ]. However, if an editor edits multiple articles, and the editor is approved at a small number of articles, the quality value of the editor deeply depends on the quality of the texts. In this paper, we apply h-index [... to improve this method. ...] the accuracy of article quality assessment in our method outperforms the existing peer-review based method."
• "Social Interactions vs Revisions, What is important for Promotion in Wikipedia?"^[25] From the abstract: "[We look] at the process of election for administrator in the English Wikipedia community. We modeled the candidates according to their revisions and/or social attributes. [...] Our model combining knowledge contribution variables and social networking variables successfully explain 78% of the results which is better than the former models. It also helps to refine the criterion for election. If the number of knowledge contributions is the most important element, social interactions come close second to explain the election. But being connected with the future peers (the admins) can make the difference between success and failure, making this epistemic community a very social community too."

Supplementary references and notes:

2015-05-27

Summer, summer, summertime

Contribute  —  

Share this

•  E-mail
•  Facebook
•  Twitter
•  LinkedIn
•  Reddit
•  Digg

By Serendipodous

As usual for the time of year, pop culture rules this week. The start of summer vacation in the US means a focus on summer movies, particularly blockbuster sequels Avengers: Age of Ultron, Pitch Perfect 2 and Mad Max: Fury Road, though the meaning of the term "summer movie" has become increasingly vague as blockbusters open earlier and earlier. Pop cultural competition the Eurovision Song Contest had a similar impact, as did summer TV shows Game of Thrones and The Flash.

For the full top-25 list, see WP:TOP25. See this section for an explanation of any exclusions. For a list of the most edited articles of the week, see here.

As prepared by Serendipodous, for the week of May 17 to 23, 2015, the ten most popular articles on Wikipedia, as determined from the report of the most viewed pages, were:

Rank	Article	Class	Views	Image	Notes
1	Mad Max: Fury Road		1,515,128		Numbers are up 35% for this action film starring Tom Hardy in the title role, which debuted on Australia on May 14 and in the United States the next day. Despite topping this list, and receiving rapturous reviews (it currently stands at 98% at Rotten Tomatoes), it has not managed to claim the top spot at the box office, being hobbled by Pitch Perfect 2 and then Tomorrowland. But while box office is a big mover of Wikipedia views, controversy is the Bagger 288, and when a mildly antediluvian men's rights activist from the modestly named website returnofkings.com declared that no one should see "Mad Max: Feminist Road", the media smelled some blood and went into a predictable spin. While he admits he hasn't actually, well, seen the film, the guy's gripes are apparently that Max doesn't speak in the trailers, while Charlize Theron does (remind me, how many lines did Max have in The Road Warrior?) and that Theron "Barks orders at Mad Max. Nobody barks orders at Mad Max". I guess he must have missed Tina Turner's immortal line in Beyond Thunderdome, "You can shovel ****, can't you?". Anyway, the only apparent result of this "controversy" is that more women have gone to see the film, which should give it the legs to recover its $150 million budget.
2	David Letterman		879,202		The American talk show host finally retired this week after 33 years on late night television, first on NBC's Late Night with David Letterman and then CBS's The Late Show with David Letterman. In the decades following his defeat in the brief war of succession for the throne of Johnny Carson, Letterman may not have achieved the ratings of the eventual victor, his onetime friend Jay Leno, but was always seen by opinion makers as the true heir to Carson's legacy, not least by Carson himself. While Leno contented himself with puerile caricatures and flat one-liners, Letterman was prickly, genuine, and fiercely topical, often barely concealing disdain for guests he didn't like. His humour bordered on the surreal; sometimes he would blow up cabs or drop televisions out of windows for no reason, while another time he turned two local souvenir shop owners named Mujibur and Sirajul into overnight celebrities just, well, because. With Leno and Letterman passing into the night, the battle has passed to the next generation, with each handing their respective chairs to Jimmy Fallon and Stephen Colbert.
3	Mad Max		815,965		The 1979 film starring Mel Gibson that started the absurdly influential Mad Max franchise, now resurrected after three decades in hibernation.
4	Eurovision Song Contest 2015		752,700		Numbers have almost doubled for this year's event, probably due to its being the diamond jubilee of this most peculiar of international competitions. Politics threatened to rear its ugly head again after it seemed likely that Russia would win the popular vote, and thus the dubious privilege of hosting next year. Thankfully the prospect of one of the most gay-friendly events in the world being held in a country with stringent anti-gay laws was ultimately undone by eventual winner, Sweden.
5	Avengers: Age of Ultron		717,191		The latest instalment in the Marvel Cinematic Universe crossed the banner $1 billion worldwide mark this week.
6	Game of Thrones (season 5)		703,082		Numbers are up this week, which isn't surprising, since the latest episode, "Unbowed, Unbent, Unbroken" featured certified sadist Ramsay Bolton raping virginal Sansa Stark to the rage of a number of viewers, particularly the feminist blog "The Mary Sue", who stated they would no longer review the show.
7	The Flash (2014 TV series)		656,073		This spinoff from the hit series Arrow marks DC Comics' second attempt to create a TV universe, after the late and much lamented DC Animated Universe. Numbers are up this week for the season finale, which aired on May 19.
8	Stephen Curry		636,375		On May 22, during a Western Conference Finals game against the Houston Rockets, the basketball player for the Golden State Warriors broke the record for the most three-point shots in a playoffs, in just 13 games.
9	Payback (2015)	Future	632,394		WWE's latest pay-per-view pantomime was held on May 17 at the Royal Farms Arena in Baltimore, Maryland.
10	Pitch Perfect 2		424,773		This American musical comedy film, and sequel to 2012's Pitch Perfect which generated the improbable hit song "Cups", debuted in North America May 15. Starring Anna Kendrick (pictured), it has grossed $187 million worldwide as of May 27 on a budget of just $29 million, only a fifth that of Mad Max: Fury Road. Interestingly, while that film has suddenly become lauded for its feminism, this one, which is geared specifically towards women and has consistently beat it at the box office so far, has been criticised for fat jokes, racial stereotypes and other easy, regressive forms of humour.

Reader comments

2015-05-27

MediaWiki blows up printers

Contribute  —  

Share this

•  E-mail
•  Facebook
•  Twitter
•  LinkedIn
•  Reddit
•  Digg

By Legoktm

...allegedly. In a post to wikitech-l, Steven Walling pointed out that the TV show CSI: Cyber had used a screenshot of MediaWiki's HTML output and claimed it was responsible for blowing up printers.

In other related news:

MediaWiki 1.25 released

MediaWiki 1.25 was released (announcement) during the 2015 Lyon Hackathon. It features numerous changes that have been live on Wikimedia sites for a few months, including bringing live preview out of beta, numerous API format and documentation improvements, and major performance improvements in the backend and frontend.

Deployment cadence is changing

Greg Grossmeier announced on wikitech-l that the deployment cadence will be changing to make it less confusing for users and faster for developers to get new code to Wikipedias. The new schedule is:

• Tuesday: New branch cut, deployed to "group0" wikis (test wikis and mediawiki.org)
• Wednesday: deployed to non-Wikipedias
• Thursday: deployed to Wikipedias

It will take effect starting with the 1.26wmf9 branch (June 9th).

Developers meet at Lyon hackathon

MediaWiki and Wikimedia developers, users and more met up in Lyon during the 2015 Wikimedia Hackathon. There were various meetings, sprints, and random projects worked on over the weekend. The author is still jetlagged from this event, so you can read more about what different projects were presented at the showcase at phab:T96378.

Reader comments

If articles have been updated, you may need to refresh the single-page edition.

Keep up with The Signpost on Twitter, Facebook or Mastodon.

Home

About

Archives

Newsroom

Subscribe

Suggestions