The Signpost

Opinion essay

Wikipedia in Academe – and vice versa

Mike Christie has been editing Wikipedia since he was stuck in a hotel in South Korea with jetlag in early 2006. He works primarily on featured articles, and has been involved for about a year with WMF initiatives related to education. He attended the Wikipedia in Higher Education Summit in Boston in July 2011. He would like to thank all the editors who gave helpful feedback on earlier versions of this article.

The views expressed are those of the author only, and do not necessarily represent those of The Signpost or its staff. Responses and critical commentary are invited in the comments section. The Signpost welcomes proposals for op-eds. If you have one in mind, please leave a message at the opinion desk.


What we're good at – and what we're not

United States Education Program campus ambassadors at Georgetown University, Washington D.C. posing with a repurposed quotation of philosopher Pierre Teilhard de Chardin.

Wikipedia has come a long way by relying on self-organized volunteer labour. It's hard for most current editors to imagine what it was like years ago, when pages like Neurology consisted of just two sentences, and browsing the encyclopedia frequently led you to articles so weak they could be improved in a few seconds without sources, just by typing in information you knew off the top of your head. There's still an enormous amount of work to do, but it's now rarer to randomly stumble across articles that can be improved without significant research.[1]

One result of this change is that you can get a sense of what interests Wikipedians by looking at which areas have good coverage. It's not just video games and military history; there are specialists – some professionals in their field and some who are pursuing a hobby interest – who focus on highly detailed areas, such as Banksia, lemurs, or early English ecclesiastical history. You can also get an idea of what doesn't particularly interest Wikipedians by looking to see what areas are poorly covered: psychology, for example, or major topics in art.[2]

We'd all like to see Wikipedia's coverage improve in the areas where it's weakest, but we're all volunteers. Not everyone focuses on content; and those who do generally have areas of expertise. With standards for sourcing higher than they used to be, it's no longer enough just to be interested; you also have to have access to good sources. For topics such as Architecture, or History of Sweden, it's not even enough to have good sources; to do the job right you have to have a truly broad understanding of the literature – which sources are fundamental to the subject, which theories are regarded as mainstream, the ideas that need to be covered in the main articles, and what can be relegated to subarticles or ignored completely. In other words, there are many articles that a non-expert simply can't improve above a certain point. This doesn't appear to be a problem caused by poor editor retention. It's certainly true that we need to do a better job of keeping new editors engaged, but simply increasing editor participation won't automatically focus attention on underserved areas.[3]

Wikipedia needs academics

How can we attract experts in these topics to edit Wikipedia? A professor of physics who spends time improving an article on weak isospin isn't going to get a publication credit that will help his or her career. And Wikipedians don't respond well to a display of credentials; an editor who says "I'm a professor and I know this is right" is going to be met with "show me the citation", which they may find hostile, particularly when the fact in question is something they regard as so basic as to need no evidence. On the other hand, a dispute may arise not because a reviewing editor ignores credentials but because they lack expertise: broad statements can be hard to source, and it may be that only an expert can judge accurately whether a source fully supports a given assertion.[4] These are real concerns, but despite this, some academic groups are already beginning to take the initiative in improving these topics. The Association for Psychological Science (APS), for example, recently launched an initiative to ensure, among other things, that Wikipedia's "articles about psychological research and theory are accurate, up-to-date [and] complete".[5] The American Sociological Association included a "Call to Duty" in their November 2011 newsletter, with two goals: "first, to improve the sociology entries in Wikipedia by making it easier for sociologists to become involved in writing and editing them, and second, to facilitate professors giving Wikipedia-writing assignments to students in their courses”.[6] This is a promising start, but it would be sad if the experts who might arrive as a result of these initiatives are driven away, as many new editors are driven away, by the difficulty of learning how to engage with Wikipedia.

Academic and Wikipedia editor Jon Beasley-Murray (Jbmurray), an early pioneer of encouraging engagement between Wikipedia and higher education who documented his experiences in the essay "Was introducing Wikipedia to the classroom an act of madness leading only to mayhem if not murder?", and co-authored the paper "Opening up the academy with Wikipedia" with fellow academic Wikipedians

In addition to proposals such as the APS's initiative, there is an ongoing project by the Wikimedia Foundation to get academics to engage with Wikipedia – the Global Education Program (GEP). This grew out of a project called the Public Policy Initiative, in which academics in public policy programs were recruited to use Wikipedia as a teaching tool – they asked their students to improve Wikipedia articles in the area of public policy. Existing editors were recruited to help the students, and the result was a success – the students added nearly nine million bytes of data to the encyclopedia,[7] and a statistical analysis of the quality of the added material showed that the average article worked on improved dramatically.[8] The next step was to expand beyond public policy: the USEP (the US branch of the Global Education Program) now recruits academics in all areas of study, and matches them with Wikipedians who are willing to act as ambassadors, to teach and guide the students and instructors. There has been a good deal of interest in the program, and the fall 2011 semester saw 77 classes signed up as part of the USEP – so many, in fact, that it's straining the ability of the limited number of volunteer ambassadors to effectively assist the students.[9] In addition to the USEP, there is a signup page for school programs that did not form via the USEP that lists another 55 classes, and there are also active classes that do not declare themselves at either page.

One of the goals of the USEP is to recruit new editors,[10] but I would be very surprised if it's turning out to be a cost-effective recruitment method. The students edit because they have to, so the retention rate is abysmal. We do have some successes at turning students into editors, but they are rare – too rare to justify the significant resources that the WMF and the community are putting into the program. It could also be argued that the use of Wikipedia as a teaching tool is a contribution to the public good, and a demonstration of the value of Wikipedia. Both these things are true, but neither point should be allowed to preempt the primary goal of improving our encyclopedic content. I believe the USEP's significance lies instead in the fact that it is the first time that there has been an attractive way for academics to engage with Wikipedia. It is the academics who can make the investment in the USEP worthwhile for the encyclopedia.

The future of the education programs

Right now, the US program is focused on scaling: recruiting large numbers of classes, increasing ambassador recruitment to support those classes, and training campus support staff to teach students and instructors as much as possible about editing so that they don't cause more harm than good. I want to be very clear that "more harm than good" is not a theoretical problem; the Indian EP (which has the potential to be an even bigger asset to the encyclopedia than the USEP) required a great deal of cleanup work, as reported recently in the Signpost, and it is easy to find editors complaining about individual students or classes making a mess of articles this past semester. This sort of impact on the community is only worthwhile if we are gaining significantly from the program – and the biggest return on investment we can get here is the academics, not the students. In fact, the current focus on growth is actively harmful – as supervision by ambassadors becomes more and more stretched, we will have thousands of students contributing to Wikipedia without truly understanding it. The burden of coping with this will fall on new page patrollers, recent changes patrollers, and the editors who already have the students' articles on their watchlists. Even initiatives such as the Association for Psychological Science's call to action are likely to end up using the USEP approach, rather than having the professors edit directly, partly because the template for using Wikipedia in the classroom exists and is being widely publicized.[11] There are other education programs for the English-language Wikipedia, such as the Canadian program, and as far as I can tell the WMF regards growth as a high-priority goal for all such programs.

The USEP, and any other similar programs, need to change their focus. Instead of trying to scale the program to the largest possible number of classes, we should be focusing all our resources on working with academics who are receptive to the idea of becoming editors. We can provide a good deal of support to their classes, but in return we should be asking for them to participate, as part of the Wikipedia editing community, in curating the articles in their specialities. Writing a good article or featured article on Wikipedia requires more than knowing a topic inside out; you have to understand what the experienced editors here understand about what makes a good encyclopedia article. We have that, and the academics have knowledge – and now, finally, thanks to the global education program, we have something they want – an innovative and productive way for them to teach their students. Any instructor who gets a couple of good articles under their belt will understand Wikipedia well enough that they will be a much better instructor for their students, and will in turn reduce the burden for the ambassadors supporting their class. We need more subject matter experts, and we need the education programs in the US and elsewhere to help us acquire them.

Global Education Program Director Frank Schulenburg speaking on the topic of "Wikipedia in higher education: a new model for teaching" at Wikipedia meets Antiquity, a Wikipedia Academy held at the University of Göttingen in June 2011

How do we do this? Here's my prescription.

  1. The community needs to engage with the Global Education Program, and particularly with the USEP, the largest of the country EPs. These programs are largely driven off-wiki, since they are WMF operations rather than community initiatives. Two key WMF staff members are Frank Schulenburg, the Global Education Program Director, and Annie Lin, the Global Education Program Manager. I suspect the best place to discuss these programs with the WMF is the USEP talk page.
  2. The quality metrics which were put in place for the Public Policy Initiative need to be restarted for the USEP and other programs. The USEP will have a cost to the community in terms of support, and we need to understand the benefit to article quality to be sure that it's worthwhile. Currently metrics for quantity exist, such as pages that measure student editing statistics; these aren't inherently harmful but anyone who has spent time reverting poor student contributions this semester will appreciate that a quality metric is at least as important.[12]
  3. The USEP should change its focus in two ways: from growth to quality, and from students to academics.
    • Growth without support will lead to stress on the community and poorer quality. Instead we should work out what it means to support a class properly – there are far too few active online ambassadors to properly review and help with the current class load; the ratio currently seems to be about one ambassador to two classes, which means many students will never hear from an ambassador at all.
    • Students will disappear after a single year; academics, if we help them, will return year after year. Any academic can run a class using Wikipedia, with or without formal involvement with the USEP, but the USEP should focus its resources on instructors who show willingness to engage with editors on Wikipedia; who participate in reviewing and managing their own students' contributions; and most of all who show interest in working on articles themselves. In return we can supply ambassadors as teaching assistants – mentors, in Wikipedia terminology.

If we manage the influx of academic interest correctly, Wikipedia will acquire an institutional connection to academia that will be a source of new content for our articles and an intellectual resource to assist with long-term growth. Wikipedia does not need to beg for respectability any more; it is already widely used by academics as a starting point for research, and sometimes for more than that.[13] We need to accept our respectability, and plan to learn from – and teach – the academic community.

Notes

  1. ^ I'm talking about content here – it's easy to find articles that need copyediting or formatting.
  2. ^ Recent quotes by editors at the Featured Article Candidates talk page include "the quality across the board in psychology is horrific ... most of the psych articles we could consider "important" on any scale in the psych realm are utterly dismal." and "in my area of the visual arts ... most articles on major topics are crap (Indian art, Italian Renaissance sculpture, in fact anything to do with sculpture, Baroque, Rococo, Romantic art etc etc)" (diff). For the psychology articles you can get an idea of the assessed quality from the Psychology articles quality chart.
  3. ^ See the Editor Trends Study for more background on editor retention.
  4. ^ For example, a Wikipedian who is a professional neuroscientist recently commented that "For topics with a large literature, where most of the statements synthesize dozens if not hundreds of sources ... it is simply futile to demand that readers with no subject matter knowledge be able to verify articles on large topics – only an expert reviewer can properly do it. I seek [...] an acknowledgement that referencing requirements should be tuned to the breadth of a topic -- the larger a topic, the lesser the need for detailed page-referencing of every line, and the greater the need for reviewers with enough expertise to have a good sense of whether an article is accurate and comprehensive without having to consult sources regarding every line."
  5. ^ See the APS's Wikipedia Initiative page.
  6. ^ See the November 2011 issue of Footnotes.
  7. ^ See the wrap-up blog entry for details.
  8. ^ See the statistical analysis of the PPI for details; you may also be interested in looking at the assessment rubric or an example assessment page.
  9. ^ See, for example, some extended discussion on this topic at the USEP talk page.
  10. ^ See the "increase participation" goal at the strategy wiki; that page is linked from one of the USEP project pages.
  11. ^ See this response to the APS initiative for an example of an instructor's response. The drive to expand the USEP was visible at the Wikipedia in Higher Education Summit, held this summer in Boston.
  12. ^ The PPI quality metric was quite labour-intensive, which may make it more difficult to implement. The Foundation does acknowledge that quantity metrics are inadequate.
  13. ^ Steve Kolowich at Inside Higher Ed recently remarked on the use of text plucked from a Wikipedia article in a symposium paper by the president of the American Sociological Association.

















Wikipedia:Wikipedia Signpost/2011-12-12/Opinion_essay