The Signpost
Single-page Edition
WP:POST/1
29 October 2012

News and notes
First chickens come home to roost for FDC funding applicants; WMF board discusses governance issues and scope of programs
WikiProject report
In recognition of... WikiProject Military History
Technology report
Improved video support imminent and Wikidata.org live
Featured content
On the road again
Recent research
WP governance informal; community as social network; efficiency of recruitment and content production; Rorschach news
 

2012-10-29

First chickens come home to roost for FDC funding applicants; WMF board discusses governance issues and scope of programs

FDC staff publish initial judgements on applications

The first round of the Wikimedia Foundation's new financial arrangements has proceeded as planned, with the publication of scores and feedback by Funds Dissemination Committee (FDC) staff on applications for funding by 11 entities—10 chapters, independent membership organisations supporting the WMF's mission in different countries, and the foundation itself. The results are preliminary assessments that will soon be put to the FDC's seven voting members and two non-voting board representatives. The FDC in turn will send its recommendations to the board of trustees on 15 November, which will announce its decision by 15 December. Funding applications have been on-wiki since 1 October, and the talk pages of applications were open for community comment and discussion from 2 to 22 October, though apart from queries by FDC staff, there was little activity.

A total of US$10.4M was requested in round 1—almost the entire FDC budget for the first two six-monthly rounds. Figure 1 shows the requests. Wikimedia Deutschland, by far the largest chapter, topped the nation-based requests with $1.8M, followed by Wikimedia France at $1.0M, and Wikimedia UK at $0.9M.


Fig. 1—Funding requests in millions of US dollars (the foundation's $4.5M bid is excluded for better scaling).


Criteria

A key part of the foundation's new financial arrangements involves the encouragement of good governance and transparency in the chapters, which are authorised to use the WMF's trademarks and to an increasing extent share responsibility for upholding the foundation's international reputation. The scores and accompanying feedback for round 1 are the first taste of just how this might play out as the FDC process evolves. The three staff members—Meera, a consultant at the Bridgespan Group, which is assisting in the setting up of the FDC; Anasuya Sengupta, director of global learning and grantmaking for the WMF; and Winifred Olliff, grants administrator for the WMF—provided this information for 13 criteria (labelled A–M). Their scores were based on a 1–5 Likert-scale: the minimum score of 1 indicated "weak or no alignment with the criterion"; 3 indicated "moderate alignment"; and the maximum of 5 indicated "strong alignment". Figure 2 below shows the average score given for each of the 13 criteria. The graph spans the minimum to maximum scores, and the bars are colour-coded for the five dimensions:


Fig. 2—Average scores handed out by the staff for each criterion, grouped by colour into the five dimensions. Poor, moderate, and strong alignments are marked by the dotted red lines.


Impact (A, B, and C; blue)

Not surprisingly, criterion A gained a relatively high average score of 4.0, since it primarily measures the lip-service paid to the movement's global targets in the application form. A good case for B is more challenging to make, at it involves the potential to fulfil the targets; predictably, this scored a lower average of 3.6. Criterion C is a judgement of likely impact on the ground, which is harder to convincingly argue. C had an average of 3.4, approaching the "moderate" threshold. Germany and the WMF itself each received the best scores for the three impact criteria: 5, 5, and 4 respectively. Also highly rated on likely impact were Argentina, the Netherlands, and Austria.

Ability to execute (D, E, and F; red)

Criterion D mainly concerns the human resources, skills, and capacity, and E the entity's record of achievement. These both scored an average of 3.3, with two entities marked down for their past records by comparison with their current capacities: Sweden ("a track record of success in many initiatives ... , but new initiatives are on the table which are more uncertain"), and the foundation more obviously, with a 5 for current capacity, and only 3 for past record ("WMF and the programs in question are well staffed with diverse capabilities", but "These initiatives have had mixed results."). Poor scorers were Australia (2 for both criteria, with concerns about challenges in hiring staff and a lack of clear indicators of past project performance) and Switzerland ("Significant concerns about entity's ability to execute on a plan of this magnitude, given staff and volunteer capacity").

On the whole, the leadership of the entities was rated slightly better, at 3.5—midway between moderate and strong. But there were two outliers: there was a complaint that the board of Wikimedia France "has communicated challenges around leadership and governance which may affect the entity's ability to execute successfully on the plan in the short term." And for Australia, stability of leadership "over the past few years" was an issue, as was the response "through the beginning of the FDC process". Leadership in Austria and the UK was rated in "moderate" alignment (3), and for the rest, moderate to high (4).

Efficiency of funding (G and H; yellow)

In terms of whether the funding request was reasonable given the proposed initiatives (G), "some initiatives are likely to be over- or under-budgeted" for seven entities (including the foundation and Germany), and for France, with a 2, the judgement was: "significant over- or under-budgeting, and/or has not thoroughly accounted for costs". On track-records of using funds efficiently, Australia was hammered for "underspending and financial conservatism", and Hungary for underspending "because of lack of execution and because of overestimation of costs."

Benchmarks for success (I, J, and K; green)

This drew the lowest average scores, with moderate or less than moderate alignment. France, Israel, Hungary, and Australia scored poorly. The benchmarking assessments may be a wake-up message for chapters to examine their procedures for benchmarking, with complaints about lack of detail common in the assessments. Curiously, one chapter scored higher on having a "feasible" plan to track the proposed metrics (K) than in having "a plan to track the proposed metrics" (J).

Benefits for the broader movement (L and M; purple)

The entities were ranked a disappointing average of 3.3 for criterion L (ability to replicate the plan elsewhere in the movement), suggesting that this aspect needs more attention when plans are conceived. In a criterion rather subtly distinct from this, criterion M, the potential of plans to bring benefits to the movement, gained an average of almost 4; the foundation scored a 5, and 4s were handed out for all but two of the remaining 10 entities.


Fig. 3—The Signpost has produced this graph to show the total scores of the entities (in parentheses, x-axis labels), though this measure should be interpreted cautiously since it reduces all 13 criteria to a level weighting. The colour-coding by dimension is the same as in Fig. 2.


The queries by the three FDC-related staff members on application talk pages, and the scores and feedback provided to the FDC, suggest that insufficient detail was provided by some of the entities for some criteria. The staff have flagged the ability to execute plans, the efficiency of spending, and measures of success as areas that need the greatest improvement by applicants. The FDC will consider the staff input when it meets to decide on what to recommend to the board of trustees on 15 November.

WMF board meeting

Between 24 and 27 October, the WMF board of trustees met in San Francisco for a retreat and several meetings to discuss the foundation's programmatic scope and to decide on a number of governance issues.

The board approved the legal fees assistance program, which the community supported in a RfC process in September. The program is designed to safeguard editors threatened legally by third parties due to fulfilling their Wikimedia community governance roles such as adminship or ArbCom membership.

The trustees approved resolutions concerning the conduct of two board committees. The audit committee was reformed to reflect the reforms of WIkimedia's financial structure, and the human resources committee charter was approved, after the board voted through the language committee's framework pre-meeting on 16 October. The changes to the institutional frameworks have been accompanied by new office appointments across board committees.

Matt Halprin, who has served since 2009 as one of the four board-appointed trustees, will not stand for re-appointment at the end of his current term in December 2012. He has driven governance reforms such as the board governance committee, organizing the annual evaluation of trustee performance and the board's working procedures, and making the voting of trustees on board resolutions publicly transparent.

The board also looked at the proposals by Sue Gardner, the foundation's executive director, to strengthen the focus of the WMF and put more resources into core projects such as the visual editor, tackling the switch from desktop to mobile devices among readers and editors, and editor retention.

In brief

Matt Halprin at WMF board session, Berlin, March 2012

2012-10-29

In recognition of... WikiProject Military History

WikiProject news
News in brief
Submit your project's news and announcements for next week's WikiProject Report at the Signpost's WikiProject Desk.
WikiChevrons
WikiChevrons with Oak Leaves
WikiChevrons (left) are the project's standard barnstar, while the project's coordinators can award modified WikiChevrons (right) based on recommendations by project members
Editors who bring three or more articles to A-class status receive a medal
For editors who build a large number of A-class articles, the medal can be adorned with oak leaves, swords (pictured), and diamonds
Barnstars exist for some special topics, like this award for contributing to content about the Napoleonic Era
Service awards are awarded for specific project tasks and drives
The "Military Historian of the Year" award, a modification of the standard Wiki Award, recognizes editors who have contributed the most to Wikipedia's coverage of military history during a given year

This week, we're checking out ways to motivate editors and recognize valuable contributions by focusing on the awards and rewards of WikiProject Military History. Anyone unfamiliar with WikiProject Military History is encouraged to start at the report's first article about the project and make your way forward. While many WikiProjects provide a barnstar that can be awarded to helpful contributors, WikiProject Military History has gone a step further by creating a variety of awards with different criteria ranging from the all-purpose WikiChevrons to rewards for participating in drives and improving special topics to medals for improving articles up to A-class status to the coveted "Military Historian of the Year" award. We interviewed Grandiose, secretlondon, Nick-D, Ian Rose, Crisco 1492, Marcus Qwertyus, Hchc2009, and Dank.

How long have you been a member of WikiProject Military History? Do you prefer working on articles related to particular subjects, people, or time periods?

Grandiose: In terms of content creation, the Spanish Civil War has been my pet project and has produced my only featured article (Nyon Conference) and most of my good articles. I was drawn to it as a topic area – I'd never before had an interest in Spanish history or the inter-war period in particular. My other contributions have been diverse – including Henry VIII of England and the Livonian War. I joined the project in April 2011, although I'd been contributing in the topic area since the beginning of the year. I was made a Military History co-ordinator in September 2012.
secretlondon: I thought I should join as I've been writing on this topic. However I've not been collaborating with anyone through this nor has it had any impact, really.
Nick-D: I've been a member of the project since 2006. I mainly work on topics relating to World War II and the Military history of Australia, both of which are well represented on Wikipedia. I also work on articles on articles which cover modern militaries, which unfortunately still vary widely in coverage and quality (there's lots of opportunities for editors who would like to take on a big topic to work on getting these articles up to scratch).
Ian Rose: I've been a member of the project for about five years, and my main focus is Australian military aviation, particularly biographies.
Crisco 1492: I've been writing content on military topics for about a year and a half, but I just became a formal (registered) member last month. Most of my military articles are related to Indonesia, especially its national revolution.
Marcus Qwertyus: I started my career as a MilHist contributor in March 2009. I compensated for my total lack of knowledge of the topic by only making WikiGnomish edits like adding infoboxes to armored vehicle articles. As I got more invested in the project, I took charge of documenting the Pentagon's various ambitious and rather ignominious attempts to develop and procure a modern ground combat vehicle fleet for the U.S. Army. The most recent program, called the Ground Combat Vehicle, birthed DARPA's crowdsourced Fast Adaptable Next-Generation Ground Vehicle program which will be a contest for the masses to design and eventually build an amphibious combat vehicle for the U.S. Marine Corps. This project may even establish crowdsourcing as the solution to the military's procurement woes if all works out well. I hope to have time to participate when the contest opens up this January.
Hchc2009: I've been a member for a couple of years - I think I first got involved in late 2009. I tend to focus on medieval and early-modern topics. Working on content is heavily dependent on sourcing, and I can only fit so many books in my house... :)

Tell us about the project's contest department. When did it start and did you take inspiration from any other WikiProjects? How are contributors rewarded for participating in the monthly contests? Has it actually helped motivate the project's members to improve articles?

Ian Rose: The contest has been running for around five years. I wasn't involved in the initial set-up but I gather that part of the inspiration came from the LGBT wikiproject. Originally points were only awarded for MilHist-centric article assessments, i.e. B-Class and A-Class, however a while back I suggested that we adopt a broader scoring base and award points for GA and FA assessments as well. This has been the norm ever since, except we also include C-Class assessments since Milhist adopted that assessment level. The highest point-scorer each month is awarded the WikiChevrons, and the runner-up the Writer's Barnstar. While it's a little difficult to quantify exactly how much the contest motivates people, the number of entrants and entries continues to remain fairly constant after all these years, so it certainly seems to have its place. Personally speaking I always like to get at least one entry in each month, so I know it motivates me... ;-)
Grandiose: I don't take part, because of the bureaucratic overhead to my contributions. However whilst the number of contributors isn't very high it doesn't take many to make something like that successful. If it it motivates even two or three people to go out when they didn't before, it's probably worth it.

The project also hands out "WikiChevrons", service awards, and a variety of other goodies throughout the year. Please describe some of these awards and why the project started offering them. Are new recipients of the awards honored in the project's newsletter or in any other ways?

secretlondon I got a chevron, which was nice. It's a bit like a barnstar. Someone also awarded me a red army badge which was nice too.
Nick-D The project's coordinators award medals for editors who develop articles to A-class status (an A-class medal is awarded for every three successful nominations, with a sliding scale of medals to acknowledge editors with an excellent track record) and medals to recognise the contributions of people who review A-class nominations and editors with a long-running history of outstanding contributions. There's also an annual military historian of the year contest (awarded through a popular vote) and editors can award one another project-specific medals as they see fit. The project's monthly newsletter (The Bugle) reports on all the project-level awards. I think that this has been successful in motivating contributions to articles within the project's scope, as well as encouraging participation in project-level activities.
Ian Rose: In addition to what Nick has covered above, there's the WikiChevrons with Oak Leaves, which is the project's highest mark of esteem for an editor. It cannot be self-nominated, and must be approved by a majority of project coordinators (who themselves are not eligible for the award while holding office).
Marcus Qwertyus: I have never been motivated by the prospect of receiving a chevron or other prestigious award. I have been awarded one military-themed barnstar in my Wikipedia career. This little bit of micro-gratitude has been enough to show that I am a valued contributor to the project. I think every editor that has invested a considerable amount of time in the pursuit of improving a WikiProject's coverage should be shown some appreciation in the form of at least one barnstar.
Hchc2009: To me, the awards are one way (amongst others) of recognising the wide range of activities that go into creating high quality articles, especially reviewing. Across the wiki we often overlook these, and the awards are a formal way of the community saying "thanks" to those involved.

Has WikiProject Military History hosted any backlog elimination drives like the Guild of Copyeditors and WikiProject Wikify? How do these backlog drives compare to the four ongoing "Operations" hosted by WikiProject Military History? What are the benefits and limitations to these kinds of goal-oriented activities?

Grandiose: Although the project keeps track of clean-up requirements, there have been few backlog drives directed at them. The one big exception is the B-Class assessment backlog – the project's attempt to assess all start-class articles against our B-class criteria, which will help identify what needs doing and whether any is C-class – which, spearheaded by Adam, has reduced outstanding articles from 27,000 to around 22,000. Quite an achievement, even if that figure has proven stubborn. The Operations vary in their success – there is no doubting the success of Operation Majestic Titan, but ultimately that's because there's a group of editors determined to work on that area, fancy name or not. The value in the others (like Operation Normandy) is in keeping track of where we are. Perhaps as the deadlines tighten the organisational element might come to the fore, but we'll have to see. I don't see any downsides; arguably other areas might lose out but that's the nature of a volunteer-based encyclopedia.

The project's large assortment of awards is likely related to the sheer size of WikiProject Military History's community. Would contests, awards, and drives like these be feasible for smaller projects? What are the best ways to motivate contributors regardless of a project's size and scope?

Ian Rose: I think awards, contests and drives should generally help motivate people, regardless of a project's size. Given that a smaller project has templates such as MilHist to draw from, there seems little to lose by trying...
Crisco 1492: Theoretically, but in reality some projects may never be able to handle it. My main project (my wife, if you will) is WikiProject Indonesia, and the project now consists of essentially four or five editors. A formal system of rewards and contests with so few editors would essentially be rearranging the deck chairs on the Titanic: we'd be so busy with administration that we'd never actually write the articles. A fairly large WikiProject, or one with a more general scope, may have more success.
secretlondon: DYK has been my motivation, together with scoping out what needed to be written and reducing the number of red links.

Anything else you'd like to add?

Crisco 1492: One of the main reasons for my continuing participation in the WikiProject's internal review program is the recognition: one cannot expect an editor to write 6000 words on a soldier / general and not hope for some general recognition. It doesn't have to be anything big, just a way of saying that what we do is appreciated. Carrots, not sticks: that's the way to retain editors.
Dank: Writers stop writing if readers stop reading. Although a half a billion people read Wikipedia each month, that attention isn't enough by itself to keep writers from losing interest, in my experience ... but knowing that people care enough about what you're producing to review it and give you feedback that you find useful generally is enough, at least for a while.


Next week, we'll sing for our supper. Until then, listen to the advice in our previous reports.

Reader comments

2012-10-29

Improved video support imminent and Wikidata.org live

TimedMediaHandler coming this week

A 2011 video featuring Jimmy Wales. Depending on when you read this, you'll either see the old video player or the new one, instantly recognisable by way of the large play button in the middle of the video.

The TimedMediaHandler extension (TMH), which brings dramatic improvements to MediaWiki's video handling capabilities, will go live to the English Wikipedia this week following a long and turbulent development, WMF Director of Platform Engineering Rob Lanphier announced on Monday (and later clarified).

The extension, which has been under development for the best part of two years, will introduce a new interface with subtitle support and a simple "|start=2.3|end=5" syntax for extracting video segments. Other features listed include "multi-format multi-bitrate transcoding with auto source selection, ... gallery and search pop-up players, viral iframe sharing / embedding, etc." although it is unclear how many of these will be available at launch. Deployments to other wikis and Commons have been pencilled in for the coming fortnight.

Readers with longer memories will note that some of these features have already surfaced in the 2009-released mwEmbed gadget, the development of which preceded work on TimedMediaHandler. The two have in common a number of features, most notably their choice of default interface – "Kaltura" video player, which was developed starting in January 2008 with assistance from the for-profit company of the same name and demoed at Wikimania 2009 (see contemporary Signpost coverage). The development path since mwEmbed has focussed on performance, security and code review concerns. Accompanying work has focussed on serving video more efficiently and it is likely that any TMH deployment will also make the possibility of accepting a larger number of video input formats a more attractive option to Wikimedia decision makers.

The deployment, should it go according to plan, is likely to be warmly welcomed by developers and readers alike. Among more seasoned developers, however, the smiles will surely be borne less from joy and more from relief that a project spanning four-and-a-half years of legal concerns, technical debates over code quality, endless technical delays and an uncertain payment structure has finally come to fruition.

In brief

Signpost poll
Wikivoyage
The Wikivoyage migration: the priority is that... (a) ...we migrate it perfectly (32%); (b) ...we migrate it quickly (19%); (c) ...we migrate it in the timescale and with the quality we said we would (41%); (d) other (8%)
You can now give your opinion on next week's poll: In my opinion, videos are...?

Not all fixes may have gone live to WMF sites at the time of writing; some may not be scheduled to go live for several weeks.

  • Wikidata.org goes live: Wikidata.org (interface in English), a new repository designed to host interwiki links, launched this week and will begin accepting links shortly. The site, which is one half of the forthcoming Wikidata trial (the other half being the Wikidata client, which will be deployed to the Hungarian Wikipedia shortly) will also act as a testing area for phase 2 of Wikidata (centralised data storage). The longer term plan is for Wikidata.org to become a "Wikimedia Commons for data" as phases 2 and 3 (dynamic lists) are developed, project managers say.
  • IE8 users face 18 JavaScript-free hours: Visitors to the English Wikipedia using Microsoft Internet Explorer 8 (IE8) were left without any site or user JavaScript on Friday for approximately 18 hours after a bug was introduced by WMF developers working on editor engagement (bug #41428). Responding, WMF Quality Assurance Lead Chris McMahon described how the issue has "prompted a lot of discussion within WMF and we are changing some priorities and procedures as a result", including introducing more stopgap protection before a full beta cluster-based JavaScript testing suite comes online. IE8 accounts for roughly 13% of web traffic and 8% of current Wikipedia traffic. In related news, other English Wikipedia editors experienced similar problems due to a JavaScript problem in version 5 of the Article Feedback tool.
  • Windows 8 app released: Early adopters of the newly released Windows 8 operating system will be able to download a Wikipedia app from the Microsoft app store, it was announced this week (Wikimedia blog). The app, which integrates with system search, shares much of its infrastructure with the other WMF developed apps for the Android and iOS platforms but adopts a design style more in-keeping with the Windows 8 GUI. Peak Windows 8 usage is expected to lie in the hundreds of millions of devices, with the operating system being installed not only onto traditional desktop and laptop computers but also a rapidly increasing number of tablets and smartphones. In related news, the WMF also reported strong visitor growth in its Wikipedia Zero project, which aims to make accessing Wikipedia exempt from data charges in developing countries.
  • Understanding the international design process: In an insightful post on the Wikimedia blog, WMF User Interface/Experience (UI/UX) developers Arun Ganesh and Pau Giner described the way they conducted user testing for the Universal Language Selector (ULS) project, which helps users translate MediaWiki's interface and select their preferred script input method. The inherent need for the project to be tested on editors from languages other than English presented its own challenges, Ganesh and Giner write, describing how (for example) it prompted developers to adopt a digital form of traditional "paper prototype" that could hence be tested on users remotely. Relatedly, i18n specialist Niklas Laxtröm also blogged this week, describing the problem MediaWiki faces trying to properly localise interface messages involving variable numerical elements.
  • WMF hires: This week saw the announcement that English Wikipedia Bot Approvals Group member Brad Jorsch (User:Anomie) has been hired to work on "the robustness of the beta cluster" and help with code review (wikitech-l mailing list). New staff who have been accidentally omitted from the pages of the Signpost over the past month include Željko Filipin as a QA Engineer, Michelle Grover as a QA contractor for the Mobile Team and Andre Klapper in a bug oversight role similar to that of former bugmeister Mark Hershberger, who left the Foundation in May to pursue other interests. Jorsch has also been granted merge permissions over core, as have two other volunteers, bringing the number of volunteers with merge rights temporarily to eight (seven plus Jorsch).

    Reader comments

2012-10-29

On the road again

This edition covers content promoted between 21 and 27 October 2012
Two new road articles were promoted this week
Pierre Monteux
A frog (Australian Green Tree Frog)
The Cathedral Church of St Michael, a grade I listed building in Coventry
An SNCB Class 77, a new featured picture
Fogo island, a new featured picture
Mitra stictica shell, a new featured picture
Saint Francis in the Desert

Thirteen featured articles were promoted this week:

  • Pierre Monteux (nom) by Tim riley, Cg2p0B0u8m, and Brianboulton. Monteux (1875–1964) was a French (later American) conductor who began regularly conducting in 1907, later rising to prominence after directing several world premieres. During his fifty-year career Monteux led numerous orchestras in the US, France, and the UK; also teaching numerous students. Although he disliked recording, he produced numerous records. He preferred German works.
  • "Episode 14" (Twin Peaks) (nom) by Grapple X, Idiotchalk, and TBrandley. "Episode 14" is a 1990 episode of the American mystery television series Twin Peaks. Written by Mark Frost and directed by David Lynch, it follows FBI special agent Dale Cooper's investigation into the murder of a schoolgirl; this episode reveals who the killer, a demon, has possessed. It was well received.
  • Hoodwinked! (nom) by Jpcase. Hoodwinked! is a 2005 American computer-animated film. Inspired in part by the Japanese film Rashomon, it parodies the fairy tale genre by retelling the folktale Little Red Riding Hood as a police investigation. The independently-funded film had a small budget, which influenced the less-realistic animation. The animation was heavily panned upon the film's release, although the film's script and cast were widely praised.
  • "Missing My Baby" (nom) by AJona1992. "Missing My Baby" is a song from American recording artist Selena's third studio album, Entre a Mi Mundo (1992). Composed by her brother, the song was intended to help Selena enter English-language markets. It is a mid-tempo R&B ballad influenced by urban and soul music which describes the singer's love for a boy. The track, though never released as a single, charted in 1995.
  • Francis Marrash (nom) by Bryan P. C. C.. Marrash (1836–1873) was a Syrian writer and poet of the Nahda movement; most of his works dealt with science, history and religion. In his youth he travelled extensively before practicing medicine. While enrolled at a medical school in France he began losing his sight, leading him to write more frequently. Middle Eastern historian Matti Moosa considered Marrash to be the first truly cosmopolitan Arab intellectual and writer of modern times.
  • Frog (nom) by Cwmhiraeth and Thompsma. Frogs are a diverse and largely carnivorous group of amphibians which may date back to 265 million years ago. They are characterised by a stout body, protruding eyes, cleft tongue, limbs folded underneath and the absence of a tail. Frogs are a keystone group in the food web, becoming food to numerous species – including humans. Approximately 4,800 species are known, although some 120 are thought to have become extinct since the 1980s.
  • Interstate 696 (nom) by Imzadi1979. Interstate 696 is an auxiliary Interstate Highway in the US state of Michigan, detouring around the city of Detroit through the city's northern suburbs in Oakland and Macomb counties. Construction on the 28-mile (45 km) long highway began in 1961, taking 28 years to complete owing to various disagreements between municipalities, the City, and the Detroit Zoo over routing.
  • Interstate 80 in Iowa (nom) by Fredddie. Interstate 80, a transcontinental Interstate Highway in the US, traverses across the state of Iowa for 306 miles (492 km). It enters the state at the Missouri River, meets with – and later splits from – I-35, before eventually leaving Iowa over the Mississippi River. Construction of the interstate began in 1958 and took fourteen years. By the 1980s, it had fallen in disrepair, although funds had been allocated for repairs.
  • Folding@home (nom) by Jesse V.. Folding@home is a distributed computing project for disease research that simulates protein folding, computational drug design, and other types of molecular dynamics. The project is powered by the idle processing resources of thousands of volunteered personal computers. It assists with medical research into many diseases. Folding@home is one of the world's fastest computing systems. Since its launch in 2000, it has assisted numerous research papers.
  • Blakeney Point (nom) by Jimfbleak. Blakeney Point is a National Nature Reserve situated on the north coast of Norfolk, England. Its main feature is a 6.4 km (4 mi) spit of shingle and sand dunes, but the reserve also includes salt marshes, tidal mudflats and reclaimed farmland. The area has long been inhabited by humans, while it also serves as a major breeding spot for birds. The point is known for its birdwatching and sailing, although such activities may endanger the birds.
  • Barber coinage (nom) by Wehwalt. The Barber coinage consisted of a dime, quarter, and half dollar designed by US Bureau of the Mint Chief Engraver Charles Barber. Minted between 1892 and 1916, all depicted a head of Liberty, facing right. Meant to replace the Seated Liberty design, the Barber coins received mixed reception among the populace. Most remain relatively easy to find, although the 1894 dime struck at the San Francisco Mint is a great rarity.
  • Halo: Reach (nom) by David Fuchs. Halo: Reach is a 2010 first-person shooter video game developed by Bungie and published by Microsoft Game Studios for the Xbox 360 console. It takes place in the year 2552, where humanity is locked in a war with the alien Covenant; players control an elite soldier named Noble Six. Reach serves as a prequel to the Halo series and includes more somber music. The game was a critical and commercial success, garnishing US$200 million within a day of its launch.
  • God of War: Chains of Olympus (nom) by JDC808. God of War: Chains of Olympus is an action-adventure video game developed by Ready at Dawn and SCE Santa Monica Studio released in 2008. The fourth installment in the God of War series and a prequel to the original God of War, it follows the Spartan Kratos as he helps Greek gods. The game was a commercial and critical success, being called the best PlayStation Portable game by GamePro in 2010.

Ten featured lists were promoted this week:

  • Peter Sellers on stage, radio, screen and record (nom) by Schrocat and Cassianto. The British actor and comedian Peter Sellers (1925–1980) performed in many genres of light entertainment between 1948 and his death. He was known internationally for his role as Chief Inspector Clouseau in The Pink Panther film series.
  • List of SEPTA Regional Rail stations (nom) by Dream out loud. SEPTA Regional Rail, the commuter rail system serving Philadelphia, US, has 13 lines, with 153 active stations divided between seven fare zones. The system was built in the early 20th century.
  • Dick Howser Trophy (nom) by Bloom6132 and Muboshgu. The Dick Howser Trophy is bestowed annually to the US college baseball player of the year. It has been awarded annually since its establishment after Howser's death in 1987. The most recent winner is Mike Zunino.
  • List of films of the Dutch East Indies (nom) by Crisco 1492. A total of 106 feature films are known to have been produced in the Dutch East Indies (modern day Indonesia) between 1926 and the colony's dissolution in 1949, starting with Loetoeng Kasaroeng.
  • Grade I listed buildings in Coventry (nom) by HJ Mitchell. There are 19 Grade I listed buildings in the City of Coventry, dating from between 1043 to the 1950s. Among these is the former prison of Mary, Queen of Scots.
  • One Direction discography (nom) by AdabowtheSecond. British–Irish boy band One Direction has released a studio album, an extended play, seven singles, a video album and seven music videos. Their debut studio album was Up All Night, while another album is due in November.
  • Jessica Mauboy discography (nom) by Oz. Australian R&B recording artist Jessica Mauboy has released three albums, thirteen singles, and sixteen music videos. She made her debut in 2007 with the live album The Journey; her best-received album as of writing is 2010's Get 'Em Girls.
  • List of frequent David Lynch collaborators (nom) by Grapple X. American filmmaker David Keith Lynch has collaborated with several individuals on multiple occasions over the course of his career. The most prolific of Lynch's frequently used actors was Jack Nance.
  • List of 2000s Christian Songs number ones (nom) by Toa Nidhiki05. A total of 44 singles reached the number one position on the Christian Songs chart, compiled by Billboard magazine, between its launch in 2003 and 2009. MercyMe was the most successful group, with seven of their singles topping the chart during the 2000s; Casting Crowns spent the longest time atop the chart.
  • List of Ryder Cup matches (nom) by NapHit. The Ryder Cup is a golf competition contested by teams from Europe and the US. The biannual event takes place alternatively in the US and Europe. The most recent match was won by Europe, although the US team has historically been the most successful.

Nine featured pictures were promoted this week:

One featured topic was promoted this week:

  • Nebula Award (nom) by PresN, with six articles. The topic covers the Nebula Awards, a set of awards given annually for the best science fiction or fantasy works published in the United States.

One featured portal was promoted this week:

  • Indonesia (nom) by Crisco 1492, with 20 selected articles, 20 selected biographies, 20 sets of did you know hooks, 20 selected pictures, and an in this month section. Indonesia is the fourth most populous country on Earth and the most populous Muslim-majority nation.
The Taj Mahal at sunset


Reader comments

2012-10-29

WP governance informal; community as social network; efficiency of recruitment and content production; Rorschach news

A monthly overview of recent academic research about Wikipedia and other Wikimedia projects, edited jointly with the Wikimedia Research Committee and republished as the Wikimedia Research Newsletter.

Wikipedia governance found to be mostly informal

A paper in the Journal of the American Society for Information Science and Technology, coming from the social control perspective and employing the repertory grid technique, has contributed interesting observations about the governance of Wikipedia.[1] The paper begins with a helpful if cursory overview of governance theories, moving towards the governance of open source communities and Wikipedia. That cursory treatment is not foolproof, though: for example, the authors mention "bazaar style governance", but attribute it incorrectly—rather than the 2006 work they cite, the coining of this term dates to Eric S. Raymond's 1999 The Cathedral and the Bazaar. The authors have interviewed a number of Wikipedians and identified a number of formal and informal governance mechanisms. Only one formal mechanism was found important—the policies—while seven informal mechanisms were deemed important: collaboration among users, discussions on article talk pages, facilitation by experienced users, individuals acting as guardians of the articles, inviting individuals to participate, large numbers of editors, and participation by highly reputable users. Notably, the interviewed editors did not view elements such as administrator involvement, mediation or voting as important.

The paper concludes that "in the everyday practice of content creation, the informal mechanisms appear to be significantly more important than the formal mechanisms", and note that this likely means that the formal mechanisms are used much more sparingly than informal ones, most likely only in the small percentage of cases where the informal mechanisms fail to provide an agreeable solution for all the parties. It was stressed that not all editors are equal, and certain editors (and groups) have much more power than others, a fact that is quickly recognized by all editors. The authors note the importance of transparent interactions in spaces like talk pages, and note that "the reported use of interaction channels outside the Wikipedia platform (e.g., e-mail) is a cause for concern, as these channels limit involvement and reduce transparency." Citing Ostrom's governance principles, they note that "ensuring participation and transparency is crucial for maintaining the stability of self-governing communities."

Social network analysis of Wikipedia community

This paper looks at the relationships between Wikipedians from the social network analysis perspective (nodes are defined as authors, and links as indicators of collaboration on the same article), treating Wikipedia as an online social network (similar to Facebook).[2] The authors note that while Wikipedia is not primarily a social network site, it has enough social networking qualities to justify being seen as such. They find that Wikipedia can be seen as a very good source of information about online relationships between actors, due to the transparent and public nature of its data. The authors present a brief overview of previous work with a similar approach. Rather unsurprisingly, the authors find that in the very early days of Wikipedia, editors were much more likely to know one another and collaborate on articles than in the later years. They find that the number of editors is highly correlated to the editors' familiarity with one another, and is more relevant than the number of articles, as they find that from 2007, when the number of editors roughly stabilized, so did their levels of connectedness through collaboration.

The paper shows that with very few exceptions (low activity, specialized editors) all Wikipedia editors are connected to one another, and there are no isolated groups (or topic areas). The authors also find that the Wikipedia collaborations can be analyzed using the small-world network approach (suggesting that the distance between editors, defined as the average path length, with links being articles contributed to, is very small). The article focuses primarily on the mathematical side of social network analysis, and unfortunately offers little commentary or analysis of the findings. The validity of the results can also be questioned, as the authors treat bots and semi-automated accounts as "regular authors"; considering that the majority of Wikipedia articles have been edited by bots or editors using scripts, the finding that editor A can be connected to editor B through the fact that they both edited different pages which in turn were edited by the same bot or script-equipped editor is hardly surprising.

Wikipedia's article on the Rorschach inkblot test found to have a limited effect on the test's results

One of the inkblot images from the original Rorschach test, illustrating the Wikipedia article about it (here shown without the list of popular responses that according to Schultz and Brabender appears to influence test results)

Earlier this month, the Journal of Personality Assessment published a paper titled "More Challenges Since Wikipedia: The Effects of Exposure to Internet Information About the Rorschach on Selected Comprehensive System Variables".[3] Summarizing past events (well-known to Wikipedians) from the point of view of psychologists adhering to the Rorschach test as a diagnostic tool, they write: "The availability of Rorschach information online has become of even greater concern in the last few years, since James Heilman, an emergency-room physician from Canada, posted images of all 10 Rorschach inkblots on the popular online encyclopedia, Wikipedia (Cohen, 2009; Wikipedia, 2004[sic]). This Wikipedia article also describes “common responses” to each blot, which frequently correspond to percepts that would be scored Popular under the current coding rules of Exner’s (2003) Comprehensive System (CS)." They remark that "Although many psychologists decried the publishing of the Rorschach inkblots on Wikipedia, before this study, no published studies had examined whether viewing the inkblots and other Rorschach information posted on Wikipedia would impact examinees’ scores." (As reported last year in this newsletter - see "Psychologists gauge impact of Wikipedia's Rorschach test coverage" - one of the authors had coauthored a study that had investigated the rise in prominence of information about the test on the Internet due to Wikipedia, but not tested its impact on the test itself.)

Before reporting their own results, the authors cite an unpublished dissertation,[4] which had compared test subjects' Rorschach results before and after reading the article. Its tentative results suggested a "significant increase in shading responses [which] then likely affected the corresponding increase in [one variable], but otherwise indicated "that the majority of CS variables do not appear to be affected by exposure to information in the Wikipedia article."

The authors' own study involved 50 participants, half of whom had to read an excerpt of the Rorschach test article (while the control group read one of the Philadelphia Phillies article) before trying to "fake good" on the test, impersonating a character which would have a huge incentive to achieve certain results in the test ("Jack is a 35-year-old father of two wonderful children ...The judge ordered that Jack have a psychological evaluation done to determine whether or not he should be given custody of his kids.")

Among the test features defined in the "CS" system, only "Populars" was found to differ significantly "between the control and experimental groups [...] likely due to the fact that the Rorschach [Wikipedia article excerpt] provided pictures of each of the inkblots, along with "common responses," which, in many cases, corresponded to those responses that are actually coded as Popular according to the CS. However, the Wikipedia information on its own did not appear to directly impact other variables associated with perceptual accuracy."

Commenting on the paper, Heilman told this research report:

That reading about the Rorschach before testing affects scores in a group of "normal" individuals is not really surprising. This analysis, however, does not show that the availability of information regarding psychological tests affects clinical important outcomes.

Efficiency of Wikipedia in editor recruitment and content production

A paper titled "Is Wikipedia Inefficient? Modelling Effort and Participation in Wikipedia"[5] will be presented at next year's HICSS '13 conference. The main research concern of the authors is whether the saturation observed in the growth of Wikipedia is due to the maturity of the project or is rather caused by editorial obstacles and inefficient collaboration processes. To address this question, they try to investigate the efficiency of collaboration in 39 language editions of Wikipedia. Two different processes are studied. 1) editor recruitment; the ability of Wikipedia projects to attract editors from the pool of potential editors and 2) the article creation process. For each of these two processes corresponding input and output parameters are chosen and by applying a set of Data Envelopment Analysis the relative efficiency of language projects is calculated. For the editor recruitment process the input parameter is the size of the population speaking the language, having access to Internet and being at a tertiary-level of education and the output is the number of Wikipedia editors contributing to the Wikipedia edition of that language. It is shown that the efficiency of some language editions, e.g. Estonian, Hungarian, Norwegian, and Finnish, are much higher than some other language editions, e.g., Malaysian, Arabic, and Chinese. A decreasing return to scale is reported for all of the studied projects; however, the effect is more pronounced for larger ones. In other words, larger projects can be considered as inefficient in attracting new editors. For the production process, the number of Wikipedia editors is considered this time as the input and 3 outputs: number of edits, number of articles, and number of Featured articles. Here, the results generally suggest that for the larger projects the returns to scale are systematically decreasing, showing the difficulties of maintaining the efficiency of the workflow as the project grows. Some projects, such as the Malaysian and Persian Wikipedias, are not as successful in editor recruitment but are still efficient in creating articles given the capacity of their human resources. As for the quality of articles, it is shown that in larger projects like French and German, the focus is more on increasing the quality of the existing articles, whereas in intermediate-size projects, e.g., Russian and Italian, the main effort is still on increasing the number of articles.

The paper notes a positive correlation between efficiency in the number of edits and the efficiency in number of articles and featured articles. Among the limitations of the study, the authors name the time period of the analysed data, being limited to one month, and the possible flaws in the demographic data used to estimate the input of the editor recruitment process. Excluding contributions from unregistered users due to technical reasons could also have induced biases in the results. Since the article starts by raising the question of efficiency of Wikipedia in general, it ends up by comparing different language editions to each other and presenting the results in only relative terms. The English Wikipedia, which could be a benchmark for such comparisons, is entirely excluded from the study. More importantly, applying the data envelopment analysis, which is originally introduced for evaluating activities of not-for-profit entities participating in public programs, on Wikipedia activity data is not well justified.

Student use of Wikipedia

How students find and evaluate information is a perpetual concern for librarians, who act as educators and guides to finding the best resources for student information needs as well as collection curators. Since the arrival of Wikipedia, librarians have grappled with how the site fits in with and compares to a more traditionally published and reviewed collection, and how best to help students understand and use Wikipedia. This study is an up-to-date addition to the body of literature on this subject.[6] Colón-Aguirre and Fleming-May use a coded qualitative interview approach to understanding undergraduate opinions about Wikipedia, compared to their use of and attitude towards traditional library resources.

The authors conducted interviews with 21 undergraduate students in one college in a large public university in the United States. Based on student responses about their research habits, the authors divided their respondents into three categories: avid library users, occasional library users, and library avoiders. While all categories of students used Wikipedia, there were differences in purpose; avid library users used Wikipedia to gather background information before turning to library-supplied resources like books and journals, while library avoiders relied more on Wikipedia and were lost if they could not find the information they needed on the site or via Google searches. Most of the students interviewed reported getting to Wikipedia via Google or other search engines, and the authors do not report any deep awareness by the students of how the site works or how to evaluate articles; awareness of ability to contribute was not mentioned. Student use of the library versus Wikipedia was also influenced by their perceptions of library resources being difficult to use (both in-person stacks and subscription online resources), particularly compared to the ease of using Wikipedia and online searching; students were also swayed in whether they used the library by their assignment requirements and faculty advice, including professors who advised against using Wikipedia as being "not credible" and required using library resources specifically.

The authors conclude that librarians need to work more with teaching faculty to craft research assignments, and that hands-on instruction in the use of the library does aid student comfort with research. This short article will be most of interest to practicing librarians and undergraduate instructors, who will doubtless see reflections of their own students in the student interviews. Wikipedians who are involved in academic classroom education and outreach will also find this study interesting, if for no other reason than to reinforce the importance of helping students become more knowledgeable about the ways that Wikipedia works with and differs from traditional academic publications.

In brief

  • "Conflict positively influences group performance": Investigating the question "Does conflict matter in the success of mass collaboration?", a paper in the Chinese Journal of Library and Information Science[7] investigates conflict on Wikipedia, analyzing it from the social network analysis perspective (nodes are defined as individuals, and links, as indicators of conflict), and differentiating between positive and negative types of conflict. Their goal is to increase understanding of the conflict mechanism in the mass collaboration setting. The authors find that "that participation positively influences task complexity, conflict, and group performance; task complexity positively influences group performance but negatively influences conflict; and conflict positively influences group performance".
  • Generating a lexical network from Wiktionary: [8] The researcher has created an open source tool – available at http://dbnary.forge.imag.fr/ – that extracts a lexical network (including definitions, translations, synonyms, antonyms, etc.) from Wiktionary data in RDF format, that can be used in existing semantic tools. The author notes that because Wiktionary – unlike traditional dictionaries – treats homonyms (words that share the same spelling and pronunciation but have a different meaning) on single pages with multiple etymology sections, it has not been possible to properly attribute the senses and lexical relations to the proper etymologies (i.e. lexemes).
  • How are article edits and page views related? We still don't know.: [9] This paper attempts to explore the relationship between the "production" and "consumption" of Wikipedia content: the edits that build articles, and the page views from readers. For broad topic areas on English Wikipedia (such as articles in Category:Dance and its subcategories), the pattern of edits mirrors the overall trend of editing activity—rising exponentially until peaking around 2007, with a linear decline in edit rate since then. Page views for these topic areas, by contrast, show an approximately linear rise page views since late 2007 (which is the earliest period for which we have article traffic statistics). According to the authors, this pattern "conforms to a two-phase evolution framework: one of production followed by consumption", although they do not attempt to establish a causal link between the article content maturation and readership. Unfortunately, the lack of earlier data on article traffic makes it hard to learn much from the relationship between edit rate and article traffic, without taking a more fine-grained approach to identify articles or topic areas whose early phases of rising and peaking edit rates are also covered by page view data.
  • More WikiSym reports: Two more reports from August's annual WikiSym conference were published this month, by the recipient of a travel grant from the UK Wikimedia chapter,[10] and by a Natural Language Processing (NLP) researcher[11] who dubbed the conference "WikipediaSym" because "the conference submissions were mostly inclined towards the information analysis and social aspects of using wikis, in particular Wikipedia, and there were very few submissions on the actual applications of wikis (or wiki-like systems) and the open collaboration context". (See also the overview report in the last issue of the research report)
  • German centrality: A discussion paper examined "Centrality and Content Creation in Networks [in] The Case of German Wikipedia". [12]
  • Systemic bias: Slides of a presentation by a librarian at the University of Massachusetts Amherst (and active Wikipedian) concern "Systemic Bias in Wikipedia: What It Looks Like, and How to Deal with It".[13]
  • Few users who edit Middle East/North Africa articles are from the region: A brief conference paper titled "The vocal minority: Local self-representation and coediting on Wikipedia in the Middle East and North Africa"[14] (presented in a slightly different form at a Workshop at 2012 ACM Web Science Conference in June) analyzed the talk pages of English Wikipedia users who had edited articles geotagged in that region (MENA) "to assess the self-declared locational affiliations of the authors (i.e. where they live, work or were born)" and found that "there exists few authors claiming to be from the MENA region, except for Israel, Iran and to a much lesser extent Egypt."
  • Article Feedback tool as means of "peripheral participation": A paper to be presented at CSCW '13[15] describes the main findings from the early tests of the Article Feedback v5 on the English Wikipeida, from the lens of legitimate peripheral participation theory. The study reviews the costs and benefits of expanding reader contributions to Wikipedia, using both quantitative and qualitative methods. The results, according to the authors (members of the Wikimedia Foundation team working on the tool), indicate that peripheral contributors add value to the encyclopedia as long as the cost of identifying low quality contributions remains low.
  • Dynamics of read and edit rates on Wikipedia: The ECCS'12 Conference on Complex Systems saw the presentation of a paper titled "From Time Series to Co-Evolving Functional Networks: Dynamics of the Complex System 'Wikipedia'"[16], reporting on research about the "access-rate time series and edit-interval time series" of articles on the English Wikipedia, and about " three organizational and dynamical networks ...: (i) the network of direct links between Wikipedia articles, (ii) the usage network as determined from cross-correlations between access-rate time series of many pairs of articles, and (iii) the edit network as determined from co-incident edit events. The major goal is to find correlations between components of these three networks that characterize the dynamics of information spread in the complex system".
  • Wikipedia articles compared to open source software projects: A paper titled "Similarities, challenges and opportunities of Wikipedia content and open source projects"[17] argues that "the evolution of Wikipedia pages and the OSS projects share some commonalities in terms of their evolutionary patterns; in particular, it was found that a predefined, cubic model could be used to explain several of the similarities in 'abandoned' or 'completed' projects and Wikipedia pages."

References

  1. ^ Schroeder, A., Wagner, C. (2012). Governance of open content creation: A conceptualization and analysis of control and guiding mechanisms in the open content domain. Journal of the American Society for Information Science and Technology 63(10):1947–59 DOI Closed access icon
  2. ^ Hirth, M., Lehrieder, F., Oberste-Vorth, S., Hossfeld, T., Phuoc T.-G. (2012). Wikipedia and its network of authors from a social network perspective. 2012 Fourth International Conference on Communications and Electronics (ICCE) DOI Closed access icon
  3. ^ Douglas S. Schultz, Virginia M. Brabender: More Challenges Since Wikipedia: The Effects of Exposure to Internet Information About the Rorschach on Selected Comprehensive System Variables. Journal of Personality Assessment. Advance online publication. DOI Closed access icon"
  4. ^ Randall,W. A. E. (2010). Rorschach reliability with exposure to Internet-based images and information (Unpublished doctoral dissertation). Massachusetts Professional School of Psychology, Boston, MA.
  5. ^ Crowston, K., Jullien, N., Ortega, F. (in press) Is Wikipedia Inefficient? Modelling Effort and Participation in Wikipedia. Forty-sixth Hawai'i International Conference on System Sciences (HICSS-46), PDF Open access icon
  6. ^ Colón-Aguirre, M. and Fleming-May, R. (in press). "'You just type in what you are looking for': Undergraduates' use of library resources vs. Wikipedia". The Journal of Academic Librarianship. Advance online publication. DOI Closed access icon
  7. ^ Wu, K.,Zhu, Q.,Vassileva, J.,Zhao, Y. (2012) Does conflict matter in the success of mass collaboration? Investigating antecedents and consequence of conflict in Wikipedia. Chinese Journal of Library and Information Science, 2012, 5(1):34-50 PDF Open access icon
  8. ^ Sérasset, G. (2012) Dbnary: Wiktionary as a LMF based Multilingual RDF network. Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC'12) PDF Open access icon
  9. ^ Capiluppi, Andrea; Duarte Pimentel, Ana Claudia; Boldyreff, Cornelia. (2012) "Patterns of creation and usage of Wikipedia content". 14th IEEE International Symposium on Web Systems Evolution (WSE 2012) DOI Closed access icon
  10. ^ Gavin Baily: Wikisym 2012 Report, Wikimedia UK wiki, October 2012
  11. ^ Bahar Sateli (2012-10-09) Wiki-NLP Integration at the WikiSym'12 Conference, semanticsoftware.info blog
  12. ^ Michael E. Kummer, Marianne Saam, Iassen Halatchliyski, and George Giorgidze: Centrality and Content Creation in Networks – The Case of German Wikipedia. ZEW Discussion Paper No. 12-053 PDF Open access icon
  13. ^ Laura Quilter. (2012) "Systemic Bias in Wikipedia: What It Looks Like, and How to Deal with It". Open Access Week 2012, University of Massachusetts Amherst PDF Open access icon
  14. ^ Bernie Hogan, Mark Graham, Ahmed Medhat Mohamed. (2012) The vocal minority: Local self-representation and coediting on Wikipedia in the Middle East and North Africa. Symposium & Workshop on Measuring Influence on Social Media (#Influence12) PDF Open access icon
  15. ^ Aaron Halfaker, Oliver Keyes, Dario Taraborelli. (in press) Making peripheral participation legitimate: Reader engagement experiments in Wikipedia. 16th ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW'13) PDF Open access icon
  16. ^ Mirko Kämpf, Jan. W. Kantelhardt, Lev Muchnik. (2012) From Time Series to Co-Evolving Functional Networks: Dynamics of the Complex System 'Wikipedia'" European Conference on Complex Systems (ECCS'12) PDF Open access icon
  17. ^ Andrea Capiluppi (2012) Similarities, challenges and opportunities of Wikipedia content and open source projects. Journal of Software: Evolution and Process. Advance online publication. DOI Closed access icon


Reader comments
If articles have been updated, you may need to refresh the single-page edition.

















Wikipedia:Wikipedia Signpost/Single/2012-10-29