Wiki Loves Monuments evaluation sees diminishing returns and increasing cost

The Wikimedia Foundation's first two program evaluations of 2015 have been published on Meta. These examine the annual Wiki Loves Monuments (WLM) and other photo competitions that have been held around the globe, with an eye towards finding what worked and what did not. Although WLM is an international contest, it is organized separately in each country, with separate budgets and contests in each before the winners advance to the global finals. It was first held in the Netherlands in 2010, and the success there encouraged organizers in other countries to join.

The evaluations reveal that in the last three years (2012–14), WLM has possibly fallen victim to its own success and seen diminishing returns: the average total number of uploads has decreased from a high of 6,266 images in 2012 to 2,714 in 2014 even as the average money spent per upload has increased from US37c to 90c. While the total number of images increased from 2012 to 2013, it was less than half that in 2014. The number of images in use on Wikimedia programs has dropped both in total number and percentage, and the cost per used image has gone up from $3.03 in 2012 to $6.31—although this is an improvement on 2013's $6.61. Cost per participant was, on average, about $25. About 1% of the total image uploads were later rated as "quality" or "valued" images.

The users participating skewed heavily towards people who had never edited Wikimedia sites: over the three studied years, about 1,400 current and 14,000 new users participated. The conversion rate into continuing editors, as measured by having at least one edit three months after the competition, is 2.4%. Extending this to twelve months after the competition (for programs that ended before February 2014) shows that the programs netted 16 "active" new editors, or 0.3%—those who made more than five or more edits in the studied period.

The overall data analysis by the WMF suggests that "When planning a photo event, it may be useful to try to balance group size with both new and experienced users to increase use and ensure high quality uploads." For funding, the evaluators recommended that the WMF be "cautious about the investment level" amidst the contest's diminishing returns. E

• Wikipedia store relaunched: The Wikipedia Store completed a relaunch this week and is now again open for online orders. The Wikipedia Store is a small web portal operated by the Wikimedia Foundation through which people may buy merchandise related to Wikipedia and the Wikimedia movement. Proceeds from sales go directly to the Wikimedia Foundation. Summarizing the changes on the Wikimedia blog and the Foundation-l mailing list, fundraising associate (and, alongside other familiar faces from the Foundation, often-time merchandise model) Victoria Shchepakina stated that:

Purchasable highlights include a free-knowledge t-shirt prominently featuring the Creative Commons CC-0 logo, literally plantable pencils ("who says knowledge can’t grow on trees?"), and your correspondent's personal favorite item, the "rabbit hole" t-shirt, featuring a visual depiction of the WikiWalk. In support of the relaunch, a merchandise giveaway allowing Wikimedians to nominate other users for free merch is currently also underway. R

• Language translations: Ever wonder how many language articles there are within the Wikipedia projects as a whole? In a post to the Foundation-l mailing list, meta-Wikipedian and linguaphile Millosh highlighted the breadth and status of language article across the Wikipedia projects, compiled into the meta-wiki article "Names of Wikimedia languages" as part of an effort to encourage translation work within the Wiktionary projects. Clicking through the page leads to a matrix presenting that Wikipedia language project's other-language coverage. For example, contrast the completed coverage of other languages achieved by the English Wikipedia or French Wikipedia, with the spottier coverage of the Arabic Wikipedia or the Hindi Wikipedia, and with the mostly-absent coverage of Kongo Wikipedia or the Yiddish Wikipedia. With 250 languages expressed in 250 other languages there are 62,500 entries in all amongst the Wikipedias; where a complete list of other-language translations is appropriate, like on Wiktionary, this corresponds with a further 250 translated expressions per article, bringing the total to 15,625,000 entries between all of the projects. Inquiries (including on how to help) are directed at the project's talk page. R
• Material from the Wikimedia Foundation's quarterly reviews made available: Minutes and slides have this week been made available from the quarterly reviews (for the period January–March 2015) conducted by the various Foundation departments. As the Signpost reported in March, a quarterly review structure, originally used mainly by the engineering teams, has been extended more broadly across all departments by executive director Lila Tretikov, to better align reporting periods with the Foundation's generally quarterly planning periods. The four summaries that have been made available are for the Community Engagement and Fundraising teams (the fundraising team is currently being consolidated into the new Advancement Department); the Mobile Web, Mobile Apps, and Wikipedia Zero teams; the Parsoid, Services, MediaWiki Core, Tech Ops, Release Engineering, Multimedia, Labs, and Engineering Community teams; the Editing, Collaboration, and Language Engineering teams; the Legal, Finance, HR, and Communications teams; and the Analytics, User Experience, Team Practices and Product Management teams. Of note is the editing team, who responded to community concerns and delayed an A/B test of the VisualEditor for new users on the English Wikipedia, as they did not believe that a new autofilled citations feature was ready for it. In related news, loading times for the VisualEditor against the standard wikitext edit window are now comparable or faster. R and E
• Board of Trustee election voting rules: As report last week in the Signpost, preparations are currently underway for the 2015 Wikimedia Foundation elections. A discussion of interest occurred this week on the Foundation-l mailing list as to whether or not movement staff and contractors not meeting the current editorial requirements for participation in the elections should be extended suffrage. At this stage such a change, were it to occur, would only go into effect before the 2017 election. R
• Scholarship Committee results for Wikimania 2015 announced: 104 people were awarded a travel scholarship to help defer the costs of attendance of this year's iteration of the annual Wikimania, the movement's biggest conference, to be held in Mexico City on July 15-19. 13 were sponsored by Wikimedia Germany; other chapter-supported scholarships will be announced separately by the various chapters at later times. This year's scholarships "involved a major re-design of the application and selection process". Posted highlights include 28% female, 73% from the global south, and 26% previous recipients from Wikimania 2014. R
• Wikimedia Netherlands annual report: Wikimedia Nederland, the movement's Dutch affiliate chapter, have released their annual report covering the year 2014. R
• Wiki-Edu monthly report: The Wiki Education Foundation have released their monthly report for the month of March 2015. R
• English Wiktionary milestone: The English Wiktionary reached 4,000,000 entries this week. The milestone entry was "cundidos", the Spanish past participle for cundir, claimed by creator Type56op9 to be "a particularly uninteresting one". R
• Female Wikipedian mailing list: A new mailing list called Systers Wikipedia, hosted by the Anita Borg Institute for Women and Technology, has been created. According to the list's FAQ page "Systers Wikipedia is for women Wikipedians to discuss topics that relate to being a woman and being a Wikipedia editor. We also allow simple how-to questions because some women feel uncomfortable asking these types of questions on forums dominated by men. Another reason this list was set up is to give women editors a refuge from Wikipedia's often hostile editing environment." This comes a few months after a proposal to create a female-only space on Wikipedia (See previous Signpost coverage). G

← Previous "News and notes"

Next "News and notes" →

In this issue

29 April 2015 (all comments)

Wikimania

News and notes

In the media

Featured content

Traffic report

Recent research

Technology report

+ Add a comment

Discuss this story

These comments are automatically transcluded from this article's talk page. To follow comments, add the page to your watchlist. If your comment has not appeared here, you can try purging the cache.

• Why is this article trying to monetize photos from participants. Gives me a horrible feeling, the community/participants are not a factory plant in what every employee needs to work a minimum number of hours. The goal of Wiki Loves Monuments is to get all monuments with a good picture on Wikipedia. The first time a contest as such is organised the low hanging fruits are done first, but in this article it is forgotten to mention that getting the low hanging fruits is not the core goal of Wiki Loves Monuments. The goal of Wiki Loves Monuments is to get a photo of every monument. The more monuments get a picture, it becomes much harder to ge a picture of the other monuments. Please, if you want to write a story about a subject, do not only write about too easy thoughts without thinking it through. This article is failing in describing the actual situation and misses totally what Wiki Loves Monuments is about. Romaine (talk) 04:20, 1 May 2015 (UTC)[reply]

@Romaine: This article is a simple summary of the WMF's evaluation. I would take up your concerns with them! In their defense, however, I would note that donors' funds are being used to pay for these uploads. While the cause is noble (and one I personally laud), the WMF needs to consider how much of an impact their funds are having in order to avoid waste. Ed ^{[talk] [majestic titan]} 05:24, 1 May 2015 (UTC)[reply]

Measurables are in the air. Surely we should be concerned with the costs of the programs that our readers' money is funding, no? That is what the analysis is all about. ResMar 05:32, 1 May 2015 (UTC)[reply]

I agree with all of you. I think cost analysis is relevant to justify donor money. However the above summary is too much focused on money. There are figures in the WMF report that point to the effectiveness of WLM in improving Wikipedia that could have been included, such as that only 13% of images are used in any Wiki project and that only 0.03% became featured. Together with the 0.3% editor retention it shows that WLM is still wasting energy by focusing on quantity rather than quality and usefulness. --ELEKHH^T 09:36, 1 May 2015 (UTC)[reply]

But perhaps there is more to it than that.

• The images are available for all time, for everyone. The cost of a commercial photograph starts at several hundred dollars.
• There is awareness raising value from the events. For example if someone makes their flikr repository CCBYSA-4 there is a huge hidden gain.
• Though we like "active editors" since they do "most of the work", there is no guarantee that this will remain the case. A million editors who make one good edit a month would be fantastic.
• There are millions of minor monuments that should be recorded and aren't. For example the other side of the marker on the right here.

All the best: Rich Farmbrough, 12:17, 1 May 2015 (UTC).

If the article is simple a summary of WMF's evaluation, I think both fail in describing it correctly. Wiki Loves monuments is organised to have a full coverage of the worlds cultural heritage, not just of the most popular or easy monuments. It is a nice benefit that we broke the record of the world's largest photography competition, but that is not the core goal. The core goal is to have the world's heritage covered in pictures on Wikipedia, which means a cumulative grow in number of photos with each contest. The comparison of the budget compared to the number of uploads is also strange, like there is a strong relationship between them. The number of uploads depends on so many parameters that are outside the control of an organising team, and so many depends on just having luck. Yes, luck, that is an underestimated parameter with any contest and with many other things on Wikipedia as well. The only thing an organising team can do, is doing their best they can.

"I would note that donors' funds are being used to pay for these uploads" -> This is not true. None of the uploads is paid. The donors' funds are being used to organise a large photography in what thousands of volunteers participate with taking and uploading photos to help Wikipedia improve. But as you are playing the "donors"-ball, ask this question: do the donors want only the most popular monuments to be covered on Wikipedia, or all cultural heritage monuments? – I speak with many many people, including donors, and none of them expect us to cover only the most popular monuments. Everyone expects that Wikipedia has them all.

It is fine to evaluate a contest in comparison, but such should be done in comparison with other ways of getting the exact same results. The current set-up is too much focussed on the wrong goal. The goal is not spending less money. The goal is getting the best possible results for the money spend, to get the same results.

Also the report, and this article, totally exclude other side effects that Wiki Loves Monuments has. For example that the contest has resulted in a image donation of 480.000+ images of monuments.

"Together with the 0.3% editor retention it shows that WLM is still wasting energy by focusing on quantity rather than quality and usefulness." -> Sorry, this is nonsense. Like the organising teams have any choice in what kind of photos participants upload. And there is no focus on quantity. And there is no focus on editor retention. That WMF sets itself this goal, fine, but this is not the goal of Wiki Loves Monuments. Wiki Loves Monuments has as goal to have a better coverage of the world's heritage. If you want to compare Wiki Loves Monuments with what you would like, fine, but that is not the goal of Wiki Loves Monuments and comparing with other people's goals makes no sense. Romaine (talk) 07:01, 2 May 2015 (UTC)[reply]

• Am I the only person that finds it bizarre that the Wikipedia Store, which sells many items presumably mostly bought by editors, was launched without any large notification on the actual wiki? Just an unofficial mention in the Signpost? Sometimes it feels like Wikimedia is off on its own planet, with whatever 1000 people apparently fly around the world each year to tiny Italian villages to discuss things WM is launching that year, and is vaguely surprised each time they are reminded about the several dozen thousand people that create their content every day. --PresN 04:26, 1 May 2015 (UTC)[reply]
  • In fairness to the WMF, where is the best place to notify the entire community without pinging thousands of talk pages? Ed ^{[talk] [majestic titan]} 05:26, 1 May 2015 (UTC)[reply]

The mailing list, which did indeed run notification, to the very tightest core of Wikimedian power users. You would be wrong in that assumption, that it sells to editors—it seems to me to be a most very literal gift shop, or pitched as such. Without seeing their traffic stats (perhaps I should?) I cannot say whether or not that is the right approach, but that is the approach. Also incongruous at least to me: in a time where we have long since moved away from Wikipedia-centrism, why "Wikipedia Store" and not "Wikimedia Store"? I think it has to do with name-brand recognition, which reinforces my notion that the gift shop is mostly literally that, for passerby. ResMar 05:33, 1 May 2015 (UTC)[reply]

Well, I would have gone with the global notification tool that puts a line on the top of people's watchlists, which is the general wiki-wide way to do it. Also: "The mailing list, which did indeed run notification, to the very tightest core of Wikimedian power users" -> "with whatever 1000 people apparently fly around the world each year to tiny Italian villages". If it's not aimed at editors, then whatever. --PresN 17:55, 1 May 2015 (UTC)[reply]

• With regard to the comparative time to open a Wikipedia article in VE, for editing, versus the wikitext editor, VE is still significantly slower at opening a large article (specifically, Barack Obama), based on testing by ten Wikipedia editors. -- John Broughton (♫♫) 16:12, 1 May 2015 (UTC)[reply]