Article Feedback reversal: watershed moment?; plus code review one year on

The Wikimedia Foundation this week aborted a plan that would have seen version 5 of the Article Feedback tool (AFTv5) rolled out to all English Wikipedia articles (Editor-Engagement mailing list). As a result of fairly damning community feedback (see previous Signpost coverage), the extension, which adds a box to the bottom of articles asking for comments, will now only appear when the article has been added to a certain category. According to a revised release plan, the tool will continue to receive updates, though the focus will be on making it available to other wikis.

Together with last month's "undeployment" of the Moodbar extension and its associated Feedback dashboard, the move marks the end of the line for two of 2011's bigger projects. "As an experiment, Moodbar was a fair success", wrote the WMF's Brandon Harris on 6 February, "but we have come to the conclusion that it will require a fair chunk of development work (on the Feedback Dashboard side) to make it fully usable as a mechanism for new user engagement... [which will only now be as] part of the upcoming Flow initiative".

Despite the suggestion of a future revival of the Moodbar at a later date, the outcomes can only be demoralising from a developer standpoint: the Article Feedback tool was ultimately rejected despite an incredibly energetic community engagement campaign, and the Moodbar simply never took off, despite filling an even more obvious need. It would be tempting, then, to think that the English Wikipedia community rejects those tools that are seen to create burdens and embraces those that are seen to empower (the VisualEditor, Lua, Wikidata). However, the success of the Teahouse points to the dangers of drawing overhasty conclusions on this point. In any case, with AFTv5 almost entirely switched off, there will be much for WMF team leaders to ponder over the coming weeks.

In late September, the Signpost published an independent analysis of code review times, an analysis it repeated in November. To the 23,900 changesets analysed the first time and 9,000 added in the revised edition, a further 20,000 have since been added. Across those 51,380 changesets, developers (human and bot) have contributed some 73,000 patchsets and 167,000 reviews. This report is designed to supersede the preceding analysis, bringing the analysis up-to-date in time for the first anniversary of the Git switchover. The methodology remains the same, though the list of WMF deployed extensions has been updated and changing Gerrit practice has required a slight revision to some figures; interested users should consult the preceding reports. As with all data, the possibility for error is always present, though the figures presented are robust at the margins.

The undeniable conclusion is that code review times have stabilised at a good but far from perfect equilibrium. The headline figure – median review time for a proposed change to WMF-deployed code – only crept up slightly after October's low of 2 hours, 20 minutes, reaching 3 hours, 29 minutes in January. Over the same period, the 75th percentile was unchanged at approximately 22 hours. Early indications for February suggest no great shift. Fears expressed a year ago that code review would grind to a halt once a pre-review system was brought in appear, then, to be unfounded, at least in aggregate terms.

Unfortunately, however, the composition of those aggregate times is also stable: staff get their patches reviewed 2 to 3 times quicker than volunteers (illustrated right). Even if staff write smaller patches – and there is no particular reason to think that they do – that multiple seems stubbornly high. All of the top five most prolific all-time first-reviewers for core code are staff; between them, they have provided 40% of the first-reviews over the last 12 months, though that figure is tracking downwards at a healthy rate. In total, staff have provided ~70% of first reviews for core code – also tracking downwards – a percentage which rises to ~80% if WMF-deployed extensions are also included (the all-time top 19 reviewers for such extensions all being staff). Thus, staff still do more of the reviewing and get their own code reviewed quicker: but at least more staff are now becoming proficient reviewers.

Not all fixes may have gone live to WMF sites at the time of writing; some may not be scheduled to go live for several weeks.

• 

  Translation interface tweaked: The Wikimedia Foundation's Language Engineering team has revealed another iteration of the translation interface. Developed during a recent sprint, the collection of UI tweaks presented, each of which has been tested on different languages, is intended to make translation easier. The team is planning additional features to be deployed, including a dedicated proofread view.
• Parsoid: challenges and design decisions: In a comprehensive blog post, WMF developer Gabriel Wicke rounded up his team's progress on Parsoid, the new parser that will ultimately power the still-in-development VisualEditor. Wicke, attempting to answer the question "why has this not been done before", lists several technical challenges (foreseen and unforeseen) that have held back progress on the project.
• Lua comes to remaining wikis: Lua programming language is going to be rolled out (via MediaWiki's Scribunto extension) to all remaining Wikimedia wikis on Wednesday 13th this week (full deployment schedule) following a successful roll out to almost 20 wikis, including the English Wikipedia, on 18 February. Currently, about 80 templates have already been converted to Lua on the English Wikipedia while a central Lua page was set up at Wikipedia:Lua. The new "Module:" namespace localization is still pending, but template-makers are welcome to help out with further template conversions through the whole WMF universe once fully rolled out. The roll out was covered in a blog post by Sumana Harihareswara (WMF).

← Previous "Technology report"

Next "Technology report" →

In this issue

11 March 2013 (all comments)

From the editor

News and notes

Featured content

Arbitration report

WikiProject report

Technology report

+ Add a comment

Discuss this story

These comments are automatically transcluded from this article's talk page. To follow comments, add the page to your watchlist. If your comment has not appeared here, you can try purging the cache.

Ed. note: this talk page was cleared after publication, per our usual practice. Comments made before publication are available here. Ed ^{[talk] [majestic titan]} 07:26, 13 March 2013 (UTC)[reply]

Article Feedback tool

It's good to see that the WMF has listened to the feedback it received about this tool through the RFC. This was a worthwhile experiment, but IMO it didn't work and it's good that it's going to be changed to opt-in. Nick-D (talk) 07:22, 13 March 2013 (UTC)[reply]

LUA templates

Whilst there are over 80 templates in Category:Lua-based templates, it should be noted that about 20 of these are sandboxes of their parent templates, plus one or two user sandboxes, leaving about 60 legitimate templates. An optimist on the run!   08:17, 13 March 2013 (UTC)[reply]

Thanks for the clarification. - Jarry1250 ^{[Vacation needed]} 14:09, 14 March 2013 (UTC)[reply]

Code review

I can understand why you focus on the first review rather than on the final result (rejection or merge), but I'm not so sure of this WMF vs. non-WMF distinction, because I think the point should rather be how good we are at bringing flesh blood. We'd need a third line for non staff (please try to include WMDE in staff), non-mediawiki patches: non-WMF staff teams may skew the numbers, as well as core volunteer developers (whose patches' quality is probably, on average, better than the WMF's, given the higher and longer MediaWiki experience they often have, although more are being added lately). As for the merges, the number of open commits is still increasing (as far as I can see), and 80 % of them are non-WMF. --Nemo 09:24, 13 March 2013 (UTC)[reply]

Historically, the staff vs. non-staff distinction has been the one that has been the one most prominent in arguments in various fora, which is why I focus on it . The argument that a few volunteer developers have networked sufficiently to improve their review times to the level of staff has been put before (e.g. by Amgine) and I will look into testing it for next week.

The number of open commits is not a particularly useful measure under Gerrit, because if you've got a review, it's up to you to do something. At the moment, we have no idle timeout, and there is no reason to have one. - Jarry1250 ^{[Vacation needed]} 14:09, 14 March 2013 (UTC)[reply]

You're right on everything except that your analysis doesn't measure staff vs. non-staff but rather WMF vs. non-WMF, which is similar but not equivalent. For open commits, the figure above doesn't change if you consider only open commits without reviews (which is probably the recommended measure, see mw:Gerrit/Navigation#Commits_lists). --Nemo 14:32, 17 March 2013 (UTC)[reply]

I would also be interested in open commits. While I do not have a full knowledge how code review works, I am having a patch submitted myself which hasnt got a final review for almost a month...--Kozuch (talk) 10:48, 15 March 2013 (UTC)[reply]

MoodBar

"the outcomes can only be demoralising from a developer standpoint" -- I wouldn't assume that. In fact, I would definitely ask folks on the Product and Engineering teams what their opinion was before speaking for them. Some of us do love MoodBar and the Feedback Dashboard (I helped start the response team) but part of being professionals in software development is that you can't be afraid to kill your babies, if you catch my drift. Steven Walling (WMF) • talk 18:24, 13 March 2013 (UTC)[reply]

Yes, "outcomes": I was including AFTv5 in that (not sure if you noted that). I think it's inferable enough from various developer comments that undeploying AFTv5 was demoralising. Not necessarily very demoralising – as you say, being a professional means having to accept the odd reversal – but demoralising nonetheless, especially on AFTv5 where it was never really deployed in the first place. Disappointing, you know. That said, I welcome contrary comments which can be posted here or sent privately to me via my username @gmail.com . I will do my best to run them next week. Thanks, - Jarry1250 ^{[Vacation needed]} 14:09, 14 March 2013 (UTC)[reply]

I have to say i do not find the writing in this piece to be the type of objective reporting expected of a news source. There seem to be several opinions, such as the moodbar "filling an even more obvious need", a statement which the authors did not even bother to explain, and obvious speculation about the possible motives of the community in rejecting these features. I went back and re read that secion trying to figure out if you were just quoting someone as it seemed more like a bunch of opinions there in the last paragraph. Beeblebrox (talk) 00:20, 15 March 2013 (UTC)[reply]

I'm sorry you don't like the style, Beeblebrox. To answer your specific points, of course there is an amount of speculation about motives: this is completely typical of any news outlet. People care about motivations, and yet they can only ever be inferred from what people say and do (in this case the AFTv5 RFC) and I'm happy to take corrections. And yup, I did get some analysis in there in the last paragraph, but phrased overtly as such. To expand upon the need point, it's been long established that new users have historically been under-supported and lack communications pathways. Or do you disagree? - Jarry1250 ^{[Vacation needed]} 19:05, 19 March 2013 (UTC)[reply]

AFT on request

So I take it from this we can place AFT on certain articles on an opt-in basis... is there a page with more detail on how to do this somewhere? -- phoebe / (talk to me) 03:25, 14 March 2013 (UTC)[reply]

The first link to the EE mailing list may be of assistance. - Jarry1250 ^{[Vacation needed]} 14:09, 14 March 2013 (UTC)[reply]

To answer this question for the lazy, including me :) -- the newsletter says: "Going forward, editors who wish to enable AFT5 on articles they watch can simply add 'Category:Article_Feedback_5' to these articles -- and the feedback form will automatically appear at the bottom of these pages." -- phoebe / (talk to me) 19:05, 18 March 2013 (UTC)[reply]

AFT question

Will there be automatic anti-spam, anti-abuse measures on opt-in articles containing AFT? --74.202.39.3 (talk) 22:18, 18 March 2013 (UTC)[reply]

AFT from watchlist

I used to be able to see all the feedback from all my watchlisted articles using a link off my watchlist to Special:ArticleFeedbackv5Watchlist. But that doesn't seem to work anymore. Is it gone forever? --99of9 (talk) 06:15, 20 March 2013 (UTC)[reply]