The Signpost

Technology report

MediaWiki 1.20 and the prospects for getting 1.21 code reviewed promptly

Code review statistics bounce back in October after difficult September

In late September, the Technology report published its findings about (particularly median) code review times. To the 23,900 changesets analysed the first time (the data for which has been updated), the Signpost added data from the 9,000 or so changesets contributed between September 17 and November 9 to a total of 93,000 reviews across 45,000 patchsets. Bots and self-reviews were also discarded, but reviews made by a different user in the form of a superseding patch were retained. Finally, users were categorised by hand according to whether they would be best regarded as staff or volunteers. The new analyses were consistent with the predictions of the previous analysis.

Our investigation found that September represented a particularly poor month for code review (across both extensions and "core" MediaWiki code) but that this loss was more than picked up in October, which was the best month on record for code review. Specifically, 50% of patchsets submitted during October were reviewed just two and a half hours after submission, and 75% within 18 hours. The 95% percentile remains stubbornly high at nearly two weeks, suggesting that finding reviewers for certain types of patch remains hard.

The staff–volunteer divide highlighted in the last report remains. The median patchset was reviewed twice as quickly if you were a staff member working on an extension in October rather than a volunteer, and although it is too early to tell conclusively, there seems to be a similar gap for contributors to "core" and/or WMF-deployed extensions. 44% of all-time first reviews come from five reviewers (all staff), though this figure is down from 55% at the time of the last report, suggesting a significant diversification in the last 7 weeks. On a positive note, the percentage of all-time first reviews coming from volunteers has also increased – from 14% to 25% – as the Foundation gives a large number of volunteers more reviewing power.

As with any statistics, these figures should be taken with a degree of caution. The full dataset is available upon request.

In brief

Not all fixes may have gone live to WMF sites at the time of writing; some may not be scheduled to go live for several weeks.

At the time of writing, 18 BRfAs are active. As usual, community input is encouraged.

















Wikipedia:Wikipedia Signpost/2012-11-12/Technology_report