External videos
Wikipedia Wars?, BBC, 24:30[1]

Carl Miller is the Research Director of the Centre for the Analysis of Social Media (CASM) at Demos, a London think tank. He has reported several segments on the BBC's Click program, including Wikipedia Wars? about possible editing on Wikipedia by the Chinese government. His interests include how social media is changing society, digital politics and digital democracy, information warfare and online disinformation, digital and citizen journalism, and "fake news". He is a Visiting Research Fellow at King's College London.

This interview covers aspects of information warfare, and the strength and weaknesses of Wikipedia's defenses against editing by governments

Signpost: So how did you get a job investigating government editing of Wikipedia?

Carl Miller: For almost a decade now I've worked at a think tank called Demos. It's basically a research-focussed charity that works on understanding current, important live political questions.

In 2012, myself and colleagues were convinced that the rise of social media would totally transform society and also how it could be researched. So we founded the Centre for the Analysis of Social Media, a mix of Demos researchers and technologists based at the University of Sussex that would work together to study digital life.

The Centre researched all kinds of things that digital technology was influencing and changing: conspiracy theories; literacy; radicalisation, political campaigning, trust, and major events.

In 2018, I took a step back to try to tie all the changes we were seeing into a larger picture. It became a book, called The Death of the Gods, and focussed on how power was shifting across society in often overlooked or invisible ways.

One of the areas on which the book concentrated was information warfare. This was part of the undeniable rise of the power of states online, and I saw that the interests of states were often colliding with those of online communities and their norms and cultures that had emerged and grown online.

After the book, I teamed up with the BBC to take some of the most important issues and stories I'd come across, and put them on TV. I've gone out to Kosovo to meet fake news merchants. I've tried to recreate myself with data, and looked at how tech can reinvent democracy. And now, of course, I've looked at whether information warfare has extended beyond Twitter and Facebook.

SP: In August 2018 you wrote an article in The New Statesman, Wikipedia has resisted information warfare, but could it fight off a proper attack? mostly using the example of the Russian government, saying that Wikipedia has good defenses against some types of organized POV pushing editing, but not against other types. John Lubbock, from WikiMedia UK, responded in another article in The New Statesman, It'd be a lot harder for foreign governments to hack Wikipedia than you think stressing the time it takes to become an administrator, our strong and diverse community of editors, and the transparency of each edit ever made on Wikipedia. Can you describe what types of attacks Wikipedia is prepared for and what types it is not?

CM: I think Wikipedia is stunningly good at protecting itself from vandalism – from outright breaches in its content policies, or patently malicious attempts to deface content. But the possible threat from organised actors is, I think, substantially different. Not vandalism but entryism.

This would be where any organised and determined group – state or otherwise – would attempt to strategically change the content of Wikipedia over a long span of time. They would create and support large groups of people to join its open community, build reputations and prestige within it, run for positions of office and actually use the processes that exist to change both the content that Wikipedia hosts and the policies that govern it.

Of course, as John rightly points out in his article, there will be other editorial communities that will push against this. But it would turn into a long and exhausting struggle and I'm just much less confident that – especially across the non-English Wikipedias – that a volunteer community would necessarily outlast one that has significant levels of state support behind it.

It's probably also worth re-stating the caveats of our investigation at this point. We cannot be sure who were behind the edits that we found, or why they were done. We don't know how widespread the practice is, and there was nothing that directly tired these edits back to the Chinese government. However, I think the evidence that we found points to the possibility of the threat; one I don't think should be ignored.

SP: What countries or government agencies do you suspect have the largest Wikipedia editing programs? the most dangerous editing programs?

CM: We just don't know. Attributing covert state-actor activity on the Internet is almost impossible for independent researchers and journalists at the best of times. Wikipedia is open for researchers to look at, but vast. It may exist, but I certainly haven't seen a detection regime on Wikipedia that is at least partly automated, systematic and multi-lingual. Unless we have that, we have no ideas what part of Wikipedia are being targeted and what kinds of changes are happening, much less who are behind these edits or why.

I think it is very clear, however, that Wikipedia is one of the most priceless patches of real-estate anywhere on the Internet. It has billions of page views, is highly trusted, populates Google's knowledge boxes, is extremely highly ranked by Google's Search, and gives answers to Siri and a number of other digital assistants. It's an important service in its own right, and other even more important services depend on it too. Given its value, it is likely there is a range of actors that have an interest in its manipulation.

SP: When can you say that a country has "gone over the line" in its interactions with Wikipedia? If you were to propose an international anti-infowar treaty, what rules would you include?

CM: I think your readers are far better placed than me to know where this line is – but I think what crosses the line is that it is a country, rather than a volunteer community that is making the edits. A state strategy of edits for geo-politics seems to run clearly against the basic reasons why Wikipedia was set up in the first place. Unfortunately, as with influence operations elsewhere, the distinction most key is unhelpfully the one that is least clear to us: intention.

An international treaty on this – like any international treaty – would have to be predicated on the idea that this kind of activity is mutually damaging. In other words, that it is in the interests of all states to step back from strategic editing activity. Whether that is actually the case, of course, remains to be seen.

SP: There was an interesting moment on the BBC program when you asked Taiwanese editors “What does Wikimedia global think about all of this?” and they started giggling. Were they telling you that the WMF doesn't realize the extent of the problem, that they don't want to address the problem, or that they are powerless to address the problem?

CM: You'll need to speak to the editors themselves to learn exactly what they meant by that. But (and this of course is only my interpretation) I got the sense that, yes, they believe that their complaints had been overlooked, principally because of the problem of language. Much of the editing had to do with the pruning of language to change its implication and even subtle connotation. These changes might be highly significant in Mandarin Chinese, but – I got the sense – were often difficult to communicate to an audience that only spoke English. Some of the editing activity and the abuse Taiwanese Wikipedians received was being lost in translation.

SP: I'm sure you've communicated with people at the WMF. What do they think of your analysis? What would you suggest that they do?

CM: Here, it's worth stating WMF's full response, which we didn't have a chance to do within the broadcast:

They went on to say:

"In terms of what you mention about concerns raised by our Taiwanese community, I wouldn't say it's fair to characterize the Foundation's actions as overlooking concerns. We take issues raised by our communities seriously, and we continue to engage with our Taiwanese community."

SP: Ultimately, how do you think the problem of governmental editing of Wikipedia will be resolved?

CM: Since the discovery of significant, state-based manipulation of their platforms in around 2016, the tech giants have built information integrity, safety and ‘conversational health' teams very rapidly. If you look at what these teams do, they're engaged in strange kind of arms race with a range of actors – including states – that want to game, manipulate and subvert their platforms. They call the problem ‘coordinated and inauthentic' activity.

They use data science and scaled heuristics to model what this activity looks like on the platform at scale, and how it's different from non-suspicious activity. That is an extremely dynamic task, as attackers try to camouflage their activity, each platform's defenders constantly have to develop new ways of detecting it. Then they have constant operational interventions and multi-layered challenges to remove manipulative content, ban accounts, and de-rank suspicious activity.

In one way, the challenge for Wikipedia is the same as for the tech giants, and in another way it is completely different. I think it is possible to build out similar approaches to detecting suspicious editing activity: creating models to identify possible geo-political intent, and finding patterns and anomalies that point to irregular, unusual or inorganic behaviour. At the very least, a system like this would be able to notify Wikipedia's editors about the places they need to patrol the most.

The new part of the challenge, however, is how to do this without the business models of the tech giants behind you. Integrity teams in platforms of equivalent sizes to Wikipedia number from the hundreds into the tens of thousands of full-time staff. Wikipedia's model is completely different.

This I think is an area where charitable foundations and trusts need to step in to support a new, non-profit capability to protect not only Wikipedia but the other parts of the open web that doesn't have the commercial ability to build out protective teams as part of a salaried staff. A new kind of citizen information integrity needs to be carved out, that leverages the enormous passion, skill and knowledge of Wikipedia's editor-base, with the constantly updating, dynamic kind of detection-to-mitigation tools that the tech giants have developed. That is where I would love the conversation to head next.

SP: What will happen if your solution, or at least some solution, isn't implemented?

CM: This case might represent something bigger: how do the parts of the Internet without a business model respond to the rise of online manipulation?

Either some kind of public-interest, non-profit alternative to the tech giants' mitigation strategies needs to be found, or, in my opinion, we'll see this kind of split emerge where the commercial internet maintains some kind of protection, and everything else becomes more vulnerable.

References

For related coverage, see: In focus and Community view