Wikipedia as an anchor of truth

Op-Ed

Wikipedia as an anchor of truth

Wikipedia has been criticized as being inherently unreliable, and we ourselves warn users not to rely uncritically on the information in Wikipedia; it is ironic to see it now used as an anchor of truth in a seething sea of disinformation. AI models are prone to hallucinating, that is, giving false answers with confidence and corroborative detail to things that simply are untrue. Can using Wikipedia help to at least spot these mistakes, and are the new search engine AIs using them in ways that will actually help prevent hallucination?

DuckAssist and Wikipedia

Following in the footsteps of Bing, the Internet search engine DuckDuckGo has rolled out DuckAssist, a new feature that generates natural language responses to search queries. When a user asks DuckDuckGo a question, DuckAssist can pop up and use neural networks to create an instant answer, a concise summary of answers found on the Web.

A problem plaguing large language model-based answerbots and other chatbots are so-called hallucinations, a term of art used by AI researchers for answers, confidently presented and full of corroborative detail giving seemingly authoritative verisimilitude to what otherwise might appear as an unconvincing answer – but that are, nevertheless, cut from whole cloth. Using another term of art, they are pure and unadulterated bullshit.

Gabriel Weinberg, CEO of DuckDuckGo, explained in a company blog post how DuckAssist uses sourcing to Wikipedia and other sources to get around this problem.^[1]

“

DuckAssist answers questions by scanning a specific set of sources – for now that's usually Wikipedia, and occasionally related sites like Britannica – using DuckDuckGo's active indexing. Because we're using natural language technology from OpenAI and Anthropic to summarize what we find in Wikipedia, these answers should be more directly responsive to your actual question than traditional search results or other Instant Answers.

”

Keeping AI agents honest

The problem of keeping AI agents honest is far from solved. The somewhat glib reference to Wikipedia is not particularly reassuring. Experience has shown that even AI models trained on the so-called "Wizard of Wikipedia", a large dataset with conversations directly grounded with knowledge retrieved from Wikipedia,^[2] are not immune to making things up.^[3] A more promising approach may be to train models to distinguish fact-based statements from plausible-sounding made-up statements. A system intended for deployment could then be made to include an "is that so?" component for monitoring generated statements, and insisting on revision until the result passes muster. Another potentially useful application of such a system could be to flag dubious claims in Wikipedia articles, whether introduced by an honest mistake or inserted as a hoax. (Editor's note: this has been attempted, with some success, here.)

References

^ Weinberg, Gabriel (March 8, 2023). "DuckDuckGo launches DuckAssist: a new feature that generates natural language answers to search queries using Wikipedia". spreadprivacy.com. DuckDuckGo. Retrieved March 9, 2023.
^ Emily Dinan; Stephen Roller; Kurt Shuster; Angela Fan; Michael Auli; Jason Weston (28 September 2018). "Wizard of Wikipedia: Knowledge-Powered Conversational Agents". ICLR 2019 (International Conference on Learning Representations). Retrieved 11 April 2023.
^ Dziri, Nouha; Milton, Sivan; Yu, Mo; Zaiane, Osmar; Reddy, Siva (July 2022). "On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?" (PDF). Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics. doi:10.18653/v1/2022.naacl-main.38. S2CID 250242329. Retrieved 11 April 2023.

← Previous "Op-Ed"

Next "Op-Ed" →

In this issue

26 April 2023 (all comments)

Op-Ed

Discuss this story

These comments are automatically transcluded from this article's talk page. To follow comments, add the page to your watchlist. If your comment has not appeared here, you can try purging the cache.

There was a recent wikimedia-l discussion of systems such as Google's RARR (Retrofit Attribution using Research and Revision) to prevent LLM hallucination: paper, Twitter thread, video. That system is clearly not currently part of Google Bard. Sandizer (talk) 15:26, 26 April 2023 (UTC)[reply]
I'm skeptical that models can simply be trained to distinguish between factual and non-factual information. To do that successfully, I think they would actually need to be able to internally represent semantic content, reason about it, and verify that against trusted prose. Something like Cyc might be the seed of the first part of that; having all of it might be equivalent to artificial general intelligence, which I expect is decades away. -- Beland (talk) 02:10, 1 May 2023 (UTC)[reply]
While it is not quite clear how it does it, GPT-4 answers questions of the nature "is this statement factually correct?" with higher accuracy than a random oracle, especially if you also ask it to explain in detail why it thinks so. It helps if it can access the Internet, but a current weakness is that it cannot discern which sources are reliable and which are not. GPT-4 also appears capable to a considerable extent of reaching correct logical conclusions from statements provided in natural language. Again, researchers do not quite understand this, but apparently the patterns and meta-patterns needed to pull this off are sufficiently represented in the corpus on which it has been trained. I am not so optimistic about how far off AGI is; I expect that it will take less than a decade before AI models can not only reliably translate statements in natural language into formalisms like Cyc and OWL, but even devise extensions to these frameworks to represent aspects currently not covered. --Lambiam 14:58, 8 May 2023 (UTC)[reply]
Quote from the article: "A system intended for deployment could then be made to include an "is that so?" component for monitoring generated statements, and insisting on revision until the result passes muster." Ding ding ding, bingo, give lolly. "A long time after inventing automobiles, humans began to realize slowly that perhaps all four of the wheels could have brakes on them, and some sort of so-called 'Seat-Belt' might possibly keep the humans' gelatinous innards from interacting with the dashboard. Humans thought about and talked about such newfangled concepts for quite some time before they gradually decided to start tentatively pursuing them." Lol. But seriously, if LLMs themselves cannot provide the "is that so?" component (as Beland mentioned), then humans need to get serious about chaining (shackling) the LLMs in series behind various things that can provide it. For example, an LLM will gladly hallucinate a totally fake reference citation, pointing to a fake/made-up book. Humans should already be capable of building some software that says, "If I can't find that book in WorldCat or in Google Books or in other database-full-of-real-books-X, within the next X milliseconds or seconds, then you're not allowed, Mr LLM confabulator, even to release your answer to the human who asked you the question, at all." It wouldn't be the full ontologic sanity check that Beland mentioned, but there's no excuse not to have at least this low-hanging fruit to start with, ASAP. Quercus solaris (talk) 01:50, 2 May 2023 (UTC)[reply]

The Signpost: doing it for free since 2005.

Home

About