The Problem with Wikipedia

Jason Scott recently gave a talk about The Great Failure of Wikipedia. The audio is available at the Internet Arcive: The Great Failure of Wikipedia (April 8, 2006). Scott uses specific examples to discuss the problems that face Wikipedia. Despite the appearance of veracity and authority, Wikipedia faces significant challenges before it embodies “the availability of the sum
of human knowledge to everyone on Earth for free.”
Wikipedia remains a great place to be an information tourist, but falls short as a serious information resource. The anti-expert bias that Scott notes in the Wikipedia editorial process will continue to keep the actual Wikipedia from becoming anything more than a novelty for information professionals.
In the NY Times, Randall Stross writes: Anonymous Source Is Not the Same as Open Source “Wikipedia, the free online encyclopedia, currently serves up the following: Five billion pages a month. More than 120 languages. In excess of one million English-language articles. And a single nagging epistemological question: Can an article be judged as credible without knowing its author? Wikipedia says yes, but I am unconvinced.”
At Concurring Opinions, Laura Heymann notes a case in the US Court of Federal Claims that discussed the reliability and admissibility of Wikipedia articles: Concurring Opinions: Wikipedia in the Courts: “In an opinion released in February, the U.S. Court of Federal Claims scolded a special master in a vaccine injury case for sua sponte supplementing the record with ‘medical ‘articles’ on afebrile seizures’ that she located on the Internet.”
Lore Sjöberg, Wired: The Wikipedia FAQK: “The Wikipedia philosophy can be summed up thusly: ‘Experts are scum.'”
danah boyd offers some insight on the Wikipedia editorial process: on being notable in Wikipedia: “People wanted “proof” that i was notable; they wanted proof of every aspect of my profile. Then, when people in my field stood up for my entry in the discussion for deletion, they were attacked for not being Wikipedians.”
Do any readers have academic Lexis/Westlaw or Hein access? Could you run a search to see if any law review articles cite to wikipedia, and if so, how many?
Previously: Wikipedia Woes, Wikipedia and Authority
(edited 4/24 to add Concurring Opinions, NYT and Wired links)

Wikipedia and Authority

I initially posted this as a comment, following up to a comment by “Y456two” on Wikipedia Woes, but here it is as its own entry, because, well, it is rather long.

This portrays a fundamental misunderstanding of what Wikipedia is. Adding editors amounts to turning Wikipedia into the Encyclopedia Brittanica. Why would you want to do that? Don’t we already have an Encyclopedia Brittanica?

No– it represents the fundamental gap that separates what Wikipedia is from what it seeks to become. A user driven Wikipedia edited by panels of subject experts in various fields will be both more comprehensive AND more authoritative than a traditional encyclopedia.

If you believe that the masses are not smart enough to make their own judgements about the veracity of what they read, then, yes, absolutely, we should have a heavily regulated Internet, publishing industry, and media (sounds like China, don’t it?)

I do think that heavy internet users and information professionals over-estimate the information literacy of the average internet user, but private editorial control on a private web site is a long way from state regulation. Why do we trust articles in the NY Times more than the Washington Times or the West Podunk Pioneer Press? A reputation for accuracy and veracity. Why would one prefer to buy from a seller on eBay with a +300 feedback rating than one with no feedback rating? A reputation for being an honest dealer.
What do we know about the authors of a wikipedia entry? Why is it authoritative? We only know that wikipedia as a whole is generally accurate. But because each article is written by a different group of authors, researchers do not have an easy way of figuring out which articles are accurate and which contain blatant falsehoods or smaller inaccuracies.
Adding a series of editorial boards comprised of acknowledged experts in various fields to monitor wikipedia entries will go a long way towards increasing the accuracy and trustworthiness of wikipedia as a whole. And it is possible to do this without becoming a Britannica clone– in fact, doing so would take advantage of the same internet and collaborative technologies and processes that make wikipedia possible. It just happens to also acknowledge the fact (and, yes, it is a fact) that some people simply have more knowledge and experience in various subject matters than others. In the wikipedia model, these boards would not be simply appointed from the get-go, but could be composed of flexible memberships, with new members joining either by distinguished work in academia or business as well as by distinguished contributions to wikipedia itself.
At the very least, Wikipedia could post a list of the contributors who wrote or edited each article. This would make it possible for researchers to find out more about the authors of each individual article and make an educated decision whether to trust the accuracy of the wikipedia article.

I could say that the ‘blogosphere’ needs editors. I could claim that the problem with blogs is that there isn’t some credentialed editor who controls what is posted.

Unlike Wikipedia, the “blogosphere” is not a single entity. Individual blogs have attributes that establish their reputation for accuracy and veracity. For example, you can read my biographical information and see that my posts carry less intellectual heft than those of Prof. Goldman, for example. Unlike the millions of individual blogs posted by named or pseudonymous authors, Wikipedia presents itself as a centralized authority and strips away many of the signs that make it possible for an individual researcher to decide whether a single article is reliable. We can’t look to the author’s biography. We can’t judge the publisher’s credibility, because this publisher will post anything. We can’t look at the professionalism of the page design. The Wikipedia brand takes credibility from articles that justifiably grant credibility and it also lends credibility to articles that are not worthy of it.
The problem with Wikipedia is that it lends its brand to anyone. In the trademark context, a trademark owner who nakedly licenses a mark to anyone without keeping track of the quality of goods sold under that mark may lose the right to defend the mark. Since a trademark is meant to protect consumers and indicate the source of a good or service, nakedly licensing the mark strips away value from the mark. By allowing anyone and everybody to edit entries on wikipedia, wikipedia may squander any credibility it has attained.

As for Eric Goldman, I suppose he would be surprised to know that Usenet continues to thrive and be useful to millions of users every day.

I would challenge the idea that Usenet continues to thrive. I have yet to even load a Usenet news reader on my Powerbook, which means that I haven’t delved into that thriving medium in at least nine months and haven’t missed it a bit. People may still use newsgroups, but they have long since ceased to be relevant. How many average internet users can recognize that “alt.nerd.obsessive” denotes a newsgroup?

Here is the heart of the issue: do we trust people?

We trust people to the extent that the people have as full information as possible to make decisions. As another analogy, this is the driving principle behind securities law– we have a policy bias towards requiring publicly traded corporations to disclose information– because this allows investors to make informed decisions. The more that identifying information is witheld, the less reason we have to trust

Wikipedia Woes

Wikipedia is one of the best sites on the internet– volunteers compile information about esoteric topics and the entire compilation is a giant guide to the universe. The beauty of the site is that the internet community has created a vast encyclopedia without a single editor.
Nature compared Wikipedia and Encyclopedia Britannica and found that the upstart contains only slightly fewer errors: Internet encyclopaedias go head to head: “The exercise revealed numerous errors in both encyclopaedias, but among 42 entries tested, the difference in accuracy was not particularly great: the average science entry in Wikipedia contained around four inaccuracies; Britannica, about three.”
Wikipedia is becoming more frequently cited as a trusted source, despite potential for inaccuracies and often amateur writing and organization (just like this blog!) Evan Brown reports at Wikipedia and the courts: “lthough not everyone is convinced that Wikipedia can be trusted to always tell the truth, it is interesting to note that in the past year or so several courts, including more than one federal circuit court, have cited to it to fill in background facts relevant to cases before them. ”
The problem with Wikipedia is that the internet community has created a vast encyclopedia without a single editor. Entries can contain factual inaccuracies or present topics in a skewed, biased manner. Wikipedia needs editors. Who chooses the experts for a particular field?
At the Volokh Conspiracy, Orin Kerr finds an interesting relationship between the level of general interest in a subject and the accuracy of that subject’s Wikipedia entry: Checking in on Wikipedia’s Patriot Act Entry:

I have found Wikipedia entries to be quite helpful when the topic is something esoteric. It seems that when fewer people care about a topic, the better the entry tends to be. When lots of people care about something, lots of people think they know something about it — or at least more people feel strongly enough that they want to get their 2 cents worth into the entry. When lots of people have strong opinions about a topic, even uninformed ones, the Wikipedia entry for that topic ends up being something like Tradesports betting odds on who Bush would pick to replace Justice O’Connor. It’s an echo chamber for the common wisdom of the subset of people who use the site more than anything else. And if the views in the echo chamber happen to be way off, then so is the entry.

This suggests that the common wisdom may be entirely backwards. Instead of greater interest leading to greater accuracy, the more people who have a strong interest in a topic, the more likely it is that discredited or inaccurate theories will find their way into that topic’s Wikipedia entry. Vocal critics of a widely accepted theory may be more likely than well-respected experts to spend time crafting the Wikipedia entry, so that the end result is that the Wikipedia entry is more likely to reflect the generally discredited minority view.
In an op-ed piece in USA Today, John Seigenthaler discussed A false Wikipedia ‘biography’: “I had heard for weeks from teachers, journalists and historians about “the wonderful world of Wikipedia,” where millions of people worldwide visit daily for quick reference “facts,” composed and posted by people with no special expertise or knowledge — and sometimes by people with malice.”
Mike Godwin thinks that this problem is not limited to Wikipedia, but is endemic of the Internet as a whole: Wikilibel: “To me, the notable thing about this incident is that it seems to have given John and others doubts about Wikipedia in particular, when in fact the problems he sees are endemic to the Web and the Internet at large.”
Unlike posting a random website on the internet at large containing the same defamatory text, posting the information at Wikipedia gives it credibility. The first place most internet users look to assess the credibility of a piece of information is the source. Because Wikipedia contains a growning number of thorough, accurate and well-written entries, Wikipedia as a whole is gaining a reputation as a trusted source for information. According to the Wikipedia entry about Wikipedia, “Articles in Wikipedia are regularly cited by both the mass media and academia, who generally praise it for its free distribution, editing, and diverse range of coverage.” An incomplete, incorrect or defamatory article posted to Wikipedia gains from the authority of the accurate entries.
Eric Goldman believes that Wikipedia Will Fail Within 5 Years: “Wikipedia inevitably will be overtaken by the gamers and the marketers to the point where it will lose all credibility. There are so many examples of community-driven communication tools that ultimately were taken over—-USENET and the Open Directory Project are two that come top-of mind.”
Unless Wikipedia starts to implement a strong editorial policy, the entire project will become suspect because of entries like the one about Siegenthaler. Wikipedia is at a critical point in that it has enough entries and reputation that by continuing to allow anyone to edit any entry may harm the future development of the project.
As with any controversial topic these days, some lawyers are already preparing a Wikipedia Class Action.