Friday, January 13, 2006

Grading Wikipedia: A Closer Look at the Scores

Last month the journal Nature published a study comparing the accuracy of the online, user-edited encyclopedia Wikipedia with the traditional research staple the Encyclopedia Britannica. In a recent article in the New York Times, George Johnson goes behind the Nature numbers and reveals that the competition between the two is simply too close to call.

The Nature study found an average of four errors per article in Wikipedia, compared to three in Britannica. But a close examination of those errors leads Johnson to ask, "Just what counts as an error?"
In many of the situations that Johnson investigates, the error represents little more than a matter of judgment. For instance, Britannica referred to "Croton" as the home of the Greek mathematician Pythagoras. Other options included "Crotone", "Crotona", and "Kroton".

Without digressing too far into the metaphysics of truth it seems to me that, if not now then someday, tools like Wikipedia may legitimately act as their own authority. A trivial example: What is the proper pronunciation of the town "Edinburgh". The answer, of course, is it depends. If you live in Scotland it's Edin-burrow (well, it is if you're an American in Scotland) and if you live in Indiana it's likely to be Edin-berg. Which is "right" and which one is an "error"? You can argue all you want about historical roots and proper pronunciation but I don't suspect there really is a "right" answer here.

What's the point of all these burrows and bergs? For years institutions like the Encyclopedia Britannica have operated as a factual authority, lending the descriptor of "truth" to what were really nothing more than discretionary judgments. Measured and well-supported, but judgments not truths nonetheless.

Wikipedia will not fundamentally change that. There is never going to be one "true" spelling for the birthplace of Pythagoras, or one "true" pronunciation of "Edinburgh." But what there will be is a vote by a majority, rather than a fiat issued by the authority.

Johnson titles his article "The Nitpicking of the Masses vs. the Authority of the Experts." What the Nature study really represents is an evaluation of the Authority of the Masses vs. the Authority of the Experts. It won't be long until resources like Wikipedia can lay legitimate claim, in matters of factual judgment, to a more persuasive authority than even a journal like Nature.


Post a Comment

Links to this post:

Create a Link

<< Home