Friday, January 13, 2006

Sorting the Wheat from the Chaff: The Wisdom of "Digging"

That’s the biggest challenge for an individual crawling through the modern World Wide Web. With the explosion of available information on almost every topic imaginable the modern information consumer must be efficient and discerning. There’s far too much to read it all, and we’d all like to productively invest the time we do spend reading and digesting quality information.

Earlier this week I wrote about the tension between large, active user bases and traditional notions of “truth,” in my discussion of facts and errors in Wikipedia and the Encyclopedia Britannica. Today, in a nice open-ended post on Gene Smith’s , Smith raises a related issue. Rather than worrying about the “truth” of online information, a la Wikipedia, Smith wonders how we can discern between the “quality” and the “popularity” of information.
read more...
The Problem

Smith gently criticizes , one of the most popular social bookmarking and blogging sites, as being overly attuned to popularity, and not focused enough on quality. You can argue that sites like Digg represent the ascension of quality information – users vote up quality stories and commend them to the attention of the time-strapped information consumer – but I, like Smith, am skeptical of this claim because I believe there is a difference between quality and numeric popularity (i.e., a large number of votes).

But therein lies the rub: How do you distinguish between the two? Smith proposes an admittedly vague “betting” scheme that would pit different stories against each other. Another solution is the one that Google employs – effectively weighting certain votes (in the form of links) more heavily than others, a highly undemocratic system but one which works in many situations.

But neither of these solutions address the problem that the Wikipedia / Britannica debate makes clear. Any label, whether it be one of “fact” or of “quality”, that derives its warrant largely from strength of numbers (whether they are links, or votes, or bets, or anything else that can be tabulated) is inherently threatened by what John Stuart Mill termed the “tyranny of the majority.”

Do You Vote Before You Think?

In his essay , Mill points out that the democratic majority is dangerous just because it does not require a ruling tyrant in order to act tyrannically. “Society can and does exercise its own mandates; and if it issues wrong mandates instead of right…it practices a social tyranny more formidable than many kinds of political oppression…”

While it is true that this risk – the runaway majority - is a risk run by every committedly democratic institution the online democracy, embodied by sites like Digg, pose additional dangers above and beyond those that attend more traditional democratic activities, such as political elections. First, and most obviously, is the problem of anonymity and the attendant risk of cheating. Much of the voting that happens online is far from transparent and, unlike in political elections, there are no recounts.

Secondly, and more debatably, is the relative online presence of the “herd” mentality, which Smith’s post draws attention to. At first blush it appears to me that, in an environment of hyperlinks, dynamic content, and instant voting there is a tendency to receive votes that are not as well considered. The effort of voting (pasting a link or “digging” a story) is minimal and the consequences of “mis-voting” are seemingly inconsequential.

But the aggregate result of unconsidered “votes” may yet be substantial. “Popularity” is frequently a self-fulfilling and self-referential phenomenon and the ease of voting, combined with the “herd” mentality (“if 463 people think this story is cool then it must be cool. I’m going to vote for it too”) might make a substantial impact on the ultimate popularity of a quantum of information, independent of its quality.

I’m not suggesting that the majority of Digg users vote multiple times, follow the crowd, or are anything less than considerate community members who vote for quality, however they personally define it. But, certainly, some voters do fit this profile.

The question is how many? If the number is significant – and even a relatively small minority can be significant where the tolerance for error is small – then we have reason to share Smith’s concern that there may be disconnect between popularity and quality.

The Solution (?)

The United States does not possess anything like a voter qualification test. The only thing we do have is a law that requires, in most political elections, that the voter be 18 years or older in order to vote. One (but not the only) rationale for this requirement is that individuals, having reached the age of majority, can by and large be expected to vote intelligently and responsibly. You can take issue with whether or not people in this country vote at all “intelligently”, but that at least is the claim.

In addition almost any election that requires you to walk into a voting booth and fill out a ballot is also going to require proof that the election is applicable to you in some meaningful way. Frequently, but not always, this is established through a residency requirement. For instance, citizens of Massachusetts can’t vote in the school board elections in Kansas, no matter how much they might wish it. Similarly, citizens of foreign countries can’t vote in American presidential elections. And so on.

Online we have nothing of the sort. Anybody who can use a mouse and has an internet connection can vote on sites like Digg. While there are (necessarily incomplete) protections in place to discourage multiple voting and other forms of cheating, there is nothing remotely resembling a voter qualification requirement, either of the residency or the age of majority variety.

Does this ultimately matter? I’m not sure, nor am I sure what form such an online voter qualification would take. Screening or barring voters because they are “unqualified” would be an incredibly tricky proposition and one that I’m uncertain would do much to improve the situation.

But in a firmly democratic environment such as Digg, where the “popularity” and “quality” of the “elected” information diverges, it’s certainly some food for thought….

4 Comments:

Anonymous Anonymous said...

Nice post Tim. I think the dilemma is equally along the lines of defining what "quality" information is, and realizing that it changes from person to person. For example, I don't read People or Us or any other gossip magazine, so information regarding Brad and Angelina's new baby is, except for the cultural reference value, completely meaningless to me. Thus, articles promoted by Digg voters may, in fact, reach their target audience: each digg is merely a confirmation that a certain person found it interesting. It says nothing more, and nothing about the quality of the particular article--and here's the key: I'm not sure that Digg would pretend that it does. They are merely operating under the paradigm of popularity, a service without membership requirements, and it probably does its job of presenting the most pertinent information in the most populist fashion.

Therefore, it's possible that even the maligned votes motivated by "well others found this interesting so i'm going to vote for it as well" are valid, or even desired, for a strictly democratic service. The interpretation would be that people want to know about what other people know about, so each vote for a particular article is both a vote for its interest AND for its value as a piece of cultural reference. There are other voting schemes that disallow the voter from seeing results until they have voted themselves, but these fail with services such as Digg whose usefulness comes in pre-selecting information for the casual, or even active, viewer.

When you say that the problem is quality, you're right, but you really mean quality to YOU. And I'm in the same boat. I want "quality information", defined by what I would, without persuasion, declare read-worthy after having read it. And that's pretty tricky.

Fri Jan 13, 08:03:00 PM EST  
Anonymous Anonymous said...

Oh, and I forgot to mention: you've got to read the CODES in the articles. They're ALL AROUND US.

Fri Jan 13, 08:04:00 PM EST  
Blogger Tim Kanwar said...

Good points – the definition of “quality” is, as you point out, a highly subjective one. Quality content for me may not be the same as quality content for you, or for the readers of People, or the users of Digg. That’s entirely true.

Still, I think what I’m suggesting is not that the enshrinement of one particular conceptualization of “quality” but, rather, the recognition that there is a meaningful difference between quality and quantity that numerosity alone doesn’t capture.

And with that in mind I’m not sure I entirely agree that every vote for a particular nugget of information represents a nod of the head in favor of its “value as a piece of cultural reference.” I entirely agree that, for some items of information, the fact that other people are talking about it at the water-cooler (or reading about it, or blogging it, or what have you) is significant, and a manner of quality in its own right. But I don’t think that negates the problem of voting without considering the value of the underlying object.

To make this point clearer, let me try a different hypothetical. Imagine that, instead of people following the herd, that everybody voted (or Dugg) blindly. Of, say, one thousand items, the one that was the most “popular” following a tabulation of blind votes would stand little to no chance of being of the best “quality” under almost any understand of that word.

So do I think online voting is blind? Not necessarily. But I’m not convinced that it is quite as efficient as you suggest. You suggest that “each dig is merely a confirmation that a certain person found it interesting.” And if the voter did find it interesting herself then a Digg is likely to be, in essence, a vote in favor of popularity (“Others found it interesting, and that’s interesting/relevant to me, so I’m going to vote for it.”).

I’m just not sure this is always the reality of the situation. While I don’t think anybody is voting with a literal blindfold over their eyes, I do think that, especially on a site as populous as Digg, there are some users who are voting with metaphorical blinders on.

But, of course, this is a proposition that neither of us can marshal much in the way of empirical evidence for. But it’d be neat if there was a good way to test this.

Anybody have any suggestions?

Fri Jan 13, 09:23:00 PM EST  
Anonymous Anonymous said...

Editors?

Seriously, with so much web 2.0 we are in the dangerously deep and muddy water of ignoring a millenium of media history. Editors are not an evil. The question is how do you use them? What do you use them for? What role can they play in the citizen media context?

The sad, sorry fact is that so many commentators and technologists are naive and untutored in the history of media...making experiments like Digg amusing...but far more limited than a publication that is only edited...this is too bad. a sorry indictment...but a deserving one.

Tue Jan 17, 12:12:00 PM EST  

Post a Comment

Links to this post:

Create a Link

<< Home