Monday, November 12, 2012

IMDB voting is a joke

I go to the Internet Movie Data Base today to see how voters take to "The House I Live In," an Oscar-contending documentary about the vast amount of money spent prosecuting non-violent drug users in America. I see it has an unspectacular voter rating of 6.3 out of 10.

Then I look at the voter breakdown. 129 out of 235 voters (55 percent) have given the movie a 10.  62 voters (another 26 percent) have given it an 8 or a 9. So more than four out of five voters have given it a rating of 8 or above, yet the overall "weighted" rating is 6.3. How can this be?

The explanation on IMDB is a combination of obfuscation and gobbledygook. The site won't explain how it assigns different weights to votes, though some users have speculated in message boards that it automatically throws out 1s and 10s or devalues them until a certain number of people have voted. There's also speculation that people who regularly vote on films get their votes counted more heavily, as if prolific voters were more honest or intelligent.

When you search the FAQ section for an explanation, you get a longer version of the paragraph below:

IMDb publishes weighted vote averages rather than raw data averages. Various filters are applied to the raw data in order to eliminate and reduce attempts at 'vote stuffing' by individuals more interested in changing the current rating of a movie than giving their true opinion of it. The exact methods we use will not be disclosed. This should ensure that the policy remains effective.

The site justifies weighting answers by comparing its ratings to those used in assessing automobiles: A car that gets a 5 for looks, a 5 for gas mileage, a 5 for price and a 1 for safety may not get an overall average of 4, because safety is more important than the other factors. But how does one distinguish among people who are rating only one thing, the overall effectiveness of a movie?

Nobody wants a ballot box to be stuffed. (Well, Vladimir Putin and certain African dictators do, I guess.) I never vote on IMDB myself, because my reviews give my opinions, should anyone want them. No doubt rabid fanboys toss 10s around like confetti when the latest Batman or Avengers movies come out, and clever hackers can confound almost any system.

But when the mast majority of voters give a grade of 8 or higher to a serious documentary, and the overall grade is 6.3, something's wrong here. Do we need to get the electoral college involved?




6 comments:

Anonymous said...

"Nobody wants a ballot box to be stuffed. (Well, Vladimir Putin and certain African dictators do, I guess.)"

[cough]Obama[/cough]

Anonymous said...

@2:24 Still bitter, Bro?

John said...

Anonymous 3:39, maybe he's just paying more attention than you?

Anonymous said...

Rotten Tomatoes is much more reliable than IMDB...good article.

Anonymous said...

Trust me, my friend: I have years of experience with the IMDb's idiotic "policy". It's a scourge, and a blight. Unfortunately, a lot of people take the site seriously, or are too obtuse to click-through to the actual vote breakdowns and see the real averages. It's a deeply flawed paradigm that needs fixing. One of the issues: The Top 1000 Voters. I've seen posts by people claiming they're one of these Top 1000 "elites," and it's truly sad to see them wallowing in their perceived "power". Until the IMDb staff diminishes the "weight" of the Top 1000, you will continue to see good films skewed downward by a single negative vote, or artificially skewed upward by a single positive vote from one of these 1000 no-life IMDb "elites". P.S. To the conspiracy theory kooks posting before me: 332-206...live it, learn it, love it.

shawn said...

I'm pretty sure sites like www.imdbstarmeter.com are able to make fake votes on their website. This is a joke and there system is simply flawed.