Curmudgeon Gamer
Curmudgeoning all games equally.
12 February 2007
Correlation between Sales and Review Scores
There is a wealth of data in the Next-Gen.biz post today. Regrettably, I did not see a spreadsheet of data to play with. I have made one which I will upload later. Using my copy of the data I made a scatter plot of sales versus review scores. I excluded Madden since it is an aberration, an outlier. It would be at 6500 on the horizontal axis and 85% on the vertical one, so you can see how far out it would be. (Click for a larger version.)
The correlation coefficient shows whether there is a correlation between two sets of data. The closer it is to 1.0, the more closely correlated the data are. The closer to 0.0, the less a correlation. The correlation between unit sales and revenue, for example, is 0.97, which shows that higher sales is closely correlated with higher revenue.

According to OpenOffice, the correlation coefficient between unit sales and score is 0.29. I think we can speak of 0.29 as being a bit low. There is a closer correlation between revenue and review score at a 0.38 correlation coefficient.

Interestingly, Madden skews the numbers a great deal. Taking it out of the data, the correlation coefficient between unit sales and data is 0.34 and the correlation coefficient between revenue and review score is 0.46.

Which leads me to think that there is more than just my intuition to tell me that "consumers know quality when they see it" isn't quite the real picture. They may know quality, but that doesn't mean they want to spend their money on it.

Labels: , , ,

--Matt Matthews at 18:55
Comment [ 8 ]

Comments on this post:

Quality is a relative term, and desire more so.

Just because review scores for a game are middling doesn't mean a large number of people won't want it.

The Dynasty Warriors franchise could use a lot of improvements that Koei sees no financial reason to implement. The games themselves get lower and lower published scores with each iteration, and the players themselves get a degree of annoyed. But they still buy the games because they are still fun, and there isn't even much serious competition outside of Koei's own products.

At the same time, I recall Ravensoft's X-Men Legends and Ultimate Alliance games doing pretty well in reviews, despite always being buggy (there are bugs in Alliance that were present in Legends 1) and honestly getting a bit repetative and boring.

From looking at the graph, the main thing seems to be that games below a certain score (a bit below 60%) aren't going to be blockbusters. But being above 60% certainly doesn't mean you will do better. (Some high reviews got even lower sales.)

Which fits what I've expected... It is only a matter of potential. Being a "good" game doesn't mean you will sell well, it just means you have the potential to sell well that "bad" games generally don't have.

By Anonymous Baines, at 13 February, 2007 13:05  

I believe this is where we should also at least mention the possibility of bias in the review scores. To what degree can even the 60% threshold be considered an accurate measure of quality? As Matt's wondered before, what is "gameplay" anyway? How do you get a grade below 60%? I'm betting a fair amount are either bugged or have awful game mechanics by most any measure. Are review scores accurately measuring gameplay or does the 60% threshold reflect a game's ability to call itself what I might call something approaching "a finished product"?

Note that the "bad games" are, with the possible exceptions of "Rampage: Total Destruction" and "Happy Feet" (don't know those, and I'm assuming on Catz), games based on a preexisting, non-game related (or originally non-game) franchise.

So I'm stealing here from baines' deal -- above a certain point, can review scores be considered an accurate measure of gameliness? I believe Matt compared reviewers to movie or book reviewers. After a while, you have to find one that mirrors your values, because the others' opinions are essentially worthless past a certain point.

Finally, though Madden seems to be a legitmate aberration, to what degree, if any, does the Madden-factor sell other games? And, of course, what the heck is the Madden-factor? Yes, yes, it's the NFL marketing juggernaut. I've got that, and it's got me. But what is the factor in more precise terms? The perfect confluence of non-game franchise and great game? The concept that football is made for TV, and the media overlap and resonate well? Dunno.

By Blogger rufbo, at 13 February, 2007 13:24  

Sorry that this isn't a direct response to the thoughtful comments by Baines and Ruffin.

The whole "consumers know quality when they see it" line is quite poor. The ideal would be "a great game will have great sales", yet I think there are plenty of examples of "great game that sells poorly".

The data we're talking about presupposes the good sales!

That is, it's more tailored to the question of "if a game sells well, then how likely is it to be a great game?"

To answer the question the other way, you need sales data for every great game (using whatever measure of greatness you want) and then ask: Given that these are great games, how likely were they to sell well?

I realize everyone else may have already thought of it in this light, but there you go.

By Blogger jvm, at 13 February, 2007 13:43  

Madden will presumably stick out for years to come. Not only for being the football brandname, but for no longer having viable competition.

It is like Dynasty Warriors, which sells insanely well in Japan and presumably well enough in the US. There isn't yet viable competition, so short of becoming absolute garbage it will maintain the bulk of sales from people seeking that gameplay.


As for the "a great game will have great sales" quote, I admit I pretty much immediately discounted it. I like too many niche games to even think that, and there has been too much licensed (or popular franchise) drek that has done well.

Products that are critical darlings but poor sellers isn't even a videogame-only phenomena. How often are blockbuster movies also the best movies? And books? Heck, comics are pretty much sold to a niche market these days, and they have niche "critically praised" books that do horrible numbers, while some of the bigger sellers can also be some of the worst written.

By Anonymous Baines, at 13 February, 2007 14:44  

It is like Dynasty Warriors, which sells insanely well in Japan and presumably well enough in the US. There isn't yet viable competition, so short of becoming absolute garbage it will maintain the bulk of sales from people seeking that gameplay.

1.) I'm not familiar with Dynasty Warriors, but am suspicious it doesn't sell like Madden in the US+Japan market.
2.) I should check out past sales of Madden pre-exclusive license, but I bet it kicked everyone's butt, not only NFL licensed options but all ['Merican] football games for a while. I played ESPN's NFL game. It, past a few innovation, wasn't sliced bread.

So, the question remains, "Why do Germans love David Hasselhoff?"

As does this question: "Why do people enjoy playing [American] football on consoles?"

More precisely, I want to know what about Madden makes it score absolutely off the charts in ways that I can only assume Dynasty Warriors ain't. I've got that it's fantasy role-playing ("Look, I'm Randy Moss!"). I've got that football is a game made for TV (and that TV has formed itself well around the game), and this helps the game translate well. I've got that the NFL is stomping all other major league professional sports into dust.

But why is Madden off the charts? Why can't other games, especially but not limited to sports games, catch this lightning in a jewel case?

Products that are critical darlings but poor sellers isn't even a videogame-only phenomena. How often are blockbuster movies also the best movies? And books? Heck, comics are pretty much sold to a niche market these days, and they have niche "critically praised" books that do horrible numbers, while some of the bigger sellers can also be some of the worst written.

This also is, I believe, an elaboration of one of my earlier questions. In brief, to what degree are reviews and sales part of a circular, self-referential system? To what degree are sales contingent on creating and selling communities [including reviewers]?

This is a start into the inertia issue. MGS rocks until it doesn't. I'll offer as a parallel that Duke's men's basketball team was ranked pretty high this season at the start, iirc, but they're in danger of not getting invited to the NCAA tournament now. Why did so many people vote "Duke" when they stunk, and Coach K was telling us they stunk? Communist plot? Communist plot that's shared, in spirit, with game reviewers?

s/Communist/Capitalist/g, natch.

By Blogger rufbo, at 13 February, 2007 16:22  

Could it be simple indoctrination by virtue of its age?

Madden's been around since 1987 or 1988, right? Sure, not in its current form, but it is practically the only game still on the market that someone in grade school in the early 1990s could still be playing in college (or afterward) this year.

People watch the Superbowl year after year because it's a tradition. Why not Madden in the same way?

By Blogger jvm, at 13 February, 2007 16:49  

No, Dynasty doesn't do Madden numbers. Madden itself is several factors combined.

One of the biggest is just how many Americans want to play a football videogame. That means a lot will buy football games, from hardcore to the most casual. Madden is nearly as mainstream as PC solitare, and similar to casual games like Popcap's.

A second is how established Madden (and EA) is in the eyes of the general public. Given a choice, they go for Madden because it is the name they know.

A third is the lack of competition. This was insured when EA got the lock on the NFL rights (and I still wonder whether EA provoked that deal by claiming a $20 football title by nature had to be garbage and was hurting the NFL name.) If that hadn't happened, Madden might not be doing so well by this point and NFL2K might be getting equal or better.

And look at how long it took competition to really start to stand a chance of taking the Madden throne. How many years, with how many companies/groups/titles trying before anything became a serious contender? (It would have been interesting to see just what happened to the numbers for everyone after serious competition was established.)

Basketball isn't as popular and its market is a lot more fragmented, so it doesn't do Madden numbers.

By Anonymous Baines, at 13 February, 2007 23:28  

Madden's solely an NFL juggernaut extension? Interesting. Heck, even if we concede 50% to the ESPN version, that's a lot o' NFL on consoles.

Why no generic football with "Washington vs. Dallas" a la Steve Walsh Football pre-NCAA license?

Why doesn't the NBA do half as well? Why doesn't FIFA do just as well worldwide? Or does it?

I just find it strange that only one commodity has such much pulling power, that there's only one aberration. Of course two makes them exceptions and three an alternative rule, but if it's not sports that's precisely the key (it's pretty apparently not), what is it? I realize people love the NFL > NBA, etc, but not only why do they, but why does it translate so well to the console *in this one example only*?

Matt -- I'm guessing most Madden players have no idea it was around on the C=64, etc. There's probably a good correlation between the polygon'd Madden dominance and now, but I wouldn't overvalue it. I suppose my point is that, as baines suggests, if it wasn't Madden, it'd be another NFL franchise that, aggregated, gives us about as many sales.

Why? And then to bring it back on point, how do reviews of Madden measure the only game in town?

(Personally I think the market is ripe for an unlicensed PRO FOOTBALL (R) game here.)

By Blogger rufbo, at 14 February, 2007 11:24  

Contact Us

Subscribe to
Posts [Atom]

 Feedburner

Playing

Warm bile sold separately:

Browse Curmudgeon Gamer Memorial Library

Blogroll:

Internet game search:


Archives:
Classic: 02/2002 to 10/2005
Google
 
Web curmudgeongamer.com

This page is powered by Blogger. Isn't yours?