Phantomimic's Blog

The Mystery of the 3 Star Reviews

6/8/2012

I was looking at the patterns of 5, 4, 3, 2, and 1 star reviews of books on Amazon and I noticed something interesting that I want to share with you.

When you look at a book’s Amazon page there is a graph that displays the number of total reviews a book has, distributed by the number of stars. Taking my own book, the Sun Zebra, as an example, the graph looks like this:

This is an example of the type of graph you get when readers like a book. There are many 5 star reviews and a much lower number of reviews with fewer stars. However, even 42 is not a lot of reviews for statistical purposes, so I will concentrate here on books that have 200 or more reviews.

The first such book I want to show you is “Wool” by Hugh Howey. Having a book with this many reviews and almost a 4.9 average is a significant accomplishment for an author and a sure sign the vast majority of readers loved the book. A large bar of 5 star reviews is characteristic of very popular books.

The converse is also true. For example, Robert Jordan’s “Crossroads of Twilight” is an example of a book a lot of readers bought but the majority didn’t like. Here you find a situation where you have a large bar with 1 star reviews.

I also wanted to see how these graphs looked in books with intermediate ratings. For example, could I find books with large bars of 4 and 2 star reviews? After skimming over 4,000 books on Amazon that I searched by “popularity” and “reviews of 1 star or more,” to my surprise I found none. The most common way in which a book gets an average rating of 4 is because the number of 4, 3, 2, and 1 star reviews counteract the effect of the 5 star reviews such as in the case of “Loving Frank” by Nancy Hogan.

Likewise, I could not find any book with a rating of near 2 with a prominent bar of 2 star reviews. The way a book ends up with an average rating of 2 is that the number of 5, 4, 3, and 2 star reviews bring up the average counteracting the effect of the 1 star reviews such as the book “Trace” by Patricia Cornwell.

I was especially interested in 3 star reviews because these indicate that, in theory, the book is neither very good nor very bad. Surely I would find a book with 200 or more reviews and a prominent 3 star column, like this example that I made up:

That was not the case.

The way a book ends up with a rating of around 3 on Amazon is if a lot of people like it but an equally large number of people hate it. Such is the situation of “Fifty Shades of Grey” by E. L. James. Here there are 2 large bars at opposite ends of the scale: 5 stars and 1 stars.

However, the situation doesn’t have to be this extreme. Take the book “Mile 81” by Stephen King. Here the 5, 4, 3, 2, and 1 star reviews are nearly all the same, and the graph is a near flat line.

I want to state that the reason I didn’t find it is that I may have missed it (I only checked 4,000 books and there are more than one million books on Amazon). So please if you know of any book with 200 or more reviews and a prominent 3 star bar that stretches past all the others, please leave a comment and let me know.

But let’s get to the point of this post. If my observations are true, what does this say about the psychology of the reviewer? Why does an average book achieve a neutral rating of “3,” not by the majority of the reviewers giving it a 3, but rather by half of the reviewers rating it above 3 and the other half rating it below 3? Is this a reflection of our polarized society where we can’t find a middle ground on anything?

What do you think?

Note: the links on the books are provided just so you can check them out. I do not advocate you buying them except, of course, my own. : ^ )

***
If you like this blog you can have links to each week's posts delivered to your e-mail address. Please click here.

16 Comments

Carla Sarett

6/11/2012 05:39:20 am

This is actually a truism of market research across many product categories, and not at all surprising. People don't bother to write a book review for a book that they feel is "average." This happens in polling for political candidates, perfumes, etc.

Rolando link

6/11/2012 10:51:06 am

Thanks Carla,this indeed may also apply to books and would explain the lack of a majority of 3 star reviews. However, I also found the same thing for the 4 and 2 star reviews, although those are closer to the edges. I think this tendency is not very good for our society. It just reinforces the subliminal perception that there is no middle ground.

Lia London link

6/15/2012 01:28:58 am

Exactly what I was going to say, Carla. If it the book is just "meh", there's not motivation to write a review (unless asked to). If you love it, you want others to enjoy it so you star it up. If you hate it and want to warn other buyers not to waste their money, you star it down.

Rolando link

6/16/2012 07:12:15 am

Thanks for your comment Lia. I guess this only happens in a "self-selecting" population. If you could round up readers and tell them they "have to" write a review my guess is that all the "meh" reviews would weigh in. The effect would be to bring high ratings down and high ratings up.

Yvonne Hertzberger link

6/12/2012 03:15:44 am

Very interesting observation. We do seem to want to polarise out impressions.

Rolando link

6/12/2012 09:25:09 am

Thanks for your comment Yvonne. It seems we are turning into a "yes or no" society that is unwilling to embrace the complexity of all the shades of grey out there.

Saraj link

6/13/2012 10:17:32 am

Interesting observation. I think it shows also that people are more likely to post a review or opinion if they have strong opinions for or against something. Those who are 'meh' about a book largely don't share a review.

Rolando link

6/14/2012 12:11:07 pm

And that is not good. If only the most passionate speak up they are perceived as the spoke persons for those who don't.

Jeri link

6/14/2012 09:48:08 pm

Personally, when I read book reviews on Amazon, I always start by reading the two star reviews. Many readers do rate according to a "love it" or "hate it" extreme, but I've found that the reviews for three, two, and one star books contain more specific critical analyses than do the four and five star rave reviews. Rave reviews tend to stay along the lines of simply stating how great or awesome the book was.

As I start to review more books on my blog, I find myself entertaining the motto, "Giver of the three star review." But who wants to be deemed average? The same can be said for public school. Noboby wants to be deemed average and by skewing the middle ground, excellence has achieved a new level of low.

Anyway, lots of great food for thought. I've been churning this issue over in my mind for quite some time as I try to devise a rubric for rating books. Another thing that comes to mind are movie reviews I read in Entertainment Weekly. If a movie gets a C what they really means is the quality will be more akin to a D or a D-.

Overall, rating tendecies nowadays reflect a lack of critical thinking skills. And I wonder why I left the classroom... Our society doesn't value a critical mindset.

Rolando link

6/16/2012 07:07:07 am

Thanks for your post Jeri. I think our society has lost it's appreciation for "average," and that is sad. Seldom will someone be average in everything, but this is no reason to be ashamed of those things where we are average. We may have also developed the notion that things are achieved by two extremes cancelling each other instead of people trying to reach a middle ground. We are back to the thesis, antithesis, synthesis dialectic model of philosophers like Hegel.

Steve link

6/18/2012 05:13:49 am

This sort of rating trend has been especially obvious in restaurants since forever. People love to inform others about incredibly good, or incredibly bad experiences. The middle of the road doesn't get a report. This is actually sad, when you consider that consistency is at the top of the restaurant review checklist. People actually want the dead center, but they don't bother pointing it out when they find it.

Rolando link

6/18/2012 12:39:23 pm

Thank you for your comment Steve. It seems to be a very prevalent phenomena for self-selecting groups of people but when people see the results it creates the mirage that we either love or hate things.

Ren link

6/19/2012 04:16:02 am

I think it just boils down to a "3 star" rating amounting to a "Eh" response - which is effectively what the number should suggest. People who didn't feel strongly one way or another about the book.

People who don't feel strongly one way or another are also rarely likely to leave reviews...because....well, they just don't feel strongly about the book.

Extremes are generally the norm when it comes to feedback - people who love something or hate something are more often than not the only people who will take the time to give feedback. Most customer service polling companies have to account for this when collating data - I think their same lessons learned can (and should) be applied here.

Rolando link

6/19/2012 10:52:15 am

Thanks for your comment Ren. I think the trick is to avoid self-selection. Ratings from self-selecting populations are prone to this bias.

DJ Morel link

6/22/2012 03:31:05 am

Thank you for the fascinating analysis of Amazon reviews. I used to review books for The Seattle Times. I was assigned the books, so had to review them no matter what I thought. I had the hardest time writing reviews for the books I thought were average. If I loved the book, it was fantastic to share it with a wider audience. If I hated it, I could spell out exactly what I didn't like. If my reaction was "meh," it was really hard to explain why I felt that way.

Just giving a book a star rating is a whole lot easier. Amazon doesn't let you do this, but Goodreads does. Interestingly, Mile 81 on Goodreads does have a graph where the three rating is the longest bar. (You have to click on rating details, twice, to bring up the graph.) Perhaps even more interesting, Fifty Shades of Gray and Crossroads of Twilight have graphs on Goodreads that slant more positive than what's on Amazon.

Rolando link

6/22/2012 08:09:45 am

Thanks for your comment DJ. I had not thought about it that way. Average reviews can indeed be harder to write and that may explain why there are not so many. I checked the books on Goodreads but it is difficult to figure out what the distribution of the ratings is because Goodreads doesn't have a graph.

Your comment will be posted after it is approved.

The Mystery of the 3 Star Reviews

Leave a Reply.

Blogroll

Categories

Archives