Customer reviews: do we give them 2 stars out of 5?
These days on the Web I’m seeing stars … As I book my hotel for my next trip, look for a bar to meet some mates in London, consider a film to watch in the evening, I search for those little indicators of quality that will guide me to making the best choice …
The power of consumer reviews is undeniable in influencing purchasing behaviours. However, I think it’s time we got our telescopes out and took a peek into the firmament to see what is driving reviews and ratings behaviour – in particular to assess whether review sites actually give a balanced perspective on the good, not so good and sometimes bad businesses or products listed thereon.
The 5-star-ratings model is the de facto standard for review sites. But is the distribution of stars uniform over the 5-point range? It would appear not … as this distribution from Yelp in September shows.
A whopping 67% of all reviews on this very popular site gave either 4 or 5 stars. A measly 15% of reviews gave businesses 1 or 2 stars. Why is this the case? and should we be worried about it?
I’ve just finished reading an intriguing report by Jupiter Research on the use of Web 2.0 technologies in Retail that highlighted a couple of very salient findings. For a start Jupiter underlined just how central reviews and ratings are to online purchasing decisions today as it found that 77% of US online shoppers look at product reviews and ratings.
But what really caught my attention was the difference in consumer behaviour after a positive and a negative shopping experience online. Consumers are, according to the research, more likely to take a survey (43%) after a good experience than a bad one (17%), more likely to tell friends about good experience (31%) than a bad one (21%), and more likely to write a product review about a product they liked (9%) than one they didn’t (4%).
So it does appear that as consumers we really only want to tell the good news and this results in reviews being skewed towards the top end of the scale. If this pattern of behaviour holds true then I would argue that the 5 point scale is potentially flawed as it presents an illusion of objectivity and a paucity of insight – If 67% of the restaurants in a town I want to visit have 4 or 5 stars, how do I then choose between them?
And this gets me onto a second concern which is the trouble an aggregated star rating system has in conveying the richness of a product or service experience. Julian Blom at LBi sent me this picture that illustrates the problem beautifully.
In this case a ‘neutral’ three star product rating is given based on the reviews of two consumers with polar opposite opinions. This example shows that any aggregated rating system suffers from the degree of freedom it has from the original consumer experience.
Compound this distance from the original experience with the fact that consumers tend towards giving positive rather than negative reviews and I can’t help thinking that star-based review systems may have some fundamental limitations.
Looking beyond the instant gratification of a star rating, an alternative approach to getting deeper insight into a product or service experience, and the quality of the business providing it, is available through sites such as Get Satisfaction that enable consumers and businesses to interact openly around product or service issues.
Posted by Ivan Croxford on December 6, 2008
Tags: consumer reviews, Get Satisfaction, star ratings, Yelp
I’m a digital strategist and I like building new businesses. This blog is an opportunity for me to air some of the insights, issues and themes that I come across in the course of my work. I’d love for some/any of these to be picked up as part of the broader conversation on digital disruption.
What intrigues me about consumer behaviour is the contradiction we see.
As you rightly point out Ivan, the research indicates that we are likely to respond after having a positive experience than a negative one, and we as consumers are therefore more likely to provide a positive review if prompted.
However, I seem to recall attending some research on buying behaviours earlier in the year, where I saw those participating stating that they would look more favourably on sites/products that have both positive and negative reviews – a far more balanced view and one that can be trusted.
So whilst we don’t like providing negative feedback about a product or service, we actually like to hear both sides of the story.
The challenge it would seem for any business or consumer review site, is to somehow encourage the negative as well as the positive.
But since in the U.K we don’t like to complain, we just tend to vote with our feet – we need to be somewhat creative in obtaining the true feeling of our consumers!
[...] Ivan Croxford looks at the problems of user reviews and star [...]
Hi Ivan
We use shopping.com’s customer rating tool, with a button on our checkout. Customers rate our service then submit another follow up review a week later when they get emailed.
We find it works pretty well and proudly display our badge at the top of our homepage (even though we no longer use SDC for leads). I guess the challenge is people wondering if the ratings are legit. It’s a tough one to get around. Some people also think that we write the positive feedback that we put on our feedback page: http://www.arenaflowers.com/customer_feedback
We put that up there totally unedited and untidied up on purpose – we feel this will help people believe it’s real. To write in a zillion different styles, with distinct capitlasations, spellings, grammar, idiom etc would be very difficult so (hopefully) it’s more convincing that these are real comments. In any case, we wouldn’t have the time to make this stuff up!
Merry Christmas!
Hi Ivan, just ran across your new blog! Thanks for the link to mine.
I can say from user research that I’ve been involved with at frog design that people are definitely suspicious of sites that lack negative reviews, but from what you are saying they are contributing to the bias! From my own experience I can say that I’ve predominantly written about products I like rather than dislike, so I guess I’m not helping either…
But what’s interesting is that this goes against conventional wisdom in retailing that an unhappy customer will tell many more friends about their bad experience than a happy customer will. (e.g. http://tinyurl.com/7dt8ng )
Who knows how accurate that is or ever was, but the contrast is striking. Perhaps the level of effort required to post something online screens out the casual complainers who would otherwise bring it up in the normal flow of conversation.
Greetings Ivan!
As I understand you! This problem very much disturbs also me. At us the same problems in shops…
Quick follow-up, I just happened to be reading an eMarketer report that matches this exactly. It states that “customers who contribute product reviews usually do not write unless they have something positive to say. Some 87% of respondents reported writing reviews that were either positive every time (36%) or most of the time (51%).”. Report is from April 2008, entitled “Consumer Interactions: Social Shopping, blogs and reviews.”
Really great comments on this post. Thanks all for your contributions. Taking into account all that’s being said:
- tendency towards positive reviews,
- consumer need for balanced, even negative, feedback, and
- concerns over ‘legit’ reviews (both from a business and consumer perspective)
it really does seem to me that the five star system isn’t the best vehicle for consumers to get insight into businesses, their services or products. Question is then, what’s a better alternative?
Why don’t you take the plunge into the real 1.0 world and book that hotel for your next trip, look for a bar to meet your friends in Tokyo, choose a film to watch in the evening in the knowledge that you might uncover something you didn’t expect?
Hi Ivan,
You make interesting points, all of which are very valid and questions I get all the time as I work on building my new consumer review based website http://www.rantorrave.com . Looking at review sites out there, I believe tripadvisor offer the most transparency; not simply for the quantity of reviews but more importantly on the ranking system that supports reviews. I don’t believe you can have one without the other. Alot of sites give a business with one 5 star review a higher ranking than a business with twenty 4 star reviews. However the most important factor of all is trust; therefore balance is essential in terms of positive and negative reviews.
The s