September 29, 2005

Product Review Mini-Series: Numbers

Review Statistics

Yesterday, we examined the features of various product review formats.
Today, we will compare the output at several major sources of reviews.

How do the various user-review resources stack up against each other?

I do not have a definitive answer, but I did gather some data while researching cameras (Canon SD500, Canon SD400, Canon SD300, Canon S2 IS, Canon G6, Canon S70). Admittedly, with a small sample size and one product category, the results might not be ultra-representative... this is just an exploratory experiment to see if anything interesting turns up.

+ + +

First, can we learn anything by looking at the distribution of product ratings?

Ratings #
5 star	186	89	44	25	28
4 star	53	37	22	11	5
3 star	12	3	3	2	4
2 star	15	1	3	2	2
1 star	25	1	0	0	1
TOTAL	291	131	72	40	40
Ratings %
5 star	64%	68%	61%	63%	70%
4 star	18%	28%	31%	27%	12%
3 star	4%	2%	4%	5%	10%
2 star	5%	1%	4%	5%	5%
1 star	9%	1%	0%	0%	3%
TOTAL	100%	100%	100%	100%	100%

All of these cameras were highly rated, so there is not as much difference as there could be. If I had it to do all over again, I would have tried to find products with divided opinions. The only notable difference here is the relatively high amount of negative ratings at Amazon.com. Diversity of opinions is valuable, so at first glance this appears to be a plus for Amazon.

However, there might be some differences in the reviews hosted by retailers and reviews hosted by independent sources. While some would say there is plenty of incentive for retailers to have uniformly positive reviews, in some ways, opposite forces are at work at Amazon.com. I saw several instances where a reviewer's negativity was directed at the retailer, for issues ranging from pricing to shipping, and not the actual product.

+ + +

How about the average length of reviews at these sources? Any differences?

Words
Average	211	146	928	104	119
Words #
1 to 71	60	41	8	18	17
72 to 127	74	35	14	12	9
128 to 258	79	37	9	8	10
259 to 4673	78	18	41	2	4
TOTAL	291	131	72	40	40
Words %
1 to 71	21%	31%	11%	45%	43%
72 to 127	25%	27%	19%	30%	23%
128 to 258	27%	28%	13%	20%	25%
259 to 4673	27%	14%	57%	5%	10%
TOTAL	100%	100%	100%	100%	100%

Wow. It is hard not to notice the eye-popping length of reviews at Shopping.com. What this means is that if a shopper is looking for as much in-depth information as possible, it is impossible to ignore Shopping.com.

To varying degrees, the can't-ignore-dynamic applies to all of the major review sources. In other areas, such as product attribution, it is possible for a site to have comprehensive coverage, relative to the other sites, so a shopper only needs to visit one site. But there is no way for one site to acquire a monopoly on unique perspectives. Thus, for any given product, any of these sites could prove to be indespensible.

+ + +

more later...

Product Review Mini-Series: Numbers (1)
Posted by Sean O'Rourke on September 29, 2005 at 7:58 AM
Archived at Product Reviews

Comments

Posted by Marc at September 29, 2005 3:04 PM