Thu, 23 Aug 07

Responses to feedback

First of all I’d like to thank the 18 people that left feedback about the new, either in the comments on my post or in replies to emails sent out by various people in the company. I’ve tried to collate the feedback (paraphrasing the points) and provide answers to each point in turn.

Moving forward, we’ve created a public google group to discuss Membership is open to all so feel free to sign-up. Richard has a few reservations about exposing ourselves to the potential problems that an open forum can bring, so this is kinda-like a private invite that the techcrunch crowd love so much. I really appreciate Richard giving us the freedom to experiment with the group and would love to see people joining up to discuss how we can make a better place.

Anyway, on with the feedback…

Points raised by four different people

I want it to be obvious when you couldn’t find any products for my search query. Currently, as no message is displayed, it’s unclear whether my search didn’t work or the site is broken.

This has been fixed and is now live on

I question whether the google map, on the homepage, is useful.

Hmm, this is obviously a bit subjective. While four people have questioned its usefulness (including one person that wanted to kill the individual that came up with it – err, yeah, sorry about that), three people have said that it really helps to understand where the reviews come from. I’m open-minded but think we’ll probably keep it there for now and possibly remove it in the future if the bounce rate (from the homepage) becomes something that we care about.

Points raised by three different people

I like the google map.

Yay. Actually, you should probably see the point above.

I don’t always have enough time to read the reviews that flash up as part of the google map.

Agreed. Each review is now displayed for 10 seconds, rather than the previous 5. The development is complete, we’re just waiting for a deployment.

I find the supercategory pages (linked from the tabs at the top) hard to navigate as the categories on the left of the screen don’t appear in any useful order.

The categories are now in alphabetic order. We also only display the five most popular (as decided by the reevoo gods) categories when you first visit the page, giving you the option to show all if you wish.

I want to be able to click the review on the homepage to see all reviews of that product.

I felt quite strongly about this too. It didn’t seem right that a product page (with the reviews on) was a minimum of two clicks away from the homepage. We’ve fixed this up and it’s now live. If any of the reviews that pop-up on the homepage take your fancy then click the name of product (in the review box on the right hand side) to visit that page.

I want to know what order the products appear in on the search results and category pages.

They appear in the top-secret reevoo ranking order. Muhahaha. Actually, it’s not that top secret. It’s a combination of the review score over time (how ‘good’ it is), the number of times people see the reevoomark for that product on our partner sites (how ‘popular’ it is) and the number of prices we get from (how ‘current’ it is).

Points raised by two different people

I want to understand the relationship between the overall score and the score breakdown for a product. For example, the Ericsson K800i is rated 9/10 yet it only scored above average in one category (‘features’).

A bit of an explanation is required I feel. The overall review score you see (9/10 in this instance) is made up of some averaged averages (hmmm, that’s not too helpful). OK, so a reviewer will be asked a series of questions about their purchase. Some of these questions will require text answers, while some will require a score out of ten. For each review, we calculate the average of all of these scores out of ten. We then calculate the average score for a product based on each of these average review scores. This is what will give us the 9/10 for the k800i.
The score breakdown is based on an average per question for this product versus the average per question for every product the same category as this one (hmmm, again, that’s not very clear either). As a contrived example, let’s assume that we have two mobile phones, each with one review. The first mobile phone (a very basic model) is awarded 6/10 for ‘features’. The second mobile (a newer, all singing, all dancing model) is awarded 10/10 for ‘features’. We calculate the average ‘features’ score for mobile phones as 8/10 (6 + 10 / 2). We use this data to state that the first mobile is ‘below average’, while the second mobile is ‘above average’, in the ‘features’ department. Does that make sense?

I don’t really understand the score breakdown.

See the above explanation and the one in the Faq. If this is still unclear to folks then we should flesh it out a little further.

I’d like to see ratings against similar products in the ‘Other products like this’ section.

I’ve added a story to our development schedule that should hopefully see this feature added in the near future.

I’m having problems navigating the Faq

Unfortunately I’ve never been able to replicate this problem. Are people still experiencing it?

I want to be able to filter the search results so that I’m not too overwhelmed when I search for a relatively generic term (e.g. phone)

We’ve added the ability to filter your search results by product category. Having just searched for ‘phone’, I get the option to filter by ‘Cordless phone’, ‘Cordless phone with answering machine’, ‘landline phones’, ‘mobile phone accessories’, ‘mobile phones’ and a further 24 categories.

I don’t understand what the quartile link is that appears on the left hand side of a category page.

And neither do I :-) Don’t worry though, because we’ve removed it.

I want to know why the comments on the google map don’t always appear where they should.

This’ll be because the description of the location is completely separate from the location itself. The description of the location is entered by the reviewer when they complete the review. Although we ask for the town, they could actually enter anything they wish. As we didn’t think this data would be reliable enough to plot points on the map, we use the first bit of the delivery postcode instead.

I want products to have a human friendly name, Canon 4000 SLR instead of Canon EOS 400D, for example.

I’m a bit confused by this as the EOS 400D seems to be how Canon (it’s sad but that’s a great url) refer to it. I’m also slightly confused as it seems we label this particular product as a Canon 4000 SLR when it appears on the homepage, but as the EOS 400D on the product page. Does anyone else have any examples of misnamed products?

I want it to be obvious that the scores are out of ten.

This has been raised a couple of times internally and I personally agree that it’s not very clear. I’m not sure that we’ve come to any decisions yet though.

I want to be able to buy a product from your site.

You can! Well, in some cases. We use to provide us with links to retailer sites that you can purchase the product from. Just click on the shopping basket at the top of the product page, select the retailer you’re interested in and go buy it.

NB1. This is how we make money on

NB2. As much as I think this sounds like one of the made up entries you see in some Faqs (“I love your site/product and want to give you some money, please tell me how”), I can assure you that it’s not.

I’m not sure about the language of some of the commentary on the site.

Hmm, this isn’t too specific so I’m not sure there’s much I can say really…

I prefer the selected tabs (no underline) to the unselected tabs (with an underline).

I agree that there is some inconsistency but I think we really wanted to make it obvious that the tabs were indeed links. I believe there may also be some accessibility issues with using images as links.

I find it difficult to navigate the super category pages due to the long list of links down the left hand side.

As mentioned in one of the responses above, we now only display the five most popular categories (as decided by the reevoo gods) when you first visit the page, giving you the option to show all if you wish. This should hopefully make it a little more manageable.

Points raised by individuals

I don’t like the spelling errors in the reviews.

I’m afraid there’s absolutely nothing we can do about this. Apart from rejecting some reviews we don’t touch them. Although, maybe we could do our bit for north/south relations and see if one is more literate than the other :-)

I don’t immediately relate your favicon to your branding.

That’s probably a fair point. I think that that little logo used to be one of our strong identifiers but I’m wondering if it’s now the orange colour instead. I’ll pass the point on and see what others think.

I want all of the same type of product to appear in the same category, e.g. in-ear headphones and ‘normal’ headphones should appear together.

I agree that this isn’t currently very good. We use a company called Encodex for our product data. Although we try to sanitize that data (removing useless facets like ‘quartile’ for example) we still have someway to go. We were actually discussing this yesterday and a story should be on its way into the development schedule pretty soon.

I couldn’t find any Sony mobile phones.

I’m not sure why as they are certainly there now.

I’m not sure that the about us page is professional enough.

It looks OK to me but then I have a feeling that it may have been changed since this suggestion was made. Do people still have concerns about it?

I find it hard to differentiate between the light blue text and the darker blue hyperlinks on certain pages, e.g. the Faq

I understand the concern but I’m not sure whether it’s something we’re currently worried about. Maybe we need to look at whether people ever click on the links from the Faq page. If they don’t then it either suggests that they are not interested in the content or that they can’t see the links.

I can’t easily get back to the home page. There’s a little ‘home’ link in the breadcrumbs but nothing more obvious.

You can always click on the reevoo logo that appears on the top left to get back to the homepage.

I want to see at a glance whether a score is an average for the product or from just one review.

The actual suggestion was to make the product scores a different colour. Although we could do it, I’m not sure about the benefits. After all, even the scores from the individual reviews are actually an average of all the scores from that review (explained in more detail above).

I want to know whether you round up or down and whether you use decimal points in your scores.

Good question (read – I don’t know the answer off the top of my head). There’s probably a moderate amount of work to get the definitive answer so I’ll look into this separately and maybe add it to the Faq or to a page in the group.

I searched for something and got five pages of results. Only four of those pages had any content.

This was a schoolboy error on our part. The paging was calculated on 10 products per page yet we were displaying 12. Oops. All fixed now.

I’d like to be able to remove a filter independently of the rest.

Interesting suggestion and definitely something that has been discussed in the past. I’ll get it raised as a story and see if it gets prioritised.

I’m able to close the balloons on the google map.

Hmm, so you are. Although I think we’ve removed the functionality to close a balloon by clicking anywhere on the map, I don’t believe we can remove the ability to close them altogether.

I’d like to have a list of recently viewed products.

Can’t you use the back button? Sorry :-) This has been discussed in the past. The best I can do right now is get it raised as a story and see if it gets prioritised.

I searched for orange and got a Creative Zen Micro in my search results but couldn’t understand why.

That’ll be because we have some top-secret (read, legacy) content that is searched but not displayed. There’s a story for this that’s waiting to be prioritised.

I want people to be able to leave a review for a product. This would be especially useful for products that currently don’t have reviews.

We used to offer this functionality but withdrew it in December 2006. I’m pretty certain that it’s not something that we’re considering re-implementing.

I want to see the list of categories (currently appearing under the search box) higher up the homepage.

I agree that it isn’t immediately obvious (especially with a lower screen resolution) how you might navigate into the site from the homepage. As with assessing whether or not the google map should stay on the homepage, I think we’ll need to look at how many people disappear straight from the homepage and decide whether we want to play around with the layout in order to reduce it.

I want it to be made very clear that I can’t leave a review on the site.

The three step diagram at the bottom of the homepage was intended to convey this idea. Maybe people don’t look at it because it’s too low down the page (it appears ‘below the fold’ for the 45% of our visitors that have their screen resolution set to 1024×768), or maybe it just isn’t very clear?

I was confused by the separation of the review content from the google map, on the homepage.

I agree and actually think I may have made it slightly more confusing, as there is now a small delay between the balloon appearing on the map and the review showing on the right hand side of the homepage… I guess we’ll look into this as part of the slightly bigger issue of the homepage in general.

I’m not sure the plus and minus icons will be clear enough for people that need textual signposts to help them.

I assume you are referring to the accessibility of the site to visually impaired users. If so, is our use of the alt tag not sufficient? We currently have the alternative text of the plus image as ‘good points’, and the alternative text of the minus image as ‘bad points’.

I struggled to find the paging on a supercategory page, as it was hidden right at the very bottom of the page.

Two things should help this situation. The first is that the long list of categories on the left side of the page now default to be only five long, and the second is that we’ve removed the paging :-)

I want to know how many times a review has been voted useful or useless when ordering reviews by ‘useful’.

I’ll add a story to the development schedule and see whether it gets prioritised.

I don’t understand what ‘elsewhere on the web’ means and why it sometimes displays with nothing below it.

This has been added in an attempt to make one of the first places to come to when you want product information. We currently use the yahoo search api to search a preset list of domains (e.g., and engadget) for information related to the product you are looking at. We also currently search Yahoo answers and are planning to add many more in the near future.

I only seem to be able to filter by one facet at a time.

Um, you can definitely filter by multiple facets. Take All bluetooth and Mp3 enabled Nokia mobile phones that have a sliding front as an example.

I sometimes get strange results in the pricing table. Take this HP Cartridge, that lists ten entries for ‘Cartridge SAVE’ all at £23.00, as an example.

I believe this is to do with the data that we are receiving from (our pricing provider). Every link in that list has a different Url but they all end up at the same page, maybe suggesting that have that link registered multiple times with

I find that the high number of anonymous reviews lessens my trust in the promise that all reviews are from genuine customers.

Fair point. I guess all I can do is assure you that we will only invite confirmed purchasers of a product to review said product. Honest. We want to allow people to leave anonymous reviews in the hope that we’ll get more coverage than if people were forced to enter their details. Having said that I’m not sure we’ve ever measured the difference between the two approaches (i.e. requiring some personal details versus allowing people to remain anonymous).

I don’t know for sure what the bold statements at the bottom of a review represent. I guess they are general comments but I’m not 100% sure.

They are indeed general comments although I agree that it’s not immediately obvious. Maybe we need a little icon, like the plus and minus icons, that people can immediately identify. I’ll raise it as a story and we’ll see if it gets prioritised.

I have to scroll down to see all of the homepage.

Hmm, I’ve noticed this too. With a resolution of 1024×768 (which is the resolution of choice for 45% of our visitors) the three panel display at the bottom of the home page is completely cut off. Maybe we should at least try to fit the homepage within this resolution. I’ll float the idea and see if it generates a story.

I can’t see the beginning of each link under the browse all categories heading on the homepage.

This was identified as problem with Internet Explorer 6 and has since been fixed

I always want the number of search results to appear next to the name of the product category on the supercategory page. If the product category is too long then the number of results drops to the next line (e.g. Audio Home Systems in the Phones and Computing supercategory)

I agree that this doesn’t look great, although I’m not sure it impacts the user experience too much. I’ll get a story written up for it.

I see the product specification panel appear below all of the reviews on some product pages.

This was found to be a problem on pages where the yahoo answers that we received were too long for their container. The problem only occurred in Internet Explorer 6 and has now been fixed.