The Top Ten Comparison Shopping Engines Compared – Q2 2011

Click here to register and download the most recent CSE Rankings from the previous quarter.

Welcome to the 10th quarterly edition of our CSE rankings. Over the past 2+ years we’ve analyzed our merchants’ data to provide a comprehensive overview of what one can expect being listed on the various CSE’s, and see how they compare to each other.

What’s New

• Like we did with our rankings last quarter, we’ve split up the rankings into 4 specific categories which include apparel, electronics, home & garden and sporting goods (free registration required.)

• Google’s Panda update made its impact felt to retailers and CSE’s alike, resulting in potential losses of organic/natural traffic for the CSE’s.
• Throughout the quarter a handful of CSEs invoked a string of CPC rate hikes that kept rolling out like a domino effect, perhaps in response to the Panda update.

• A great analysis on the effect of Panda and its effect on the CSE’s can be found on Become’s blog.

For a more in-depth look at our methodology please take a look at our original rankings which provide further details into the sections below.

The Criteria

Our study is based directly from our merchants’ data and how they performed on the CSE’s.

We aggregate and calculate how each engine performed against its competitors and present them based on the following criteria:

•  overall traffic driven
• average amount of revenue generated
• conversion rate
• COS (Cost of Sale) % or return on spend
• average CPC (cost-per-click) rates
• responsiveness rate among the engines
• quality of tools found within the login

For the first two categories, the results are based on an index of 100, with the rest expressed as a percentage of the leader.

A few things to note in order to avoid confusion:

• “Google” refers to Google Product Search and does not consider Google’s organic listings or their paid offerings like Adwords or Product Listing Ads
• Similarly, Bing Shopping refers to just the shopping portal that Bing offers, and not Bing the shopping engine
• Amazon Product Ads is a separate entity from the Amazon Marketplace. The primary difference is that Product Ads lead a customer off of Amazon’s site, whereas on the Marketplace the purchases are made directly on Amazon’s site

Traffic

After opening its gates to a handful of new categories, Amazon Product Ads has soared as this quarter’s traffic leader, just slightly ahead of last quarter’s leader Shopzilla.

As one of the youngest CSE’s, Amazon has managed to grown its Product Ads program very successfully from its infant stages, similar to the way its grown its marketplace program.

Revenue

While the paid CSE’s saw rate hikes nearly all across the board–Google continues to separate itself from the pack with its free-per-click model as our highest revenue driver for this quarter.

The next three are close, and the top-tier engines continue to dominate total revenue driven to our clients.

Below the charts are broken into two parts, dark blue represents data from Q2 2011, and light blue reflects data from the previous quarter, Q2 2011.

Conversion Rate % (orders/clicks)

The free CSE’s, Bing and Google Shopping saw increases in conversion rate this quarter (Google by the tiniest of margins) while the others saw drops all around–though some were more extreme than others.

We speculate this may be due to the Panda update, with a loss of organic traffic resulting in slightly less qualified visitors.

COS (Cost divided by Revenue) %

In spite of the rate increases (seen below) we actually see that a majority of the paid CSE’s (4 out of the 7) saw lower COS %–with just slight increases for Become and Shopzilla.

Average CPC (Cost-per-click) Rate

Other than Amazon, all the paid CSE’s saw a slight tick in CPC rates this quarter which was to be expected.

Since the rate increases didn’t take place until we were well into Quarter 2, the increase wasn’t quite as drastic as it potentially can be next quarter.

Shopping Engine Responsiveness Rating (ranking from 1-10, 1 being best)

Taken from previous study: In spite of being a smaller engine Become.com once again tops our rankings as being the most responsiveness by not only answering all questions we have in a timely manner but by taking initiatives as well–though the same can be said about most of the top CSE’s on our list.

Merchant Tools (ranking from 1-10, 1 being best)

Taken from previous study:

For purposes of our rankings Google Product Search is represented by its cousin Google Analytics. The tools that an engine provide are a crucial part of running a successful campaign.

While most CSE’s provide similar tools for tracking cost and revenue, some go above and beyond to provide the necessary resources to push a campaign’s success to its limits.

For instance Shopzilla allows a merchant to not only see product and category performance but be able to bid specifically on those as well.

Amazon Product Ads is the only paid CSE that does not have its ROI tracker so the only way to track orders generated from clicks made there is through a 3rd party tracker.

Also lacking from the Product Ads login is the ability to run SKU reports over custom-time frames, which is also crucial for monitoring how a specific product is doing.

Being a relative newcomer to the CSE’s they have come a long way, but there’s still a gap between them and the longer established CSE’s.

Movers and Shakers

PriceGrabber’s Cost of Sale % dipped below 15% for our clients in Q2. It appears that their Yahoo Search Direct integration and partnership with Marin are leading to more qualified traffic.

Shopzilla dropped from 3rd to 6th in the overall rankings and is 9th in Cost of Sale %. We were hoping that the recent sale would help but raising rates on 30% of their categories definitely hurt our client’s ROI.

Conclusion

In the end we ranked each shopping engine from each of our metrics from 1-10 and assigning a point value for each category (higher being better).

We doubled the COS % value making that out of a possible 20 points, and weighted traffic + revenue as 1.5 each for a maximum of 15, summed it up, and got the following:

1. 

2. 

3. 

4. 

5.

6.

7. 

8. 

9. 

10.

Not too many surprises this time around, Google remains our leader with Nextag and Pricegrabber close together in second, followed by Amazon, Shopzilla, and Bing.

With Bing and TheFind always having been great options to get started on the CSE’s because they’re free to use–they’re now becoming even more viable options as CPC rates increase across the paid engines.

However like anything else, balance is the key here, and any well managed CSE campaigns (or ecommerce campaigns in general) rely heavily on balance.

Breaking Down the Rankings by Category

Through special demand by our readers asking us to break down the CSE’s by specific categories, we have created separate studies that concentrate on apparel, electronics, home & garden, and sporting goods