Landing Page Testing – Unexpected Results

by Matt Sullivan on October 28, 2011

I recently went through split testing a new landing page to determine if it would be worth rolling it out moving forward. The test was pretty simple: an email marketing send was randomly split into three groups, each going to a different landing page. While the email content was identical, the actual links behind the call-to-action varied. Aside from the layout changes on the landing pages, the text and images remained identical.

Landing Page Testing - Control Version

Control Version - Click for Full Size

Landing Page Testing - Right Version

Right Version - Click for Full Size

Landing Page Testing - Left Version

Left Version - Click for Full Size

The “Control” landing page was the current format. Essentially the layout was two-columns, with the left-column being the source of the copy appearing above the conversion form.

In the next two versions, the layout changed by pulling the form into the smaller of the two columns. There were two versions of this landing page, testing out the form in both the left & right columns.

Also, the new layouts added three new content areas into the bottom of the page: a link to the company’s iPhone/Android app, a simple sign-up to download an ebook, and a newsletter sign-up.

Being the gambling man that I am, I asked for predictions from the rest of the marketing department. Almost unanimously, the Right Version was selected as the likely winner. (I personally was drawn to the Left Version, just because it was so different and made the form the real focus of the page.)

The email activity (i.e. Open Rate & Click-thru Rate) was close enough to identical across all three test groups. It was the downstream numbers that really started to prove the value of the test.

The Right Version yielded 1 additional conversion than the control, so nothing statistically significant there. But the Left Version yielded a 217% increase in conversions! 

There were no additional conversions in the terms of ebook downloads or newsletter sign-ups due to the format change, but we have seen these offerings perform well in non-email campaigns.

But, the ultimate lesson is that just because you’re landing page is working doesn’t mean it’s working as well as it could. Furthermore, asking your co-workers to pick the winner won’t work, mostly because your co-workers aren’t your customers.

The next iteration of my landing page optimization is going to be simple: button colors. The orange & blue combo is part of our corporate color pallet, Performable (now part of Hubspot) did a great blog post that something as simple as button color can have a statistically significant effect on conversion rates.

With so much effort being put into getting someone to your landing pages, you want to do everything you can to make sure that your potential leads convert into actual leads.

Comments on this entry are closed.

Previous post:

Next post: