Avinash Kaushik - world leader in web business optimisation
Document Actions
Improving website design using multivariate testing methodology
Explains how to devise a multivariate testing methodolology to improve site design. Gives examples and a case study and highlights multi-variate testing tools.
Multivariate testing example
In the Website design best practice guide I write for E-consultancy I give many recommendations to improve site effectiveness. Although expert review comments will almost always improve results, no one can second guess user behaviour. So, for key site pages with high traffic volume such as the home page, category pages and registration pages there is no substitute for multi-variate testing.
What is multivariate testing?
Multivariate testing enables simultaneous testing pages for all combinations and variations of page elements that are being tested. These enables selection of the most effective combination of design elements to achieve the desired goal.
This contrasts with AB Testing where, in its simplest form A/B testing refers to testing two different versions of a page or a page element such as a heading, image or button. The alternatives are served alternately with the the visitors to the page randomly split between the two pages. Hence it is sometimes called ‘live split testing’.
You may also want to see Avinash Kaushik's overview of the advantages and disadvantages of website design testing alternatives and other great tips on where to start with improving website design effectiveness.
Design goals for multi-variate testing
The design goals of multi-variate testing are to increase page or site effectiveness against key performance indicators including:
- Clickthrough rate
- Conversion rates
- Revenue per visit
Example of multi-variate testing methodology
An example of the multivariate testing approach is shown in the table below.
Table : Multivariate testing example
|
Test |
Headline |
Button |
Body copy message |
|
Test 1 |
Existing |
Existing |
Existing |
|
Test 2 |
New |
Existing |
Existing |
|
Test 3 |
New |
Existing |
New |
|
Test 4 |
Existing |
New |
Existing |
|
Test 5 |
New |
New |
Existing |
|
Test 6 |
Existing |
New |
New |
|
Test 7 |
Existing |
Existing |
New |
|
Test 9 |
Existing |
New |
New |
A more sophisticated multivariate test may test more than one new element, for example 3 alternative headlines may be tested. Marketing Experiments has some good examples of more sophisticated multivariate tests.
The DVD rentals site LoveFilm was tested with the multivariate testing tool Maxymiser. I have shown the test results in the graphic or you can read the multivariate testing case studies on their site. As an example, this showed that:
- Call-to-action button, ‘Start your FREE trial’ performed better than ‘two week free trial’
- Page headline ‘the best way to rent DVDs’ worked better than ‘The new way to rent DVDs’ or ‘Unlimited DVD rentals’ to your door.
The site has since gone through some further changes and it is an excellent example of how to quickly communicate an online customer value proposition to site visitors. It combines a simple, value-led home page with more detailed information to overcome objections and reassure prospects about the company.
Best tools for multivariate testing
Some of the best known providers of multivariate testing tools include:
- Maxymiser (www.maxymiser.com)
- Optimost (www.optimost.com)
- Offermatica (www.offermatica.com). Examples of methodology.
- Google Website optimiser (free) (http://services.google.com/websiteoptimizer)
- SiteSpect (www.sitespect.com)
Of course, most will be interested in how to use the free Google Website optimiser tool, for tips see these articles from GrokDotcom:
Please bookmark this article if you found it useful (just two clicks).
Please add your comments, experiences or link suggestions to inform me and other visitors to my site.
I've set it up to be quick, but you can't add true hyperlinks - just text.
Thanks, Dave Chaffey.
