Projects

Data-Driven Website Optimization Using Multivariate Testing

Testing a new technology and approach to understanding user-behavior for improved digital communications.

Executive Summary

Digital technologies and best practices evolve rapidly and modern web programs must have the ability to adapt quickly in this changing environment. However, new tools must be systematically and rigorously evaluated to ensure that they are adding value to the larger program. Multivariate testing (MVT) is an established protocol for sites that are designed to optimize sales or where conversations are clear and unambiguous. However the usefulness of MVT on sites that are largely informational in nature is still an open question. Moreover, MVT must be an efficient use of resources and compare favorably with other user experience approaches to justify its incorporation into the overall digital program.

Multivariate testing (MVT) is testing several elements within one webpage to determine which versions of those elements, put together, would best achieve the objective of the webpage.  Adding it into our digital strategies integrates a lean and flexible approach to dealing with what could be only one flaw or error in design (what the website looks like), site architecture (how the webpages are organized), or content curation (how the content is presented).

This project is testing whether incorporating MVT to our digital process leads to efficiencies in how a website is optimized for communications. To calculate the impact of MVT, the team is tracking the following key performance indicators:

Total reach for each page version:

  • Change in traffic for final version (Has traffic increased or decreased since the winning version of testing been published?)
  • Findability of page version (How is the page version likely to perform in search?)
  • Total conversion rate  for each page version (Is the user accomplishing the task?)
  • Total effort hours spent on project by site improvement method
    • Total cost of testing tools

A project supported by the: HHS Ignite Accelerator

Team Members

Achaia Walton (Project Lead), ASPA
Nicholas Garlow, ASPA
Beth Liu, ASPA
Karen Silver, ASPA

Milestones

December 2013: Project selected into the HHS Ignite Accelerator
January 2014: Time in the Accelerator began
April 2014: Time in the Accelerator ended

Project Sponsor

Andrew Wilson, Content Manager, Digital Strategy Group, ASPA