Multivariate testing usually requires to use specialized MVT solutions wich are numerous on the market.
MVT platforms allow to implement and manage tests without - or at least with very little- IT department intervention.
This article focus on MVT tools functionalities and deployment. Multivariate test principles and benefits must be seen at multivariate test definition.
The first task is to design the experiment by choosing pages to be tested, variables and variations of the variables. For complex projects consulting services from the solution provider may be neeeded.
The next stage is to declare on the platform the variables to be tested and for each variable to create or indicate the different versions to use. It is generally done on wysiwyg intuitive interfaces by just pointing, clicking or importing images.
In the demo below we can see how variations are easily created:
A persistent cookie is stored in the visitor’s browser to assign the same combination for a next visit and to track post-visit conversion.
During the MVT test deployment results can be delivered in real time and the test stopped when results are statistically significant.
Some solutions offer continuous optimization service which allows to dynamically adjust site or page with winner components after test results and to add continuously new variable values.
Costs vary widely according to functional coverage, test volumes and consulting needed.
Some platforms have monthly fees, others charge for each server call or visitor particpating to the experiment and others mix these elements.
Many offer free trial for amonth and first 10 or 20 000 participant visitors.
For a SMB, monthly fee may be from 50 to $100 for approximately 20 000 participants and a cost of about $.005 per participant in excess.
For hundreds of thousands of participants, the cost would be of several hundreds dollars. As seen in multivariate testing definition, number of participants needed for significant results must not be underestimated.
For big enterprises which use dedicated account manager, consulting services and high volumes of data and participants, the cost may be several thousands dollars per month.
Sometimes, free MVT functions are integrated in web analytics or CMS solutions. But they are usually less easy to use and have hidden costs.