Tutorial: A/B-Testing with Inxmail


Urbano Blanco    12/05/15

Content Marketing





The subject line and the sender often play a decisive role as to whether a marketing email or newsletter is opened or not. With the help of A/B split testing you can, for example, find out which subject lines have the highest opening rate.



How does it work? For example:

  1. A marketing email or newsletter is to be sent to 1000 recipients.
  2. A number of recipients - for example, around 10% of the mailing list - are randomly chosen for the A/B split test. A mail with the first subject line (A) is then sent to 5% (50) of the random group and a mail with the second of the chosen subject lines (B) is sent to the other 5%.
  3. The results are evaluated after one day: which subject line achieved the higher opening rate?
  4. The remaining 900 recipients are then sent a mail with the more successful subject line.

The selection of the 10 % recipient group is completely random and none of the recipients are aware that they are taking part in an A/B split test. It can therefore be concluded that the test is carried out on a representative sample.



Tutorial: Workflow in Inxmail: A/B Split Test with dispatch to remainder

  1. Prepare the mailing list in the usual manner.
  1. In order to convert the selected mailing to a split test mailing, first mark it in your list and then click the “Create split test” button on the toolbar. Afterwards click “Convert” in the dialogue box that appears.
  1. The selected mailing will now be converted and the Split Test Mailings Assistant will open for the next steps.








4. Create a new split test

  1. Give your split test a title (for internal use)
  2. Select “Subject line test“






5. Subject lines and choice of strategy for the A/B test mailing


Once the mailing has been converted and a new split test created, the next screen enables one to individually configure three aspects of the test (see screen shot below).


Section 1: Execute subject line with the following mailing

The split test mailing has already been selected. It just remains to choose which two subject lines (A and B) are to be used for the test.


Section 2: Strategy and recipient distribution

In this section you can stipulate whether the split test should be carried out with or without dispatch to the remainder as well as which version should be distributed to which recipients.


Section 3: Choose most successful version for dispatch to remainder

Here you can stipulate how the most successful version is to be selected for dispatch to the remainder of recipients on the mailing list. There are two possibilities:

  • Option I: Determine the most successful version manually and start dispatch to remainder.

You determine which of the versions was most successful and then choose the time and date of the mailing to the remainder.

  • Option II: Automatically determine and send the most successful version. 

Click on the calendar symbol and the time dropdowns to select a date and time for the automatic evaluation of the most successful subject line and the subsequent dispatch of that mail version to the remaining recipients.

Now select the key performance indicator which should be used to define the most successful mailing, in our case here, “open rate”. If, however, the main criteria for the success of a mailing should be the number of clicks, please select the option “click rate” and then enter the relevant link.




6. Dispatch the A/B Split Test Mailing


So long as dispatch of the test versions has not yet started, it is still possible to add new recipients to the core mailing list.

The final step is to decide whether the random percentage of recipients for the subject line test are to be selected from the whole mailing list or from particular target groups within that list.

If wished, you can then enter one or more email addresses of persons to whom notifications of any test status changes will automatically be sent.

Dispatch of the test versions can now start.








A good software solution such as Inxmail can, with little effort, help you obtain important information about the behaviour of your prospects and thus optimise your mailings. A/B testing is just one of the options available. As an Inxmail Solution Partner, effective world can provide you with professional support and training for all your email marketing needs. The software includes customisable newsletter templates with which you can design your content to be responsive-ready. This important option can positively affect your open rates and also prevent recipients unsubscribing just because the newsletter cannot be read on a mobile device.