How to do your test right

There is a national craze for tests, with everybody sure that results will show their incredible wisdom.

Regardless of the educational dialogue not taking place, testing is an important part of fundraising, not just once but over time and at various times. During the Direct Marketing Association (DMA) Nonprofit Federation Conference, Lynn Mehaffy and Amy Sukol of Lautman Maska Neill & Company offered advice about conducting worthwhile tests and then getting useful results from them. Testing without strategy or purpose is just a useless exercise.

To do tests right:

  • Determine what is meant to be achieved.
  • Only test one thing at a time.
  • Split data proportionally and randomly.
  • Test enough quantity to provide readable, replicable results.

All that is preliminary to conducting the testing. After the testing is done comes the evaluation. How do you pick a winner from the data gathered?

  • Set a goal for the test, that is, have a hypothesis.
  • Look at test results in relation to the desired goal: Did response rate increase? Did the average gift increase? Did the organization lower costs? Did the cost to acquire a donor decrease?
  • Ask: Was the test result statistically significant?