Beware of list owners who load your test with good names and give you garbage
You will occasionally run into the dishonest list owner who gives
you gold-plated names for your 5,000-letter prospect test. This produces
dishonestly tremendous results on your test. You then order the rest
of the names in the select you think you tested. Your 100,000-letter
rollout then crashes because the broker sent you weaker names than you
tested, perhaps threw in all the non-donors and people who may not have
contributed in years.
What can you do about this?
Answer: Not much, except make sure you’re using a great list broker who knows the lists she’s recommending and knows who the charlatans are. Very few list owners will pull this scam on you because they want a good reputation so that marketers will continue to rent their lists in the future. If you’re in the business of renting lists, nothing will kill your business faster then getting a reputation for loading up tests. It would be like a casino cheating people. It wouldn’t be long before no one showed up to play at that casino any more. Still, there are a few steps you can take that will minimize your risk.
One is to roll out in stages. Never take for your rollout more than five times the number of names you tested. Another strategy I used with a list owner who I knew was loading up tests was only to order tests, never rollouts. That way I knew I was only getting the very best names, often getting 15 percent response rates on the test names he supplied.
I always look at test results very carefully. If I notice something
out of the ordinary, such as a very high response rate, or a higher-than-
expected average contribution, I’ll order another 5,000 or 10,000
names in the same select and test again.