The cover story (World of Temptation) in the July issue of Research magazine focused on how "DIY" online survey tools are "an increasingly tempting option for conducting research at minimal cost." 

In the article, Forrester Research analyst Brad Bortner cautions about the dangers of using DIY research tools.  According to Bortner, "In some cases there will be no control over design, no control over the sample. At best, a bad sample might occasionally be used... At worst, organisations might be flooded with misleading results and take bad decisions - and the confidence in research as a decision-making tool could ultimately collapse."

In his blog post on the topic of DYI surveys (When DIY Surveys Become DYI Surveys), Vovici's Jeffrey Henning opined that if survey tool providers were to write a Surgeon General's style warning about the dangers of Do-It-Yourself research, it would warn that DIY surveys done poorly cause erroneous conclusions, bad decisions, and may cost you customers.

As a research professional, I, too, have concerns about the use of DYI survey tools.  And no, it is not because the increasing use of DIY survey tools could mean less business for me (since one of the services I offer is online consumer research.) 

My biggest concern relates to questionnaire design.  As Sue Brooker, deputy chair of the MRS standards board, said in the Research magazine article, "A good questionnaire isn't just a series of good questions. People using these tools have no idea about the tenets of good research. We know that just one word can make a difference to the result you get... Everybody thinks they can write a good questionnaire, and they can't."

A second concern relates to the analysis and interpretation of the data.  It is not enough to simply summarize the data.  The President of The NPD Group recently talked about data analysis in an American Marketing Association podcast. He said there are three questions you have to ask when analyzing the findings from a research study: 1) What? [What does the data say?], 2) So what? [What does the data mean to us?] and 3) Now what? [Where do we go from here/how do we use what we have learned?].  The "so what?" and "now what?" questions are what transform data into actionable insight.  As A.G. Lafley former Chairman and CEO of Procter & Gamble, wrote in his book The Game-Changer, "Imaginatively connecting and distilling what you see and hear is how observation becomes insight." 

Here is some advice on how you can keep DIY (Do-It-Yourself) surveys from becoming, in Jeffrey Henning's words, DYI (Do-Yourself-In) surveys.

• Learn how to write a good questionnaire.

There are numerous online resources that provide valuable tips on questionnaire design.  Vovici is doing a webinar on August 25  on the Zen and Art of Questionnaire Design.  Zoomerang has an excellent white pape titled "10 Tips to Improve Your Surveys".  Send me an e-mail and I will send you a couple of other good resources on questionnaire development.

• Know when to hire a professional. 

Doing a simple straightforward 5 question survey is relatively easy to do and does not require a lot of experience in questionnaire design.  But if the questionnaire is complex (for example, requires the use of skip patterns) or the topic is sensitive, you may want to consider hiring a research professional to design and develop your questionnaire for you.  You may also want to consider hiring a professional to help you interpret the data to answer those all important questions of "so what?" and "now what?"   There is something magic that happens when your people have the opportunity to collaborate with an outside resource who has years of research experience and industry knowledge.