This is part three of a four-part newsletter dedicated to specific screener writing strategies that will help to enhance screeners in order to maximize the quality respondents for a qualitative market research study. See Part 1: Maximize the Quality of Your Participants and Part 2: Setting the Stage for Potential Flexibility.
There are some types of questions that may work well for quantitative questionnaires but not as well for screeners. These types of questions make it difficult to qualify respondents because the response data tends to be lengthy and complex. We recommend that they be used sparingly within screeners for qualitative studies.
For instance, many screeners include likert scales to assess respondents’ attitudes and behaviours, as illustrated below in the example regarding personal style..
For each of the following statements, please indicate your level of agreement using a 5-point scale, where 5 means you agree completely and 1 means you disagree completely. When it comes to clothing and shoes:
- I create my own style and am not influenced by other people. _________
- I express my personality through the clothing and shoes I choose to wear. _________
- I am a very creative person. _________
- I like to stand out from the crowd. _________
- I like to push the boundaries in everything I do. _________
While these types of quesitons can be useful in ensuring that respondents’ ideologies align correctly with the target market, overuse can lead to unecessarily high dropout and disqaulificaiton rates. The reason for this is due in part to the nature of respondent subjectivity (e.g. something as simple as a current mood can affect the way a respondent answers), respondent fatique from long lists, and the decreasing likelihood that respondents will chose the desired combination of ratings as the number of statements increases.
As a general rule of thumb it is recommended that likert scales be used sparingly, if at all, and whenever they are used for qualificaiton purposes, that they be limited to a maximum of 3 – 5 statements.
Other examples of questions that we recommend avoiding or minimizing are as follows:
These are formula-based questions that take the answers provided by respondents and give them a specific code that is usually linked to a personality/behavioural trait. Respondents might qualify on all other parts of the screener, but based on the algorithm categorization, be disqualified. In our experience these questions have a high rate of disqualification and we suggest avoiding them.
If they must be used, please ensure that the algorithm is airtight. If let’s say 100 respondents qualify based on all of the other questions in the screener, age, household income product usage, family composition etc. how could it be possible that once the algorithm is applied related to personality traits or behavioural questions that all 100 respondents then disqualify? This may perhaps indicate an issue with the algorithm itself.
Long lists of products/brands
These are a deterrent to respondents and time consuming for recruiters. If lists of products/brands must be listed, we suggest using no more than the top 10 in the market, including only necessary ones for qualification purposes. Providing respondents with an option to answer freely in an open-ended text box, can often flush out any additional products/brands and will cut down on respondent fatigue.
Questions such as those relating to ethnicity or sexual orientation are best left up to the respondent to decide. This means using an open-ended question to let them express themselves freely without restrictions to answer on the screener.
Market research recruiters understand that it is important to get the most information possible out of screener questions and this can be achieved with a short and concise screener. Respondents find a drawn-out screening questionnaire tiresome and qualifying respondents on the recruiters end can be a tedious task. Keep it short, ask the right questions, and avoid the lengthiness that leads to high dropout rates.
Stay tuned for next month’s newsletter about realistic expectations, and understanding the market.