Screening respondents to increase data quality in consumer tests

Sammendrag

Product tests are often conducted to understand consumer opinions and perceptions. A screener is often used to determine which respondents are consumers. Our goal was to determine how filtering consumers who gave low-quality screener responses affects test results. Respondents in Finland (n = 343) and Turkey (n = 342) completed an online questionnaire. The questionnaire began with a screener. Respondents who consumed category products regularly were classified as consumers. Points were allocated to respondents who gave dubious answers to screener questions. Respondents with too many points “failed”; others “passed”. Regardless, all respondents completed an online test of three product concepts and a short version of the portrait values questionnaire. Screener-passing consumers better discriminated the product concepts and had larger directional differences between the concepts than a conventional panel comprised of all consumers. Fixed-sized panels composed first of screener-passing consumers, then of all consumers, were simulated using the bootstrap procedure. Screener-passing simulated panels were more discriminating of the concepts and had larger between-concept differences than conventional simulated panels. These empirical results indicate the size of the benefit realized by incorporating data quality checks into screening. If consumers who give dubious screener responses are dropped, then other consumers can take their places or their places can be left unfilled, yielding either better differentiation or cost savings. Further study is needed to develop best practices in screening that can be used in different countries and cultures.