"Don't know" (DK) Option
"Don't know" (or ‘‘No opinion’’) option is a non-substantive response alternative. These can also be implicit or explicit. An implicit don't know option is an admissible answer not explicitly provided to the respondent, which requires an interviewer to record it. An explicit don't know option can be directly provided as a different response alternative to the respondent.
Theoretical arguments
- DK may be selected because of truly not having an attitude, lack of motivation, wish to avoid giving an answer or uncertainty of which exact point represents best their opinion (Alwin and Krosnick 1991).*
- If some respondents cannot answer the question, offer explicit DK (Dolnicar 2013).*
- Explicit DK increases the likelihood of false negatives (Gilljam and Granberg 1993).*
- Providing DK leads to less valid and informative data than omitting it (Krosnick et al. 2002).*
- DK provision encourages respondents not to provide undesirable or unflattering opinions (Krosnick et al. 2005).*
- DK option should be explicitly provided if there is a good reason to believe that respondents truly have no opinion on the issue in question (Kunz 2015).*
- Explicit DK leads to incomplete data, it is better use implicit DK (Saris and Gallhofer 2014).*
Empirical evidence on data quality
YES means that there is an effect on quality either positive or negative impact. NO means that there is no effect.
*DeCastellarnau, A. Qual Quant (2018) 52: 1523. doi: 10.1007/s11135-017-0533-4
- Providing an explicit DK option has a comparable reliability to not providing it [Wiley-Wiley reliability] (Alwin 2007) → NO*
- Explicit DK leads to higher data quality [MTMM validity, method effect and residual error] (Andrews 1984) → YES*
- Explicit DK increases missing data and lowers reliability. Implicit DK lowers missing data and increases reliability [Item non-response and Coefficient alpha] (De Leeuw et al. 2016) → YES*
- Explicit DK does not reduce acquiescence or recency responses [Acquiescence and Satisficing bias] (McClendon 1991) → YES*
- There is no support towards offering DK to improve reliability [True-score reliability] (McClendon and Alwin 1993) → NO*
- Validities lower when offering DK explicitly [MTMM construct validity] (Rodgers et al. 1992) → YES*
- The provision of the DK option does not have a significant effect on measurement quality [True-score MTMM reliability and validity] (Saris and Gallhofer 2007) →NO*
- Explicit or implicit DK does not affect reliability or validity [True-score MTMM reliability and validity] (Scherpenzeel and Saris 1997) → NO*
YES means that there is an effect on quality either positive or negative impact. NO means that there is no effect.
*DeCastellarnau, A. Qual Quant (2018) 52: 1523. doi: 10.1007/s11135-017-0533-4
References
Alwin, D.F. (2007). Margins of Error: A Study of Reliability in Survey Measurement. Wiley, Hoboken
Alwin, D.F., Krosnick, J.A. (1991). The reliability of survey attitude measurement: the influence of question and respondent attributes. Sociol. Methods Res. 20, 139–181. doi: 10.1177/0049124191020001005
Andrews, F.M. (1984). Construct validity and error components of survey measures: a structural modelling approach. Public Opin. Q. 48, 409–442. doi: 10.1086/268840
De Leeuw, E.D., Hox, J.J., Boeve, A. (2016). Handling do-not-know answers: exploring new approaches in online and mixed-mode surveys. Soc. Sci. Comput. Rev. 34, 116–132. doi: 10.1177/0894439315573744
Dolnicar, S. (2013). Asking good survey questions. J. Travel Res. 52, 551–574. doi: 10.1177/0047287513479842
Gilljam, M., Granberg, D. (1993). Should we take don’t know for an answer? Public Opin. Q. 57, 348–357. doi: 10.1086/269380
Krosnick, J.A., Judd, C.M., Wittenbrink, B. (2005). The measurement of attitudes. In: Albarracin, D., Johnson, B.T., Zanna, M.P. (eds.) The Handbook of Attitudes, pp. 21–78. Lawrence Erlbaum, Mahwah.
Kunz, T. (2015). Rating scales in Web surveys. A test of new drag-and-drop rating procedures. Technische Universität, Darmstadt [Ph.D. Thesis]
McClendon, M.J. (1991). Acquiescence and recency response-order effects in interview surveys. Sociol. Methods Res. 20, 60–103. doi: 10.1177/0049124191020001003
McClendon, M.J., Alwin, D.F.(1993). No-opinion filters and attitude measurement reliability. Sociol. Methods Res. 21, 438–464. doi: 10.1177/0049124193021004002
Rodgers, W.L., Andrews, F.M., Herzog, A.R. (1992). Quality of survey measures: a structural modeling approach. J. Off. Stat. 8, 251–275.
Saris, W.E., Gallhofer, I.N.(2007). Design, Evaluation, and Analysis of Questionnaires for Survey Research. Wiley, Hoboken
Saris, W.E., Gallhofer, I.N.(2014). Design, Evaluation, and Analysis of Questionnaires for Survey Research. Wiley, Hoboken
Scherpenzeel, A.C., Saris, W.E. (1997). The validity and reliability of survey questions: a meta-analysis of MTMM studies. Sociol. Methods Res. 25, 341–383.
Alwin, D.F. (2007). Margins of Error: A Study of Reliability in Survey Measurement. Wiley, Hoboken
Alwin, D.F., Krosnick, J.A. (1991). The reliability of survey attitude measurement: the influence of question and respondent attributes. Sociol. Methods Res. 20, 139–181. doi: 10.1177/0049124191020001005
Andrews, F.M. (1984). Construct validity and error components of survey measures: a structural modelling approach. Public Opin. Q. 48, 409–442. doi: 10.1086/268840
De Leeuw, E.D., Hox, J.J., Boeve, A. (2016). Handling do-not-know answers: exploring new approaches in online and mixed-mode surveys. Soc. Sci. Comput. Rev. 34, 116–132. doi: 10.1177/0894439315573744
Dolnicar, S. (2013). Asking good survey questions. J. Travel Res. 52, 551–574. doi: 10.1177/0047287513479842
Gilljam, M., Granberg, D. (1993). Should we take don’t know for an answer? Public Opin. Q. 57, 348–357. doi: 10.1086/269380
Krosnick, J.A., Judd, C.M., Wittenbrink, B. (2005). The measurement of attitudes. In: Albarracin, D., Johnson, B.T., Zanna, M.P. (eds.) The Handbook of Attitudes, pp. 21–78. Lawrence Erlbaum, Mahwah.
Kunz, T. (2015). Rating scales in Web surveys. A test of new drag-and-drop rating procedures. Technische Universität, Darmstadt [Ph.D. Thesis]
McClendon, M.J. (1991). Acquiescence and recency response-order effects in interview surveys. Sociol. Methods Res. 20, 60–103. doi: 10.1177/0049124191020001003
McClendon, M.J., Alwin, D.F.(1993). No-opinion filters and attitude measurement reliability. Sociol. Methods Res. 21, 438–464. doi: 10.1177/0049124193021004002
Rodgers, W.L., Andrews, F.M., Herzog, A.R. (1992). Quality of survey measures: a structural modeling approach. J. Off. Stat. 8, 251–275.
Saris, W.E., Gallhofer, I.N.(2007). Design, Evaluation, and Analysis of Questionnaires for Survey Research. Wiley, Hoboken
Saris, W.E., Gallhofer, I.N.(2014). Design, Evaluation, and Analysis of Questionnaires for Survey Research. Wiley, Hoboken
Scherpenzeel, A.C., Saris, W.E. (1997). The validity and reliability of survey questions: a meta-analysis of MTMM studies. Sociol. Methods Res. 25, 341–383.
