The Choice of Response Scales
In the previous section we have introduced a procedure to develop a valid request for an answer. An essential next step is to present response scales for the respondents. There are many possibilities to chose these scales. Here an essential difference will be introduced between item specific scales and agree/disagree scales.
Item specific (IS) scales
An item specific response scale is a scale which provides as a scale for the answers gradations of the evaluations the researcher is interested in. In the European Social Survey these scales are frequently used. Below we present a set of questions of the third round using such scales asking the effects of immigration.
An item specific response scale is a scale which provides as a scale for the answers gradations of the evaluations the researcher is interested in. In the European Social Survey these scales are frequently used. Below we present a set of questions of the third round using such scales asking the effects of immigration.
Each of these questions asks a specific consequence of the immigration. The first is directed to the good or bad effects on the economy, the second to the enrichment or undermining of the cultural life and the third to the better or worse living surroundings. The scales are also expressed in the same terms as used in the request for an answer.
Agree/disagree scales
In case of agree/disagree scales the response scales are for all items the same; they go from disagree strongly to agree strongly. Another difference is that here statements are evaluated. Examples of such scales also are asked in round 3 of the ESS to compare the quality of the different forms of the requests for an answer. The requests for the same topics are presented below.
In case of agree/disagree scales the response scales are for all items the same; they go from disagree strongly to agree strongly. Another difference is that here statements are evaluated. Examples of such scales also are asked in round 3 of the ESS to compare the quality of the different forms of the requests for an answer. The requests for the same topics are presented below.
The statements present specifc positions on the scale from good to bad or undermine or enrich and better or worse. In all three items a negative opinion has been chosen. In the item specific scales no position is chosen but the respondent has to make a choice on the scale.
In both forms an 11 point scale has been used to keep this aspect the same. How ever different random samples of the different countries received Agree/disagree scales for the same requests with 5 and 7 categories but otherwise the structure was the same.
In both forms an 11 point scale has been used to keep this aspect the same. How ever different random samples of the different countries received Agree/disagree scales for the same requests with 5 and 7 categories but otherwise the structure was the same.
A comparison with respect to quality
This research was done in order to evaluate the reliability validity and the quality of these scales. For the way to determine the reliability and validity we refer to another section. All three measures vary between 0 and 1 where reliabilty is higher when the random errors are less The validity is higher when the systematic errors related with the method used are smaller and the validity is the product of the reliability and the valdity. In the table below the results of this experiment in round 3 of the ESS has been shown.
This research was done in order to evaluate the reliability validity and the quality of these scales. For the way to determine the reliability and validity we refer to another section. All three measures vary between 0 and 1 where reliabilty is higher when the random errors are less The validity is higher when the systematic errors related with the method used are smaller and the validity is the product of the reliability and the valdity. In the table below the results of this experiment in round 3 of the ESS has been shown.
This table shows very clearly that the Item spcific scales were much better than the Agree/disagree scales. The table also shows that the Agree/disagree scales with more categories were not better than the often used 5 points scale. The same results hold for most of the countries involved while in other experiments done in a similar way also the same conclusion has to be drawn which is:
Item specific scales (IS) are much better than the Agree/disagree scales in most countries.
Item specific scales (IS) are much better than the Agree/disagree scales in most countries.
This conclusion is based on the experiments reported in the following two published papers:
Saris, W.E., Revilla, M., Krosnick, J.A., and Shaeffer, E. M. (2009) "Comparing Questions with Agree/Disagree Response Options to Questions with Item-Specific Response Options". RECSM Working Paper 1. Published in the Survey Research Methods
Revilla, M., Saris, W. and Krosnick, J.A. (2013) Choosing the number of categories in agree-disagree scales, Sociological Methods and Research. Published online before print December 10, 2013,
doi:10.1177/0049124113509605.
Saris, W.E., Revilla, M., Krosnick, J.A., and Shaeffer, E. M. (2009) "Comparing Questions with Agree/Disagree Response Options to Questions with Item-Specific Response Options". RECSM Working Paper 1. Published in the Survey Research Methods
Revilla, M., Saris, W. and Krosnick, J.A. (2013) Choosing the number of categories in agree-disagree scales, Sociological Methods and Research. Published online before print December 10, 2013,
doi:10.1177/0049124113509605.
