The Three Steps Procedure: A Procedure to Design Valid Questions
Formulating questions for survey research is a different matter than formulating questions in conversations. In survey research a question should measure the concept that the researcher wants to measure. This is called the validity of a question. Often the concepts that people like to measure are "concepts by postulation" or complex concepts. That means that the researcher wants to measure the concept by a combination of different questions. In this section we concentrate on "concepts by intuition" that is a simple concept that can be measured by a single question for which the meaning is rather obvious. To guarantee that a question is a valid question for a "concept by intuition" Gallhofer and Saris (2014) have developed a three steps procedure.
The three steps procedure
The first step requires to determine to which more basic class of concepts the concept of interest belongs. This is relevant because there are so many different concepts that one cannot develop a procedure for all concepts that can be imagined. However, it becomes clear that there is a limited number of basic concepts which cover most social science concepts. The basic concepts are indicated in the first column of the table below. For these basic concepts one can suggest how one can formulate questions. In that way one can solve the problem for most of the concepts in the social sciences. We can say so because a linguistic analysis of assertions or statements have shown that all basic concepts can be specified in three basic forms of sentences. These basic forms have been summarised in the table below. For examples and the basis for this table we refer to the publication of Saris and Gallhofer (2014).
The first step requires to determine to which more basic class of concepts the concept of interest belongs. This is relevant because there are so many different concepts that one cannot develop a procedure for all concepts that can be imagined. However, it becomes clear that there is a limited number of basic concepts which cover most social science concepts. The basic concepts are indicated in the first column of the table below. For these basic concepts one can suggest how one can formulate questions. In that way one can solve the problem for most of the concepts in the social sciences. We can say so because a linguistic analysis of assertions or statements have shown that all basic concepts can be specified in three basic forms of sentences. These basic forms have been summarised in the table below. For examples and the basis for this table we refer to the publication of Saris and Gallhofer (2014).
The second step in this procedure is the selection of a structure of an assertion from the possibilities for the chosen basic concept and to formulate an assertion for the concept that one wants to measure as a specific case. For example to study job satisfaction. This is a feeling about jobs. The table shows that one can consider three different forms of an assertion
my job(x) is(I) nice (f)
or I (x) like (F) my job(y)
or my job (x) gives me(P) a nice feeling(f)
One of them should be chosen to continue to develop the question. Let us say we choose the first form.
The third step is to transform the assertion formulated into one of the different requests for an answer that are possible.
There are again different procedures for this transformation. For example:
Is your job nice?
Can you tell me how nice your job is.
I would like to ask you whether your job is nice.
The first form is a question, the second an instruction and the third is a statement. Because often not the form of question is used but always an answer is expected the term "a request for an answer" is a better term to describe these sentences than "a question".
This text gives a brief picture of the procedure that has been worked out to assure that the question will measure what the researcher likes to measure. It will be clear that one has to add much more components to this simple request for an answer, such as the response scale, an introduction and instructions, but first the basic structure must be good.
In other sections we discuss the additional components to be added to the request for an answer and the development of complex concepts.
For the details of the three steps procedure and the complete process of designing requests for answers and evaluation of them with respect to quality, we refer to
Saris and Gallhofer (2017) and Revilla et al. (2017).
my job(x) is(I) nice (f)
or I (x) like (F) my job(y)
or my job (x) gives me(P) a nice feeling(f)
One of them should be chosen to continue to develop the question. Let us say we choose the first form.
The third step is to transform the assertion formulated into one of the different requests for an answer that are possible.
There are again different procedures for this transformation. For example:
Is your job nice?
Can you tell me how nice your job is.
I would like to ask you whether your job is nice.
The first form is a question, the second an instruction and the third is a statement. Because often not the form of question is used but always an answer is expected the term "a request for an answer" is a better term to describe these sentences than "a question".
This text gives a brief picture of the procedure that has been worked out to assure that the question will measure what the researcher likes to measure. It will be clear that one has to add much more components to this simple request for an answer, such as the response scale, an introduction and instructions, but first the basic structure must be good.
In other sections we discuss the additional components to be added to the request for an answer and the development of complex concepts.
For the details of the three steps procedure and the complete process of designing requests for answers and evaluation of them with respect to quality, we refer to
Saris and Gallhofer (2017) and Revilla et al. (2017).