History of the SRF
The small house was the office of the SRF in Amsterdam
The SRF (in Dutch "SSO") was founded in 1984 in the Netherlands by Prof. Dr. Willem E. Saris. The aim of the foundation was the improvement of social science research by application of statistics. Although the term sociometry has been used for network analysis, it was nevertheless decided to use the name sociometric research in order to connect the activities of the Foundation with the similar research done in psychometry and econometry.
The creation of the Graduate school in Psychometry and Sociometry IOPS in the Netherland by an initiative of the pscychometrist Prof. Dr. G. Mellenberg and sociometrist Prof. Dr. W. Saris around the same time.
The promotion of sociometrics started by organizing a conference on Sociometric Research with as special guests Robert Groves (later Director of US census office) and William Belson (professor of survey methodology in Great Britain). In the proceeding one can also see the first papers of our later colleagues, Albert Satorra, Peter Neijens and Jan de Ridder.
- Middendorp, C.P., Niemöller, B. and Saris,W.E. (1982). Sociometric Research. Amsterdam, SRF.
- Saris, W.E. and Gallhofer, I.N. (1987). Sociometric Research I: data collection. London. MacMillan Press.
- Saris,W.E. and Gallhofer I.N. (1987) Sociometric Research II: data analysis. London. MacMillan Press.
Structural equation models (SEM)
Because the study of measurement errors requires the possibility to distinguish between observed variables with errors and the same variable without errors, statistical procedures had to be used that could make a distinction between the two variables and allows for the estimation of the size of the measurement errors. This was possible with Structural Equation Models (SEM).
The first contribution of the SRF to the social science research was an introduction to this statistical approach which has been used widely through the years.This book was written by Willem Saris and Dr H. Stronkhorst (later, the director of the social section of the Dutch Science foundation (NWO) and also the European Science foundation (ESF)).
Because the study of measurement errors requires the possibility to distinguish between observed variables with errors and the same variable without errors, statistical procedures had to be used that could make a distinction between the two variables and allows for the estimation of the size of the measurement errors. This was possible with Structural Equation Models (SEM).
The first contribution of the SRF to the social science research was an introduction to this statistical approach which has been used widely through the years.This book was written by Willem Saris and Dr H. Stronkhorst (later, the director of the social section of the Dutch Science foundation (NWO) and also the European Science foundation (ESF)).
Testing Structural Equation models
Worried about the testing procedures in structural equation modelling, the SRF worked for years together with Professor Dr.Albert Satorra (Universitat Pompeu Fabra, Barcelona) to improve these procedures. Through the years Albert Satorra has been the statistical conscience of the SRF. The cooperation with him has been extreme important for the SRF
Worried about the testing procedures in structural equation modelling, the SRF worked for years together with Professor Dr.Albert Satorra (Universitat Pompeu Fabra, Barcelona) to improve these procedures. Through the years Albert Satorra has been the statistical conscience of the SRF. The cooperation with him has been extreme important for the SRF
Prof.Dr Albert Satorra and Prof.Dr. Willem Saris at the SRF.
They also have developed different procedures to evaluate structural equation models.
- Satorra, A. and Saris, W.E. (1986). Power of the likelihood ratio test in covariance structure analysis. Psychometrika. 50: 83–90. doi: 10.1007/BF0229415
- Saris, W.E., Satorra, A., and Sörbom, D. (1987). Detection and correction of structural equation models. Sociological Methodology. 105–131. doi: 10.2307/271030
- Saris, W.E., Satorra, A., and de Pijper, W.M. (1991). A comparison of several approximations to the power function of the likelihood ratio test in covariance structure analysis. Statistica Neerlandica. 45 (2): 173–185. doi:
10.1111/j.1467-9574.1991.tb01302.x
- Saris, W.E.; Satorra, A. and van der Veld, W.M. (2009). Testing Structural Equation Models or Detection of Misspecifications?. Structural Equation Modeling: A Multidisciplinary Journal. 16 (4), 561–582. doi: 10.1080/10705510903203433
William van der Veld
A program (JRule) was developed for these tests performed using the program LISREL by
Dr. William van der Veld (Radboud University, Nijmegen). This program can be obtained for free from the author.
A similar program (JRule) has been developed by Daniel Oberski (http://daob.nl/software/) for
use in combination with the program Mplus.
In R it is part of the miPowerFit function.
Dr. William van der Veld (Radboud University, Nijmegen). This program can be obtained for free from the author.
A similar program (JRule) has been developed by Daniel Oberski (http://daob.nl/software/) for
use in combination with the program Mplus.
In R it is part of the miPowerFit function.
Prof.Dr Zvi Namenwirth , University of Connecticut
Procedures for analysis of existing texts
Before starting with the SRF, Irmtraud Gallhofer evaluated together with others different procedures for text analysis.
Before starting with the SRF, Irmtraud Gallhofer evaluated together with others different procedures for text analysis.
- Namenwirth,J.Z., Saris, W.E., Gallhofer, I.N. and Kleinnijenhuis, J. (1978). In search of semantic characteristics for machine coding, Methoden en Data Nieuwsbrief van de Sociaal-Wetenschappelijke Sectie van de Vereniging voor Statistiek, MDN 1
- Namenwirth, J.Z.,Gallhofer I.N., and K. Niemöller (1978). Reader on text-analysis, MDN 1
- Gallhofer, I.N., Saris W.E. and de Valk, B.M. (1978). Een begrippenapparaat voor de beschrijving van redeneringen van politici, Acta Politica 3
- Gallhofer, I.N., Saris, W.E. and E.L. Morton. (1978). A validation study of Holsti’s dictionary procedure, Quality and Quantity 12 (2), 131–145. doi: 10.1007/BF00144066
Dr. Irmtraud Gallhofer
After that general orientation in text analysis Irmtraud Gallhofer in cooperation with other researchers studied procedures for text analysis with respect to decision making. In this research the emphasis was on the question in how far the formal model of decision making is applied in important decisions as foreign policy decision making.
- Gallhofer, I.N. and Saris, W.E. (1985). Un procedimiento de codificación para la investigación empírica de la toma de decisiónes políticas, Revista Internacional de Sociología 43, 163-182.
- Gallhofer, I.N., Saris, W.E. and Melman, M. (1986). The Empirical Decision analysis procedure. In I.N. Gallhofer, W.E. Saris and M. Melman (Eds.), Different Text-Analysis Procedures for the Study of Decision-Making. Amsterdam: Sociometric Research Foundation
- Gallhofer, I.N. and Saris, W.E. (1988). A coding instrument for empirical research of political decision-making. In: W.E. Saris and I.N. Gallhofer (Eds.), Sociometric Research I, London: Macmillan.
- Gallhofer, I.N., Saris, W.E. and Schellekens,M. (1988). General recognition of political arguments. In: L. Beach, B. Rohrmann, C. Vlek and S.Watson (Eds.), Advances in Decision Research. Amsterdam: North Holland, Elseviers publishers.
- Gallhofer, I.N. (1989). Graphische Darstellungsweisen von Entscheidungsbäumen. In: J. Bellers and W. Woyke (Eds.), Analyse Internationaler Beziehungen. Budrich, Opladen: Leske.
- Gallhofer, I.N. and Saris, W.E. (1989). Three methods for analyzing decision-making using written documents. In: H. Montgomery and O. Svenson (Eds.), Process and Structure in Human Decision-Making. London: Wiley.
- Gallhofer, I.N. and Saris, W.E. (1989). Decision trees and decision rules in politics: the Empirical Decision Analysis Procedure. In: H. Montgomery and O. Svenson (Eds.), Process and Structure in Human Decision-Making. London: Wiley.
After this fundamental research the attention was directed on texts of minutes of governments concerning arguments of individual members of the government and the collective decision making of the government.
Finally two books were published that present a complete overview of the developed procedures and the results of the analyses.
Finally two books were published that present a complete overview of the developed procedures and the results of the analyses.
- Gallhofer, I.N. and Saris, W.E. (1996). Foreign policy decision making: A Qualitative and Quantitative Analysis of Political Argumentation. London, Praeger
- Gallhofer, I.N. and Saris, W.E. (1997). Collective Choice Processes: A qualitative and Quantitative Analysis of foreign policy decsion-making. London, Praeger
Prof.Dr B.Hamblin
University of Arizona in Tucson
Continuous scales and variation in response functions
Application of SEM showed how large the errors were in survey data. Inspired by the work of Prof.Dr Bob Hamblin we directed our research on procedures to improve the measurement in the social sciences by substituting the usual category scales by continuous scales.
Application of SEM showed how large the errors were in survey data. Inspired by the work of Prof.Dr Bob Hamblin we directed our research on procedures to improve the measurement in the social sciences by substituting the usual category scales by continuous scales.
- Saris W.E (1982) . Different questions, different variables. In C.Fornell (ed), A second generation of multivariate analysis.Volume 2 Measurement and evaluation. pp.78-95, New York , Praeger.
- Saris W.E (1982) Individual response functions and correlations between judgements. In: W.E. Saris, I.N. Gallhofer (eds.), Sociometric Research, Volume I. Data Collecting and Scaling. London. McMillan Press.
- Saris W.E. (1988). A measurement model for psychophysical scaling. Quality & Quantity, 22 (4), 417-433. doi: 10.1007/BF00171631.
- Saris W.E. (1998). Words are sometimes not enough to express the existing information. In M. Fennema, C. van der Eijk, H. Schijf (Eds.), In Search of Structure: essays in social science and methodology. Amsterdam: Het Spinhuis. 63-73. ISBN: 90-5589-114-2
Prof. Milton Lodge University of NewYork at Stony Brook
During a visit to Prof Dr Milton Lodge we detected that people could provide very precise responses but using different scales. This variation in response functions will be perceived as measurement error if it is not detected. In order to prevent this problem it was suggested to use two fixed reference points normally at the end points of the scales so that there could be no doubt what they meant. In this research many people have been cooperated and the work has been summarized in the book mentioned below:
- Saris, W. E. (1988). Variation in response functions: a source of measurement error in attitude research. Amsterdam: Sociometric Research Foundation. ISBN: 9070947080.
Marius de Pijper
Computer assisted data collection
Studies were undertaken as early as 1982 to improve measurement by use of computer assisted data collection, especially in order to remove the effect of the interviewer in the data collection. In this reseach our programmer Marius de Pijper played a very important role.
Studies were undertaken as early as 1982 to improve measurement by use of computer assisted data collection, especially in order to remove the effect of the interviewer in the data collection. In this reseach our programmer Marius de Pijper played a very important role.
- de Pijper W.M., and Saris W.E. (1986). The formulation of interviews using the program Interv. Amsterdam. Sociometric Research Foundation, pp.168.
- Saris, W. E. (1991).Quantitative Applications in the Social Sciences: Computer-assisted interviewing. Thousand Oaks, CA: SAGE Publications, Inc. doi: 10.4135/9781412983723
The telepanel system used from 1986- 1996 for interviewing without interviewers
Also the University of Amsterdam has used the developed system for several years with support of the Dutch Science Foundation (NWO). This sytem made it possible for several PhD students to collect data and to do experiments which they needed for their dissertation.
- Saris,W.E. and de Pijper, W.M. (1986). Computer assisted interviewing using home computers. European Research, 14, 144-150.
- Saris W.E. (1998). Ten years of interviewing without interviewers: The telepanel. In Couper M.P., R.P.Baker, J.Bethlehem, C.Clark, J.Martin, W.L.Nicholls II, J.M. O’reilly (Eds.) Computer assisted survey information collection.New York Wiley, 409-431. ISBN: 978-0-471-17848-4.
- Saris W.E. (1993). CASIP: A complete automated system for information processing in family budget research, Eurostat report, 230.
Attitudes and non attitudes and the Choice questionnaire
Given the well-known fact that most people do not know much about many problems discussed and inspired by the decision theoretical approach a new research line was developed to develop a decision aid for people to participate in decision making with respect to complex issues. In this context a decision aid, the Choice questionnaire, was developed by Willem Saris, Peter Neijens and Jan de Ridder. In this procedure the respondents had to evaluate necessary information about the issue before they were asked to make their choice. Peter MNeijens wrote his dissertation about this approach and some articles were published.
Given the well-known fact that most people do not know much about many problems discussed and inspired by the decision theoretical approach a new research line was developed to develop a decision aid for people to participate in decision making with respect to complex issues. In this context a decision aid, the Choice questionnaire, was developed by Willem Saris, Peter Neijens and Jan de Ridder. In this procedure the respondents had to evaluate necessary information about the issue before they were asked to make their choice. Peter MNeijens wrote his dissertation about this approach and some articles were published.
- Saris, W.E., P. Neijens and J.A. de Ridder (1984). Kernenergie: ja of nee? Een weloverwogen oordeel van de Nederlandse bevolking. Amsterdam: SSO.
- Neijens, P. and W.E. Saris (1986). Evaluation of a decision aid in a nationwide discussion. In: B. Brehmer, H. Jungermann, P. Lourens & G. Sevon (Eds.), New directions in research on decision making. Amsterdam: North-Holland.
- Neijens, P. (1987). The Choice Questionnaire. Design and evaluation of an instrument for collecting informed opinions of a population. Amsterdam: Free University Press.
- Neijens, P. de Ridder, J.A., and Saris, W.E. (1992). An instrument for collecting informed opinions. Quality and Quantity. 26, 245–258. doi: 10.1007/BF00172428
With Paul Sniderman of the Stanford University a reader was published with papers of American and European researchers about attitudes and nonattitudes that were earlier discussed at a conference the SRF organized in Amsterdam.
- Saris W.E. and P.Sniderman (2004) Studies in Public Opinion: Attitudes, nonattitudes, measurement error and change. Princeton, Princeton University Press.
Prof.Dr. Frank Andrews
University of Michigan
Estimation of the quality of survey questions
After all these studies Willem Saris realized that one indeed could improve survey measures but one could never reduce the errors to zero. So he decided to change his research in the direction of estimating the size of the systematic and random errors in the measures in order to be able to correct for measurement error. At that point he recognized the importance of the work on this topic done by Frank Andrews (1984). We continued in fact his research. Because of the early death of Frank Andrews he has not seen how much his research has been appreciated and further developed through the years.
After all these studies Willem Saris realized that one indeed could improve survey measures but one could never reduce the errors to zero. So he decided to change his research in the direction of estimating the size of the systematic and random errors in the measures in order to be able to correct for measurement error. At that point he recognized the importance of the work on this topic done by Frank Andrews (1984). We continued in fact his research. Because of the early death of Frank Andrews he has not seen how much his research has been appreciated and further developed through the years.
The SRF started this new research line by organizing a conference with representatives of different countries because we realized that such a research could never be done by a small research group. In that conference a research program has been developed in cooperation with 11 different research groups in Europe. The research group (IRMCS) met yearly to discuss different statistical issues about the research and the applications in the different countries.
First the approach to evaluate measurement instruments, the so-called Multitrait-Multimethod (MTMM) experiment has been evaluated. The papers written in this context have been brought together in a reader:
Finally different models for the analysis of the data of MTMM experiments were tested with the final conclusion that the classical model and the equivalent True Score model were the best to be used in the analysis:
First the approach to evaluate measurement instruments, the so-called Multitrait-Multimethod (MTMM) experiment has been evaluated. The papers written in this context have been brought together in a reader:
- Saris, W.E., v. Meurs, A. (Eds.), Evaluation of measurement instruments by meta-analysis of multitrait multimethod studies. North-Holland.
- Saris,W.E. and Andrews, F.M. (1991) Evaluation of measurement instruments using a structural modeling approach. In: P. Biemer, R. Groves, L. Lyberg, N. Mathiowetz, S. Sudman (Eds.): Measurement errors in surveys. John Wiley & Sons.
- Saris, W.E. and Münnich, A. (Eds). (1994). Multitrait Multimethod Approach to Evaluate Measurement Instruments. Budapest, Eötvös University Press, 280.
- Saris W.E., Veenhoven R., Scherpenzeel, A.C., and Bunting B. (Eds) (1996). A comparative Study of Satisfaction with Life in Europe. Budapest: Eotvos University Press, 305.
Finally different models for the analysis of the data of MTMM experiments were tested with the final conclusion that the classical model and the equivalent True Score model were the best to be used in the analysis:
- Corten, I.W., Saris, W.E., van der Veld, W.M., Aalberts, C.E., and Kornelis, C. (2009). Fit of different models for Multitrait-Multmethod Experiments. Structural Equation Modeling. 9(2): 213–232. doi: 10.1207/S15328007SEM0902_4
- Saris, W.E. and Aalberts, C. (2003). Different explanations for correlated disturbance terms in MTMM studies. Structural Equation Modeling. 10 (2): 193–214. doi: 10.1207/S15328007SEM1002_2
One of the rare official fotos of the IRMCS research group with as permanent members from left to right, no1 Dr. Irena Bartova, no2 Prof.Dr. Brendan Bunting,no.3 Prof.Dr Anuska Ferligoy,no4(front) Prof.Dr Jaak Billiet,no4 (back) Dr Jan Rehak no.5 Prof.Dr Akos Munnich, no.6 Prof.Dr.Willem Saris, no.Dr Anna Andreenkova no.8 Dr Tina Hlebec no.9.Dr Annette Scherpenzeel, no10 Prof.Dr JuanManuel Battista-Foquet
Dr. Annette Scherpenzeel now working at SHARE (Munich)
Prediction of the quality of questions
Probably the most important development in this research took place when we realized that one can never study the reliability and validity of all questions. This would be too expensive and time consuming. The alternative suggested was to develop a procedure to predict the reliability and validity of these questions on the basis of the coded characteristics of the questions. If these predictions were good enough, the prediction procedure could also be used for questions that were not studied so far but of which the same characteristics were coded. To proof this a lot of experiments were needed which could be collected on the basis of the grant obtained for the Telepanel mentioned above. This research was done mainly by Annette van Scherpenzeel. It turned out that the prediction of the quality of the questions, determined in the MTMM experiments, on the basis of the characteristics of the questions, was quite good.
This led to the idea to design a computer assisted expert system which uses all available information of data quality to predict the quality of new questions. This program, called Survey Quality Predictor or SQP was first developed by him in MS-DOS and later transformed in a Windows version by Oberski, Kuipers en Saris (SQP 1.0)
Probably the most important development in this research took place when we realized that one can never study the reliability and validity of all questions. This would be too expensive and time consuming. The alternative suggested was to develop a procedure to predict the reliability and validity of these questions on the basis of the coded characteristics of the questions. If these predictions were good enough, the prediction procedure could also be used for questions that were not studied so far but of which the same characteristics were coded. To proof this a lot of experiments were needed which could be collected on the basis of the grant obtained for the Telepanel mentioned above. This research was done mainly by Annette van Scherpenzeel. It turned out that the prediction of the quality of the questions, determined in the MTMM experiments, on the basis of the characteristics of the questions, was quite good.
- Scherpenzeel, A., and Saris, W.E. (1997). The validity and reliability of survey questions: a meta analysis of MTMM studies. Sociological Methods and Research. 25(3): 341–383. doi:10.1177/0049124197025003004
- Saris, W.E., van Wijk, T., and Scherpenzeel, A. (1998). Validity and reliability of subjective social indicators: the effect of different measures of association. Social Indicators Research. 45: 173–199.
This led to the idea to design a computer assisted expert system which uses all available information of data quality to predict the quality of new questions. This program, called Survey Quality Predictor or SQP was first developed by him in MS-DOS and later transformed in a Windows version by Oberski, Kuipers en Saris (SQP 1.0)
- Saris W.E., van der Veld, W. and Gallhofer, I.N. (2004). Development and improvement of questionnaires using predictions of reliability and validity. In Presser et al. (Eds) Methods for testing and evaluating survey questionnaires. Hoboken, Wiley, 275-299.
- Saris W. E., and Gallhofer, I. N. (2007). Estimation of the effects of measurement characteristics on the quality of survey questions. Survey Research Methods, 1, 31─46.
- Saris W.E. and Gallhofer, I.N. (2007) Design, evaluation and analysis of questionnaires for survey Research. Hoboken, USA. Wiley.
The ESS cooperation
Given the work SRF had done on improvement of survey research Willem Saris was asked in 1999 to become a member of the methodological group to prepare the European Social Survey. The plan was to hold the same survey in all countries in Europe who were willing to cooperate in this project. Willem Saris convinced the methodology group that such a large project required control of the quality of the data and study of the comparability of the questions across countries. He suggested to add to the main questionnaire a supplementary questionnaire for this research in the form of MTMM experiments. In preparing for this large scale application of MTMM experiments a new design, The Split-ballot MTMM design, for MTMM experiments was developed in order to reduce the number of repetitions of the same questions. This work was done in cooperation with Prof.Dr. Albert Satorra and Prof Dr. Germa Coenders.
Because of the lack of validity of the proposals for measurement instruments specified even by experts in the different fields, Dr. Gallhofer and Prof. Dr Saris created the “three steps procedure for designing survey questions”. The suggestion is that the use of this procedure guarantees the validity of the proposed questions
Given the work SRF had done on improvement of survey research Willem Saris was asked in 1999 to become a member of the methodological group to prepare the European Social Survey. The plan was to hold the same survey in all countries in Europe who were willing to cooperate in this project. Willem Saris convinced the methodology group that such a large project required control of the quality of the data and study of the comparability of the questions across countries. He suggested to add to the main questionnaire a supplementary questionnaire for this research in the form of MTMM experiments. In preparing for this large scale application of MTMM experiments a new design, The Split-ballot MTMM design, for MTMM experiments was developed in order to reduce the number of repetitions of the same questions. This work was done in cooperation with Prof.Dr. Albert Satorra and Prof Dr. Germa Coenders.
- Saris, W.E., Satorra, A. and Coenders, G. (2004). A new approach to evaluating the quality of measurement instruments: the split-ballot MTMM design. Sociological Methodology. 311–347.
Because of the lack of validity of the proposals for measurement instruments specified even by experts in the different fields, Dr. Gallhofer and Prof. Dr Saris created the “three steps procedure for designing survey questions”. The suggestion is that the use of this procedure guarantees the validity of the proposed questions
- Gallhofer, Irmtraud (2004). Operationalization of social science concepts by intuition. Quality and Quantity. 38: 235–258.
This work for the ESS took so much time that we had to dedicate all our time to this research. This was a unique opportunity to apply the developed knowledge in practice but it prevented to continue the work of the Sociometric Research Foundation. Because we also decided to move to Spain the SRF was liquidated and we continue our research with partially the same group of people under the name Research and Expertice Centre for Survey Methodology (RECSM), first at he busuness school ESADE and later at the Universitat Pompeu Frabra, both in Barcelona.
The RECSM research group that worked for the ESS. From left to right: Prof Dr. Willem Saris, Dr Diana Zavala Rojas, Dr Paolo Moncagatta, Dr Wiebke Weber , Dr Melanie Rivella and Dr Daniel Oberski.
Research and Expertise Centre for Survey Methodology (RECSM)
During this participation within the ESS in every round (every 2 years since 2001) Willem Saris, Irmtraud Gallhofer and others controlled and improved the quality of the questions using the predictions made first with SQP Dos and later with SQP 1.0. A report about the construction of the core questionnaire of the ESS can be found on the website of the ESS.
Besides that, SB-MTMM experiments have been performed in each round which were analyzed and have been reported to the CCT. Many papers have been published on the basis of this research while all PhD students working in RECSM have finished now their dissertation at least partially on the basis of research for the ESS or at least data of the ESS. A very important results was also obtained which suggests that standard agree/disagree questions have a much lower quality than questions specified with Item specific scales. Also another important result was that it was detected that increasing the number of categories of a Agree/ disagree scale above a 5 point scale does not help to improve the quality of the responses.
During this participation within the ESS in every round (every 2 years since 2001) Willem Saris, Irmtraud Gallhofer and others controlled and improved the quality of the questions using the predictions made first with SQP Dos and later with SQP 1.0. A report about the construction of the core questionnaire of the ESS can be found on the website of the ESS.
Besides that, SB-MTMM experiments have been performed in each round which were analyzed and have been reported to the CCT. Many papers have been published on the basis of this research while all PhD students working in RECSM have finished now their dissertation at least partially on the basis of research for the ESS or at least data of the ESS. A very important results was also obtained which suggests that standard agree/disagree questions have a much lower quality than questions specified with Item specific scales. Also another important result was that it was detected that increasing the number of categories of a Agree/ disagree scale above a 5 point scale does not help to improve the quality of the responses.
- Saris, W.E, Revilla, M., Krosnick, J., and Schaefer, E.M. (2009). Comparing questions with agree/disagree response options to questions with item specific response options. Survey Research Methods. 16. 561–582.
- Revilla, M., Saris, W. and Krosnick, J.A. (2013). Choosing the number of categories in agree-disagree scales, Sociological Methods and Research. Published online before print December 10, 2013
Dr. Daniel Oberski
After three rounds of the ESS were done and the MTMM experiments analyzed, there were 1005 earlier questions evaluated and 2695 ESS questions. Based on these 3700 questions Daniel Oberski and Thomas Gruner created a new version of SQP (SQP2.0) based on the work of the RECSM group around Willem Saris . The advantages of the new version of SQP is that the program is online so that users can add their own questions in the data base and receive evaluations of the quality of the questions with respect to reliability and validity. Users get also information about possible improvements of the questions. The importance of this program is that the evaluation is not limited to the questions that have been evaluated but the program can be used for any questions for which the characteristics are coded in the system. All these questions and quality predictions are available to all users free of charge. In this way the knowledge base of questions and quality predictions is growing day by day. Momentarily the program contains already 67.000 questions introduced by more than 2000 unique users of the program.
- Saris W., Oberski, D., Revilla, M., Zavala, D., Lilleoja, L., Gallhofer I. and T. Gruner (2011). The development of the program SQP 2.0 for the prediction of the quality of survey questions, RECSM Working paper 24
- Saris W.E. and Gallhofer, I.N. (Second Edition, 2014) Design, evaluation and analysis of questionnaires for survey Research. Hoboken, USA. Wiley.
- DeCastellarnau A. and W.Saris (2014). A simple procedure to correct for measurement errors in survey research. ESS Edunet
- Saris, W.E. and Revilla, M. (2016). Correction for measurement errors in survey research: necessary and possible. Social Indicators Research. 127 (3): 1005–1020.
- Revilla M., Zavala, D. and Saris, W.E. (2017). Creating a good question: How to use the cumulative experience.
In Wolf C. ,D Joy T.Smith Y. Fu (Eds.) Survey Methodology. - Van der Veld, W. and Saris, W.E. (2017). Measurement Equivalence Testing 2.0. In Davidov, E., P.Schmidt, J.Billiet, B. Meuleman (Eds.), Cross-Cultural Analysis. Methods and Applications, Second Edition.
Besides all the work done for the ESS all people working from the start in our research group finished their dissertation based on methodological problems we were confornted with or on data that were collected by the ESS. The following dissertations have been published:
- Daniel L Oberski. Measurement error in Comparative Surveys. (2011) PhD dissertation at the University of Tilburg.
- Melanie Revilla. Impact of the mode of data collection on the quality of survey question in social sciences. (2012) PhD dissertation at the Universitat Pompeu Fabra.
- Wiebke Weber. Behind Left Right. The meaning of right-left orientation in Europe. (2015) PhD dissertation at the Universitat Pompeu Fabra.
- Paolo Moncagatta. Studies in support of Democracy. (2016) PhD dissertation at the Universitat Pompeu Fabra.
- Diana Zavala Rojas. Measurement equivalence i multilingual comparative survey research. (2017) PhD dissertation at the Universitat Pompeu Fabra.
- André Pirralha. The relationship between political participation and life satisfaction. (2017) PhD dissertation at the Universitat Pompeu Fabra.
- Laur Lilleoja. The internal and external validity of the theory of basic human values. (2017) PhD dissertation at the Tallinn University.
