Estimation using pooled data (EUPD)
The problems of the analysis of the two-group SB-MTMM experiments in European Social Survey were due to the fact that in many experiments the three traits in the experiment were three indicators for the same concept. It was not expected by Saris et al. (2014) that this problem would occur so frequently.
However, due to the design of the experiments in the ESS, the correlations between the traits were very similar. Saris and Satorra (2018) have shown that this leads in the two-group SB-MTMM experiments to a model that is empirically not identified. The solution for such identification problems is that extra information or extra restrictions is used in order to make the estimation possible.
In the ESS where in many countries the same experiments are done one expects that the reactions of the people across countries is approximately the same, otherwise the responses would not be comparable. This means that one expects across countries that the response function (the loadings) are approximately the same. This characteristic of these experiments led to the specification of the estimation procedure Estimation Using Pooled Data or EUPD.
The EUPD procedure works in two steps:
Step 1: The estimation of the SB-MTMM model based on the pooled data across all groups which provides an estimate of the matrix of the loadings which is expected to be approximately the same across groups
Step 2: The estimation of the SB-MTMM model in each country restricting the matrix with the loadings to the values that have been estimated in the pooled data analysis (step1). This step also includes a test with Jrule of misspecifications in the model for each country. If the misspecification are sufficiently serious a correction in model is introduced.
This approach has been described and illustrated in:
Saris, Willem & Albert Satorra (2018): The Pooled Data Approach for the Estimation of Split-Ballot Multitrait–Multimethod Experiments, Structural Equation Modeling: A Multidisciplinary Journal, 25:5, 659-672, DOI: 10.1080/10705511.2018.1431543
There was also an alternative for this approach suggested using Bayesian estimation:
Helm Jonathan Lee, Laura Castro-Schilo, Diana Zavala-Rojas, Anna DeCastellarnau & Zita Oravecz (2018) Bayesian Estimation of the True Score Multitrait–Multimethod Model With a Split-Ballot Design, Structural Equation Modeling: A Multidisciplinary Journal, 25:1, 71-85, DOI: 10.1080/10705511.2017.1378103
In order to determine which paper is better for the data of the ESS a comparison of the two approached has been done which showed that the EUPD has considerably smaller Root Mean Square Errors than the Bayesian approach:
Saris, Willem & Albert Satorra (2019): Comparing BSEM and EUPD Estimates for Two-Group SB-MTMM Experiments
Structural Equation Modeling: A Multidisciplinary Journal 26 (5), 745-749, DOI: 10.1080/10705511.2019.1576046
However, due to the design of the experiments in the ESS, the correlations between the traits were very similar. Saris and Satorra (2018) have shown that this leads in the two-group SB-MTMM experiments to a model that is empirically not identified. The solution for such identification problems is that extra information or extra restrictions is used in order to make the estimation possible.
In the ESS where in many countries the same experiments are done one expects that the reactions of the people across countries is approximately the same, otherwise the responses would not be comparable. This means that one expects across countries that the response function (the loadings) are approximately the same. This characteristic of these experiments led to the specification of the estimation procedure Estimation Using Pooled Data or EUPD.
The EUPD procedure works in two steps:
Step 1: The estimation of the SB-MTMM model based on the pooled data across all groups which provides an estimate of the matrix of the loadings which is expected to be approximately the same across groups
Step 2: The estimation of the SB-MTMM model in each country restricting the matrix with the loadings to the values that have been estimated in the pooled data analysis (step1). This step also includes a test with Jrule of misspecifications in the model for each country. If the misspecification are sufficiently serious a correction in model is introduced.
This approach has been described and illustrated in:
Saris, Willem & Albert Satorra (2018): The Pooled Data Approach for the Estimation of Split-Ballot Multitrait–Multimethod Experiments, Structural Equation Modeling: A Multidisciplinary Journal, 25:5, 659-672, DOI: 10.1080/10705511.2018.1431543
There was also an alternative for this approach suggested using Bayesian estimation:
Helm Jonathan Lee, Laura Castro-Schilo, Diana Zavala-Rojas, Anna DeCastellarnau & Zita Oravecz (2018) Bayesian Estimation of the True Score Multitrait–Multimethod Model With a Split-Ballot Design, Structural Equation Modeling: A Multidisciplinary Journal, 25:1, 71-85, DOI: 10.1080/10705511.2017.1378103
In order to determine which paper is better for the data of the ESS a comparison of the two approached has been done which showed that the EUPD has considerably smaller Root Mean Square Errors than the Bayesian approach:
Saris, Willem & Albert Satorra (2019): Comparing BSEM and EUPD Estimates for Two-Group SB-MTMM Experiments
Structural Equation Modeling: A Multidisciplinary Journal 26 (5), 745-749, DOI: 10.1080/10705511.2019.1576046
