THE POWER OF THE TEST ON MISSPECIFICATIONS
If a parameter is correctly fixed on a specific value or correctly restricted to other parameters, the EPC nevertheless may deviate from zero by chance and consequently the test statistic MI as well. Fortunately the distribution of the MI under this condition is known to be a x2 distribution with 1 degree of freedom. This distribution has a means of 1 (-df).
To test if the MI is not significantly different zero one normally choses the critical value for which this hypothesis is rejected, with a 5% chance of a wrong decision (α level of the test) which means that the hypothesis is rejected if the obtained MI value is larger than 3.84. The blue line in the figure below represents this distribution with df= 1.
This way of testing whether a parameter is correctly specified or not is too simple because the test can be so sensitive for a specific parameter that even an EPC= 0.01 or even an EPC =0.0001 will lead to rejection of the hypothesis that parameter is correctly specified. On the other hand we also would not like to accept that the parameter is not miss-specified if an EPC = 0.7 was observed. This means that we have to have information about the probability that the misspecification is detected for a critical value of the EPC. This probability is called “the power of the test”. This critical value is the value of the EPC for which we would like to detect with high probability that the parameter is miss-specified.
The power of the test can be obtained from the tables of the non-central -distribution (or using any computer-based routine) as:
Prob ( x2(1, ncp) ˃ cα) (1)
Where cα is the critical value of an a-level test based on x2 distribution with df= 1 and x2(1, ncp) is the non-central chi-square distribution with non-centrality parameter (ncp).
The noncentrality parameter (ncp) shifts the mean of the x2 - distribution with df= 1 with the value of the ncp to the right. In the figure above the non-central x2-distribution with a ncp= 8 is represented by the red graph. This distribution is very flat and the mean of this distribution is equal to df + ncp.
If that would be the situation for a test to detect a miss-specified parameter and we would reject the null hypothesis if MI > 3.84 then the probability that we would make the proper decision is the area under the red curve at the right side of the value 3.84. This probability is called the power of the test. In this figure the power of the test is 0.81.
We also see that there is a chance that we make an error using this test because if the MI < 3.84 we accept the model. The probability that this decision is wrong (beta error) is represented by the area under the red curve at the left side of the value 3.84 which is in this case .19. The power = 1- beta error or the beta error = 1- the power.
In order to determine the ncp of a test we need the value of MI and EPC which can be obtained directly from any SEM program nowadays but one also has to determine the critical value (d) of the EPC for which we would like to have high power to detect that the parameter is miss-specified.
ncp=(MI/EPC2)d2 (2)
Saris et al. (2009) suggested that for a standardized structural parameter and for a correlated error term a misspecification of 0.1 is relevant enough to be detected by a test. For factor loadings, one may follow the standard approach where loadings smaller than .4 are ignored. These values are merely suggestions and one could use other values for d that are more appropriate within a specific theory of interest.
In the next section we present the procedure suggested by Saris et al. (2009) how to use the presented tools for the test of misspecification of single parameters in structural equation models.
In order to avoid cumbersome calculations Van der Veld, Saris, Satorra (2008) have developed the program Jrule which can take over these calculations.
