​The Power of the Test on Misspecifications

If a parameter is correctly fixed on a specific values or correctly restricted to other parameters, the EPC nevertheless may deviate from zero by chance and consequently the test statistic MI as well. Fortunately the distribution of the MT under this condition is known to be a χ²-distribution with 1 degree of freedom. This distribution has a mean of 1 (=df).

To test if the MI is not significantly different from zero one normally choses the critical value for which this hypothesis is rejected, with a 5% chance of a wrong decision (α level of the test) which means that the hypothesis is rejected if the obtained MI value is larger than 3.84. The blue line in the figure below represents this distribution with df=1.

​This way of testing whether a parameter is correctly specified or not is too simple because the test can be so sensitive for a specific parameter that even an  EPC= 0.01 or even an EPC =0.0001 will lead to rejection of the hypothesis that parameter is correctly specified. On the other hand we also would not like to accept that the parameter is not miss-specified if an EPC = 0.7 was observed. This means that we have to have information about the probability that the misspecification is detected for a critical value of the EPC. This probability is called “the power of the test”. This critical value is the value of the EPC for which we would like to detect with high probability that the parameter is miss-specified.  

The power of the test can be obtained from the tables of the non-central χ²-distribution (or using any computer-based routine) as:

Prob(χ²(1,ncp) > C_α)                                      (1)

​

where C_α is the critical value of an α-level test based on a χ²-distribution with df=1 and χ²(1, ncp) is the non-central χ²-distribution with non-centrality parameter (ncp).

​

The non-centrality parameter (ncp) shifts the mean of the χ²-distribution with df=1 with the value of the ncp to the right. In the figure above the noncentral χ²-distribution with a ncp=8 is represented by the red graph. This distribution is very flat and the mean of this distribution is equal to df + ncp.

​​If that would be the situation for a test to detect a miss-specified  parameter and we would reject the null hypothesis if MI > 3.84 then the probability that we would make the proper decision is the area under the red curve at the right side of the value 3.84. This probability is called the power of the test. In this figure the power of the test is 0.81.

We also see that there is a chance that we make an error using this test because if the MI < 3.84 we accept the model. The probability that this decision is wrong (beta error) is represented by the area under the red curve at the left side of the value 3.84 which is in this case .19. The the power = 1- beta error or the beta error = 1- the power.

In order to determine the ncp of a test we need the value of MI and EPC which can be obtained directly from any SEM program nowadays but one also has to determine the critical value (d) of the EPC for which we would like to have high power to detect that the parameter is miss-specified.
​
With the choice of the value of d the ncp can be computed using equation (2):

ncp=(MI/EPC²)d²                                      (2)

Saris et al. (2009) suggested that for a standardized structural parameter and for a correlated error term a misspecification of 0.1 is relevant enough to be detected by a test. For factor loadings, one may follow the standard approach where loadings smaller than .4 are ignored. These values are merely suggestions and one could use other values for d that are more appropriate within a specific theory of interest.
 
In the next section we present the procedure suggested by Saris et al. (2009) how to use the presented tools for the test of misspecification of single parameters in structural equation models.

In order to avoid cumbersome calculations Van der Veld, Saris, Satorra (2008) have developed the program JRule which can take over these calculations. 

References:
Saris, W. E., Satorra, A., & Van der Veld, W. M. (2009). Testing structural equation models or detection of misspecifications? Structural Equation Modeling, 16(4), 561- 582.

Van der Veld, W. M., Saris, W. E., & Satorra, A. (2008). JRule 3.0: User's Guide.

<< two statistics

Jrule >>