Jrule, a Procedure and Program to detect Misspecifications
We suggest switching from goodness-of-fit testing, based on the CHI2 test and/or Fit indices, to searching for possible misspecifications in the model, using the MI, the EPC and the power of the MI test. The approach we propose distinguishes the following four possible situations shown in the Table below, which results from combining the significance or not of the MI test and the high/low power of the MI test: The decisions to be made in the different situations defined on size of the modification index (MI) and the power of the test.
High power |
Low power |
|
Significant MI |
Inspect EPC (EPC) |
Misspecification present (m) |
Non-significant MI |
No misspecification (nm) |
Inconclusive (I) |
hen MI is significant and the power of the MI test is low, we conclude that there is a misspecification because the test is not very sensitive (low power) and nevertheless a significant value of the MI has been obtained. This is the cell in the Table labelled “m” for misspecification.
Using a reversed argument, the decision is also simple if the MI is not significant and the power of the MI is high. In that case, the conclusion is that there is no misspecification, so the corresponding cell is labelled “nm”.
The situation is more complex if the MI is significant but the power of the MI test is high. In that case it may be a serious misspecification, but it may also be that the MI is significant due to a high sensitivity of the test for this misspecification. Therefore, in that situation, we suggest looking at the substantive relevance of the EPC: If the EPC is rather small, one concludes that there is no serious misspecification. This makes sense because, generally, we do not want to adjust our model for a standardized coefficient of .001 even though this coefficient is significant. However, when the EPC is large, for example larger than the critical value for this type of parameter, it is concluded that there is a relevant misspecification in the model
The fourth and last situation is that MI is not significant and the power of the MI test is also low. In that case it should be concluded that the test lacks sufficient information to make a decision. This case is labeled as inconclusive “I”.
Because this procedure requires computations on the basis of the numerical results obtained from the estimation of the model, programs have been developed to be applied on the output of different programs to provide for each parameter the decision with respect to its status in terms of the four options mentioned in the table above.
For more information about this approach and an application on data analyzed with the LISREL program we refer to the paper of Saris, Satorra and Van der Veld (2009).
For further illustration of this approach using the program Jrule (Van der Veld, Saris and Satorra. (2008)
Nowadays there are several other versions of the original program available, see below:
The program JRule for LISREL
The manuel JRule
The program JRule for R - The function is called 'miPowerFit'
Using a reversed argument, the decision is also simple if the MI is not significant and the power of the MI is high. In that case, the conclusion is that there is no misspecification, so the corresponding cell is labelled “nm”.
The situation is more complex if the MI is significant but the power of the MI test is high. In that case it may be a serious misspecification, but it may also be that the MI is significant due to a high sensitivity of the test for this misspecification. Therefore, in that situation, we suggest looking at the substantive relevance of the EPC: If the EPC is rather small, one concludes that there is no serious misspecification. This makes sense because, generally, we do not want to adjust our model for a standardized coefficient of .001 even though this coefficient is significant. However, when the EPC is large, for example larger than the critical value for this type of parameter, it is concluded that there is a relevant misspecification in the model
The fourth and last situation is that MI is not significant and the power of the MI test is also low. In that case it should be concluded that the test lacks sufficient information to make a decision. This case is labeled as inconclusive “I”.
Because this procedure requires computations on the basis of the numerical results obtained from the estimation of the model, programs have been developed to be applied on the output of different programs to provide for each parameter the decision with respect to its status in terms of the four options mentioned in the table above.
For more information about this approach and an application on data analyzed with the LISREL program we refer to the paper of Saris, Satorra and Van der Veld (2009).
For further illustration of this approach using the program Jrule (Van der Veld, Saris and Satorra. (2008)
Nowadays there are several other versions of the original program available, see below:
The program JRule for LISREL
The manuel JRule
The program JRule for R - The function is called 'miPowerFit'
