Testing the Fixed Effects Restrictions? A Monte Carlo Study of Chamberlain’s Minimum Chi-Squared Test

Chamberlain (1982) showed that the fixed effects (FE) specification imposes testable restrictions on the coefficients from regressions of all leads and lags of dependent variables on all leads and lags of independent variables. Angrist and Newey (1991) suggested computing this test statistic as the degrees of freedom times the R2 from a regression of within residuals on all leads and lags of the exogenous variables. Despite the simplicity of these tests, they are not commonly used in practice. Instead, a Hausman (1978) test is used based on a contrast of the fixed and random effects specifications. We advocate the use of Chamberlain (1982) test if the researcher wants to settle on the FE specification and we check this test’s performance using Monte Carlo experiments and we apply it to the crime example of Cornwell and Trumbull (1994).


Abstract
showed that the fixed effects (FE) specification imposes testable restrictions on the coefficients from regressions of all leads and lags of dependent variables on all leads and lags of independent variables. Angrist and Newey (1991) suggested computing this test statistic as the degrees of freedom times the R2 from a regression of within residuals on all leads and lags of the exogenous variables. Despite the simplicity of these tests, they are not commonly used in practice. Instead, a Hausman (1978) test is used based on a contrast of the fixed and random effects specifications. We advocate the use of Chamberlain (1982) test if the researcher wants to settle on the FE specification and we check this test's performance using Monte Carlo experiments and we apply it to the crime example of Cornwell and Trumbull (1994).
1 Introduction Chamberlain (1982) showed that the fixed effects (FE) specification imposes testable restrictions on the coefficients from regressions of all leads and lags of the dependent variable on all leads and lags of the independent variables. Chamberlain suggested the estimation and testing of these restrictions using a minimum chi-squared (MCS) method with the test statistic being the minimand. Angrist and Newey (1991) demonstrated that this MCS method has 3SLS equivalents and that the resulting over-identification test statistic is equivalent to the MCS test statistic suggested by Chamberlain (1982). In addition, they showed that in the standard fixed effects model with remainder disturbances having a scalar identity covariance matrix, this MCS test statistic can be obtained as the sum of T terms. Each term of this sum is simply the degrees of freedom times the R 2 from a regression of within residuals for a particular period on all leads and lags of the independent variables. They applied this test to the union-wage effect using data from the NLSY random subsample of civilian men observed over the period 1983-1987. They failed to reject the fixed effects specification for the union-wage example. Next, they applied it to the estimation of the returns to schooling in a conventional human capital earnings equation. They found that the fixed effects estimates of the returns to schooling in the NLSY were roughly twice those of OLS. However, the over-identification test rejected the fixed effects restrictions.
Unfortunately, this careful testing of the FE restrictions has not been the usual practice in empirical work. In fact, the standard practice is to run a Hausman (1978) test. The latter statistic is based upon a contrast between the FE and random effects (RE) estimators. The RE estimator is an efficient estimator under the null hypothesis that the conditional mean of the disturbances given the regressors is zero, while the FE estimator is consistent under the null and alternative hypotheses. Not rejecting this null, the applied researcher reports the RE estimator. Otherwise, the researcher reports the FE estimator, see Owusu-Gyapong (1986) and Cardellichio (1990) for two such applications 1 . Rejecting the null of the Hausman test implies that the RE estimator is not consistent. This does not necessarily mean that the FE restrictions are satisfied. Therefore, a natural next step would be to test the FE restrictions before settling on this estimator as the preferred one.
The aim of this paper is to check the performance of Chamberlain's test for the FE restrictions. This is done via Monte Carlo experiments. In onedesign we let all regressors be correlated with the individual effects at every point in time, i.e., a Chamberlain reduced form (case 1). In another design, we let one regressor be correlated with the individual effects, whereas the other is not. Section 2 describes the model, the Chamberlain (1982) and the equivalent Angrist and Newey (1991) reformulation, and sets up the Monte Carlo design. Section 3 presents the Monte-Carlo results, while section 4 applies this test to the crime example of Cornwell and Trumbull (1994). Section 5 concludes.
The error terms u i,t and µ i are uncorrelated with x i,1 , ..., x i,T , and with each other by construction. Let y 0 i = (y i1 , ..., y iT ) and The restrictions between the reduced form and structural parameters are given by , and ι T is a vector of ones of dimension T . Chamberlain (1982) suggested estimation and testing be carried out using the minimum chi-square (MCS) method. Let b π be a consistent estimator of π, MCS estimates of β and λ are computed by minimizing . The minimand is a χ 2 goodness of fit statistic for the restrictions on the reduced form with 2T 2 − (2T + 2) degrees of freedom. Angrist and Newey (1991) showed that the minimand can be obtained as the sum of T terms. Each term of this sum is simply the degrees of freedom times the R 2 from a regression of the Within residuals for a particular period on all leads and lags of the independent variables.
Rather than 76 we get variances between 60 and 71 for T = 5, depending on the experiment performed. Also, rather than 436 we get variances between 281 and 358 for T = 11, depending on the experiment performed. The worst case is for T = 20, where the variance varies between 694 and 908 rather than 1516. A similar phenomenon was observed by Bowsher (2002) for the Sargan over-identification test in the context of dynamic panel data GMM estimation. Table 1 reports the frequency of rejections in 1000 replications for the MCS Chamberlain test and its associated Angrist-Newey version. Since the null hypothesis is always true, this represents the empirical size of the test. For T = 5, the size of the MCS and AN tests overstate the 5% level for ρ = 0.2 yielding 10.9 to 12.9% rejections. This improves as ρ increases to 0.5 yielding rejections between 7.2 to 9.3%. For ρ = 0.9, the corresponding rejections are between 3.3 to 4.8%. The results get better when T increases to 11. The size of the MCS and AN tests at the 5% level for ρ = 0.2 yield 6.6 to 8.9% rejections. For ρ = 0.5 the empirical size is between 4.7 to 6.1%. For ρ = 0.9, the corresponding size is understated varying between 2.3 to 3.2%. For T = 20, the empirical size of the MCS and AN tests understate the 5% level. The only exception is for ρ = 0.2 for the AN version of the test. This is not surprising given the understating of the true variance of the test statistic as T gets large. The last part of Table 1 reports the conflict between the MCS and AN tests in 1000 replications at various levels of significance. As clear from the table, there is at most 2.2 % conflict in the decision rendered by these two identical tests at the 5% level.
4 Empirical example Cornwell and Trumbull (1994), hereafter (CT), estimated an economic model of crime using panel data on 90 counties in North Carolina over the period 1981-1987. The empirical model relates the crime rate (which is an FBI index measuring the number of crimes divided by the county population) to a set of explanatory variables which include deterrent variables as well as variables measuring returns to legal opportunities. All variables are in logs except for the regional dummies. Here we focus on the explanatory variables that were significant in the fixed effects specification of CT. This was their prefered estimator. These variable include the probability of arrest P A (which is measured by the ratio of arrests to offences), the probability of conviction given arrest P C (which is measured by the ratio of convictions to arrests), the probability of a prison sentence given a conviction P P (measured by the proportion of total convictions resulting in prison sentences); the number of police per capita as a measure of the county's ability to detect crime (Police); the population density, which is the county population divided by county land area (Density); percent minority, which is the proportion of the county's population that is minority or non-white; regional dummies for western and central counties. Opportunities in the legal sector are captured by the average weekly wage in the county by industry. These industries are: transportation, utilities and communication (wtuc); manufacturing (wmfg). Table 2 shows the fixed effects and MCS estimates. Compared to the FE estimates of CT, removing the insignificant variables does not change the results much, neither in magnitude nor in significance. The MCS estimates are slightly different from the FE estimates, with Police for example having an estimate of 0.305 for MCS as compared 0.412 for FE. The Chamberlain MCS test for the restrictions imposed by (4) yield a χ 2 value of 415.7 which is significant. Hence, the null is rejected, and the FE assumption may be inappropriate for the crime example.

Conclusion
The random effects (RE) model in panel data is usually criticized for imposing restrictive conditions requiring the independence of the individual effects and the regressors. Rejection of the RE model by a Hausman (1978) test does not necessarily mean that the researcher should adopt a FE specification. Instead, we argue that one should run the Chamberlain (1982) test or its Angrist-Newey (1991) alternative to check that the restrictions imposed by a FE model are valid. Our Monte Carlo results show that these tests yield the same decision and are in conflict at most 2.2 % of the time. One caveat, is that like the Sargan overidentification test for dynamic panels, the MCS test tends to understate the true variance of the test statistic as T gets large. This is because as T gets large, the number of testable restrictions increase and the variance of the test statistic is understated. We illustrate the Chamberlain MCS test for the crime data of Cornwell and Trumbull (1994) finding that the FE restrictions are rejected by the data. When the Chamberalin test rejects the FE specification, and by default the reduced form implied by it, relating the individual effects to all the regressors at every point in time, we suggest careful examination of which regressors may or may not be correlated with the individual effects. In this case, one should be willing to entertain a more restricted model where only a subset of the regressors are correlated with the individual effects as proposed by Hausman and Taylor (1981). This would impose less restrictions than the general Chamberlain model and is also testable with a Hausman test. Alternatively, one could question the endogeneity of the regressors with the disturbances and not only with the individual effects. This endogeneity leads to inconsistency of the FE estimator and invalidates the Hausman test performed based on the fixed effects versus the random effects estimator, see Baltagi (2008). This requires an instrumental variable approach resulting in a fixed effects 2sls rather than a simple fixed effects estimator. The latter approach was actually considered in the application by Cornwell and Trumbull (1994). More generally, one could question the constancy of the parameters assumption over time which is underlying the panel data model and the fixed effects and random effects specification, see Crepon and Mairesse (2008) for an excellent survey as well as extensions of the Chamberlain model.