| DESCRIPTION | FORECASTS | CASES | DATA | MANUSCRIPTS | CONTRIBUTORS |
A Note on the Expert Panel
The individuals on our legal expert panel are law professors or appellate practitioners who specialize in one or more areas of the Supreme Court's docket, and who will predict the outcomes of up to three cases in their area of specialty during the upcoming Term. Their voluntary participation is essential to the project design, and we are highly grateful for their assistance.
The primary purpose of this study is to compare the collective predictive accuracy of the entire group of human legal experts with the outputs of our statistical forecasting model -- we are less interested in internal comparisons among the legal experts. Therefore, although we will acknowledge all legal experts alphabetically at the end of the study, we will not reveal the names of the particular experts who predict particular cases. Moreover, because we continue to add experts in various subject matter specialties in response to new grants of certiorari (and will do so for a few more months), we will not publish the list of legal experts until after the entire panel is complete.
We can, however, make the following statements about the existing panel of legal experts in the study. As of October 14, 2002, the panel consists of 60 individuals; 52 of these are law professors and 8 are appellate attorneys. All panelists devote a significant portion of their scholarship and/or practice to Supreme Court cases and issues. 29 of the legal experts were law clerks to Supreme Court justices.
The "legal" method is more decentralized--three different legal experts predict the outcome of each of the Term's decisions. For each case, the three predictors will be drawn from a pool of 75-100 prominent legal scholars and appellate lawyers, each of whom is an expert in some area of the Supreme Court's docket and many of whom clerked at the Court. For each pending case, we will ask three people with the relevant subject matter expertise to predict the outcome. Because we are interested only in the collective accuracy of the legal experts, not in individualized comparisons between different legal experts, we will not reveal any individual panelist's vote(s) or the accuracy thereof. Click here [b] if you would like more information about the expert panel.
The individuals on our legal expert panel are law professors or appellate practitioners who specialize in one or more areas of the Supreme Court's docket, and who will predict the outcomes of up to three cases in their area of specialty during the upcoming Term. Their voluntary participation is essential to the project design, and we are highly grateful for their assistance.
The primary purpose of this study is to compare the collective predictive accuracy of the entire group of human legal experts with the outputs of our statistical forecasting model -- we are less interested in internal comparisons among the legal experts. Therefore, although we will acknowledge all legal experts alphabetically at the end of the study, we will not reveal the names of the particular experts who predict particular cases. Moreover, because we continue to add experts in various subject matter specialties in response to new grants of certiorari (and will do so for a few more months), we will not publish the list of legal experts until after the entire panel is complete.
We can, however, make the following statements about the existing panel of legal experts in the study. As of October 14, 2002, the panel consists of 60 individuals; 52 of these are law professors and 8 are appellate attorneys. All panelists devote a significant portion of their scholarship and/or practice to Supreme Court cases and issues. 29 of the legal experts were law clerks to Supreme Court justices.
A Note on the Statistical Forecasting Model
Our forecasting procedure proceeds by fitting classification trees (Breiman, et al. 1984) to the decisions of each justice using covariates that are observable before oral argument. We assume that the probability of a liberal ruling for a particular justice depends on case-specific quantities such as: the court of origin, the direction (liberal/conservative) of the lower court ruling, the type of petitioner, the type of respondent, the issue area of the case, and whether there was lower court disagreement, among other things. In addition, we assume that some justices decisions depend on the expected decisions of a subset of other justices.
Fitting the model to the training data produces a function which takes the case characteristics mentioned above as input and returns predicted liberal/conservative decisions for each of the nine justices. Putting the case characteristics of the 2002 cases into this forecasting function produces liberal/conservative forecasts for each justice for each case decided in the 2002 term. We convert these liberal/conservative forecasts to affirm/reverse forecasts by examining the direction of the lower court rulings, coded using the protocol of Spaeth (2002). A forecasted liberal decision by a U.S. Supreme Court Justice on a case that was decided in the conservative direction by the lower court is recoded as a forecasted reversal, and so forth.
We plan to write a brief note describing the technical aspects of the forecasting model in the near future.
References
Breiman L., Friedman J. H., Olshen R. A., and Stone, C. J. 1984. Classification and Regression Trees. Wadsworth.
Spaeth, Harold J. 2002. The Original United States Supreme Court Judicial Database, 1953-2001 Terms.
Our forecasting procedure proceeds by fitting classification trees (Breiman, et al. 1984) to the decisions of each justice using covariates that are observable before oral argument. We assume that the probability of a liberal ruling for a particular justice depends on case-specific quantities such as: the court of origin, the direction (liberal/conservative) of the lower court ruling, the type of petitioner, the type of respondent, the issue area of the case, and whether there was lower court disagreement, among other things. In addition, we assume that some justices decisions depend on the expected decisions of a subset of other justices.
Fitting the model to the training data produces a function which takes the case characteristics mentioned above as input and returns predicted liberal/conservative decisions for each of the nine justices. Putting the case characteristics of the 2002 cases into this forecasting function produces liberal/conservative forecasts for each justice for each case decided in the 2002 term. We convert these liberal/conservative forecasts to affirm/reverse forecasts by examining the direction of the lower court rulings, coded using the protocol of Spaeth (2002). A forecasted liberal decision by a U.S. Supreme Court Justice on a case that was decided in the conservative direction by the lower court is recoded as a forecasted reversal, and so forth.
We plan to write a brief note describing the technical aspects of the forecasting model in the near future.
References
Breiman L., Friedman J. H., Olshen R. A., and Stone, C. J. 1984. Classification and Regression Trees. Wadsworth.
Spaeth, Harold J. 2002. The Original United States Supreme Court Judicial Database, 1953-2001 Terms.
