We assessed whether district-level remand rates were associated with several types of variables:
• District-level organization of disability appeal adjudication. In the majority of districts, magistrate judges decide most or all social security cases. We wondered
whether districts’ organization and deployment of judicial resources were associated with case outcomes. If a magistrate judge decides a steady diet of social security cases to the exclusion of a more varied caseload, perhaps her decision-making tendencies more closely resemble an ALJ’s. We addressed these issues by creating variables indicating whether districts assign cases mostly or exclusively to magistrates, as well as to a mix of both types of judges.
• District-level caseload pressures. We measured the importance of caseload pressures with two variables. The first is the share of all civil cases in the district that are disability appeals. This variable measures the relative importance of the disability part of the docket. Perhaps as the percentage of a district’s civil docket devoted to social security cases grows, its remand rate falls. This would happen if judges get impatient with disability litigation and just want to clear cases off their desks.476 The second was the
number of pending cases—whether civil or criminal—per congressionally approved Article III judgeship. This variable measures the overall degree of docket pressure in the district. Since the substantial evidence standard of review makes affirmances relatively easy decisions to reach, we wondered whether rising docket pressures are associated with lower remand rates.
• Judicial skepticism of the federal government. Professors Krent and Morris used the party affiliation of the President who nominated judges as a proxy for judges’ political leanings. They determined that ideology, so measured, does not explain variation in remand outcomes.477 One district judge suggested to us, however, that judicial
skepticism of the federal government, whether ideologically inflected or not, might affect decision patterns. Following her suggestion, we obtained data on the share of criminal sentences in a district in which judges sentenced criminal defendants to more lenient terms than given by federal sentencing guidelines. We included both a variable
measuring a district’s downward departure frequency when the U.S. Attorney sponsored the downward departure, as well as a variable measuring the frequency with which
difference deals means that what Gelbach (2012) calls “settlement selection” is not a concern here, which might make causal interpretations more plausible.
476 For empirical evidence concerning such an effect in a very different litigation context, see Eric Helland &
Jonathan Klick, The Effect of Judicial Expedience on Attorney Fees in Class Actions, 36 J.LEG.STUD.171 (2007).
477 Krent & Morris, supra note 1 at 385.
downward departures occurred in the absence of government support. We included both these variables on the theory that the government might support downward departures more often in districts where judges are more skeptical of the government as a way of channeling and thus reducing judicial rejection of the government’s position.
• District-level variables constructed in an effort to measure the performance of
hearing offices within the district. We used data from ODAR that identifies the hearing
office in which newly filed court cases were initially adjudicated to create composite variables measuring the ALJ award rate from hearing offices where claims are initially adjudicated for that district, as well as the number of dispositions per day per ALJ in each hearing office.478
o We compiled information on ALJ awards and denials from data that are publicly available on the agency’s website. From this information, we estimated the hearing office-level award rate for claims filed in the hearing offices that feed into each district court.479 We speculated that higher initial award rates might be
associated with lower remand rates at the district court level.
o We also compiled information concerning the number of dispositions per day per ALJ in each hearing office.480 Because the award rate is held constant via
inclusion in the regression model of the hearing office-level award rate discussed just above, dispositions per ALJ per day can be understood as a measure of labor productivity.481 Including this measure allows us to determine whether more
productive ALJs are associated with either higher or lower remand rates.
• Labor market conditions in the years preceding a disability appeal. To the extent that some workers treat disability insurance as a substitute for unemployment or a job search,
478 Ideally, we would use lagged values of our award rate and ALJ productivity variables in our analysis in order to
properly match the timing of hearing office performance with subsequent district court appeals. However, data on these variables were available on the agency’s website only for 2010 and later years. See
www.ssa.gov/appeals/DataSets/archive/archive_data_reports.html#&ht=6. Because this means we do not have the relevant information to properly measure these variables, we used the 2010 value of each variable as a proxy for the variable’s long run value in each district court.
479 See section 4(c) of the Data Appendix for the approach we took to creating judicial district-level information
concerning the variables that are measured at the hearing office level.
480 Unfortunately, data we used for ALJ dispositions per day per ALJ in hearing offices, posted by the agency at
https://www.ssa.gov/appeals/DataSets/archive/04_FY2010/04_September_Disposition_Per_Day_Per_ALJ_Ranking _FYTD2010.xml, do not include information for national hearing centers. Consequently, our calculation of variables determined at the hearing office level does not account for any differences across the national hearing centers and local hearing offices. However, the ODAR data we were provided indicates that national hearing centers accounted for less than 5% of cases in the filing years 2010 through 2013, so we do not believe this issue is likely to be very important.
481 To be sure, this understanding may be appropriate only if ALJs have the same ratio of what are known as Type I
and Type II error rates. A Type I error is a denial of a claim that in which benefits should be awarded, whereas a Type II error is an award in response to a claim that should be denied. Consider two ALJs who must adjudicate cases with the same average characteristics. If they have the same award rate and ratio of Type I and Type II errors, then on average the two ALJs are equally accurate. If one of them adjudicates more cases than the other in a typical day, then the quicker one adjudicates more cases in a given period of time, with the same average accuracy. Thus it is reasonable to describe the quicker ALJ as more productive under those conditions.
variables measuring labor market tightness might play a role in explaining initial claiming and subsequent appeal behavior. We thus included the value of the state employment to population ratio—the ratio of estimated state employment to estimated state population—in the state where each district was located. We included the
employment to population ratio for the year before the year in which the case was filed in the district court, as well as for two, three, and four years before. Our thinking here was that the condition of the labor market in the several years before a disability appeal was filed in the district court should correspond roughly to its condition when claimants first decide whether to claim, then decide whether to pursue appeals within the administrative review system, and finally decide whether to appeal to federal court.
• Salaries of lawyers. Perhaps employment as an ALJ is particularly attractive in districts where lawyers generally earn lower salaries. If so, ALJs in low-salary areas might be particularly able lawyers. These ALJs would generate better decisions that would eventually result in lower remand rates.482
• Urbanization. Finally, we included measures of the degree of urbanization of the counties that make up each district. One interview subject suggested that federal judges in more rural districts probably remand fewer cases.483
The data we used for the regression analysis below ultimately included information on eighty-three districts. Collecting data on all the variables described above was not always simple, and in some instances we had to make do with rough proxies. The Data Appendix includes more details.484