Procesamiento de la información

This section of the report provides recommendations regarding implementation of outcome-based efficiency measure for the 11 ETA programs that are the focus of this study. These recommendations are based on analysis of available data, as well as input from members of the Expert Panel, interviews with state workforce agencies, ETA program administrators/staff, and the ETA internal work group.92

1. Recommendation #1: Use Program Expenditures Rather than Appropriations or Obligations as the Measure of Program Costs in Efficiency Measures

To generate an efficiency or unit cost measure, it is necessary to account for costs in the numerator of the measure and a unit of participation, service delivery, or outcome in the

denominator of the measure. Based on discussions with ETA officials (and a review of the literature and available data sources within ETA), three potential types of “cost” data could be considered for the numerator of each efficiency measure:

• Appropriations/Allotments – “Appropriations” are defined by the Government

Accountability Office as “Authority given to federal agencies to incur obligations and to

make payments from Treasury for specified purposes.”93 Appropriations are generally

the amount of funding made available by Congress for spending on a given program (such as the TAA program) during a fiscal year. Allotments are the amount of

92 _{The recommendations in this report differ to some extent to the recommendations in our earlier reports. The} changes are due to additional data analyses and suggestions from ETA staff and the Expert Panel.

93 _{See U.S. Government Accountability Office (2004). Principles of Federal Appropriations Law, Third Edition,} Volume 1. Washington, DC: U.S. Government Accountability Office Report GAO-04-261, SP, p. 2-5.

appropriated funds distributed to a state or grantee based upon a legislative or regulatory formula.94 Allotments exclude amounts retained by the federal government to administer programs.

• Obligations – According to the General Accounting Office (GAO), “…obligations

reflect orders placed, contracts awarded, and other similar transactions during a fiscal year. As an expression of an agency’s total financial commitments for a given period, gross obligations portray the relative size of an organization, without regard to the type of underlying budgetary resource or when resulting outlays may occur.”95 Hence,

obligations are funds that have been committed through contracts, grants, and other vehicles.

• Expenditures – Expenditures are funds paid or the amount of funds due (depending on whether a cash or accrual basis is used) for provision of goods or services pursuant to a grant or contract agreement.

Use of expenditures rather than appropriations, allocations, or obligations in calculating

efficiency measures is recommended because (1) expenditures can vary substantially from what is initially appropriated/allocated, especially at the state level (because of transfers, rescissions, and unexpended funds); and (2) expenditures reflect what is actually spent on delivery of services and capture the underlying notion of efficiency. States interviewed, ETA program offices, and the Expert Panel endorsed the use of expenditures over the other available measures of costs.

2. Recommendation #2: Use Common Measures as Starting Point for Measuring Program Outcomes in Efficiency Measures

If they were to be implemented in the short run (i.e., within three years), efficiency measures should be closely tied to the current outcome performance measures in effect under ETA’s Common Measures framework. Data is already being collected at the state and grantee

94_{Although the term “allocation” is used quite often in an interchangeable manner with allotment, in the context of} formula programs the appropriate term is allotment.

95_{United States General Accounting Office (GAO), Federal Budget: Agency Obligations by Budget Function and} Object Classification for Fiscal Year 2003, GAO-04-834, June 2004 (available at

levels on these outcomes, so the performance data needed to generate efficiency measure results would already be available (for most programs) – reducing costs and start-up time. In addition, though the report highlights some of the challenges of comparing efficiency measure results across programs and there is considerable variability across programs in terms of data quality and comparability, the Common Measures provide common definitions for outcome measures and thus increase the potential for making meaningful comparisons of efficiency measure results within individual programs (e.g., across states/subgrantees) and across at least some of the ETA programs of interest. As discussed in more detail in the report, the following are key features of the current Common Measures performance measurement system across the 11 ETA programs of interest:

• Ten (of the 11) ETA programs currently use the Common Measures: Workforce Investment Act (WIA) Adult, WIA Youth, WIA Dislocate Worker, WIA National Emergency Grants (NEG), Wagner-Peyser/Employment Service, Trade Assistance Act (TAA), Senior Community Service Employment Program (SCSEP), Indian and Native American (INA), National Farmworker Jobs Program (NFJP), and Work Incentive Grant (WIG) Programs. The Apprenticeship program (the 11th program), which has in the past measured program performance in terms of retention and earnings gains for apprentices, is in the process of shifting to tracking and reporting on the three main Common Measure performance outcomes.

• For Common Measures programs (except WIA Youth), outcome measures are entered employment, retained employment, and post-program earnings.96 • For WIA Youth, the Common Measures are placement in education or training,

credential attainment, and literacy/numeracy.

It is essential that the outcome-based efficiency measures adopted should be sensitive to goals and objectives of each of the 11 programs. There are three potential efficiency measures the 11 ETA program could implement over the short run based on the Common Measure outcomes:

• Cost per entered employment (for the WIA Youth program - cost per placement in employment or education);

• Cost per retained employment; and • Cost divided by post-program earnings.

In addition, although not currently used, cost divided by change in post-program earnings could be used, as this data is being collected by most of the programs currently (as a result of the Common Measures post-program earnings measure) and was formerly a measure that was used by the WIA Adult and Dislocated Worker programs. Because the proposed efficiency measures are based on cost and outcome data already being submitted to ETA (or soon to be reported in the case of the Apprenticeship program) through annual expenditure reports and Common

Measures reporting, states and grantees will not have to develop new data systems or submit new cost or outcome data. We recommend that ETA initially focus on monitoring results nationally and at the state/grantee levels on these four outcome-based efficiency measures for several years for most of the ETA programs, as described below. See Exhibit 7-1 for overview of programs for which the four recommended efficiency measures highlighted below should be considered:

• Recommendation 2a: Cost per entered employment should be tracked (for monitoring

purposes initially) as an efficiency measure for 8 of the 11 ETA programs: WIA Adult, WIA Dislocated Worker, WIA NEG, Wagner-Peyser, TAA, SCSEP, INA, and NFJP. The WIA Youth program could use cost per placement in employment or education (as an alternative to cost per entered employment). The WIG program should be excluded from implementing this measure because it does not enroll participants; the

Apprenticeship program should be excluded because federal funding is primarily used to promote and monitor apprenticeship programs (i.e., other sources of funding primarily pay for training-related costs) and other factors discussed earlier in this report and summarized in the next recommendation (Recommendation #3).97 The main

rationale for recommending cost per entered employment (and using cost per placement in education or training for the WIA Youth program) as an efficiency measure is as follows: (1) employment is a high priority for all programs except WIA younger youth; (2) entered employment is the simplest and most direct way to assess whether programs are achieving their goals; (3) data are already being collected on the number of entered employments under the Common Measures

97 _{In addition, the WIG program’s funding is scheduled to be eliminated in the coming program year, and so, by the} time the measure was implemented the program will have ended (see below, Recommendation #3 for additional discussion of WIG and potential suitable efficiency measures).

EXHIBIT 7-1: OVERVIEW OF RECOMMENDED OUTCOME-BASED