Legislación Internacional

Recipient reporting data quality is integral as part of the effort to bring transparency and accountability to the Recovery Act funds. While recipients we contacted during the first round of recipient reporting appeared to have made good faith efforts to ensure complete and accurate reporting, our fieldwork and analysis of first round data indicated that there were a range of significant reporting and quality issues that needed to be addressed. Collecting information from such a large and varied number of entities in a compressed time frame, as required by the Recovery Act will continue to be a huge task, and developing systems to check the data submitted under these circumstances takes on increased importance.

OMB’s updated December 2009 guidance outlined steps to address the issue of federal agency data quality checks on recipient reported data. The guidance lays out ways in which agencies can help recipients report better data. OMB reinforces that agencies, at a minimum, are to establish data quality plans that articulate their review process to focus on significant reporting errors and material omissions and ensure complete, accurate, and timely reporting of all amounts funded by the Recovery Act. In addition, agencies are now advised to provide recipients with key award information, such as recipient name and award amount. The agencies also are instructed to have recipients examine their reports for logical

inconsistencies, such as if a recipient indicates that the project is fully completed but the funds received are minimal compared to the award amount.

Federal agencies’ recipient reporting data quality review efforts continue to develop. In particular, from the federal agencies we reviewed, various processes were in place for outreach, identifying nonreporters, monitoring compliance, and identifying errors in reporting. The prime recipient report records include data on whether or not the federal agency reviewed the record during the data quality review time frames. In addition, the report includes a flag as to whether or not a correction was initiated. A

correction could be initiated by either the prime recipient or the reviewing agency. In our review of the prior quarter reports, we examined the

number and percentage of prime recipient records that were marked as having been reviewed by the federal agency. We repeated that analysis for the most current quarter—table 12 below shows the results.

Table 12: Fourth Quarter 2009 Prime Recipient Reports Reviews and Corrections Reviewed

by agency Correction Percentage

Number of prime recipient reports

No No 23 15,178

No Yes 11 7,464

Yes No 54 35,269

Yes Yes 12 7,916

Total 100 65,827

Source: GAO analysis of prime recipient reports for fourth quarter 2009 from Recovery.gov as of January 30, 2010.

Relative to the prior quarter, both the percent and number of prime reports marked as reviewed by agency were lower in the 2009 fourth quarter. In the prior quarter, 80 percent (45,825 records) of the prime recipient reports were marked as having been reviewed by the agency. As shown above in table 12, 66 percent (43,185 records) of the prime recipient reports were marked.

Education’s efforts to facilitate jobs reporting and data quality include coordinating with OMB, hosting conference calls, monitoring recipients, and conducting data quality reviews. Education provided guidance and technical assistance to states. As part of its guidance and technical assistance efforts, Education hosted three conference calls before and after the updated guidance was released to address reporting changes.

Department of Education

Education developed a quality review plan that included processes for identifying errors through various cross-checking mechanisms. According to Education officials, one example of such a mechanism is a series of comparisons of second round data to data reported in the first quarter by

using data elements such as number of jobs and job descriptions.

Similarly, Education compares various data elements in recipient reports, such as award number and CFDA numbers, with information in its grant management system to identify incorrect data elements that could make it appear that a recipient had not submitted a report. Before the federal agency review period began, officials ran daily reports and compared these reports to first quarter reports to check data, such as project status.

According to a program official, these reports were invaluable to the program offices because they made it easier for them to assist state officials in fixing errors before the actual corrections period.

To aid in identifying nonreporters, Education has implemented, through a monitoring questionnaire sent to state officials for Title I and SFSF, compliance measures, which in some circumstances also allow Education officials to identify errors before the reporting deadline. For example, Education officials we interviewed noted a situation in Illinois where LEA reporting was lower than they expected. Illinois officials also told us that they had observed that the number of LEAs shown in the system was too low, that this error was due to an incomplete upload of their report, and that they had subsequently resubmitted the report. Finally, officials told us that the department plans to do a reasonableness check on subrecipient reports from the second quarter across programs. As part of this check, officials will observe whether the number of reports for each grant appear to be consistent with the number of LEAs they expect to see and will contact state officials to discuss inconsistencies that arise.

Education officials said that the recipient reporting process has improved their knowledge of states’ cash management practices and that this information will help them to monitor states and offer them targeted technical assistance. For example, a department official told us that as a result of the Recovery Act, program officials have learned in more detail how funds flow from SEAs to LEAs in each state, and that this information has been very important in helping program staff interpret the expenditure numbers they see on the recipient reports in their monitoring efforts. For instance, a state’s reported expenditures might appear low because the state operates on a quarterly reimbursement basis and districts have been spending their own funds expecting reimbursement—without the

knowledge of the state’s cash management practices such numbers could be misinterpreted. This official told us that her specific program office will use this information to train program office staff in how to use this

information in their monitoring efforts. The official added that the

financial management information they have learned through the recipient reporting process adds another layer to the information gathered through

broader, systems-based questions they have asked during IDEA

monitoring visits over the last 5 years. Further, she believes that this more nuanced understanding of how state finances work will help department officials provide more tailored technical assistance to states.

In response to OMB’s guidance, HUD has engaged in efforts to facilitate timely and accurate jobs reporting. HUD’s efforts included outreach to recipients, developing processes for error detection, and identifying nonreporters. According to officials, HUD contacted recipients by e-mail with reminders to report and provided key information that should be included in certain data fields. In addition, HUD provided technical assistance to walk unfamiliar recipients through the reporting process.

Department of Housing and Urban Development

HUD also developed data quality review processes with cross-checking mechanisms to detect errors. For example, the Recovery Act Management and Performance System (RAMPS) compared data in four key fields from FederalReporting.gov against parameters HUD established to identify potential significant errors (see table 13).¹⁰⁰ RAMPS also flagged duplicate entries, awards entered incorrectly as contracts, and reports entered with invalid award ID numbers.

Table 13: Parameters HUD Established to Identify Significant Errors for the Capital Fund

Data field Parameter for determining significant error

Award amount Compared directly to Office of the Chief Financial Officer (OCFO) obligation, with a 0% variance allowed.^a

Total received Compared directly to OCFO disbursement, with a +/-10% variance allowed.

Total spent May not exceed OCFO obligation.

Jobs Related to OCFO obligation. The number of jobs may not exceed a +50% variance from the following:

OCFO obligation/$205K.

Source: HUD officials.

aFor the first round of recipient reports, this parameter allowed a +/-10 percent variance.

100HUD’s data quality process proposal defines a significant error in the fields award amount, total received, total spent, and jobs as any entry that varies from a HUD

comparison source value taken from corresponding HUD Office of Chief Financial Officer award records for obligations and disbursements.

Moreover, for the second cycle HUD officials told us they further

incorporated suggestions from OMB’s guidance to agencies to improve the data quality review process. The department identified some material omissions by examining fields with narrative responses to determine whether they contained a minimum number of characters as an indicator of whether the responses adequately addressed requirements for those fields. HUD officials said that the most prevalent errors in reported data were in the number of jobs field. They said that confusion persists among public housing agency officials regarding how to calculate and count jobs, which is reflected in the number of potential errors identified by the data quality review process. HUD officials said they first followed up on the most egregious errors—large overcounting or undercounting—that remained uncorrected after their initial review of the data before addressing other errors. Table 14 identifies the number of potential significant errors identified by HUD in each field for Public Housing Capital Fund formula grants and competitive grants for each quarterly reporting cycle.

Table 14: Number of Potential Significant Errors Identified by Field for the October 2009 and January 2010 Reporting Cycles

Data field

Number of potential errors, October reporting cycle

Number of potential errors, January reporting

cycle

Award amount 55 28

Amount received 639 730

Total spent 5 1

Jobs 1,437 1,118 Total 2,136 1,877

Source: GAO analysis of HUD data.

Note: A single report can have multiple significant errors identified.

HUD’s data quality process proposal defines material omissions as any grantee that failed to report at all. HUD developed a Recovery Act Non-Reporting Enforcement Plan in September 2009, which states that all recipients who fail to report in FederalReporting.gov by the end of their first applicable reporting cycle will receive a warning letter from HUD program staff. If recipients fail to report a second time, HUD will initiate further enforcement actions which can include formal or informal hearings, suspension of access to funds, or other actions. In the first recipient reporting cycle, HUD identified 152 Capital Fund formula and competitive grants for which a report was required but none was found,

according to HUD officials. Officials told us HUD provided technical assistance to housing agencies on its nonreporter list, including walking them systematically through the reporting process. HUD officials told us they also sent reminder e-mails on December 4, 2009 to the 152

nonreporting housing agencies reminding them to report in the second reporting period and warning them that HUD may take additional action if they failed to report in the next reporting period. In the second reporting cycle, HUD identified 27 grants for which a report was required but none was found, including 6 grants for which no report was found for either cycle. HUD officials told us they followed up by phone with all housing agencies for which no report was found for either cycle.

HUD officials questioned the accuracy and validity of the data submitted through recipient reports and said they were not yet comfortable relying on these data because of the level of confusion expressed by housing agency officials regarding jobs reporting and the large number of potential errors HUD identified in the “number of jobs” field. In addition, they said HUD already has most of the information collected through the recipient reports—jobs information and project status were two exceptions—and uses other systems for that information because HUD officials believe the data are more reliable than the recipient-reported data. Although HUD has received feedback from housing agencies that are confused or frustrated over the reporting process, it has not received substantive feedback from external stakeholders, such as OMB, the HUD IG, interest groups, or the media on recipient data for the Capital Fund grant program.

For the first submission of recipient reports, we noted that public housing agencies experienced problems with the process of recipient reporting and the FTE calculation. These problems appeared to continue in round two.

For example, officials at one Mississippi housing authority reported the same 6 jobs in round two that they reported in round one. We determined their calculations did not conform to the OMB guidance of December 18, and they did not recall receiving guidance from HUD or OMB on job calculations for round two. However, HUD officials told us HUD sent several e-mails to each grantee between December 23 and the end of the reporting period that included a link to HUD’s explanation of the job count and the change in job count guidance from OMB.

Officials from a Mississippi and a Pennsylvania housing authority reported using a jobs calculator produced by HUD to calculate the number of jobs they reported in the second round. HUD had posted a jobs calculator for the first round of recipient reporting, which was designed to calculate jobs cumulatively across reporting periods, but removed it prior to the second

round of reporting after OMB changed the guidance for calculating jobs.

According to a HUD official, their jobs calculator for the second round—

which reflected the changes in OMB guidance for calculating jobs—was posted for about 1 week in December 2009 before OMB requested that it be taken down in order to review it further. After reviewing the HUD guidance to housing agency officials, it appears that HUD did not take steps to instruct housing agencies not to use the jobs calculator from the first round. As a result, housing agencies may have incorrectly used the jobs calculator from the first round to calculate the number of jobs for the second round.

We recommend that the Secretary of Housing and Urban Development instruct housing agencies to discontinue use of the jobs calculator provided by HUD in the first round of recipient reporting for subsequent rounds of reporting to ensure the correct job calculation is used.

Recommendation for Executive Action

DOT officials noted that the combination of information sharing among the operating administrations (OA),¹⁰¹ prior knowledge of reporting by the recipients, and the dissemination of training and guidance helped the department and recipients comply with reporting in the second

submission of reports. DOT’s OAs engaged in several efforts of outreach to assist in accurate recipient reporting. The OAs conducted Webinars and meetings, sent e-mails, and made telephone calls to brief and train their program recipients. According to officials, they wanted to ensure that, along with keeping recipients updated on requirement changes, first-time recipients would be successful in reporting. In addition, during the first reporting period, DOT learned that the OAs, working together as a group, allowed sharing of information and mediation of reporting challenges.

Department of Transportation

To identify the nonreporters, the OAs compared the list of recipients with reports found in FederalReporting.gov. As a result, they identified 64 recipients that failed to report during the first reporting period. The OAs worked one-on-one with these recipients and received assurances from these nonreporters that they would comply with future reporting requirements.

101The department’s operating administrations overseeing the implementation of the Recovery Act include the Federal Aviation Administration, Federal Highway

Administration, Federal Railroad Administration, Federal Transit Administration, and Maritime Administration.

According to officials, DOT and its OAs took numerous steps to address the quality of the recipient reported data and identify errors. To address reporting, DOT modified contract terms and conditions to include

recipient reporting requirements and warnings that were sent to recipients reminding them that reporting data is a condition of the Recovery Act funding. Further, officials reported that each OA has developed a written process to conduct data quality reviews. To understand the reasons for identified inaccuracies in reported data, DOT relied on direct contact and follow-up with its recipients. When asked how DOT planned to use the recipient reported data, officials said they did not have plans to use the information from recipient reports in future decision making. Instead, they planned to rely on other department data collection methods.

DOT took steps to promote consistency between OMB definitions and the unique nature of FHWA programs. However, guidance provided by FHWA raised concerns with some state and local officials we interviewed. OMB encouraged federal agencies to provide supplementary reporting guidance to their recipients. To improve consistency in the guidance recipients receive, OMB requires agencies to clear any supplemental guidance through OMB. FHWA approached OMB to clarify the number recipients should report for total federal Recovery Act funds received. To meet the statutory requirement of reporting at the project or activity level, OMB and FHWA determined that recipients should report the amount awarded and the amount received as the same regardless of expenditures, since recipients report expenditures in a subsequent field.

Some state officials stated that the supplementary guidance from FHWA raised definitional and transparency issues. We also found that the states we reviewed differed in the application of the guidance. In some cases, states did not report the same number in the amount awarded and amount received column. In one case, the state response was that funding values for all amounts were taken from the state accounting system and were consistent across all of the state recipient reports. A few other states made the changes to conform to the guidance, but expressed concerns about the definition of this field and the transparency of the reporting. FHWA and OMB officials are aware that some state officials have raised concerns with this guidance and are working with state officials to try to resolve this issue. However, as OMB clearly states in its guidance, although federal agencies are able to comment and suggest changes to the reports, prime recipients are ultimately responsible for the data reported into

FederalReporting.gov. We will continue to monitor this matter.

DOE made several outreach efforts to their program recipients to ensure timely reporting. These efforts included e-mail reminders for registration and Webinars that provided guidance on reporting requirements. For the first round of reporting, DOE developed a quality assurance plan to ensure all prime recipients filed quarterly reports, while assisting in identifying errors in reports. The methodology for the quality assurance review included several phases and provided details on the role and

responsibilities for DOE officials.

Department of Energy

According to DOE officials, the data quality assurance plan was also designed to emphasize the avoidance of material omissions and significant reporting errors. More specifically, as reported by the OIG and DOE officials, the plan outlined a qualitative comparison of recipient data obtained from FederalReporting.gov to agency data obtained from the

According to DOE officials, the data quality assurance plan was also designed to emphasize the avoidance of material omissions and significant reporting errors. More specifically, as reported by the OIG and DOE officials, the plan outlined a qualitative comparison of recipient data obtained from FederalReporting.gov to agency data obtained from the