Discusión de resultados

This assessment will aim to understand the context in which the Program operates by:

• identifying the legislative goals (missions and associated objectives) of the Program and determining the Program’s ability to meet these goals;

• identifying how states implement the Program relative to the logic model developed for the Program (see Section 1.1.2);

• determining how well DOE manages and administers the Program;

• determining how well the Program and the weatherization network is delivering its services to the low-income weatherization market;

• identifying the leveraging and partnership opportunities the Program is exploiting (i.e., identifying the degree to which states and agencies coordinate the implementation of the Program with other federal, state, utility, and other programs) and determining if the Program’s regulations are enhancing and/or inhibiting leveraging and partnership opportunities; and

• determining the role the Program plays in the larger low-income energy assistance effort.

3.1.1 Data and Surveys

Information on process improvement will be collected from all states as part of the “All States PY 2007 Survey” (see Appendix E) and from the 400 agencies involved in the billing data

energy savings portion of the impact assessment (see Section 2.2.1) as part of the “Subset of Agencies PY 2007 Survey” (see Appendix H). Both surveys will also ascertain their views of how DOE manages and administers the Program and will collect detailed input on how the states and agencies implement house audits, client education, training, and monitoring as previously described in Section 2.1.1. The state survey will also collect information that define state

“implementation models.”. These surveys will be implemented after each state and agency completes PY 2006 (August to October 2007).

DOE will be surveyed using the “DOE PY 2007 Survey” (see Appendix C) to collect

information on staffing, costs, and their enforcement of state and local agency data collection, storage, and data mining capabilities. This survey will be given when PY 2006 is complete for all states (October 2007).

Open-ended interviews will be held with DOE and a subset of states and agencies over the phone or by one or more group meetings by April 2008 to solicit needed process information for this study. The sampling plan and survey instruments associated with this data collection will need to be developed; however, the survey instrument should include the following information:

• major strengths/positive traits at each implementation level (DOE, state, and agency),

• major weaknesses at each level,

• major barriers to effective operation posed by each level and outside partners,

• adequacy of current resources,

• suggested reallocation of current resources,

• suggested allocation of additional resources,

• coordination,

• communications, and

• Program administration.

Occupants from 940 housing units weatherized in PY 2007 from the 400 agencies included in the billing data sample will be surveyed using a portion of the “Occupant Survey” (see Appendix I) to determine their perceptions on how well Program services were delivered (see Appendix M for a detailed justification for this sample size). This portion of the “Occupant Survey” will be implemented immediately after these housing units are weatherized in PY 2007, with an incentive provided to the occupants to maximize the response rate. These surveys will be conducted over the telephone and will collect information such as the timeliness of the work performed (e.g., audit, measure installation, inspection), if agency staff showed up at scheduled times, the condition in which workers left the house following their work (e.g., cleanliness, debris removal), and the courtesy of the agency personnel. A “Program Services Agency Survey” (see Appendix J) will be administered at the same time as the “Occupant Survey” to agency staff and crews that worked on these houses to collect equivalent information on their perceptions of how well they delivered Program services to these houses. It is expected that the agencies will be compensated by DOE for the time it takes their staff and crews to complete this survey.

3.1.2 Analysis

The mission and objectives of the Program and DOE’s management structure and responsibilities will be described. This will include a summary of:

• the Program’s legal authorities and regulatory framework;

• the goals, objectives, and key measures of performance as viewed by Congress, the Department, and the Administration; and

• Federal, state, and local responsibilities as described by regulation and by network participants.

In addition, particular attention will be paid to the treatment of leveraging under the Program’s rules and regulations.

An analysis will be performed of the management structure, responsibilities, and resources for each of the Program’s management levels (i.e., headquarters, regions, state, and local agencies).

This analysis will focus on the operation at each management level, the allocation of human and funding resources to various functions, and how each management level perceives the adequacy of those resources for each function within and between management levels. A secondary focus of this analysis will be on the perceived barriers to effective operations posed by each

management level on other management levels and by each management level with outside partners.

Organizational activities relating to how well Program services were delivered will be identified, and a measurable indicator for each activity will be developed (e.g., if audit was performed on time, if measures were installed when scheduled). Information collected from all or a subset of agencies on all or a subset of the houses they weatherized will be used to determine average values for these indicators.

Approximately five typical models of how states administer and implement the Program will be developed based on information collected from the state survey and from reviewing state

weatherization plans submitted to DOE. How well these models work, and how well they fit the theory and logic of the Program, will be discussed using the logic model developed for the Program (see Section 1.1.2) as a guide. Lessons learned from the various approaches, model dependencies, and key issues and administrative concerns affecting model effectiveness should be identified.

A literature review of evaluations conducted on utility low-income weatherization programs will be performed so that the issue of systematic differences in savings per dollar spent between utility programs and the DOE Program, if any, can be discussed. Reasons for any identified differences will be discussed, including the Program’s case management approach to addressing customers’ needs versus the utilities’ emphasis on just installing measures (that are cost

effective), and the Program’s requirements to address health and safety issues.

The occupant’s perceptions on how well Program services were delivered will be analyzed and compared to the perceptions provided by the agency staff and crews.

The role of the Program in the overall low-income energy assistance effort will be assessed by drawing together information from the impact assessment on the Program characteristics (see Section 2.1) with information collected and analyzed in this study. Program characteristics that will be used include information on the national low-income population, the segment of the market currently being served by the Program, and the characterization of the local and state agencies and the scope of their programs.