To address the study objectives, we conducted two rounds of site visits to each of five states.9
The states were selected because they had implemented a large number of modernization initiatives—including initiatives spanning most or all of the four categories of modernization activities—and because their efforts were well established in terms of years and dispersion across a state.
Within each state, we selected a mix of locations to visit to see how modernization efforts were implemented in different types of locales—urban and rural. During the site visits, we interviewed staff from all levels of state SNAP agencies, community partners that were involved in outreach and application assistance, and other program stakeholders. We also conducted four focus groups in each state with SNAP participants. Finally, we collected extant data from each of the study states, such as administrative case records, application statistics, performance data, and other relevant materials about state modernization efforts.
1. Site Selection
The five study states were intentionally selected based on state demographic, geographic, and modernization-specific characteristics. First, we developed an index that ranked all of the states and the District of Columbia based on the number and type of modernization changes they had implemented. Then, using the natural break in the distribution of states along this index, we identified 15 states that had undertaken numerous modernization steps. For these states, we conducted a second-stage analysis to identify states that had implemented modernization changes broadly across the state, for a long period, and with minimal negative consequences. This process pointed to nine states as being the most likely to help us achieve the project’s objectives, and together with FNS we identified the final group of five10 for inclusion in the study.
The final states selected through this process were Florida, Georgia, Massachusetts, Utah, and Washington (Table I.2). These states represent four FNS geographic regions (Southeast, Northeast, Mountain Plains, and Western) and include a mixture of state populations, geographic sizes, and SNAP participation levels, along with some variation in the number and maturity of initiatives they implemented. However, because we intentionally selected states with several modernization initiatives in place, the variation among study states is not as great as the variation nationwide. Table I.2. Characteristics of Case Study States at Time of Selection (2009)
Florida Georgia Massachusetts Utah Washington
Initiatives Implemented 12 8 9 8 6
Start of Modernization Efforts 2005 2007 2006 2005 2000
Breadth of Efforts Statewide Statewide Statewide Statewide Statewide
FNS Region Southeast Southeast Northeast Mountain
Plains Western
2009 Population (thousands) 18,500 9,800 6,600 2,800 6,800
2009 SNAP participants
(thousands) 1,950 1,290 628 185 761
Notes: Changes to states’ status since 2009, although reflected in the chapters later in this report, do not appear in this table.
In Washington, some initiatives were statewide, whereas others were only regional at the time of 2009 site selection.
Within each state, we selected individual sites to participate in the study. In each state, the state capital was chosen as one of the sites. Other sites were selected to ensure a mix of urban and rural localities and multiple regions within each state, as well as diversity in implementation (for example, pilot sites and early and late adopters). To identify these sites, we worked with officials in each state. Mathematica suggested a number of locations based on our knowledge of the implementation of modernization activities across the state, as well as logistical considerations related to visiting multiple locations in a large state during only two rounds of site visits. We also took into account the suggestions of state agency staff, as well as the opinions of other knowledgeable sources, such as
10 We also originally selected Wisconsin, a county-administered state, to participate in the study. However,
community partners. In total, we visited five locations in Florida and Massachusetts, six locations in Georgia and Utah, and seven locations in Washington (Table I.3).
Table I.3. Locations Visited in Case Study States
Site Visit Location (Round Visited) Primary Rationale for Selection Florida
Site Visits: October 2011 and January 2012
Tallahassee (1) State capital
Jacksonville (1) Call center location; proximity to state capital
Ft. Myers (2) Interview partner demonstration project
Immokalee (2) Rural; migrant population; regional diversity
Miami (2) Largest urban area in the state; call center location
Georgia
Site Visits: November 2011 and February 2012 Atlanta (Fulton County) (1) State capital
McDonough (Henry County) (1) Regional diversity; proximity to other locations Albany (Dougherty County) (1) Call center location
Savannah (Chatham County) (2) Second urban location; prominent community partner Brunswick (Glynn County) (2) Headquarters for region that includes Savannah Reidsville (Tattnall County) (2) Rural; regional diversity; proximity to Savannah
Massachusetts
Site Visits: November 2011 and February 2012
Boston (1) State capital; call center location; centralized online application processing unit location
Fitchburg (1) Intake/ongoing split localized initiative
Springfield (2) Pilot location for intake/ongoing split; centralized online application processing unit
Holyoke (2) Intake/ongoing split localized initiative; rural; large Hispanic population
Northampton (2) Rural; community partner location
Utah
Site Visits: October 2011 and February 2012 Salt Lake City (1) State capital; call center location
Ogden (1) Call center location; diverse local population
Midvale (1) Centralized Imaging Operations unit location
St. George (2) Call center location; largest city in southern Utah
Cedar City (2) Serves rural population
Kanab (2) Serves rural population
Washington
Site Visits: October 2011 and January 2012
Olympia (1) State capital
Seattle (1) Largest urban area in the state
Yakima (1) Region staff; call center; document imaging hub
Wapato (1) Rural; central Washington; unique partnerships with local tribe (Yakama Nation)
Colfax (2) Unique client population (college students)
Moses Lake (2) Rural; eastern Washington, community partner location
Spokane (2) Urban; eastern Washington; region staff
Note: In Georgia, where local offices are named by counties, we list both the city of the site visit and the office name for ease of reference. The intake/ongoing split for assigning work in Massachusetts is described in Chapter IV.
2. Data Collection
In each state, we collected primary data through interviews and focus groups. We also collected extant data. Both data sources were central to the analysis presented in this report.
Primary data were collected during two rounds of site visits in each state. During each visit, we conducted interviews with SNAP staff at all levels of the agency, community partners, and other stakeholders. We also conducted four focus groups in each state, two with clients and two with potential clients (eligible nonparticipants), offering facilitation in Spanish when needed. Finally, we took tours, watched demonstrations, and conducted observations of SNAP operations. Appendix A contains a detailed description of our data collection approach. The total number of interviews and focus groups ranged from 33 in Massachusetts and Utah to 42 in Florida, and the total number of respondents ranged from 73 in Massachusetts to 149 in Florida.
We collected extant data from the study states and other relevant sources. These data help document the modernization changes made in each state, identify any trends associated with the implementation of modernization initiatives, and examine whether the modernization changes are potentially driving changes in key program outcomes. Extant data collected during the course of the study include the following:
• Monthly administrative case records
• Monthly application statistics, including data on application mode when available
• SNAP quality control (QC) data
• Annual administrative cost data
• Details on waivers implemented by the state
When possible, although availability varied by state, we also gathered performance data about specific initiatives and other descriptive documents, such as agency communications, reports, and fact sheets. Data shown in tables and figures represent all available data, through the end of 2011. Dates of data availability vary by source and by state. Appendix A describes the sources and uses of these extant data in greater detail.
3. Limitations of Research Design
The study was designed to collect comprehensive information from the five participating states to provide a detailed picture of their experiences with modernization, identify key lessons learned, and assess the potential impacts of modernization. The study was descriptive in nature and focused on a specific time period in a specific subset of states. Importantly, the study does not include a cost-benefit analysis. Although we provide descriptive information on costs and benefits, we do not evaluate here whether the benefits outweigh the costs, because balancing these aspects depends on how policymakers and other stakeholders give relative weight to aspects such as access and client satisfaction. Readers should consider four important limitations when interpreting and extrapolating the study findings.
First, our understanding about the implementation of modernization efforts in each state over the past decade is based primarily on interviews with state and local SNAP and partner staff. Some staff who were most directly involved in the modernization initiatives are no longer with the state or the community partners. Other staff could have been in different positions at the time when specific initiatives were implemented, which limited their ability to speak authoritatively on all issues. And because many years might have passed since initiatives were implemented, the information we collected during interviews is also subject to recall error. Nonetheless, the staff we spoke with were the most knowledgeable about current and past modernization initiatives and therefore the study is based on the best available information.
The second limitation is that the study design lacks the causal validity to conclude that (1) any changes in outcomes were caused by the modernization changes or (2) outcomes that have no change were unaffected by the modernization changes. However, we do identify potentially positive and negative aspects of modernization in each of the study states, and the analysis of potential impacts can prove informative when trying to understand the role that modernization potentially plays in outreach, participation, payment accuracy, and program costs.
Third, modernization is a complex and continually evolving process, which makes assessing the impacts of specific initiatives quite complicated, if not impossible. Specific initiatives are often rolled out across offices through pilots, over time, and with small, but important differences as the state learns what has worked well and what still has to be improved. This fluidity makes it difficult to pinpoint exactly when an initiative first began or when it was fully implemented. Changes can also be purposefully implemented as a package or are simply implemented at the same time with other initiatives.
Finally, the study lacks representation of county-administered states. Twenty percent of states administer SNAP at the county level (USDA, FNS 2011). In those states, some counties have instituted their own modernization initiatives. As such, we did not collect specific information regarding the challenges faced by states and counties when implementing initiatives at the county level, nor about the opportunities/advantages counties might have in terms of implementing initiatives. Potential differences could be fewer barriers to implementation, fewer stakeholders, the presence of very active local-partners, and so on. Nonetheless, the information provided in the report remains instructive to all levels of SNAP administration and highlights many of the same challenges and opportunities presented to counties in county-administered states.