Part 1 |
Save page Remove page | Previous | 1 of 2 | Next |
|
|
small (250x250 max)
medium (500x500 max)
Large
Extra Large
Full Size
Full Resolution
|
This page
All
|
■• *J fyyt*:S«*/*l*l \ United States TI«k »<^. I v% ***. ^^ an_n_ ^^ ^wMfe *Jk % =r9 The income and Eligibility Verification System (IEVS) Targeting Demonstration Agriculture Food and Consumer Sorvlce Office of Analysis and Evaluation $. ?ueteo The Cost Effectiveness of the cQt* Income and Eligibility Verification System in Arizona and Michigan Final Report Volume I \ Contract No : FCS 53-3198-8-95, Task Order 7 MPR Reference No.: 8029-600 TOE COST EFFECTIVENESS OF THE INCOME AND ELIGIBILITY VERIFICATION SYSTEM IN ARIZONA AND MICHIGAN FINAL REPORT VOLUME I April 1,1995 Authors: Nancy Fasciano Sheena McConnell Submitted to: U.S. Department of Agriculture Food and Consumer Service Office of Analysis and Evaluation 3101 Park Center Drive, 2nd Floor Alexandria, VA 22303 Submitted by: Mathematica Policy Research, Inc. 600 Maryland Avenue, S.W. Suite 550 Washington, DC 20024 Attention: Sharron Cristofar Project Director: Harold Beebout / BLANK PAGE // ACKNOWLEDGMENTS The authors would like to thank the many people who have helped with this project. At the U.S. Department of Agriculture, Food and Consumer Service, Sharron Cristofar, the Project Officer, provided valuable direction and advice for the study. The project also benefitted from suggestions from Abigail Nichols, Joe Pinto, Carolyn Foley, Ed Speshok, and Cecilia Fitzgerald of the Food and Consumer Service. John Bedwell of the Food and Consumer Service provided data on established and recovered claims. Staff at the U.S. Department of Health and Human Services provided data on the AFDC program and staff at the Social Security Administration provided information on computer matching. Our thanks go to Tom O'Conned, Joe Lonergan, and Norma Griffin. Staff at the Arizona Division of Family Support and Michigan Department of Social Services designed the new targeting strategies, implemented the demonstration, and provided valuable assistance throughout the study. The study would not have been successful without the dedication of these staff. In Arizona, special thanks go to Vince Wood, Liz Steele, Ben Dillon, Cindy Walker, Pam Esirella, and Aldona Vaitkus. We would also like to thank other staff members who provided us with data for our savings and costs measurements: Guy Wilson, Jeffrey Bowman, Michael Doyle, John Haines, Neil Young, Mary Werne, Carlos Verdugo, Ben Wilmar, Susan Olsen, and Julie Rioux. In Michigan, special thanks go to Sue Hall, Gary Miller, and Phil Michel. We would also like to thank Dick Hall, John Kelley, William Minihan, Dan Hinds, Steve Smucker, Dick O'Herron, Linda Peterson, Larry Matecki, Irma Guzman, Deb Kristopherson, and Sandra Zwemer. We would also like to extend our gratitude to the caseworkers in both states who completed the data collection forms and the local office coordinators who monitored the data collection process. Myles Maxfield at Mathematica Podcy Research, Inc. reviewed an earlier draft of the report and provided valuable suggestions. The study has also benefitted from input from Susan Allin, Ed Hoke, John Mamer, and Alberto Martini. Daisy Ewed and Royston McNeid provided expert research assistance. Marianne Stevenson and Parti Rossi input the data from the data collection forms. Daryl Hall edited the report. Report production support was provided by Bob Skinner, Sharon Clark, and Ann Miles. iii BLANK PAGE W CONTENTS Chapter Page ACKNOWLEDGMENTS iii GLOSSARY OF ACRONYMS xv EXECUTIVE SUMMARY xvii I INTRODUCTION 1 A AN OVERVIEW OF THE IEVS PROCESS 3 1. Matching 5 2. Targeting 8 3. Follow Up 9 B. PERCEPTIONS OF THE IEVS PROCESS 10 C. PRIOR COST-EFFECTIVENESS STUDIES OF IEVS 12 D. AN OVERVIEW OF THE IEVS TARGETING DEMONSTRATION AND EVALUATION 13 E. AN OVERVIEW OF THE REPORT 15 II THE ENVIRONMENT. ARIZONA AND MICHIGAN FOOD-STAMP AGENCIES 17 A CHARACTERISTICS OF THE FOOD-STAMP AGENCIES IN ARIZONA AND MICHIGAN 17 B. IEVS PROCEDURES IN ARIZONA AND MICHIGAN 20 1. Arizona 20 2. Michigan 24 C MATCHES DURING THE DEMONSTRATION 31 III EVALUATION DESIGN 33 A THE GENERAL APPROACH 33 CONTENTS (continued) Chapter m (continued) IV P-ge B. THE RESEARCH SAMPLE 36 1. The Research Sample in Arizona and Michigan 37 2. Demonstration Offices 38 3. Sample Sizes 40 C. THE NEW ffiVS MATCHING AND TARGETING PROCEDURES 40 1. Arizona 41 2. Michigan 49 D. DATA COLLECTION 52 1. Data Collection Forms 52 2. Monthly Case-Record Extracts 54 3. State Agencies' Accounting Records 54 4. Reports Submitted by State Agencies to Federal Agencies 55 MEASURES OF SAVINGS AND COSTS 57 A. SAVINGS RESULTING FROM THE IEVS PROCESS 58 1. Avoided Benefit Payments 59 2. Avoided Administrative Costs 66 3. Recovered Previous Benefit Overpayments 68 4. Unmeasured Savings • 73 B. COSTS INCURRED DURING THE IEVS PROCESS 73 1. Caseworker Follow-Up Costs 75 2. Claims Establishment and Collection Costs 77 3. Data Processing Costs 82 4. Development Costs 88 ACTION, HIT, AND MATCH RATES 91 A. ACTION RATES 92 vi CONTENTS (continued) Chapter Page V B. REASONS GIVEN FOR FOLLOW UPS THAT DO (continued) NOT LEAD TO A CHANGE IN BENEFITS 96 1. Income Was Already Recorded in the Casefile 98 2. Income Did Not Affect Benefits or Eligibility 102 3. Case Was Inactive 107 4. Caseworker Could Not Verify Income on the External Database 108 5. Income on the External D. abase Was Incorrect 109 6. Report Did Not Provide Income Information 109 C CHARACTERISTICS OF CASES AND CLIENTS FOLLOWED UP AND THOSE THAT WERE ACTED UPON AS A RESULT OF IEVS 110 1. Characteristics of Cases and Clients in the Research Sample and Those Targeted for Follow Up 115 2. Characteristics of Cases and Clients Acted Upon As a Result of IEVS 116 D. MATCH AND HIT RATES 119 VI SAVINGS RESULTING FROM IEVS 127 A. TYPES OF ACTIONS 127 B. SAVINGS FROM CASE CLOSURES, BENEFIT DENIALS, AND BENEFIT CHANGES 132 1. Savings Per Month , 132 2. Total Savings from Case Closures, Benefit Denials, and Benefit Changes 139 C. SAVINGS FROM THE DETECTION OF PREVIOUS BENEFIT OVERPAYMENTS 145 1. Detected Benefit Overpayments 145 2. Recovered Benefit Overpayments 149 D. UNMEASURED SAVINGS 150 1. Savings from Actions in Other Programs 151 2. Qualitative Evidence on Other Potential Savings 154 vii CONTENTS (continued) Chapter VI (continued) vn vm Page E SUMMARY OF SAVINGS FROM IEVS 155 COSTS INCURRED BY THE IEVS PROCESS 161 A COST OF CASEWORKERS' FOLLOW-UP 161 1. Tasks Performed by Caseworkers During Follow Up 162 2. Time Taken to Conduct a Follow Up 166 3. Estimates of Follow-Up Costs 171 B. CLAIMS ESTABLISHMENT AND COLLECTION COSTS 173 C. DATA PROCESSING COSTS 178 D. COSTS OF DEVELOPING THE MATCHING AND TARGETING STRATEGIES 182 E. TOTAL COSTS INCURRED BY THE IEVS PROCESS 183 THE COST-EFFECTIVENESS OF IEVS 189 A ESTIMATES OF THE COST-EFFECTIVENESS OF IEVS 189 1. The Savings-to-Cost Ratios 192 2. Net Savings 195 3. Total Savings 1% B. SENSmVITY OF THE COST-EFFECTIVENESS ESTIMATES TO ASSUMPTIONS 197 1. Length of Time Savings Persist 197 2. Recovery of Previous Benefit Overpayments and Costs of Claims Establishment and Collection 202 3. Hourly Cost of the Caseworkers' Time 204 4. Discounted Future Savings and Costs 204 5. Assumptions Used by Ward and Smucker (1990) 205 6. Assumptions Used by Puma (1989) 206 7. Include Only Savings to the FSP 207 vm UQ CONTENTS (continued) Chapter Page Vni C. LIMITATIONS OF OUR STUDY 207 (continued) 1. We Included Only the Savings and Costs That Accrued to the Federal and State Agencies 207 2. The Study Was Conducted in Only Two States and in Only Some Offices 208 3. The Agency Staff in Both Arizona and Michigan Knew They Were Participating in the Study 209 4. We Did Not Learn of the Outcome of Some Follow Ups of Research-Sample Cases 209 5. The Cost-Effectiveness of IEVS Matches May Be Greater When the Match is First Introduced 210 6. The Number of Follow Ups Was Small 210 7. We Cannot Determine the Precision of Our Estimates 210 D. TARGETING STRATEGIES 211 1. Applicant Targeting 212 2. Suggested Targeting Strategies 214 E- SUMMARY OF FINDINGS AND CONCLUSIONS 215 REFERENCES 217 BLANK PAGE TABLES iabie p«r JJ.1 CHARACTERISTICS OF FOOD STAMP HOUSEHOLDS IN MICHIGAN, ARIZONA, AND U.S. AS A WHOLE, 1991 19 D.2 SUMMARY OF IEVS PROCEDURES IN ARIZONA 21 D.3 SUMMARY OF IEVS PROCEDURES IN MICHIGAN 25 m.1 OVERVIEW OF EVALUATION DESIGN 34 ffl.2 PROJECT OFFICES IN THE DEMONSTRATION 39 ffl.3 SIZE OF RESEARCH SAMPLE 40 m.4 PREDEMONSTRATION AND DEMONSTRATION TARGETING STRATEGIES IN ARIZONA 42 ffl.5 PREDEMONSTRATION AND DEMONSTRATION TARGETING STRA1EGJES IN MICHIGAN 50 IV.l ESTIMATES OF THE LENGTH OF TIME HOUSEHOLDS WOULD HAVE REMAINED ON FOOD STAMPS IN THE ABSENCE OF IEVS 65 IV.2 ADMINISTRATIVE COST SAVINGS 67 IV.3 COSTS OF CLAIMS ESTABLISHMENT AND COLLECTION: ARIZONA 79 IV.4 COSTS OF CLAIMS ESTABLISHMENT AND COLLECTION: MICHIGAN 80 JV.5 DATA PROCESSING UNIT COSTS: ARIZONA 85 IV.6 DATA PROCESSING UNIT COSTS: MICHIGAN 86 V.I NUMBER OF HITS, ACTIONS, AND ACTION RATES: ARIZONA 93 V.2 NUMBER OF HITS, ACTIONS, AND ACTION RATES: MICHIGAN 94 VJ REASONS GIVEN FOR NO CHANGE IN BENEFITS OR ELIGIBILITY: ARIZONA 99 TABLES (continued) Table Page V.4 REASONS GIVEN FOR NO CHANGE IN BENEFITS OR ELIGIBILITY: MICHIGAN 100 V.5 CHARACTERISTICS OF CASES AND CLIENTS THAT ARE FOLLOWED UP AND THOSE ACTED UPON AS A RESULT OF IEVS: ARIZONA Ill V.6 CHARACTERISTICS OF CASES AND CLIENTS THAT ARE FOLLOWED UP AND THOSE ACTED UPON AS A RESULT OF IEVS: MICHIGAN 113 V.7 MATCH AND HIT RATES: ARIZONA 121 V8 MATCH AND HIT RATES: MICHIGAN 122 VI.1 TYPES OF ACTIONS RESULTING FROM IEVS FOLLOW UPS: ARIZONA 129 VL2 TYPES OF ACTIONS RESULTING FROM IEVS FOLLOW UPS: MICHIGAN 130 VI.3 SAVINGS PER MONTH FROM CASE CLOSURES AND BENEFIT REDUCTIONS: ARIZONA 133 VI.4 SAVINGS PER MONTH FROM CASE CLOSURES AND BENEFIT REDUCTIONS: MICHIGAN , 136 VI.5 TOTAL SAVINGS FROM CASE CLOSURES OR BEN ^FTT REDUCTIONS: ARIZONA 141 VI.6 TOTAL SAVINGS FROM CASE CLOSURES AND BENEFIT REDUCTIONS: MICHIGAN 142 VI.7 PREVIOUS BENEFIT OVERPAYMENTS DETECTED BY IEVS FOLLOW UPS: ARIZONA 146 VL8 PREVIOUS BENEFIT OVERPAYMENTS DETECTED BY IEVS FOLLOW UPS: MICHIGAN 147 VI.9 ESTIMATES OF SAVINGS FROM MEDICAID: ARIZONA 152 VI.10 ESTIMATES OF SAVINGS FROM MEDICAID: MICHIGAN 153 VI.11 TOTAL SAVINGS FROM IEVS MATCHES: ARIZONA 156 xu TABLES (continued) Table P«fe VI.12 TOTAL SAVINGS FROM ffiVS MATCHES: MICHIGAN 158 Vn.l TASKS INVOLVED IN FOLLOW UPS IN ARIZONA: BREAKDOWN BY WHETHER ACTION OCCURRED 164 VD.2 TASKS INVOLVED IN FOLLOW UPS IN MICHIGAN: BREAKDOWN BY WHETHER ACTION OCCURRED 165 VH.3 TASKS INVOLVED IN FOLLOW UPS IN ARIZONA: BREAKDOWN BY DATABASE 169 Vn.4 TASKS INVOLVED IN FOLLOW UPS IN MICHIGAN: BREAKDOWN BY DATABASE 170 VH.5 COST OF CASEWORKERS' FOLLOW UPS: ARIZONA 172 VU.6 COST OF CASEWORKERS' FOLLOW UPS: MICHIGAN 174 VE.7 COSTS OF CLAIMS ESTABLISHMENT AND COLLECTION: ARIZONA 176 VE.8 COSTS OF CLAIMS ESTABLISHMENT AND COLLECTION: MICHIGAN 177 VD.9 DATA PROCESSING COSTS: ARIZONA 179 Vn.10 DATA PROCESSING COSTS: MICHIGAN 180 VD.11 COSTS OF DEVELOPING IEVS MATCHES: ARIZONA 183 VD.12 TOTAL COSTS OF IEVS MATCHES: ARIZONA 184 VD.13 TOTAL COSTS OF IEVS MATCHES: MICHIGAN 185 Vm.l SAVINGS AND COSTS FROM IEVS: ARIZONA 190 VID.2 SAVINGS AND COSTS FROM IEVS: MICHIGAN 191 Vm.3 ESTIMATES OF SAVINGS-TO-COST RATIOS UNDER DIFFERENT ASSUMPTIONS: ARIZONA 198 Vm.4 ESTIMATES OF SAVINGS-TO-COST RATIOS UNDER DIFFERENT ASSUMPTIONS: MICHIGAN 199 xui TABLES (continued) Table Page Vm.5 MINIMUM NUMBER OF MONTHS THAT SAVINGS MUST PERSIST FOR MATCH TO BE COST-EFFECTIVE: ARIZONA AND MICHIGAN 201 xiv GLOSSARY OF ACRONYMS ADP Automated Data Processing (category of mainframe-computing costs) AFDC Aid to Families with Dependent Children AHCCCS Arizona Health Care Cost Containment System (department in Arizona that administers Medicaid) ARS Automated Recoupment System (computer system in Michigan that calculates recoupments) BEER Beneficiary Earnings Exchange Reports (annual earnings data) BENDEX Beneficiary Data Exchange (Title II benefits data) CIS Client Information System (client database in Michigan) DES Department of Economic Security, Michigan DSP Designated Staff Person (a caseworker in Michigan who specializes in dealing with overpayments) DSS Department of Social Services, Michigan FAA Family Assistance Administration, Department of Economic Security, Arizona FCS Food and Consumer Service, U.S. Department of Agriculture FSP Food Stamp Program GAO General Accounting Office HHS U.S. Department of Health and Human Services IEVS Income and Eligibility Verification System IRS Internal Revenue Service (also refers to the annual unearned income data maintained by the Internal Revenue Service) MESC Michigan Economic Security Commission (agency that maintains state earnings and Unemployment Insurance data) MPR Mathematica Policy Research, Inc. OARC Office of Accounts Receivable and Collections (office in Arizona's Department of Economic Security that processes claims) xv OIG Office of Inspector General (office in Michigan's Department of Social Services that investigates fraud) SDX State Data Exchange (data on Supplemental Security Income) SIPP Survey of Income and Program Participation SSA Social Security Administration SSI Supplemental Security Income SSN Social Security Number SWICA State Wage Information Collection Agency (collects quarterly state earnings data and Unemployment Insurance data, also refers to the quarterly earnings data collected by the agency) TPQY Third-Party Query (a type of request for information from the Social Security Administration) UI Unemployment Insurance (also refers to the Unemployment Insurance data maintained by the state) USDA U.S. Department of Agriculture xvi EXECUTIVESUMMARY The Income and Eligibility Verification System (IEVS) was established to verify income information reported by welfare program applicants and recipients. Misreported income can lead to errors in eligibility and benefit determination which can divert resources away from the truly needy and weaken public support for the programs. Minimizing such errors is therefore important. In 1986, the Food Stamp Program regulations were amended to require states to implement IEVS. The IEVS regulations require state welfare agencies to compare income reported by applicants and recipients of food stamps. Aid to Families with Dependent Children (AFDC), and Medicaid with income reported on six external income databases. For most IEVS matches, the state agencies create computer tapes listing welfare applicants and recipients, which are then matched to the external databases. If a match occurs-information on the client is available from the external database-the caseworker conducts follow-up procedures to investigate whether income has been misreported. These procedures may include reviewing the client's case, contacting the client, verifying information on the external database, recomputing eligibility and benefits, investigating fraud, and recovering benefits paid in error. The six IEVS external databases are: 1. State Wage Information Collection Agency (SWICA) database, which provides data on quarterly earnings reported by employers to the state 2. Unemployment Insurance (UI) database, which provides monthly information on UI receipt 3. Beneficiary Data Exchange (BENDEX) database, which provides monthly information on receipt of Social Security and other Title II benefits 4. Beneficiary Earnings Exchange Reports (BEER) database, which provides annual earnings information 5. State Data Exchange (SDX) database, which contains monthly information on receipt of Supplemental Security Income (SSI) 6. Internal Revenue Service (IRS) database which contains annual information on unearned income BACKGROUND TO THE STUDY After implementing IEVS, some state food-stamp agencies expressed concern that the IEVS regulations were inflexible and burdensome. While caseworkers followed up many matches with the external database, only a small proportion of follow ups detected errors in benefits or eligibility. As these follow ups can be time-consuming, caseworkers perceived that IEVS used a large amount of resources in relation to the savings it generated and was therefore not cost-effective. xvii In response to these concerns, interim regulations were published in 1988 permitting states to follow up only a subset of recipient matches. The process of selecting a subset of matches to follow up is known as targeting. The regulations prohibit targeting of applicants. Despite these regulatory changes, some state agencies argue that, even with targeting, matching with some databases is not cost-effective. The agencies' concerns with IEVS are largely related to the external data: some are out-of-date, some are aggregated over too long a period, some duplicate other IEVS data, and some require third-party verification. Suggested changes to the IEVS regulations include allowing states to target applicants in addition to recipients and to conduct only matches they consider cost-effective. To address the concerns of the state agencies, the Food and Consumer Service of the U.S. Department of Agriculture contracted with Mathematica Policy Research, Inc. to estimate the cost-effectiveness of IEVS matches in two demonstration states, Arizona and Michigan. The study estimates the cost-effectiveness of conducting IEVS matches using a targeting strategy compared to the situation in which the match is not conducted at all. All but one of the IEVS matches in the demonstration used a targeting strategy. THE IEVS DEMONSTRATION The IEVS demonstration took place in 7 local food-stamp offices in Arizona and 16 local food-stamp offices in Michigan between July and October 1992. The research sample included only food-stamp recipients in Arizona and food-stamp applicants in Michigan. (Some of the applicants in Michigan began to receive benefits during our study.) Prior to the demonstration, Arizona did not match recipients with the SWICA database or follow up any match with the BEER and IRS databases. This was because agency staff believed that these matches were not cost-effective. During the demonstration, Arizona reinstated the SWICA, BEER, and IRS matches and used a new targeting strategy for each match. In Arizona, we estimated the cost-effectiveness of the SWICA, BEER, and IRS recipient matches. Prior to the demonstration, Michigan followed up information from all matches. Staff in Michigan were concerned that the SWICA and IRS applicant matches were not cost-effective. During the demonstration, Michigan introduced a new targeting strategy for the IRS match and continued to conduct the SWICA applicant match with no targeting. In Michigan, we estimated the cost-effectiveness of the SWICA, UI, BENDEX, SDX, and IRS applicant matches. All except the SWICA match were targeted during the demonstration. SAVINGS AND COST MEASURES Cost-effectiveness is measured as the ratio of program savings from IEVS to the cost of matching, targeting, and follow up under IEVS. We measure the cost-effectiveness of IEVS from the perspective of the state and federal agencies that administer the Food Stamp and AFDC programs. Hence, we do not include savings or costs to the clients, employers and financial institutions that are required to verify income, or the agencies that maintain the external databases. xvin The savings from IEVS fall into four categories: 1. AvoidedBenefitPayments. Benefits may be denied or reduced on the basis of follow-up information obtained through the IEVS process. 2. Avoided Administrative Costs. If an applicant is denied benefits or a case is closed because of the IEVS process, the agency will avoid the cost of administering that case. 3. Recovered Previous Benefit Overpayments. An IEVS follow up may result in the determination that a client has received incorrect benefits. The savings to the agency is the portion of the overpayment that is actually recovered from the client. 4. Unmeasured Savings. Savings from IEVS other than those discussed above may be important but to quantify them were beyond the scope of this study. The most obvious of these is savings to other programs, such as Medicaid. IEVS may also deter clients from misreporting income and improve caseworker morale. The costs of IEVS fall into four categories: 1. Caseworker Follow-Up Costs. These involve primarily the cost of caseworkers' time in following up IEVS matches. They also include the cost of some supervisor and clerical staff time, materials and supplies, and overhead. 2. Costs of Claims Establishment and Collection. These include the costs of investigating fraud, establishing and collecting claims, and conducting hearings and prosecutions. 3. Data Processing Costs. These include payments to the agencies that maintain the external database, as well as the mainframe computer costs incurred from producing request tapes or matching extracts from the external databases against the caseload; processing response tapes and running targeting algorithms; and producing reports of the matches to be followed up. 4. Development Costs. These are the costs involved in developing and implementing the matching and targeting strategies. As they are one-time-only costs, they are not included in our measure of the cost-effectiveness of IEVS. We were required to make many assumptions in measuring these savings and costs,. Whenever a range of equally reasonable options was presented, we selected the one that led to the highest estimate of costs and the lowest estimate of savings. The estimates of the savings-to-cost ratios presented in this report are therefore low estimates of the cost-effectiveness of IEVS. xix ACTION, HIT, AND MATCH RATES The cost-effectiveness of a match depends on the action rate, the proportion of all follow ups that lead to a change in benefits, a change in eligibility, or the detection of a previous benefit overpayment. The central criticism of IEVS is that caseworkers conduct many follow ups that do not detect misreported income. Our findings support this criticism. The action rates during the demonstrations were low in both states: 12 percent in Arizona and 6 percent in Michigan. In both states, the action rate varied by database, from 7 percent for the SWICA match to 16 percent for the IRS match in Arizona, and from 4 percent for the UI match to 13 percent for the IRS match in Michigan. The IEVS regulations require the state agencies to report both the match and hit rates for each database. The match rate is the number of social security numbers (SSNs) on which information is available from the external database as a proportion of all SSNs that could potentially be matched. The hit rate is the number of SSNs that are targeted for follow up as a proportion of all SSNs for which information is available from the external database. Few states actually do report these rates (Allin 1991). The IEVS demonstration revealed that it is difficult to calculate these rates because (1) the components of the match and hit rates are measured in different units (records, SSNs, and cases), and (2) it may not be possible to observe the number of SSNs that are matched because, for example, the matching and targeting steps are combined. We were able to estimate the match and hit rates only for the IRS database in Arizona and for the SWICA, UI, and IRS databases in Michigan. Both the match and the hit rates were low. The match rates varied from 8 percent for the IRS match in Arizona to 44 percent for the UI applicant match in Michigan. The hit rate was "<nly 2 percent for the IRS match in Michigan and exceeded 20 percent only for the SWICA applicant match in Michigan, which was not targeted. Thus, the targeting strategies used in the demonstration excluded from follow up many clients on whom information was found on the external database. SAVINGS FROM IEVS When a follow up led to a change in benefits, a change in eligibility, or the detection of a previous overpayment, the resulting savings were large. For every follow up that led to an action, an average of over $1,000 was saved in Arizona and an average of over $900 was saved in Michigan. Avoided benefit payments accounted for the largest portion of these savings in both states (52 percent in Arizona and 90 percent in Michigan). Recovered previous benefit overpayments accounted for 44 percent of all savings in Arizona, but only for 4 percent in Michigan. Overpayments were more important in Arizona because a recipient case is more likely to have had previous benefit overpayments than is a case applying for benefits. Avoided administrative costs accounted for only a small portion of savings in both states (4 percent in Arizona and 6 percent in Michigan). Savings were larger if the follow up led to a change in benefits or eligibility for the AFDC program. When an error was detected in a joint food stamps/AFDC case, both avoided benefit payments and recovered benefit overpayments were, on average, higher for the AFDC program than for food stamps. xx The average savings over all follow ups (including those not resulting in an action) was $122 in Arizona and $54 in Michigan. The average savings per follow up varied considerably by database. In Arizona, average savings per follow up were about $62 for the SWICA match, $123 for the BEER nutch, and $146 for the IRS match. In Michigan, average savings per follow up were $18 for the SDX match, $63 for the SWICA match, $81 for the UI match, $86 for the BENDEX match, and $1,129 for the IRS match. COSTS INCURRED BY IEVS A follow up cost an average of $40 in Arizona and $16 in Michigan. Caseworker follow ups accounted for the largest portion of these costs: 80 percent in Arizona and 68 percent in Michigan. Costs incurred in establishing and collecting claims were also sizable, accounting for 18 percent of costs in Arizona and 31 percent of costs in Michigan. Data processing costs were small, accounting for less than 2 percent of costs in both states. Caseworkers took an average of 50 minutes in Arizona and 13 minutes in Michigan to conduct a follow up. In Arizona, the time to conduct a follow up varied little by database. However, in Michigan, it took an average of only 10 minutes to conduct a follow up of the SDX match and 19 minutes to conduct a follow up of the SWICA and IRS matches. The SWICA, BEER, and IRS matches require more follow-up time because caseworkers must obtain third-party verification of external income data and compare monthly income reported by the client in a previous quarter or year with quarterly or annual data on the external database. The UI, BENDEX, and SDX databases do not require third-party verification and contain current, monthly income information. In Arizona, the average cost per follow up of IEVS varied little by database: $40 for the SWICA database, $39 for the BEER database, and $42 for the IRS database. In contrast, the average cost per follow up in Michigan varied from $12 for the BENDEX database to $106 for the IRS database. The average cost per follow up for the other databases in Michigan was $13 for UI, $14 for SDX, and $22 for SWICA. The costs of developing the three IEVS matches and targeting strategies in Arizona were about $100,000. In Michigan, we could not estimate the cost of developing the matches as they were developed prior to our study. Staff in Michigan reported that few resources were used to implement the new IRS targeting strategy. COST-EFFECTIVENESS OF IEVS All matches in the demonstration were found to be cost-effective. For each match, the savings from the IEVS process exceeded the costs incurred by the process. Table 1 presents the savings-to-cost ratios and the net savings (savings minus costs) per research-sample case for each IEVS match. The most cost-effective match in the demonstration was the IRS match in Michigan, with nearly 11 dollars saved for every dollar spent on the match. The least cost-effective match was the SDX match in Michigan with $1.24 saved for every dollar spent. Our findings suggest that the net savings that can be realized from IEV£ matches is potentially very large. If the matches were conducted statewide, net savings per year would be about $355,000 xxi TABLE 1 THE COST-EFFECTIVENESS OF IEVS MATCHES IN ARIZONA AND MICHIGAN State Match Savings-to-Cost Ratio Net Savings per Research-Sample Case (dollars) Arizona Michigan SWICA 1.55 BEER 3.13 IRS 3.53 SWICA 2.82 UI 6.40 BENDEX 7.26 SDX 1.24 IRS 10.66 0.70 2.35 3.00 6.63 5.64 1.56 0.14 2.12 xxii for the SWICA match and over $1 million for the BEER and IRS matches in Arizona. In Michigan, net savings per year would be over $2 million for the SWICA and UI matches, over $500,000 for the IRS and BENDEX matches, and over $50,000 for the SDX match. These estimates far exceed our estimates of the costs of developing the IEVS matches in Arizona. All IEVS matches in the demonstration were found to be cost-effective even under a wide range of alternative assumptions. Even if we ignored any savings from case closures, benefit denials, or benefit reductions and just included savings from recovered overpayments, all IEVS matches in our study in Arizona were cost-effective. They were also cost-effective in Arizona even if we assume that (1) the costs of claims establishment and collection are as high in Arizona as they are in Michigan, (2) the portion of established claims that is recovered is as low in Arizona as it is in Michigan, or (3) the hourly cost of a caseworker is as high in Arizona as it is in Michigan. In Michigan, the savings from case closures, benefit denials, and benefit reductions need persist only for three and a half months for all the IEVS matches to be cost-effective. If we ignore all savings to the AFDC program, and attribute all IEVS costs to the Food Stamp Program, all matches in both states, except the SDX match in Michigan, were still cost-effective. The SWICA applicant match would be cost-effective if savings persisted for at least two-and-a-third months. This match ceases to be cost-effective only if we assume that (1) a SWICA recipient match would have detected all misreported income detected by the applicant match and (2) this recipient match occurred within two-and-a-tliird months of application. A study by agency staff in Michigan (Ward and Smucker 1990) found that the SWICA applicant match is not cost-effective. Their study differs from this study in two important ways: (1) they estimated that a follow up took an average of over 34 minutes compared to our estimate of 19 minutes, and (2^ they estimated that for applicants savings from case closures, benefit denials, and benefit reductions persisted for 2.5 months compared to our estimate of 7 months. If we estimate savings using either of Ward and Smucker's estimates, the SWICA applicant match is still cost-effective. However, it is not cost-effective if we use both of these estimates. The results of our study are consistent with the results of a previous study of applicant matching conducted by Puma (1989). He found that all applicant matches, except the SDX match, were cost-effective. Although the results are similar, Puma's study differed from ours in five important ways: (1) it examined only applicant matches, (2) none of its matches were targeted, (3) only offices in which the caseworker could receive information before certification were included in the study (this was not the case for Michigan during our demonstration), (4) it included Medicaid savings, and (5) caseworkers verified external data in only two of the nine offices, while caseworkers in our study offices verified the SWICA, BEER, and IRS data. TARGETING As this study was designed to estimate the cost-effectiveness of the whole IEVS process and not targeting perse, we cannot draw many firm conclusions about targeting strategies. Because all of the matches were cost-effective, we can conclude in some sense that all targeting strategies used in this study were successful. We cannot say whether these matches wouldhave been cost-effective had they not been targeted. However, the IEVS matches with targeting strategies that excluded many matched xxm SSNs from follow up (all of the matches in Arizona and the IRS match in Michigan) had higher than average action rates and large savings per action. Our results suggest that there is a trade-off between the cost-effectiveness of a targeting strategy and total savings from IEVS. For example, the IRS targeting strategy in Michigan targeted only eight matches for follow up during our study. The match was very cost-effective because one of the follow ups led to large savings, and as only eight follow ups were conducted, the cost of the match was low. In contrast, the cost-effectiveness of the SWICA applicant match in Michigan was relatively low because the match was not targeted and caseworkers consequently followed up all matches. However, the total savings from the SWICA match were over four times the total savings from the IRS match. This trade-off occurs because it is nearly impossible to design a targeting strategy that exempts from follow up only those cases in which there is no misreported income. The IEVS regulations prohibit applicant targeting. The rationale for this regulation is that it is less costly to detect misreported income before a case begins to receive benefits. Moreover, the Puma study is cited as evidence that applicant matches without targeting are cost-effective. However, many applicants begin to receive benefits before follow up is completed. In Michigan, caseworkers rarely finished a follow up prior to certification. Although we found that the SWICA applicant match was cost-effective without targeting, our findings suggest a targeting strategy would increase the cost-effectiveness of the match. Some targeting strategies that would probably increase the cost-effectiveness of the match while only marginally reducing savings include: (1) exclude from follow up inactive cases, (2) exclude from follow up persons under age 18, and (3) exclude from follow up cases in which the average monthly reported income over the reference quarter is the same as or similar to the income on the SWICA database. CONCLUSIONS All IEVS matches were cost-effective during the demonstrations. We cannot conclude that each IEVS match is always cost-effective. However, our results suggest that the matches were cost-effective with the targeting strategy used with each database, with the IEVS procedures used in each state, and for the types of clients that were in the research sample in each state (recipients in Arizona, and applicants and new recipients in Michigan). xxiv I. INTRODUCTION To be eligible to receive benefits from the Food Stamp Program (FSP), a household's income and assets must fall below specified limits. However, if incorrect information is provided at the time of application or later changes in a household's circumstances are not reported, it is possible for individuals who are actually ineligible for the program to receive benefits and for eligible individuals to receive an incorrect amount of benefits. Minimizing such errors in eligibility and benefit levels is important because it increases the resources available to the truly needy and strengthens public support for the program. The Income and Eligibility Verification System (IEVS) was established by Congress under the 1984 Deficit Reduction Act to minimize errors in determining eligibility and benefit levels in the Food Stamp, Aid to Families with Dependent Children (AFDC), and Medicaid programs. The IEVS regulations require state agencies to compare income reported by program applicants and recipients with income reported on six databases containing information on: earnings reported to the state by in-state employers; earned and unearned income reported to the Internal Revenue Service (IRS); and the receipt of social security, unemployment insurance, and supplementary security income benefits. Lists of welfare applicants and recipients are matched, using a computer, to the external database. If information on the external database is available for the applicant or recipient, the agency conducts "follow-up"procedures, which may include reviewing the client's case, contacting the client, verifying information on the external database, recomputing eligibility and benefits, investigating fraud, and recovering benefits paid in error. In 1986, the FSP regulations were amended to require states to implement the IEVS procedures.' These regulations required states to match all applicants and recipients to the six 'The final IEVS regulations are discussed in the February 28, 1986 Federal Register. The regulations became effective October 1986. The final IEVS regulations pertaining to the FSP are (continued...) 1 external databases, and to follow up on all cases about which the external database supplied information. After implementing IEVS under these regulations, some state food-stamp agencies expressed concern that the IEVS regulations were inflexible and burdensome. While caseworkers followed up on many matches with the external database, errors were detected in only a small proportion of follow ups. As follow ups can be very time-consuming, caseworkers perceived that IEVS used a large amount of resources in relation to the savings it generated and was not cost-effective. This issue was especially pertinent as states faced a combination of fiscal contraction and growing caseloads. The U.S. Department of Agriculture (USDA) and the U.S. Department of Health and Human Service (HHS) responded to these concerns by publishing interim regulations for comments in February 1988 permitting states to follow up on only a subset of matches that were most likely to lead to a benefit savings. The process of selecting a subset of matches to follow up is known as targeting. The interim amendments to the IEVS regulations gave the states the option to target recipient matches, although they are still required to follow up all applicant matches.2 While many states have adopted targeting strategies, their efforts to design and implement cost-effective strategies have been limited by the lack of information with which to evaluate targeting strategies. In response, the Food and Consumer Service (FCS) of USDA contracted with Mathematica Policy Research, Inc. (MPR) to conduct a study of targeting under IEVS. The objectives of this study were to (1) learn about targeting strategies used by state agencies, (2) develop improved targeting strategies, (3) evaluate the cost-effectiveness of these targeting strategies in two demonstration states, and (4) disseminate the results of the study to the state agencies. '(...continued) contained in 7 CFR, Part? 271-273; the regulations pertaining to the AFDC program are contained in 45 CFR, Parts 205-206; and the regulations pertaining to the Medicaid program are contained in 42 CFR, Parts 431 and 435. 2The interim IEVS targeting regulations pertaining to the FSP are contained in 7 CFR Part 272. 2 As we learned more about the needs of the state agencies, the focus of the study changed. As originally planned, the study was to develop and evaluate new or refined targeting strategies. Implicit in this vision of the study was the assumption that computer matching with an appropriate targeting strategy was cost-effective. However, during discussions with state agencies in the course of conducting a census of states' IEVS procedures and recruiting demonstration states, it became clear that the states questioned this assumption; some argued that some computer matches were not cost-effective even with targeting. The targeting strategies the states wished to introduce were not fine-tuned versions of existing targeting strategies, but radically different strategies that involved not following up on any matches with certain databases. In response, the study evolved into a comparison of the cost-effectiveness of matching IEVS databases using a targeting strategy with the cost-effectiveness of not following up any matches with some databases at all. This report presents the findings from the cost-effectiveness study of IEVS matching and targeting strategies implemented as part of the IEVS demonstration. In total, five new matching and targeting strategies were implemented in the two demonstration states, Michigan and Arizona. The demonstrations were conducted between July and October 1992. The rest of this chapter is organized as follows. Section A provides an overview of the IEVS process. Section B discusses the states' perceptions of the IEVS process. Prior research related to IEVS is discussed in Section C. Section D provides an overview of the demonstration and evaluation, and Section E presents the structure of the rest of the report. A. AN OVERVIEWOF THE IEVS PROCESS3 The lEVSprocessis the sequence of procedures involved in verifying client-reported information under IEVS regulations. It includes preselecting cases and databases to match, as well as computer matching, targeting, and follow up. While FSP, AFDC, and Medicaid regulations specify that certain 3The content of this section is based on the results of a state census of IEVS procedures conducted by MPR in 1991 and reported in Allin (1991). procedures be followed in the IEVS process, the procedures actually implemented vary considerably from state to state. In this section, we explain the IEVS regulations and procedures, and discuss briefly how IEVS is implemented by the states. In discussions of IEVS procedures, technical terms often have different meanings to different people. To avoid confusion, we define below some of the technical terms "sed in this report. Screening}* the preselection of a subset of cases to match. Current regulations prohibit most forms of screening. Computer matchingis the automated process of identifying information on external databases that pertains to a welfare case. A match occurs when information on a welfare case is available from an external database. Targeting is the selection of a subset of matched cases on which to follow up. Under current regulations, targeting is permitted for ongoing cases, but not for new applicants. A Air is a matched case designated for follow up. Followup refers to any actions taken to (1) check that the client-reported information in the computer match is valid, (2) verify information on the external database, (3) recompute eligibility and benefits with n formation from the external database, (4) process claims, disqualify recipients, and investigate fraud.4 The clientdatabase refers to the state agencies' database of welfare applicants and recipients that is matched to the external database under IEVS. This database may also contain information on persons who are neither applicants nor recipients, such as persons who have become ineligible to receive benefits or persons who live in the home of the applicant or recipient but are not eligible to receive benefits. A client is any person listed on the client database. The three main procedures in the IEVS system-matching, targeting, and follow up-are discussed below. 4In the regulations, the definition of follow up also includes targeting. For ease of exposition, we do not include targeting in our definition of follow up, but we do include all actions taken after a case is targeted. 1. Matching FSP regulations require states to conduct computer matches of all applicants and recipients with six external databases: 1. State WagelnformationCollectionAgency(SWlCA) Database. This database, maintained by a state agency such as the Department of Employment Security or the Department of Labor, provides quarterly earnings information that employers whose employees are covered by unemployment insurance must report each quarter. In most states, when the SWICA match takes place, the data refer to earnings information from the quarter prior to the previous quarter. 2. Unemploymentlnsurance(UI) Database. This database provides weekly data on UI benefits received in the previous month. It is often maintained by the same agency that maintains the SWICA database. 3. BeneficiaryData Exchange(BENDEX) Title II Database.5 This database, also maintained by the Social Security Administration (SSA), provides monthly information on social security and other benefits provided under Title II of the Social Security Act such as Black Lung benefits, Railroad Retirement benefits, and Medicare. At the time of the match, the data refer to benefits that will be received in the subsequent month. 4. BeneficiaryEarnings ExchangeReports (BEER) Database. This database, maintained by the SSA, provides annual earnings information compiled from information on the IRS Form W-2. If the match is made before about April, the data refer to the year prior to the previous year; if the match occurs after April, the SSA sends data from the previous year when it becomes available. 5. State Data Exchange(SDX) Database. This database, also maintained by SSA, provides monthly information on supplemental security income (SSI) benefits. At the time of the match, the SDX data refer to the subsequent month. 6. Internal Revenue Service (IRS) Database. This database provides annual information on unearned income, such as interest and dividends, compiled by the IRS from the IRS Form 1099. If the match occurs before July, the data refer to the year prior to the previous year; if the match occurs in July or later, the data refer to the previous year. Not all states comply with the IEVS regulations and conducts matches with all six of these databases. In a state census of IEVS procedures, respondents from three states reported not matching with BEER, and one respondent reported not matching with BENDEX. Both states in our demonstration 'Strictly speaking, BENDEX refers to the system at SSA through which Title II and Beneficiary Earnings Exchange Reports information is accessed. However, in this report we use the term BENDEX to refer to only the Title II information. did not follow up any cases from some matches. In addition, some states reported that they screened cases before a match: not all persons on the client database were sent to be matched to the external database. Most forms of screening are prohibited by the IEVS regulations. However, some forms of screening are permitted. For example, state agencies are not required to request UI information on all applicants and recipients. a. Nonmandated Databases The IEVS regulations require that state agencies verify social security numbers (SSNs). SSNs can be verified by matching with the Numident file, which contains a list of SSNs and names, or with another SSA database, such as BENDEX. Most states conduct a separate match with the Numident database. However, in nearly all of these states the match is not coordinated with the matches with the IEVS-mandated databases. A state may not follow up on any discrepancies between the SSN reported by the client and the SSN on the Numident file until after a match has been conducted with the other databases. Many states also conduct computer matches with databases other than the six mandated databases, such as the state's Department of Motor Vehicles.6 b. Timing of Matches The IEVS regulations require applicant matching at the "next available opportunity." This must be at least every two weeks with the SWICA and UI databases and every month with IRS and SSA data. Recipient matching is required less frequently-SWICA daut milt be matched quarterly, IRS data annually, SSA data by the second month of each certification period. Not all state agencies comply with these timing regulations. c. Matching Procedures The procedures involved in matching with the SWICA and UI databases vary by state. Typically, one or more of the following procedures is used: (1) the state agency sends a tape to the SWICA, which *More information about the use of nonmandated databases is available in an FCS report Profiles of States' Food Stamp Operations (Research and Evaluation Associates 1992). conducts the match and sends a tape with the matched cases back to the state agency; (2) the SWICA sends a tape containing earnings or UI benefit information to the state agency, which conducts the match; (3) a central computer agency conducts the match and sends reports to the state agency (this often occurs when the state agency and the SWICA are part of the same umbrella organization); or (4) workers at the local office conduct on-line matches with the databases. The matches with the BEER and BENDEX databases occur at SSA. Typically, states send a tape to SSA, which conducts the match and sends a tape with the matched information back to the state agency. In addition, SSA keeps an "orbit file" of all SSNs that were sent previously by the state agency.7 If BEER or BENDEX information regarding SSNs on the orbit file becomes available or changes, SSA sends another tape with the new information to the state agency or appends this new information to data on the match tape. Each month, SSA sends each state a tape, known as the "Treasury tape," containing data on all SSI recipients in the state. The state agency then typically matches the data on the tape against its client database. Eleven states also have direct access to BENDEX and SDX data (but not BEER data) at SSA via SSA's File Transfer Management System. Through this system, SSA can provide the states with information on SSI recipients three times a week via electronic file transmission. SSA sends a weekly tape containing information on changes in BENDEX and SDX data to those states that do not have direct access to the File Transfer Management System. In addition, all states have access to third-party query (TPQY) through which they can request a BENDEX or SDX match directly from a computer terminal at an SSA district office or by completing speciai ds they send to SSA. Information can be obtained via TPQY in about one or two weeks. The match with the IRS database occurs at IRS. The state agency sends a tape of client SSNs to IRS, and IRS matches the tape with its databases and sends a tape containing the matched information back to the state. 7The SSNs stay on the orbit file until the state explicitly requests their removal. 7 2. Targeting The 1988 interim amendments to the IEVS regulations permitted targeting of recipients, not applicants. According to the IEVS state census, most states have implemented a targeting strategy for at least one database, and over three-quarters of all states have implemented targeting strategies for at least three of the IEVS-mandated databases. The matches most frequently targeted are those conducted with the BEER, IRS, and SWICA databases. Only five states have not implemented any targeting strategies and so follow up on all matches from all IEVS-mandated databases. The types of targeting strategies used by the state agencies vary considerably by state and by database. However, the strategies typically consist of a number of rules, most of which fall into one of four categories: 1. Exempt Casesfrom Follow Up on the Basis ofIndividual or Case Characteristics. This rule exempts individuals or cases from follow up on the basis of their benefits-related and/or demographic characteristics. Examples of this type of rule include (a) follow up only on cases in which a person is currently receiving benefits and (b) do not follow up on persons under a certain age. 2. Exempt Specified Data Items from Follow Up. Under this rule, certain information items are not followed up because the information duplicates information available from other databases. 3. Use of a Tolerance Threshold. Individuals or cases are targeted for follow up if the income reported on the external database exceeds a given threshold. 4. Use ofa Discrepancy Threshold. Individuals or cases are targeted for follow up if the difference between income reported on the external database and income on the client database exceeds a given threshold. A more general variant of this rule is to follow up if there is any discrepancy between an item of information included in the two databases-the item may be the dollar amount of income, the receipt of a type of income, or the employer name. The IEVS census reported that the most common targeting strategy for the SWICA, UI, BENDEX, and SDX databases is the use of a discrepancy threshold. The size of the threshold varies by database and by state. The most common targeting strategy used for the BEER database is to examine income data that are not also contained on the SWICA database-information on earnings 8 from out-of-state activities, pensions, agricultural work, and self-employment. As the BEER data are annual and one or two years old at the time of the match, states would require an extensive benefit-history file in order to compare income from the same periods on the BEER database and the client database. Hence, most states do not use a discrepancy threshold as part of their targeting strategy for BEER. The predominant strategy for the IRS database is to follow up on matches only if the amounts of one or more of the types of unearned income are above a certain tolerance threshold. States rarely use a discrepancy threshold for the IRS database for the same reason that they do not use one for the BEER database-the data are annual and out of date. 3. Follow Up A case designated for follow up is called a hit. There are two main procedures used to send information on hits to local offices: (1) hard-copy reports (IEVS reports) and (2) messages, or "alerts,"that appear on the caseworker's computer screen. These procedures may vary by database. The caseworker is usually responsible for most of the follow up. However, clerical staff, specialized caseworkers, and fraud investigators may also be involved. Follow-up procedures involve the activities listed below: 1. Reviewing the Information in the CaseJUe. This involves checking that the client-reported information used in the computer match is valid and was correctly entered into the computer; checking that the information had not already been received from the client or via another match, and whether the computer system has not already made this check; and checking whether there is a discrepancy between the information reported by the client and the information provided on the external database. 2. Contactingthe Client. The caseworker may contact the client to (1) ask for verification of income, (2) obtain permission for the collateral contact to release information, (3) inform the client Uiat there is a discrepancy between income he or she reported and income on the external database, or (4) inform the client that an action is going to be taken as a result of the IEVS match. 3. Verifyinglnformationfrom the External Database Ifa DiscrepancyExists. This involves contacting the client and/or making a "collateral contact" with the client's bank or employer. In some states the computer system produces letters to the collateral contacts if the caseworker enters into the system the necessary information. Data on SSI, Title II, and UI benefits are considered already verified and do not require third-party verification. 4. RecomputingEligibility and Benefits Using Informationfrom the External Database In many states this can be performed by the computer. After recomputing the eligibility and benefits, the caseworker inputs the new information into the client database. 5. ProcessingOaims,DisqualifyingRecipients,andInvestigatingFraud. Processing claims involves computing total overpayments and initiating action to recover the overpayment. Depending on the state, the caseworker, a specialized worker, or the state agency's collection division is responsible for processing claims. In most states a special unit is responsible for investigating fraud. IEVS regulations require that the states complete follow-up procedures within 45 days of receipt of the matched information. If follow up is delayed because the state is waiting for information from collateral contacts, the state is permitted to follow up 20 percent of the cases in more than 45 days. Estimates of how many cases are followed up within the 45-day limit vary widely from state to state, but in most states follow up procedures are completed within 45 days for two-thirds to three-quarters of the cases designated for follow up. B. PERCEPTIONS OF THE IEVS PROCESS This section describes some of the states' concerns about IEVS and some changes to the IEVS regulations suggested by the states. This section draws upon the state census, a report by the American Public Welfare Association (1989), and discussions with state staff at potential and actual demonstration sites.7 Most of the agency staff believe that computer matching is useful and generally cost-effective. However, they perceive a number of problems with some of the information provided by the IEVS-mandated databases. The three most often cited problems are: 1. Out-of-DateData. BEER and IRS data can be up to 30 months out-of-date. SWICA data can be up to 6 months out of date. Thus, income on those databases ould refer to periods of time when the client was not receiving benefits. Moreover, verifying out-of-date 7These concerns were expressed prior to our study. 10 information is more difficult. The benefit data provided by BENDEX, UI, and SDX are kept up-to-iiate. 2. Data Are Aggregatedover Different Time Periods While clients report monthly income, income collected by the source agencies are often aggregated over a longer period of time. BEER and IRS data provide annual income data and SWICA provides quarterly income data. Thus, it is difficult to make direct comparisons between income reported by the client and income reported by the external data source. 3. Duplicated Data. Most of the information provided by BEER duplicates data provided by SWICA. The only income data that are provided by BEER, but not SWICA, are (1) self-employment income, (2) out-of-state wages, (3) federal and military employees wages, and (4) agricultural earnings. Although few states had conducted detailed cost-effectiveness studies, over half of the states responding to the state census perceived that matches with the SWICA, UI, BENDEX, and SDX databases were cost-effective. The SWICA match was the most popular-80 percent of the states viewed the SWICA match as cost-effective. Staff argued that information on earned income was useful in detecting incorrect benefit and eligibility determinations and easy to use. However, other respondents were more critical of the match and argued that the SWICA information was out-of-date and was costly to follow up because it required contact with employers. UI, BENDEX, and SDX matches were popular because the data are up-to-date, monthly, and do not require third-party verification. The least popular matches were those with the IRS and BEER databases. Only 12 percent of states perceive the BEER match to be cost-effective. Problems with these matches include: (1) the data are out-of-date, (2) the data are annual, (3) much of the BEER data duplicates data received from the SWICA match, (4) there is a long turnaround time for receiving the data, and (5) there are stringent security requirements for using these data. A common view among state agency staff is that the IEVS regulations do not allow the states sufficient flexibility to use computer matching in the most cost-effective way. This perception of the burden of the IEVS regulations is reflected in the evidence we found of noncompliance with the regulations. Neither of the two demonstration states were in full compliance with the IEVS 11 regulations prior to the demonstration. Changes in the current IEVS regulations that have been suggested include: ■ Allow states to conduct matches with only those databases viewed as cost-effective • Allow states to use screening, that is, to send only selected clients to be matched with the external database • Allow states to target applicants C. PRIOR COST-EFFECTIVENESS STUDIES OF IEVS No previous multi-state study of IEVS has estimated the cost-effectiveness of computer matching with targeting under IEVS. Greenberg and Wolf (1986) estimated the cost-effectiveness of computer matching with earnings data, but their study was conducted before IEVS was established. Puma (1989) examined the cost-effectiveness of using computers to match external data sources with data reported by applicants, but not recipients. Some states have conducted their own cost-effectiveness studies, but in general these studies lack sophistication and are difficult to interpret. Greenberg and Wolf estimated the cost-effectiveness of computer matching with state earnings databases in four sites: Mercer County, New Jersey; Camden County, New Jersey; San Joaquin, California; and New Hampshire. These sites were chosen because they had well-functioning wage matching systems. This study was conducted in 1982, prior to the introduction of IEVS. Greenberg and Wolf found that wage matching was cost-effective in all four sites, with the cost-benefit ratio varying from 1.6 to 2.5. Puma (1989) estimated the cost-effectiveness of computer matching of applicants in nine sites. Only offices in which information from the matches could be made available to the caseworker before certification were included in the study. This study was conducted in 1987, before applicant matching was mandatory. (No site did an IRS applicant match.) Puma found that the cost-benefit ratio, averaged over all matches, was greater than one in all sites. Overall, the cost-benefit ratio was just 12 under four. The SW1CA match was the most cost-effective. The SDX match was the only match that, on average over all sites, not cost-effective. During the census, states were asked to submit to MPR any cost-effectiveness studies of IEVS matching that they had conducted. Studies were received from 13 states. In general, these studies were not highly sophisticated, and among those of moderate to high quality, results varied widely. Although no match was found to be cost-effective by all studies, a majority found the SWICA, UI, BENDEX, and SDX matches to be cost-effective and the BEER and IRS matches not to be cost-effective. Michigan-one of our demonstration sites-conducted a cost-effectiveness study of matching with the SWICA database (Ward and Smucker 1990). This study is of fairly high quality. Ward and Smucker found that while matching recipients against SWICA data is cost-effective, matching applicants with SWICA data is not cost-effective. The ratio of savings to cost for the recipients varies from between 3 and 15 depending on the targeting strategy used-the higher the tolerance threshold used in the targeting strategy, the higher the savings-to-cost ratio. However, the savings-to-cost ratio for applicants is less than 0.8. This is contrary to Puma's findings that applicant wage matching is cost-effective. We discuss Puma's study in more detail in Chapter VIII. D. AN OVERVIEWOF THE IEVS TARGETING DEMONSTRATION AND EVALUATION The demonstration involved introducing a total of five new IEVS matching and targeting strategies in two states, Michigan and Arizona. The criteria used for selecting the states were that the state was interested in committing the necessary resources to conduct the study, and the state was as similar as possible to the majority of states in its IEVS system and in demographic and economic factors. Although states vary considerably in the operation of IEVS, we consider the operation of IEVS in either Michigan or Arizona to be fairly representative of the other states. The Arizona demonstration focused on recipients, while the Michigan demonstration focused on applicants. Arizona introduced three new matching and targeting strategies: (1) matching recipients 13 with the SWICA database and using a new targeting strategy, (2) following up recipient matches with the BEER database and using a new targeting strategy, and (3) following up recipient matches with the IRS database and using a new targeting strategy. Michigan introduced two new IEVS strategies: (1) not following up any match of an applicant with the SWICA database and (2) increasing the tolerance threshold for the IRS match. In the evaluation, we estimated the cost-effectiveness of the SWICA recipient match, the BEER match, and the IRS match in Arizona. In Michigan, we estimated the cost-effectiveness of the SWICA applicant match (or not conducting the match), and the cost-effectiveness of the IRS match. We also measured the cost-effectiveness of the UI, BENDEX, and SDX matches in Michigan. No changes were made to the matching or targeting strategies used with these databases. Cost-effectiveness is measured as the ratio of program savings from IEVS to the cost of matching, targeting, and following up under IEVS. Program savings, the numerator of the ratio, includes FSP and AFDC benefits that would have been erroneously paid to clients, benefits that were erroneously paid to clients in the past and are recovered, and the cost of administering cases that would have been opened or maintained on the rolls in the absence of IEVS. The cost of IEVS, the denominator of the ratio, includes the cost of matching and targeting, the cost of the caseworkers' follow ups, the cost of investigating fraud, and the costs of establishing and collecting claims. We also estimated the cost of developing new the new matching and targeting rules. However, as the development costs are one-time-only costs, we do not include them in the savings-to-cost ratio. We measure cost-effectiveness from the perspective of the federal and state governments. Hence, we do not include the costs of IEVS to the clients or to third parties, such as employers and financial institutions. We also include only directly measurable savings and costs. Hence, we do not include any measure of the savings that may result because IEVS deters clients from misreporting their income or because caseworkers' morale has improved. 14 It is important to note that cost-effectiveness is not the only possible criteria by which matching and targeting under IEVS can be judged. While the savings-to-cost ratio indicates the amount of program savings that can be expected for every dollar spent on matching and targeting, it does not provide any information about the percentage of all errors in benefit and eligibility determination that are detected. The most cost-effective matching and targeting strategy may be one that follows up on only a few cases that are likely to yield large savings, while allowing many smaller errors go undetected. E. AN OVERVIEWOF THE REPORT This volume of the report (Volume I) provides an overview of the study and discusses the main results. Volume II consists of a series of appendices that provide more details about the IEVS procedures used in the two states and the measures of savings and costs used in the study. Chapter II of this report describes the two demonstration sites, Michigan and Arizona. Chapter III describes the evaluation design. Chapter IV presents our framework for estimating costs and savings. It discusses the measurement issues and the assumptions underlying our measures. The last four chapters present the results of the evaluation. The cost-effectiveness of IEVS depends on three factors: (1) the proportions of all cases that are matched, targeted for follow up, and lead to savings, (2) the size of the savings realized as a result of follow up, and (3) the cost of matching, targeting, and following up cases. Chapters V, VI, and VII examines each of these factors, respectively. Chapter VIII presents our estimates of the cost-effectiveness of IEVS matches and discusses our overall findings from the IEVS demonstrations. 15 BLANK PAGE /I II. THE ENVIRONMENT: ARIZONA AND MICHIGAN FOOD-STAMP AGENCIES The IEVS demonstrations were conducted at two food-stamp agencies: Arizona and Michigan. These agencies were chosen to take part in the demonstration because (1) they were interested in participating in the study and willing to commit the necessary staff to the demonstration and (2) their IEVS procedures were fairly representative of IEVS procedures used in other states. In order to provide some context for the results of the evaluation, this chapter describes the key characteristics of the two food-stamp agencies. We begin in Section A by describing some general features of the FSP in the two states. Section B provides a summary of the IEVS procedures in the two states. Finally, Section C provides a description of the matches that took place during our demonstration. A. CHARACTERISTICS OF THE FOOD-STAMP AGENCIES IN ARIZONA AND MICHIGAN In Arizona, the Food Stamp and AFDC programs are fully integrated. Both programs are administered by the Family Assistance Administration (FAA) within the Department of Economic Security (DES). The administration of the Medicaid program is the responsibility of the Arizona Health Care Cost Containment System (AHCCCS) Administration, but Medicaid cases that are also FSP or AFDC cases are administered by the FAA. In Michigan, all three programs are fully integrated and administered by the Department of Social Services (DSS). In both states, most caseworkers work with cases in all three programs. In July 1992, Arizona's caseload was about 169,000. This is an average-sized state caseload; 20 states had larger caseloads. In comparison, Michigan had a caseload of about 406,000, larger than that of all but seven other states. Both states, like most others, have experienced a growth in their caseload over the past three years. Arizona experienced a particularly large growth both in absolute and relative terms. Between fiscal year 1991 and 1992, the food-stamp caseload in Arizona rose by nearly 30,000 or 21 percent, while in Michigan the caseload remained fairly steady over this time period. 17 The average number of cases per caseworker in Arizona is about 152, compared to 231 in Michigan. The IEVS Census found that the caseload per caseworker varied considerably by state, ranging from 100 to S23 with an average of 253 cases per caseworker. The quality control error rates vary considerably by state. In fiscal year 1991, the percentage of cases with errors detected in a state during the quality control process varied between 4 and 14 percent, and on average was 9.8 percent. Both of the demonstration agencies had error rates that were about average: in fiscal year 1991, the error rate was 10.9 percent in Arizona and 8.9 percent in Michigan. Table II 1 describes some characteristics of the food-stamp households in the two demonstration states and in the U.S. as a whole estimated from the 1991 Food Stamp Quality Control database. Important differences between the two states include. A smaller proportion of the food-stamp households also receive AFDC in Arizona (33 percent) than in Michigan (51 percent) and the U.S. as a whole (41 percent). Also, a smaller proportion of the food-stamp households in Arizona receive Medicaid than in Michigan or the U.S. as a whole. More food-stamp households have earnings in Arizona (29 percent) than in Michigan (15 percent) or the U.S. as a whole (20 percent). The average amount of earnings received by food-stamp households in Arizona ($196) is larger than in Michigan ($72) and in the U.S. as a whole ($117). The proportion of households with elderly persons is slightly higher in Arizona (11 percent) than in Michigan (10 percent), but it is lower in both states than in the U.S. as a whole (16 percent). Arizona has a larger proportion of Hispanic and Native American households (28 and 14 percent, respectively) than does Michigan (2 and 1 percent, respectively). Michigan has a higher proportion of African-American households (46 percent) than does Arizona (8 percent). The average length of the certification period in Arizona (6.6 months) is much shorter than in Michigan (12.6 months) and in the U.S. as a whole (9.7 months). 18 TABLE II. 1 CHARACTERISTICS OF FOOD STAMP HOUSEHOLDS IN MICHIGAN, ARIZONA, AND U.S. AS A WHOLE, 1991 Characteristic Arizona Michigan U.S. as a Whole Percent of Households wi ; AFDC Benefits Medicaid Eligibility Positive Gross Income Positive Earnings Expedited Service Elderly Persons Children under 18 No Male Adult Present Percent Distributions Income as Percent of Poverty 0 1-50 51-100 101-150 Over 150 Race of Household Head White, non-Hispanic Black, non-Hispanic Hispanic Native American or Alaskan Native Other/Unknown Average Values Gross Income Earnings Household Size Food Stamp Benefit Number of Persons with Earnings in Household Length of Certification Period (months) Sample Size 33.2 50.9 40.5 42.3 62.5 59.1 86.1 95.4 91.7 29.2 14.6 19.8 8.4 4.7 5.0 10.7 9.8 16.4 63.9 58.4 60.4 56.2 65.0 64.7 13.9 4.6 8.3 45.2 42.4 33.3 31.2 47.0 50.6 9.6 5.9 7.7 0.2 0.0 0.2 45.4 50.9 45.7 7.8 45.5 35.0 27.7 2.1 13.7 13.5 0.6 1.1 5.6 0.8 4.5 $434 $430 $464 $196 $72 $117 2.9 2.4 2.6 $191 $166 $161 0.3 0.2 0.2 6.6 12.6 9.7 1,214 1,644 63,692 SOURCE: 1991 Food Stamp Quality Control database. 19 B. IEVS PROCEDURES IN ARIZONA AND MICHIGAN The IEVS process is fairly well automated in both states. In both states, the caseworker can use the system to recompute eligibility and benefits for a defined past period. According to the IEVS Census, 28 other states have this capability (Allin 1991). As in most states (all but seven), targeting is completely automated in both states. Twelve states have some automated support for follow up-usual ly to produce a letter for the client or collateral contact. Arizona has some automated support for producing letters to the collateral contacts, but follow up is almost completely manual in both Arizona and Michigan. As in the majority of states, in Arizona and Michigan the matching process for the food stamp, AFDC, and Medicaid programs is coordinated. That is, one tape containing SSNs for individuals participating in one or more of these programs is sent to the external agency. Below, we describe the matching, targeting, and follow-up procedures used in Arizona and Michigan. For brevity, we include only a discussion of the procedures used for matches in our demonstrations. A fuller description of the IEVS procedures in the two states is given in Appendix A of Volume II. 1. Arizona The IEVS procedures in Arizona are summarized in Table II.2. a. Matching Arizona has conducted matches with all six IEVS-mandated databases. However, prior to our demonstration, Arizona had discontinued its SWICA quarterly tape match and was conducting only on-line matches with the SWICA database. And although the state was conducting matches with the BEER and IRS databases, no matches from these databases were followed up. The BEER and IRS matches were not followed up because Arizona believed they were not cost-effective. 20 TABLE D.2 SUMMARY OF IEVS PROCEDURES IN ARIZONA Maich Match Took Place Prior to Demonstration? Frequency of Match Which Clients Are Matched? Process Targeting? Form in Which Information is Sent to Caseworkers SWICA On-line Tape Yes No NA Applicants, clients at recertification. Direct on-line access and clients who report any changes in circumstances Quarterly All clients Tape match at FAA No NA Print of screen (on-line access) Hard-copy report K> BEER Yes Monthly New clients and any clients for whom Tape match at SSA there is new information on the (response tape returned system; to FAA via AHCCCS) SSA sends information on all clients whose situation has changed No matches followed up prior to demonstration Hard-copy report IRS All Clients New Clients Yes Yes Annually All clients who are active Monthly New clients Tape match at IRS Tape match at IRS No matches followed up prior to demonstration No matches followed up prior to demonstration Hard-copy report Hard-copy report NA = Not applicable. o/ During our demonstration, Arizona reinstated the quarterly SWICA recipient match and followed up matches from the BEER and IRS databases. No matches were followed up from BENDEX. Only the quarterly SWICA recipient match and the BEER and IRS matches were included in the demonstration. The client database in Arizona includes applicants, recipients, and persons who do not receive benefits but who reside in the households of applicants or recipients. Unless otherwise stated, Arizona's FAA requests information from the external database on all persons on the client database. SWICA. Employers in Arizona are required to report their employee's quarterly earnings to the Administration of Unemployment Insurance, which is a division within the Arizona DES. The SWICA database, known in Arizona as the "base wage" database, contains information on the SSN of each employee, the employees' quarterly earnings, and the employers' names and addresses. Two types of matches can be conducted with the SWICA database, an on-line match and a tape match. As the SWICA database is in effect "in-house" at the DES, staff can access the database directly via on-line commands from their computer terminals. The quarterly SWICA recipient match is a tape match that takes place at the FAA. BEER. The BEER match is coordinated with the BENDEX match. Arizona sends a tape, known as the "BENDEX request tape," to the SSA each month. This tape contains the SSNs of all new clients who have not been sent previously. The SSA matches the SSNs on the BENDEX request tape with its BEER database in addition to its BENDEX database. The FAA receives two tapes a month containing BEER information-one consisting largely of matched information on new clients and the other containing new matched information from the orbit file on clients who were previously sent to SSA. Depending on when in the month new data on clients on the orbit file is received by SSA, information on clients who were previously sent to SSA may be included on the first tape containing matched information on new clients or on the orbit-file tape. 22 IRS. Once a year, Arizona sends a tape to the IRS containing the SSNs of clients on the client database who are active for either the food stamp or AFDC programs. As not all clients on the database are sent to be matched, this is a form of screening. After a month or two, Arizona receives a tape back from the IRS with the matched information. In addition, Arizona sends a tape to the IRS each month containing the SSNs of all new clients (that is, those clients who have not been previously matched with the IRS database). About one month later, Arizona receives a tape from the IRS that contains information on any matches with the IRS database. b. Targeting In the past, Arizona conducted targeting on the matches with all six IEVS-mandated databases. However, prior to the demonstration, Arizona was targeting only its BENDEX match, UI tape match and SDX match. c. Follow Up This section explains the typical follow-up procedures in Arizona. The specific procedures do, however, vary across local offices. The manner in which caseworkers are notified of a hit varies by database. Caseworkers are notified of a hit from the SWICA, BEER, and IRS matches by a hard-copy report. The SWICA reports are sent from the state office to the local offices, where they are distributed to caseworkers. For security reasons, BEER and IRS reports are locked in cabinets. Caseworkers must sign for a report when it is removed from the cabinet and must return the report to the cabinet within a few hours. While the IEVS regulations require that caseworkers follow up a hit within 45 days, caseworkers in Arizona are requested to complete the follow-up procedures in less time. The time allowed to complete follow up varies by database. The SWICA hits must be followed up within 10 calendar days after the worker receives the report; the UI, BENDEX, and SDX hits within 14 days; and the BEER 23 and IRS hits within 30 days. Discussions with some field staff in Arizona suggest that these time limits are not always met, and in some instances, a hit may not be followed up at all. If the caseworker discovers that the client is currently receiving an incorrect benefit payment, he or she will recompute die new benefit amount using the new income information. However, if the caseworker suspects fraud or discovers an overpayment, he or she will complete a referralform (FA-S26), which is given to an overpayment writer. An overpayment writer is a caseworker who specializes in calculating overpayments and establishing claims. It may take a couple of weeks for the overpayment writer to calculate the amount of the overpayment. The overpayment writer completes a form, FA-529, with information on the case. This form is sent to the Office of Accounts Receivable and Collections (OARC), which investigates the cause of the overpayment (fraud, client error, or agency error), proceeds with any legal action, and arranges for the collection of the claim. 2. Michigan The IEVS procedures in Michigan are summarized in Table II.3. a. Matching Michigan conducts matches with all sixIEVS-mandated databases. All matches except the BEER and the SWICA recipient match are included in the demonstration. During our study period, none of the research-sample cases were subject to the SWICA recipient match and no BEER match was processed.1 Before a case is found to be eligible for benefits, only the SSN of the person who applied for benefits is entered into the client database. That is, applicant matches in Michigan do not include the SSNs of persons who reside in the same household as the applicant. Unlike Arizona, which requests information on everyone in the household, Michigan requests information on only one 'All BEER tapes received during the year are processed at one time. The annual processing did not occur during our study. 24 Match SW1CA Applicants Recipients UI Applicants Recipients BENDEX SDX IRS Frequency of Match TABLE D.3 SUMMARY OF EEVS PROCEDURES IN MICHIGAN Which Clients Are Matched? Twice weekly New applicants who have not been active within the past 105 days Quarterly All recipients who have received benefits for the past three months Twice weekly New applicants who have not been active within the past 105 days Monthly Clients who report receiving some unearned income, clients who have lost employment within the past three months, and clients who applied for welfare benefits less than three months previously Monthly New recipients and one-third of the caseload SSA sends information on all clients whose situations have changed Weekly All applicants and recipients Process Tape match at MESC Tape match at MESC Tape match at MESC Tape match at MESC Tape match at SSA Receive tapes from SSA Monthly Applicants, and recipients due for redetermination within 3 months Tape match at IRS ■Notification is sent first to the client. Caseworkers receive a printout listing those clients who were notified. NA = Not applicable. Targeting? No Yes Yes Yes Yes Yes Yes Form in Which Information is Sent to the Caseworkers Hard-copy report Hard-copy report Hard-copy report Hard-copy report Hard-copy report Hard-copy report NA' OU" person in each household for the applicant matches. For recipient cases, information is requested only on persons who are eligible to receive benefits. SWICA. Employers are required to report the quarterly earnings of employees who are covered by Unemployment Insurance to the Michigan Economic Security Commission (MESC). Michigan performs an applicant match and a recipient match with these SWICA data. Michigan's DSS does not have direct on-line access to the SWICA database, so both matches involve sending a tape to the MESC. With the exception of New York, Michigan is the only state required to pay for each match with the SWICA database. A tape of the SSNs of "new" applicants is sent about twice a week to the MESC. A "new" applicant is defined as an applicant who has not previously been active in the previous 105 days. As a client is defined as active if they have either applied for or receive welfare benefits, this screens out applicants who previously applied for benefits within the past three months. The rationale for this screen is that as the SWICA database is updated only on a quarterly basis, it will provide identical information on a person for three consecutive months. This screening rule could, however, screen out useful information on a person who applies more than once in a three-month period if these applications were made in two different quarters. The MESC conducts the match and returns the matched information on a tape within two or three weeks. Within a few days of receiving the tape, DSS produces reports of the hits and sends them to the caseworkers. UI. The MESC also collects information on UI benefits. Although DSS must pay for the SWICA match, it does not pay for the UI match. An applicant match and a recipient match are conducted with the UI database. The tape of all "new" applicant SSNs sent twice a week to the MESC to be matched to the SWICA database is also matched to the UI database. The MESC conducts the match and returns the matched information within about a week. At the beginning of each month, DSS also sends to 26 the MESC a tape containing the SSNs of all clients who report receiving some form of unearned income, who do not report receiving UI benefits but have lost employment within the past three months, or who do not report receiving UI benefits but have applied for welfare benefits less than three months previously.2 The rationale for including persons who report receiving some unearned income is th»t the client database does not include a field that contains UI benefit information; hence, any UI benefit data is entered as unearned income. BENDEX. Michigan sends a BENDEX request tape to ihe SSA during about the third week of the month. The request tape contains SSNs of all new recipients who have become active that month and have not already been matched to the BENDEX database. Michigan does not currently include applicants on the BENDEX request tape. The BENDEX request tape also contains the SSNs of about one-third of the clients on the client database (chosen by the digits in the case number).3 The SSA sends two tapes back to Michigan. Both arrive around the middle of the month. The first tape consists primarily of information on the clients included on the request tape. The second tape contains any new information on clients who were sent previously and were kept by the SSA on its orbit file. Within two or three days of receiving the tapes, Michigan produces reports of hits to be sent to the caseworkers. If, during the application process, a caseworker suspects that a client is receiving unreported Title II (or SSI) benefits, he or she can send a TPQY card to the SSA. SDX. At the end of each month, Michigan's DSS receives a tape-the Treasury tape-containing infotmation on all persons in Michigan who have applied for SSI, receive SSI, or have received SSI in the past. Because Michigan does not have direct access to the SSA File Transfer Management System, it does not electronically receive updated information on SSI recipients three times a week. 2As all clients are not sent to be matched, this is a form of screening. However, this form of screening for the UI match is explicitly permitted by the IEVS regulations. 3As SSA sends a tape from the orbit file containing data on any clients whose benefits have changed, it is redundant to send SSNs of clients who have previously been sent to SSA. However, this procedure began before SSA sent data from the orbit file and has not yet been changed. 27 Instead, each week Michigan receives a tape from SSA that contains any new information on persons on the SDX database. IRS. Around the second week of each month, Michigan sends the IRS a tape containing SSNs of all current applicants and SSNs of recipients who are due for redetermination within about three months. About two or three weeks later, the IRS returns to Michigan a tape with the matched information. b. Targeting Michigan conducts some form of targeting on all of its matches except the SWICA applicant match. The targeting strategies are implemented at the state office. There is no difference in the targeting strategies by welfare program. The following explains Michigan's targeting strategies for each database. SWICA. No targeting strategy is applied to the applicant match with the SWICA database-all matches are designated for follow up. However, discussions with agency staff in Michigan suggest that caseworkers do not currently have time to follow up a substantial proportion of the matches. UI. Michigan applies the same targeting strategy to its applicant and recipient matches with the UI database. A match is followed up only if both of the following rules are satisfied: ■ The client is currently active for a program administered by DSS. The UI database reports that the client has applied for UI benefits in the past 30 days, has received Ui Lenefits in the past 60 days, or has returned to work within the past 90 days. The first rule exempts from follow up clients who, at the time the targeting strategy is applied, have neither applied for nor receive benefits. These clients are not followed up because they cannot lead to any change in current benefits or eligibility status and because it is difficult to recover overpayments from clients who are no longer active. The second rule exempts from follow up those cases for which the receipt of UI benefits is unlikely to affect current benefits or eligibility. However, 28 it does not exempt from follow up clients who have recently stopped receiving Ul benefits because this may indicate that the client nas recently started work and has earned income. BENDEX. The targeting strategy for the BENDEX match is to follow up on matches only if both of the following rules are satisfied: ■ The client is currently active for a program administered by the DSS. • The client is currently receiving Title II benefits. The first rule exempts inactive clients from follow up. The second rule exempts from follow up clients who do not currently receive any Title II income. SDX. Matches are followed up by the caseworker only if both of the following targeting rules are satisfied: The client is currently active for a program administered by DSS. The client has applied for SSI, is currently receiving SSI benefits, has just had SSI benefits denied or terminated, or has had a change in address or living arrangements. These targeting rules exempt from follow up any clients that received SSI benefits in the past, but are no longer receiving benefits and for which the information consequently cannot lead to a change in current benefits or eligibility. IRS. The targeting strategy for the IRS match used prior to the demonstration was is to follow up only if both of the following targeting rules were satisfied: The client is currently active in a program administered by DSS. • The IRS reports levels of unearned income that exceed specified thresholds: - Interest income exceeds $100 or - Dividends exceed $100 or - Agricultural subsidies exceed $100 or • Capital gains exceed $100 or - Stock dividends exceed $100 or - Stock liquidations exceed $100 or 29 Savings bond interest exceeds $100 or Income from rental properties exceeds $100 or Royalties exceed $100 or Bond liquidations exceed $100 or Prizes and awards exceed $100 or IRA distributions exceed $100 or Profit sharing distributions exceed $100 or Real estate sales exceed $100 or State income tax refunds exceed $300 c. Follow Up The follow-up procedures in Mxhigan also vary by local office and by database. For example, in some offices certain caseworkers process applicant matches, and others process recipient matches; in other offices caseworkers process both applicant and recipient matches. Caseworkers are notified of a hit from the SWICA, UI, BENDEX, and SDX matches by a hard-copy report from the state office. For a hit from the BEER or IRS match, the state office sends a letter to the client notifying him or her that DSS has been notified of a source of income. The client is required to schedule an interview with the caseworker within a couple of weeks. If the client fails to do so, he or she is disqualified from the program and the case is closed. The caseworker receives a printout that lists those clients who have been sent a letter notifying them of the BEER or IRS information. Caseworkers are requested to complete the follow up of all hits within the 45 days specified by the IEVS regulations. They begin by checking the information in the casefile. If verification is required, the caseworker sends a letter to a collateral contact. The BEER and IRS matches are not verified until after the client has given the caseworker the letter from the state office about the match. If the estimated amount of overpayment is less than $200 or if fraud is not suspected because, for example, the agency itself made an error in benefit payments, the caseworker sends the client a letter about the overpayment. If the client does not dispute the overpayment, the caseworker enters the amount of the overpayment into a special system on the mainframe computer, the Automated 30 Recoupment System (ARS). The ARS automatically calculates the recoupment, which is the amount by which the monthly benefit is reduced to recover the overpayment. If the estimated amount of the overpayment is between $200 and $500 and fraud is suspected, the caseworker transfers the case materials to a caseworker who is specialized in dealing with overpayments, a designated staffperson (DSP).4 The DSP checks the amount of the overpayment and investigates whether there was fraud. If the investigation shows that fraud is a possibility, the DSP arranges for a hearing. It takes about a month for these procedures to be completed. If the estimated amount of the overpayment is $500 or more and fraud is suspected, the case is referred to the Office of the Inspector General (OIG). OIG conducts an investigation and arranges for any legal proceedings. If the case is referred to the OIG, it can take months or even years before the exact amount of the overpayment is established. C. MATCHES DURING THE DEMONSTRATION This section describes the matches that took place as part of the demonstration. In Arizona, two quarterly SWICA matches took place. Usually, only one match would take place during our study period. However, in order to increase the number of hits in our sample, an early SWICA match was postponed by a few months so it would take place during our study. The first SWICA match, which took place in August, involved matching the client database against earnings data for the first quarter of 1992 (January, February, and March). The second SWICA match, which occurred in September, involved matching the client database against earnings data from the second quarter of 1992 (April, May, and June). Arizona sent three monthly BEER request tapes to SSA in July, August, and September, respectively. These tapes were processed (targeted and reports produced) in August, September, and early November, respectively. Arizona processed two IRS matches during our study. The first was the annual IRS match, which includes all active clients. The request tape for this match was sent in early July. The response tape was processed in late October. The second was a monthly 4In some offices where there is no DSP, the caseworker would perform the tasks of the DSP. 31 match including only new clients. The request tape for this match was sent in early October, and the response was processed in November. In Michigan, the SW1CA and UI applicant matches took place twice a week throughout the study. The first reports of hits were produced in August for applicants sent in July. In total, 19 SWICA and UI applicant matches took place. The recipient UI match took place twice for cases in our sample, in mid-September and mid-October. The BENDEX match took place three times during our study, in August, September, and October. During the study, Michigan received four Treasury tapes and 20 weekly SDX tapes. Michigan received IRS response tapes for cases in our sample in September and October. 32 m. EVALUATIONDESIGN The evaluation was designed to estimate the cost-effectiveness of new IEVS matching and targeting procedures. Cost-effectiveness is measured as the ratio of program savings from IEVS to the costs incurred because of IEVS. Program savings, the numerator of the ratio, includes FSP and AFDC program benefits that would have been erroneously paid to clients, benefits that were erroneously paid to clients in the past and are recovered, and the cost of administering cases that would have been opened or remained on the rolls in the absence of IEVS. The cost of IEVS, the denominator of the ratio, includes the cost of the time caseworkers devote to follow-up activities; data processing costs; the cost of investigating fraud, establishing and collecting claims, and conducting hearings and prosecutions. We also estimated the cost of developing new matching and targeting rules. In this chapter, we discuss the design of the evaluation, including the general approach, the research sample, the new matching and targeting strategies tested in each state, and the data collection procedures. Table III. 1 provides an overview of the evaluation design in each state. A. THE GENERAL APPROACH Each demonstration state, Arizona and Michigan, agreed to develop and operate, for a four-month period, a total of five new versions of IEVS. In Arizona, the new versions of IEVS involved conducting three matches with new targeting strategies: (1) the quarterly SWICA recipient tape match, (2) the BEER match, and (3) the IRS match. Previously, Arizona did not conduct the SWICA tape match, and did not follow up any hits from the BEER or IRS matches. In Michigan, the new versions of IEVS were (1) discontinuing the applicant match with the SWICA database, and (2) increasing the tolerance threshold in the targeting strategy for the match with the IRS database. The objective of the evaluation is to estimate the cost-effectiveness of the new matching and targeting strategies in each state. However, it is important to note that we do not estimate the cost- 33 TABLE III.l OVERVIEW OF EVALUATION DESIGN Arizona Michigan New IEVS Procedures 1. Match and target the quarterly SWICA recipient match 2. Target and follow up the BEER match 3. Target and follow up the IRS match 1. Discontinue the SWICA applicant match 2. Increase the tolerance threshold for the IRS match Definition of Research Group 1. May be matched with 1. SWICA (UI, BENDEX, and SDX)a 2. May be matched with 2. BEER (UI, BENDEX, and SDX)* 3. May be matched with IRS 3. (UI, BENDEX, and SDX)* May be matched with SWICA, UI, BENDEX, and SDX May be matched with IRS, BENDEX, and SDX May be matched with BENDEX, and SDX New Targeting Strategies SWICA, BEER, IRS IRS Type of Case Food stamp recipients Number of Demonstration Offices 7 Food stamp applicants and new recipients 16 Sample Size (cases) 22,500 13,462 *The matches in parentheses were conducted during our study, but the results of the follow ups were not recorded. 34 effectiveness of the targeting strategy per se, nor do we estimate the cost-effectiveness of matching with a database and following up on all hits (no targeting). Instead, the evaluation is designed to address the question: Is a match against a given external database cost-effective if conducted with a particular targeting strategy? The policy decision rule implicit in this design is that the state will implement the new matching and targeting strategy if the ratio of savings to costs is greater than one, and it will not implement the strategy if the ratio is less than one. In measuring the cost-effectiveness of the IEVS matching and targeting strategies, the savings and costs are measured relative to the situation in which the match with the database did not occur, but the rest of the IEVS procedures used prior to the demonstration are continued. Hence, the IEVS program is viewed as a set of independent programs rather than a single integrated system. Implicit in this design is the assumption that the cost-effectiveness of each IEVS database is independent of every other IEVS database. In other words, matching with a given database will result in a certain amount of program savings whether or not any other databases are used. One rationale for this assumption is that many food stamp applicants and recipients have only a single source of income. Because each database investigates different income sources, it is unlikely that misreported income would be picked up by more than one database.' Due to differences in the states' interests and their implementation of IEVS prior to the demonstration, the demonstration design is fundamentally different in the two states. In Arizona, the evaluation was designed to estimate the cost-effectiveness of matching recipients with the SWICA, BEER, and IRS databases. Michigan staff were primarily interested in improving the IEVS matching and targeting of applicants. Hence, the evaluation in Michigan was designed to estimate the cost- 'The BEER and SWICA databases contain similar income information. However, in Arizona, the BEER match did not take place prior to the demonstration, and in Michigan, the BEER match did not occur during the study. 35 effectiveness of matching applicants and "new"recipients2 with the SW1CA, IT.S, UI, BENDEX, and SDX databases. In each state a sample of FSP cases were chosen to be in the research sample. We refer to cases in the research sample as "research-sample cases." The sample was designed to represent the entire state and to provide enough observations to support statistical estimates of the cost-effectiveness of IEVS. Prior to being matched to a database, each case in the research sample was randomly assigned to one of three groups. In Arizona, cases in the first group were subject to the quarterly SWICA recipient tape match, cases in the second group were subject to the BEER match, and cases in the third group were subject to the IRS match. All cases were also subject to the UI, BENDEX, and SDX matches. In Michigan, all cases were matched to the SDX and BENDEX databases. Cases in the first group were subject to the SWICA and UI match, as well as the SDX and BENDEX matches. Cases in the second group were subject to the IRS, SDX, and BENDEX matches, but not the SWICA or UI match. Cases in the third group were subject only to the SDX and BENDEX matches. The follow-up procedures were not affected by the study. However, each caseworker recorded on a data collection form the outcome of the follow up and the amount of time he or she spent conducting follow-up activities for the case. In Arizona, data collection forms were completed for follow ups of the SWICA recipient match, the BEER match, and the IRS match. In Michigan, data collection forms were completed for follow ups of the SWICA, UI, IRS, BENDEX, and SDX databases. "New" recipients are recipients who have recently applied. 36 B. THE RESEARCH SAMPLE Even though IEVS involves matching client rather than case data, follow ups are conducted on the whole case. Thus, cases, not clients, were randomly assigned. All clients within a case were assigned to the same group. As changes to the IEVS process involved both matching and targeting, random assignment took place prior to matching. In Arizona, all cases were randomly assigned at the beginning of the study. In Michigan, a case was randomly assigned once it was registered into the computer system as an applicant (a few days after the application was submitted). Once a case was assigned to a group, it remained in that group for the rest of the study. If a research-sample case was denied or closed and the household later reapplied for benefits within our study period, the case was assigned to the same research group after the second application. 1. The Research Sample in Arizona and Michigan The research sample consists of the set of cases that were included in the study. The types of cases included in the research sample differ between Arizona and Michigan. Arizona. In Arizona, a case is in the research sample if: 1. The case was open (that is, eligible to receive benefits), suspended (that is, ineligible to receive benefits for a reason that is temporary), or in the recertification process on July 1, 1992. 2. The case is open for food stamps. Cases that are open for AFDC and/or Medicaid but not food stamps are not in the research sample. 3. The case is in a local office included in the demonstration. Michigan. In Michigan, a case is included in the research sample if: 1. A new application for food stamps was submitted by a person in the case's household between July 1 and October 2, 1992. We will refer to these cases as "applicant cases". These cases include households that apply for food stamps for the first time, households that became ineligible for food stamps but then reapplied, and households who failed to complete their monthly reporting requirements but then reapplied. A case is not 37 considered an applicant at recertification. However, if a household fails to complete the recertification procedures in time, it is required to reapply for the program. In this case, a household would be considered an applicant at recertification. 2. The household applied for food stamps. Households who applied for AFDC and/or Medicaid but not food stamps are not in the research sample. 3. The case is in a local office included in the demonstration. Although every case must be an applicant case during the study period to qualify for the research sample, the application may be approved and begin to receive benefits during our study period. We refer to cases that are approved during our study as "new"recipient cases. Thus, the sample includes both applicants and new recipients. There are two major differences between the research sample in Arizona and Michigan. First, in Arizona, all the cases are recipient cases, while in Michigan, the research-sample cases are applicant cases at some time during the study period. Second, in Michigan the cases flowed into the research sample during the study period, while in Arizona, the research sample consisted of a stock of cases and did not vary in size during the study period. 2. Demonstration Offices The offices that participated in the demonstration were chosen by state staff with input from MPR. The objective was to choose offices that were representative of the state and were large enough to meet the targeted sample size. Table III.2 lists the demonstration offices in Arizona and Michigan and presents a brief description of each office. In Arizona, there were seven demonstration offices in seven counties. The offices were chosen to include both rural and urban counties and to include only offices that were under the supervision of a key member of the Arizona IEVS team. An office on an Indian Reservation was also included. In Michigan, there were 16 demonstration offices in 12 counties. The offices were chosen to include both rural and urban offices, offices serving Detroit (Wayne County), and offices that were 38 TABLE HI.2 PROJECT OFFICES IN THE DEMONSTRATION Office Arizona Phoenix Mesa Buckeye Tucson Flagstaff Winslow Window Rock Michigan Bay Muskegon Crawford Saginaw Eaton Sanilac Genesee Ionia Wexford Jackson Midland Wayne - Fullerton Wayne - Greydale Wayne - Hamtramck Wayne - Maddelein Wayne - Oakman Description/Location Urban Urban Rural Urban Urban Rural Indian Reservation Small urban Midsize urban Upper northern rural Large urban Southern rural Midstate rural Large urban (Flint) Midstate rural Lower northern rural Small urban Midstate rural West Detroit West Detroit East Detroit East Detroit West Detroit 39 under the supervision of members of the Michigan IEVS team. The rural offices were chosen to be geographically representative, including offices in upper northern Michigan, lower northern Michigan, mid-state Michigan, and southern Michigan. The urban offices include sma'I urban areas, mid-size urban areas, and large urban areas. Five offices are located in Wayne County, which accounts for over 40 percent of the total caseload. Three of these offices are located in West Detroit and two are located in East Detroit. 3. Sample Sizes The sample sizes were chosen to be large enough to estimate the cost-effectiveness of each database, but not so large as to waste project resources and caseworker time on unnecessary observations. Table III.3 shows the realized sample sizes in Arizona and Michigan. The random assignment algorithm was designed so that different numbers of cases were assigned to each group. The likelihood of a case being assigned to a group was set so that there would be enough hits in each group to measure the cost-effectiveness of each match with a similar level of statistical significance. TABLE III.3 SIZE OF RESEARCH SAMPLE Research Group Number of Cases Arizona 1. SWICA 2. BEER 3. IRS 3.856 8.507 10.137 Total 22,500 Michigan 1. SWICA, UI, BENDEX. SDX 2.460 2. IRS, BENDEX. SDX 3 86i 3. BENDEX, SDX 7j41 Totel . 13,462 40 C. THE NEW IEVS MATCHING AND TARGETING PROCEDURES This section describes in detail the new IEVS procedures used in Arizona and Michigan during the demonstration. In both states, some IEVS procedures used in the demonstration were radically different from the matching and targeting strategies that were used prior to the demonstration. In Arizona, matches that were not conducted prior to the demonstration were conducted during the demonstration. In Michigan, an applicant match was discontinued. In both states, the matching and targeting strategies do not distinguish between programs-the targeting strategy for food-stamp-only cases is the same as that for food-stamp/AFDC cases. 1. Arizona Table III.4 provides a summary of the targeting strategies used in Arizona during the demonstration and those strategies used prior to the demonstration. Most elements of the new targeting strategies used in Arizona have been used in other states. However, the targeting strategies are innovative in that they combine many elements of targeting strategies previously used separately. a. Match and Target the SWICA Database In the first new IEVS procedure, Arizona continued to conduct its SWICA on-line match and reinstated the quarterly SWICA recipient tape match with a new targeting strategy. Under the new targeting strategy, caseworkers followed up on matches only if all of the following rules were satisfied: 1. Use ofindividual and case characteristics The person is active in either the current month or was active in one of the two previous months The person is 16 years of age or older as reported on the client database in the current month3 The case received a benefit during at least one month over the reference quarter 3Current month refers to the month in which the computer job is run. 41 TABLE III.4 PREDEMONSTRATION AND DEMONSTRATION TARGETING STRATEGIES IN ARIZONA Match Predemonstration Targeting Strategy Demonstration Targeting Strategy SWICA Tape Match did not take place Follow up if all of the following are satisfied: INDIVIDUAL AND CASE CHARACTERISTICS • Person is active for food stamps in current month or was active in one of the two previous months • Person is 16 or older in the current month • Case received a benefit during at least one month over the reference quarter TOLERANCE THRESHOLD • The person's total quarterly earnings reported on the SWICA database from all employers are $3,600 or more DISCREPANCY THRESHOLD • The difference between the total prorated earnings on the SWICA database and the total earnings reported on the client database over the same quarter is 20 percent or more of the total prorated earnings on the SWICA database BEER No matches were followed up Follow up if all of the following are satisfied: INDIVIDUAL AND CASE CHARACTERISTICS • Employer identification code on the BEER database is different from the employer identification code on the SWICA database • Person was active for at least six months during the reference year • Information from the BEER database for the same employer during the same reference period has not already been received • Person is active for food sumps in current month or was active in one of the two previous months • Person is 16 or older in the current month • Case received a benefit during at least one month of the reference year •V*2 TABLE IH.4 (continued) Match IRS Predcmonstration Targeting Strategy No matches were followed up Demonstration Targeting Strategy Follow up if all of the following are satisfied: iNDtVrDUAL AND CASE CHARACTERISTICS • Person was active for at least six months during the reference year • Person is active for food stamps in current month or was active in one of the two previous months • Person is 16 or older in the current month • Case received a benefit during at least one month of the reference year TOLERANCE THRESHOLD • Total unearned income for the case, excluding UI income and prior year tax refunds, exceeds $100 ¥3 2. Use ofa tolerance threshold The person's total quarterly earnings from all employers as reported on the SWICA database are $3,600 or more 3. Use ofa discrepancy threshold The difference between the total prorated earnings on the SWICA database and the total earnings reported on the client database over the same quarter is 20 percent or more of the total prorated earnings on the SWICA database. The prorated earnings on the SWICA database are calculated by dividing the person's quarterly earnings by three to find the average monthly earnings, and multiplying the average monthly earnings by the number of months in which the person was active in either the Food Stamp or AFDC programs. The first set of rules for this targeting strategy exempts from follow up persons for whom changes in earnings will probably not affect benefits and persons from whom it will be difficult to recover overpayments. Earnings of persons younger than age 16 are not counted toward income under the FSP. If a person did not receive benefits during the reference quarter, errors in reported earnings could not have caused an overpayment. Persons who have not been active for three months are not followed up because they will typically remain inactive over the period in which benefits could be recovered. It is extremely difficult to recover overpayments from persons who are not receiving benefits. The second rule exempts from follow up some persons whose earnings are low enough that they will not affect eligibility for the FSP, although unreported earnings could still affect the level of benefits. The gross monthly income eligibility threshold for a household of three is currently $1,207. Hence, if a person was the single earner in a three-person household, he or she could earn up to $3,621 each quarter and still be eligible for benefits. As the typical working FSP household contains just under three persons and only one earner, a tolerance threshold of $3,600 a quarter will exempt from follow up a high proportion of persons whose earnings are low enough for them to be eligible for food stamps. However, it will also exempt from follow up some persons whose earnings are too 44 high for them to be eligible for food stamps. For example, a person in a one-person household would not be eligible for food stamps if he or she earned more than $718 per month, or $2,154 per quarter. Persons with earnings of less than $3,600 are excluded from the external database prior to the match. Excluding persons prior to the match, rather than after the match, conserves computer resources but has no impact on which clients are followed up. However, strictly speaking this is a screening rule rather than a targeting rule. As such, it is not in compliance with the IEVS regulations. The third rule compares client-reported income with income on the external database. When Arizona previously matched with the SWICA database, the state followed up only if the difference between the total quarterly earnings reported by the client and the total quarterly earnings reported on the SWICA database exceeded 20 percent of the total quarterly earnings on the SWICA database. Under this rule, a high proportion of matches were targeted for follow up, many of which did not lead to changes in benefits or eligibility. The problem with this targeting strategy was that it did not take into account that some persons were not active during some of the reference quarter because, for example, they had not yet been accepted into the program. In these months, their earnings were not recorded on the client database and were treated as zero earnings in the calculation of total quarterly earnings. Hence, in these cases the total quarterly earnings reported on the SWICA database were much higher than the total quarterly earnings on the client database even though the client may have correctly reported earnings while active on the program. Ideally, Arizona would like to compare earnings as reported by the client with earnings reported on the SWICA database during only those months in which the client is active. However, the SWICA database reports total quarterly earnings, not monthly earnings. The new targeting strategy addresses this problem by prorating the quarterly earnings on the SWICA database over the months in which the person was active. For example, if the client earned $2,500 over the quarter but was active for only two months, his total prorated earnings would be $1,670(2,500 + 3x2). However, 45 prorating does not completely solve the problem. If earnings were higher in the months in which the client was not active, the prorated earnings would still be higher than the client's true earnings. For example, suppose a person earned $1,000 in the first month of the quarter and $750 in the subsequent two months, and became active in the second month of the quarter. The total prorated earnings on the SW1CA database would be $1,670 [(1,000 + 750 + 750) + 3 x 2], but the total quarterly earnings as reported on the client database would be only $1,500(0 + 750 + 750). So, to further reduce the number of matches to be followed up, the targeting rule requires a 20 percent difference between the prorated earnings on the SWICA database and the earnings on the client database. b. Match and Target the BEER Database Under the second new IEVS procedure, Arizona introduced a new targeting strategy for the BEER database and followed up some hits. Under the new targeting strategy, caseworkers followed up on matches if all of the following rules were satisfied: /. Use ofindividual and case characteristics The employer identification code on the BEER database is different from the employer identification code on the SWICA database The person was active for at least six months during the reference year Information from the BEER database for the same person and the same reference period has not already been received The person is 16 years of age or older as reported on the client database in the current month The person is active either in the current month or was active in one of the two previous months The case received a benefit during at least one month of the reference year Arizona opted not to use a tolerance threshold as part of this targeting strategy. 46 Both the BEER and SWICA databases provide earnings data. However, the BEER database provides information on some forms of earnings, such as out-of-state earnings, not provided by the SWICA database. When earnings data from the same employer are provided by both databases, the data from the SWICA database are viewed as the more useful because they are more recent and are aggregated by quarter instead of by year. Hence, the first targeting rule restricts the matches to be followed up to those in which the employer reported on the BEER database differs from the employer reported on the SWICA database.4 As discussed above, there is a problem in comparing client-reported earnings with earnings from an external database when the client is not active during all of the reference period. This is an even more pertinent problem in the BEER match because the earnings reported on the BEER database cover 12 months. To prevent following up matches in which the client was not active for a substantial period of time, the second targeting rule requires the client to have been active for at least six months during the reference year. A match is made with the BEER database for all new clients and all clients who have reported a change in circumstances. This may be an actual change in name or address or a change because information provided previously, such as the person's date of birth or SSN, was incorrect on the client database. Hence, it is possible for the same person to be matched more than once even though there is no new earnings information from the BEER database for this person. Consequently, follow-up efforts on essentially the same information may result. By excluding all information that duplicates information already received, the third targeting rule prevents duplicate follow-up efforts. 4A technical difficulty with this element of the targeting rule is that the employer identification code on the BEER database is a federal employer identification code, while the employer identification code on the SWICA database is a state employer identification code. The state employer identification code was "translated'' to the federal employer identification code using a mapping between the two codes provided by the Administration of Unemployment Insurance. Unfortunately, there was not always a correct translation between the two identification codes. 47 The remaining three elements of the first rule of the targeting strategy were part of the SWICA targeting strategy and have the same purpose. c. Match and Target the IRS Database Under the third new IEVS procedure, Arizona introduced a new targeting strategy for the IRS match and followed up on the hits. The same targeting strategy was used for both the IRS annual and monthly matches. Under the new targeting strategy, caseworkers followed up on matches only if all of the following rules were satisfied: 1. Use ofindividual and case characteristics The person was active for at least six months during the reference year The person is 16 years of age or older as reported on the client database in the current month The person is active in either the current month or was active in one of the two previous months A person in the case received a benefit during at least one month of the reference year 2. Use ofa tolerancethreshold Unearned income, excluding UI income and prior year tax refunds, summed over all persons in the case is more than $100 The first rule in this targeting strategy-the exemptions based on individual and case characteristics-was also part of the BEER targeting strategy and were used in this match for the same reasons. The rationale for the second rule is that to be eligible for the FSP, a household must have total liquid assets in each month of less than either $2,000 (if the household does not contain an elderly FSP participant) or $3,000(if the household does contain an elderly FSP participant). Hence, the targeting rule specifies that only persons for whom the total unearned income for the case exceeds a threshold should be followed up. The amount of interest income that corresponds to asset 48 holdings of $2,000 will vary with the interest rate. However, with a 5 percent rate of interest a household with assets of $2,000 held all year would earn $100 in interest income. Two types of unearned income are excluded from the calculation of total unearned income: (1) UI income and (2) prior year tax refunds. UI income is excluded because the match with the UI database can provide more recent and less aggregated information about the receipt of UI benefits. Prior year tax refunds are excluded because they are disregarded when determining income eligibility for the FSP. 2. Michigan Table III.5 provides a summary of the matching and targeting strategies used in Michigan during the demonstration and those strategies used prior to the demonstration. a. No SWICA Applicant Match Under the first new IEVS procedure, Michigan discontinued its applicant match with the SWICA database. No matches with the SWICA database in the second and third research group were followed up. In the first research group, all SWICA matches were followed up. The rationale for not conducting the SWICA applicant match is that a cost-benefit study of the SWICA match conducted by Michigan staff (Ward and Smucker 1990) found that following up on SWICA applicant matches was not cost-effective. This was because no action was required, and hence no savings realized, for over 95 percent of the applicant matches. (An action was defined by Ward and Smucker as an application denied, a case closed, or a change in benefits.) The IEVS regulations explicitly state that all applicants must be matched and followed up~they prohibit the targeting of applicants. There are two rationales for this regulation. First, savings can be achieved at a lower cost prior to certification. Prior to certification, benefits can be denied, but after certification, previous benefits overpayments must be recovered. Second, Puma (1989) found that following up on all applicant matches was cost-effective. 49 TABLE III.5 PREDEMONSTRATION AND DEMONSTRATION TARGETING STRATEGIES IN MICHIGAN o Match Predemonstration Targeting Strategy Demonstration Targeting Strategy SWICA Applicants None All matches are followed up in first research group, no matches are followed up in second and third re
Object Description
Page/Item Description
Title | Part 1 |
Full-text |
■•
*J fyyt*:S«*/*l*l
\ United States TI«k »<^. I v% ***. ^^ an_n_ ^^ ^wMfe *Jk % =r9 The income and
Eligibility Verification
System (IEVS) Targeting
Demonstration
Agriculture
Food and
Consumer
Sorvlce
Office of
Analysis and
Evaluation
$.
?ueteo The Cost Effectiveness of the
cQt* Income and Eligibility Verification
System in Arizona and Michigan
Final Report
Volume I
\
Contract No : FCS 53-3198-8-95, Task Order 7
MPR Reference No.: 8029-600
TOE COST EFFECTIVENESS OF THE
INCOME AND ELIGIBILITY VERIFICATION
SYSTEM IN ARIZONA AND MICHIGAN
FINAL REPORT
VOLUME I
April 1,1995
Authors:
Nancy Fasciano
Sheena McConnell
Submitted to:
U.S. Department of Agriculture
Food and Consumer Service
Office of Analysis and Evaluation
3101 Park Center Drive, 2nd Floor
Alexandria, VA 22303
Submitted by:
Mathematica Policy Research, Inc.
600 Maryland Avenue, S.W.
Suite 550
Washington, DC 20024
Attention: Sharron Cristofar Project Director: Harold Beebout
/
BLANK PAGE
//
ACKNOWLEDGMENTS
The authors would like to thank the many people who have helped with this project. At the U.S.
Department of Agriculture, Food and Consumer Service, Sharron Cristofar, the Project Officer, provided
valuable direction and advice for the study. The project also benefitted from suggestions from Abigail Nichols,
Joe Pinto, Carolyn Foley, Ed Speshok, and Cecilia Fitzgerald of the Food and Consumer Service. John
Bedwell of the Food and Consumer Service provided data on established and recovered claims.
Staff at the U.S. Department of Health and Human Services provided data on the AFDC program and
staff at the Social Security Administration provided information on computer matching. Our thanks go to Tom
O'Conned, Joe Lonergan, and Norma Griffin.
Staff at the Arizona Division of Family Support and Michigan Department of Social Services designed
the new targeting strategies, implemented the demonstration, and provided valuable assistance throughout the
study. The study would not have been successful without the dedication of these staff. In Arizona, special
thanks go to Vince Wood, Liz Steele, Ben Dillon, Cindy Walker, Pam Esirella, and Aldona Vaitkus. We
would also like to thank other staff members who provided us with data for our savings and costs
measurements: Guy Wilson, Jeffrey Bowman, Michael Doyle, John Haines, Neil Young, Mary Werne,
Carlos Verdugo, Ben Wilmar, Susan Olsen, and Julie Rioux. In Michigan, special thanks go to Sue Hall,
Gary Miller, and Phil Michel. We would also like to thank Dick Hall, John Kelley, William Minihan, Dan
Hinds, Steve Smucker, Dick O'Herron, Linda Peterson, Larry Matecki, Irma Guzman, Deb Kristopherson,
and Sandra Zwemer.
We would also like to extend our gratitude to the caseworkers in both states who completed the data
collection forms and the local office coordinators who monitored the data collection process.
Myles Maxfield at Mathematica Podcy Research, Inc. reviewed an earlier draft of the report and provided
valuable suggestions. The study has also benefitted from input from Susan Allin, Ed Hoke, John Mamer, and
Alberto Martini. Daisy Ewed and Royston McNeid provided expert research assistance. Marianne Stevenson
and Parti Rossi input the data from the data collection forms. Daryl Hall edited the report. Report production
support was provided by Bob Skinner, Sharon Clark, and Ann Miles.
iii
BLANK PAGE
W
CONTENTS
Chapter Page
ACKNOWLEDGMENTS iii
GLOSSARY OF ACRONYMS xv
EXECUTIVE SUMMARY xvii
I INTRODUCTION 1
A AN OVERVIEW OF THE IEVS PROCESS 3
1. Matching 5
2. Targeting 8
3. Follow Up 9
B. PERCEPTIONS OF THE IEVS PROCESS 10
C. PRIOR COST-EFFECTIVENESS STUDIES OF IEVS 12
D. AN OVERVIEW OF THE IEVS TARGETING
DEMONSTRATION AND EVALUATION 13
E. AN OVERVIEW OF THE REPORT 15
II THE ENVIRONMENT. ARIZONA AND MICHIGAN
FOOD-STAMP AGENCIES 17
A CHARACTERISTICS OF THE FOOD-STAMP
AGENCIES IN ARIZONA AND MICHIGAN 17
B. IEVS PROCEDURES IN ARIZONA AND MICHIGAN 20
1. Arizona 20
2. Michigan 24
C MATCHES DURING THE DEMONSTRATION 31
III EVALUATION DESIGN 33
A THE GENERAL APPROACH 33
CONTENTS (continued)
Chapter
m
(continued)
IV
P-ge
B. THE RESEARCH SAMPLE 36
1. The Research Sample in Arizona and Michigan 37
2. Demonstration Offices 38
3. Sample Sizes 40
C. THE NEW ffiVS MATCHING AND TARGETING
PROCEDURES 40
1. Arizona 41
2. Michigan 49
D. DATA COLLECTION 52
1. Data Collection Forms 52
2. Monthly Case-Record Extracts 54
3. State Agencies' Accounting Records 54
4. Reports Submitted by State Agencies to Federal Agencies 55
MEASURES OF SAVINGS AND COSTS 57
A. SAVINGS RESULTING FROM THE IEVS PROCESS 58
1. Avoided Benefit Payments 59
2. Avoided Administrative Costs 66
3. Recovered Previous Benefit Overpayments 68
4. Unmeasured Savings • 73
B. COSTS INCURRED DURING THE IEVS PROCESS 73
1. Caseworker Follow-Up Costs 75
2. Claims Establishment and Collection Costs 77
3. Data Processing Costs 82
4. Development Costs 88
ACTION, HIT, AND MATCH RATES 91
A. ACTION RATES 92
vi
CONTENTS (continued)
Chapter Page
V B. REASONS GIVEN FOR FOLLOW UPS THAT DO
(continued) NOT LEAD TO A CHANGE IN BENEFITS 96
1. Income Was Already Recorded in the Casefile 98
2. Income Did Not Affect Benefits or Eligibility 102
3. Case Was Inactive 107
4. Caseworker Could Not Verify Income on the
External Database 108
5. Income on the External D. abase Was Incorrect 109
6. Report Did Not Provide Income Information 109
C CHARACTERISTICS OF CASES AND CLIENTS
FOLLOWED UP AND THOSE THAT WERE ACTED
UPON AS A RESULT OF IEVS 110
1. Characteristics of Cases and Clients in the Research
Sample and Those Targeted for Follow Up 115
2. Characteristics of Cases and Clients Acted Upon
As a Result of IEVS 116
D. MATCH AND HIT RATES 119
VI SAVINGS RESULTING FROM IEVS 127
A. TYPES OF ACTIONS 127
B. SAVINGS FROM CASE CLOSURES, BENEFIT
DENIALS, AND BENEFIT CHANGES 132
1. Savings Per Month , 132
2. Total Savings from Case Closures, Benefit Denials,
and Benefit Changes 139
C. SAVINGS FROM THE DETECTION OF PREVIOUS
BENEFIT OVERPAYMENTS 145
1. Detected Benefit Overpayments 145
2. Recovered Benefit Overpayments 149
D. UNMEASURED SAVINGS 150
1. Savings from Actions in Other Programs 151
2. Qualitative Evidence on Other Potential Savings 154
vii
CONTENTS (continued)
Chapter
VI
(continued)
vn
vm
Page
E SUMMARY OF SAVINGS FROM IEVS 155
COSTS INCURRED BY THE IEVS PROCESS 161
A COST OF CASEWORKERS' FOLLOW-UP 161
1. Tasks Performed by Caseworkers During Follow Up 162
2. Time Taken to Conduct a Follow Up 166
3. Estimates of Follow-Up Costs 171
B. CLAIMS ESTABLISHMENT AND COLLECTION COSTS 173
C. DATA PROCESSING COSTS 178
D. COSTS OF DEVELOPING THE MATCHING AND
TARGETING STRATEGIES 182
E. TOTAL COSTS INCURRED BY THE IEVS PROCESS 183
THE COST-EFFECTIVENESS OF IEVS 189
A ESTIMATES OF THE COST-EFFECTIVENESS OF IEVS 189
1. The Savings-to-Cost Ratios 192
2. Net Savings 195
3. Total Savings 1%
B. SENSmVITY OF THE COST-EFFECTIVENESS
ESTIMATES TO ASSUMPTIONS 197
1. Length of Time Savings Persist 197
2. Recovery of Previous Benefit Overpayments and
Costs of Claims Establishment and Collection 202
3. Hourly Cost of the Caseworkers' Time 204
4. Discounted Future Savings and Costs 204
5. Assumptions Used by Ward and Smucker (1990) 205
6. Assumptions Used by Puma (1989) 206
7. Include Only Savings to the FSP 207
vm
UQ
CONTENTS (continued)
Chapter Page
Vni C. LIMITATIONS OF OUR STUDY 207
(continued)
1. We Included Only the Savings and Costs That
Accrued to the Federal and State Agencies 207
2. The Study Was Conducted in Only Two States
and in Only Some Offices 208
3. The Agency Staff in Both Arizona and Michigan
Knew They Were Participating in the Study 209
4. We Did Not Learn of the Outcome of Some
Follow Ups of Research-Sample Cases 209
5. The Cost-Effectiveness of IEVS Matches May Be
Greater When the Match is First Introduced 210
6. The Number of Follow Ups Was Small 210
7. We Cannot Determine the Precision of Our Estimates 210
D. TARGETING STRATEGIES 211
1. Applicant Targeting 212
2. Suggested Targeting Strategies 214
E- SUMMARY OF FINDINGS AND CONCLUSIONS 215
REFERENCES 217
BLANK PAGE
TABLES
iabie p«r
JJ.1 CHARACTERISTICS OF FOOD STAMP HOUSEHOLDS IN
MICHIGAN, ARIZONA, AND U.S. AS A WHOLE, 1991 19
D.2 SUMMARY OF IEVS PROCEDURES IN ARIZONA 21
D.3 SUMMARY OF IEVS PROCEDURES IN MICHIGAN 25
m.1 OVERVIEW OF EVALUATION DESIGN 34
ffl.2 PROJECT OFFICES IN THE DEMONSTRATION 39
ffl.3 SIZE OF RESEARCH SAMPLE 40
m.4 PREDEMONSTRATION AND DEMONSTRATION
TARGETING STRATEGIES IN ARIZONA 42
ffl.5 PREDEMONSTRATION AND DEMONSTRATION
TARGETING STRA1EGJES IN MICHIGAN 50
IV.l ESTIMATES OF THE LENGTH OF TIME HOUSEHOLDS
WOULD HAVE REMAINED ON FOOD STAMPS IN THE
ABSENCE OF IEVS 65
IV.2 ADMINISTRATIVE COST SAVINGS 67
IV.3 COSTS OF CLAIMS ESTABLISHMENT AND COLLECTION:
ARIZONA 79
IV.4 COSTS OF CLAIMS ESTABLISHMENT AND COLLECTION:
MICHIGAN 80
JV.5 DATA PROCESSING UNIT COSTS: ARIZONA 85
IV.6 DATA PROCESSING UNIT COSTS: MICHIGAN 86
V.I NUMBER OF HITS, ACTIONS, AND ACTION RATES:
ARIZONA 93
V.2 NUMBER OF HITS, ACTIONS, AND ACTION RATES:
MICHIGAN 94
VJ REASONS GIVEN FOR NO CHANGE IN BENEFITS
OR ELIGIBILITY: ARIZONA 99
TABLES (continued)
Table Page
V.4 REASONS GIVEN FOR NO CHANGE IN BENEFITS
OR ELIGIBILITY: MICHIGAN 100
V.5 CHARACTERISTICS OF CASES AND CLIENTS THAT
ARE FOLLOWED UP AND THOSE ACTED UPON AS A
RESULT OF IEVS: ARIZONA Ill
V.6 CHARACTERISTICS OF CASES AND CLIENTS THAT
ARE FOLLOWED UP AND THOSE ACTED UPON AS
A RESULT OF IEVS: MICHIGAN 113
V.7 MATCH AND HIT RATES: ARIZONA 121
V8 MATCH AND HIT RATES: MICHIGAN 122
VI.1 TYPES OF ACTIONS RESULTING FROM IEVS
FOLLOW UPS: ARIZONA 129
VL2 TYPES OF ACTIONS RESULTING FROM IEVS
FOLLOW UPS: MICHIGAN 130
VI.3 SAVINGS PER MONTH FROM CASE CLOSURES AND
BENEFIT REDUCTIONS: ARIZONA 133
VI.4 SAVINGS PER MONTH FROM CASE CLOSURES AND
BENEFIT REDUCTIONS: MICHIGAN , 136
VI.5 TOTAL SAVINGS FROM CASE CLOSURES OR BEN ^FTT
REDUCTIONS: ARIZONA 141
VI.6 TOTAL SAVINGS FROM CASE CLOSURES AND BENEFIT
REDUCTIONS: MICHIGAN 142
VI.7 PREVIOUS BENEFIT OVERPAYMENTS DETECTED BY
IEVS FOLLOW UPS: ARIZONA 146
VL8 PREVIOUS BENEFIT OVERPAYMENTS DETECTED BY
IEVS FOLLOW UPS: MICHIGAN 147
VI.9 ESTIMATES OF SAVINGS FROM MEDICAID: ARIZONA 152
VI.10 ESTIMATES OF SAVINGS FROM MEDICAID: MICHIGAN 153
VI.11 TOTAL SAVINGS FROM IEVS MATCHES: ARIZONA 156
xu
TABLES (continued)
Table P«fe
VI.12 TOTAL SAVINGS FROM ffiVS MATCHES: MICHIGAN 158
Vn.l TASKS INVOLVED IN FOLLOW UPS IN ARIZONA:
BREAKDOWN BY WHETHER ACTION OCCURRED 164
VD.2 TASKS INVOLVED IN FOLLOW UPS IN MICHIGAN:
BREAKDOWN BY WHETHER ACTION OCCURRED 165
VH.3 TASKS INVOLVED IN FOLLOW UPS IN ARIZONA:
BREAKDOWN BY DATABASE 169
Vn.4 TASKS INVOLVED IN FOLLOW UPS IN MICHIGAN:
BREAKDOWN BY DATABASE 170
VH.5 COST OF CASEWORKERS' FOLLOW UPS: ARIZONA 172
VU.6 COST OF CASEWORKERS' FOLLOW UPS: MICHIGAN 174
VE.7 COSTS OF CLAIMS ESTABLISHMENT AND COLLECTION:
ARIZONA 176
VE.8 COSTS OF CLAIMS ESTABLISHMENT AND COLLECTION:
MICHIGAN 177
VD.9 DATA PROCESSING COSTS: ARIZONA 179
Vn.10 DATA PROCESSING COSTS: MICHIGAN 180
VD.11 COSTS OF DEVELOPING IEVS MATCHES: ARIZONA 183
VD.12 TOTAL COSTS OF IEVS MATCHES: ARIZONA 184
VD.13 TOTAL COSTS OF IEVS MATCHES: MICHIGAN 185
Vm.l SAVINGS AND COSTS FROM IEVS: ARIZONA 190
VID.2 SAVINGS AND COSTS FROM IEVS: MICHIGAN 191
Vm.3 ESTIMATES OF SAVINGS-TO-COST RATIOS UNDER
DIFFERENT ASSUMPTIONS: ARIZONA 198
Vm.4 ESTIMATES OF SAVINGS-TO-COST RATIOS UNDER
DIFFERENT ASSUMPTIONS: MICHIGAN 199
xui
TABLES (continued)
Table Page
Vm.5 MINIMUM NUMBER OF MONTHS THAT SAVINGS MUST
PERSIST FOR MATCH TO BE COST-EFFECTIVE:
ARIZONA AND MICHIGAN 201
xiv
GLOSSARY OF ACRONYMS
ADP Automated Data Processing (category of mainframe-computing costs)
AFDC Aid to Families with Dependent Children
AHCCCS Arizona Health Care Cost Containment System (department in Arizona that administers
Medicaid)
ARS Automated Recoupment System (computer system in Michigan that calculates
recoupments)
BEER Beneficiary Earnings Exchange Reports (annual earnings data)
BENDEX Beneficiary Data Exchange (Title II benefits data)
CIS Client Information System (client database in Michigan)
DES Department of Economic Security, Michigan
DSP Designated Staff Person (a caseworker in Michigan who specializes in dealing with
overpayments)
DSS Department of Social Services, Michigan
FAA Family Assistance Administration, Department of Economic Security, Arizona
FCS Food and Consumer Service, U.S. Department of Agriculture
FSP Food Stamp Program
GAO General Accounting Office
HHS U.S. Department of Health and Human Services
IEVS Income and Eligibility Verification System
IRS Internal Revenue Service (also refers to the annual unearned income data maintained
by the Internal Revenue Service)
MESC Michigan Economic Security Commission (agency that maintains state earnings and
Unemployment Insurance data)
MPR Mathematica Policy Research, Inc.
OARC Office of Accounts Receivable and Collections (office in Arizona's Department of
Economic Security that processes claims)
xv
OIG Office of Inspector General (office in Michigan's Department of Social Services that
investigates fraud)
SDX State Data Exchange (data on Supplemental Security Income)
SIPP Survey of Income and Program Participation
SSA Social Security Administration
SSI Supplemental Security Income
SSN Social Security Number
SWICA State Wage Information Collection Agency (collects quarterly state earnings data and
Unemployment Insurance data, also refers to the quarterly earnings data collected by
the agency)
TPQY Third-Party Query (a type of request for information from the Social Security
Administration)
UI Unemployment Insurance (also refers to the Unemployment Insurance data maintained
by the state)
USDA U.S. Department of Agriculture
xvi
EXECUTIVESUMMARY
The Income and Eligibility Verification System (IEVS) was established to verify income
information reported by welfare program applicants and recipients. Misreported income can lead to
errors in eligibility and benefit determination which can divert resources away from the truly needy
and weaken public support for the programs. Minimizing such errors is therefore important.
In 1986, the Food Stamp Program regulations were amended to require states to implement
IEVS. The IEVS regulations require state welfare agencies to compare income reported by
applicants and recipients of food stamps. Aid to Families with Dependent Children (AFDC), and
Medicaid with income reported on six external income databases. For most IEVS matches, the state
agencies create computer tapes listing welfare applicants and recipients, which are then matched to
the external databases. If a match occurs-information on the client is available from the external
database-the caseworker conducts follow-up procedures to investigate whether income has been
misreported. These procedures may include reviewing the client's case, contacting the client, verifying
information on the external database, recomputing eligibility and benefits, investigating fraud, and
recovering benefits paid in error.
The six IEVS external databases are:
1. State Wage Information Collection Agency (SWICA) database, which provides data on
quarterly earnings reported by employers to the state
2. Unemployment Insurance (UI) database, which provides monthly information on UI
receipt
3. Beneficiary Data Exchange (BENDEX) database, which provides monthly information on
receipt of Social Security and other Title II benefits
4. Beneficiary Earnings Exchange Reports (BEER) database, which provides annual earnings
information
5. State Data Exchange (SDX) database, which contains monthly information on receipt of
Supplemental Security Income (SSI)
6. Internal Revenue Service (IRS) database which contains annual information on unearned
income
BACKGROUND TO THE STUDY
After implementing IEVS, some state food-stamp agencies expressed concern that the IEVS
regulations were inflexible and burdensome. While caseworkers followed up many matches with the
external database, only a small proportion of follow ups detected errors in benefits or eligibility. As
these follow ups can be time-consuming, caseworkers perceived that IEVS used a large amount of
resources in relation to the savings it generated and was therefore not cost-effective.
xvii
In response to these concerns, interim regulations were published in 1988 permitting states to
follow up only a subset of recipient matches. The process of selecting a subset of matches to follow
up is known as targeting. The regulations prohibit targeting of applicants.
Despite these regulatory changes, some state agencies argue that, even with targeting, matching
with some databases is not cost-effective. The agencies' concerns with IEVS are largely related to
the external data: some are out-of-date, some are aggregated over too long a period, some duplicate
other IEVS data, and some require third-party verification. Suggested changes to the IEVS
regulations include allowing states to target applicants in addition to recipients and to conduct only
matches they consider cost-effective.
To address the concerns of the state agencies, the Food and Consumer Service of the U.S.
Department of Agriculture contracted with Mathematica Policy Research, Inc. to estimate the cost-effectiveness
of IEVS matches in two demonstration states, Arizona and Michigan. The study
estimates the cost-effectiveness of conducting IEVS matches using a targeting strategy compared to
the situation in which the match is not conducted at all. All but one of the IEVS matches in the
demonstration used a targeting strategy.
THE IEVS DEMONSTRATION
The IEVS demonstration took place in 7 local food-stamp offices in Arizona and 16 local food-stamp
offices in Michigan between July and October 1992. The research sample included only food-stamp
recipients in Arizona and food-stamp applicants in Michigan. (Some of the applicants in
Michigan began to receive benefits during our study.)
Prior to the demonstration, Arizona did not match recipients with the SWICA database or follow
up any match with the BEER and IRS databases. This was because agency staff believed that these
matches were not cost-effective. During the demonstration, Arizona reinstated the SWICA, BEER,
and IRS matches and used a new targeting strategy for each match. In Arizona, we estimated the
cost-effectiveness of the SWICA, BEER, and IRS recipient matches.
Prior to the demonstration, Michigan followed up information from all matches. Staff in
Michigan were concerned that the SWICA and IRS applicant matches were not cost-effective.
During the demonstration, Michigan introduced a new targeting strategy for the IRS match and
continued to conduct the SWICA applicant match with no targeting. In Michigan, we estimated the
cost-effectiveness of the SWICA, UI, BENDEX, SDX, and IRS applicant matches. All except the
SWICA match were targeted during the demonstration.
SAVINGS AND COST MEASURES
Cost-effectiveness is measured as the ratio of program savings from IEVS to the cost of
matching, targeting, and follow up under IEVS. We measure the cost-effectiveness of IEVS from
the perspective of the state and federal agencies that administer the Food Stamp and AFDC
programs. Hence, we do not include savings or costs to the clients, employers and financial
institutions that are required to verify income, or the agencies that maintain the external databases.
xvin
The savings from IEVS fall into four categories:
1. AvoidedBenefitPayments. Benefits may be denied or reduced on the basis of follow-up
information obtained through the IEVS process.
2. Avoided Administrative Costs. If an applicant is denied benefits or a case is closed
because of the IEVS process, the agency will avoid the cost of administering that
case.
3. Recovered Previous Benefit Overpayments. An IEVS follow up may result in the
determination that a client has received incorrect benefits. The savings to the agency
is the portion of the overpayment that is actually recovered from the client.
4. Unmeasured Savings. Savings from IEVS other than those discussed above may be
important but to quantify them were beyond the scope of this study. The most obvious
of these is savings to other programs, such as Medicaid. IEVS may also deter clients
from misreporting income and improve caseworker morale.
The costs of IEVS fall into four categories:
1. Caseworker Follow-Up Costs. These involve primarily the cost of caseworkers' time in
following up IEVS matches. They also include the cost of some supervisor and clerical
staff time, materials and supplies, and overhead.
2. Costs of Claims Establishment and Collection. These include the costs of investigating
fraud, establishing and collecting claims, and conducting hearings and prosecutions.
3. Data Processing Costs. These include payments to the agencies that maintain the
external database, as well as the mainframe computer costs incurred from producing
request tapes or matching extracts from the external databases against the caseload;
processing response tapes and running targeting algorithms; and producing reports of
the matches to be followed up.
4. Development Costs. These are the costs involved in developing and implementing the
matching and targeting strategies. As they are one-time-only costs, they are not
included in our measure of the cost-effectiveness of IEVS.
We were required to make many assumptions in measuring these savings and costs,. Whenever
a range of equally reasonable options was presented, we selected the one that led to the highest
estimate of costs and the lowest estimate of savings. The estimates of the savings-to-cost ratios
presented in this report are therefore low estimates of the cost-effectiveness of IEVS.
xix
ACTION, HIT, AND MATCH RATES
The cost-effectiveness of a match depends on the action rate, the proportion of all follow ups that
lead to a change in benefits, a change in eligibility, or the detection of a previous benefit
overpayment. The central criticism of IEVS is that caseworkers conduct many follow ups that do not
detect misreported income. Our findings support this criticism. The action rates during the
demonstrations were low in both states: 12 percent in Arizona and 6 percent in Michigan. In both
states, the action rate varied by database, from 7 percent for the SWICA match to 16 percent for the
IRS match in Arizona, and from 4 percent for the UI match to 13 percent for the IRS match in
Michigan.
The IEVS regulations require the state agencies to report both the match and hit rates for each
database. The match rate is the number of social security numbers (SSNs) on which information is
available from the external database as a proportion of all SSNs that could potentially be matched.
The hit rate is the number of SSNs that are targeted for follow up as a proportion of all SSNs for
which information is available from the external database. Few states actually do report these rates
(Allin 1991). The IEVS demonstration revealed that it is difficult to calculate these rates because
(1) the components of the match and hit rates are measured in different units (records, SSNs, and
cases), and (2) it may not be possible to observe the number of SSNs that are matched because, for
example, the matching and targeting steps are combined. We were able to estimate the match and
hit rates only for the IRS database in Arizona and for the SWICA, UI, and IRS databases in
Michigan.
Both the match and the hit rates were low. The match rates varied from 8 percent for the IRS
match in Arizona to 44 percent for the UI applicant match in Michigan. The hit rate was " |