The cost effectiveness of the income and eligibility verification system in Arizona and Michigan final report

              ■•
*J fyyt*:S«*/*l*l
\ United States TI«k »<^. I v% ***. ^^ an_n_ ^^ ^wMfe *Jk % =r9 The income and
Eligibility Verification
System (IEVS) Targeting
Demonstration
Agriculture
Food and
Consumer
Sorvlce
Office of
Analysis and
Evaluation
$.
?ueteo The Cost Effectiveness of the
cQt* Income and Eligibility Verification
System in Arizona and Michigan
Final Report
Volume I
\
Contract No : FCS 53-3198-8-95, Task Order 7
MPR Reference No.: 8029-600
TOE COST EFFECTIVENESS OF THE
INCOME AND ELIGIBILITY VERIFICATION
SYSTEM IN ARIZONA AND MICHIGAN
FINAL REPORT
VOLUME I
April 1,1995
Authors:
Nancy Fasciano
Sheena McConnell
Submitted to:
U.S. Department of Agriculture
Food and Consumer Service
Office of Analysis and Evaluation
3101 Park Center Drive, 2nd Floor
Alexandria, VA 22303
Submitted by:
Mathematica Policy Research, Inc.
600 Maryland Avenue, S.W.
Suite 550
Washington, DC 20024
Attention: Sharron Cristofar Project Director: Harold Beebout
/
BLANK PAGE
//
ACKNOWLEDGMENTS
The authors would like to thank the many people who have helped with this project. At the U.S.
Department of Agriculture, Food and Consumer Service, Sharron Cristofar, the Project Officer, provided
valuable direction and advice for the study. The project also benefitted from suggestions from Abigail Nichols,
Joe Pinto, Carolyn Foley, Ed Speshok, and Cecilia Fitzgerald of the Food and Consumer Service. John
Bedwell of the Food and Consumer Service provided data on established and recovered claims.
Staff at the U.S. Department of Health and Human Services provided data on the AFDC program and
staff at the Social Security Administration provided information on computer matching. Our thanks go to Tom
O'Conned, Joe Lonergan, and Norma Griffin.
Staff at the Arizona Division of Family Support and Michigan Department of Social Services designed
the new targeting strategies, implemented the demonstration, and provided valuable assistance throughout the
study. The study would not have been successful without the dedication of these staff. In Arizona, special
thanks go to Vince Wood, Liz Steele, Ben Dillon, Cindy Walker, Pam Esirella, and Aldona Vaitkus. We
would also like to thank other staff members who provided us with data for our savings and costs
measurements: Guy Wilson, Jeffrey Bowman, Michael Doyle, John Haines, Neil Young, Mary Werne,
Carlos Verdugo, Ben Wilmar, Susan Olsen, and Julie Rioux. In Michigan, special thanks go to Sue Hall,
Gary Miller, and Phil Michel. We would also like to thank Dick Hall, John Kelley, William Minihan, Dan
Hinds, Steve Smucker, Dick O'Herron, Linda Peterson, Larry Matecki, Irma Guzman, Deb Kristopherson,
and Sandra Zwemer.
We would also like to extend our gratitude to the caseworkers in both states who completed the data
collection forms and the local office coordinators who monitored the data collection process.
Myles Maxfield at Mathematica Podcy Research, Inc. reviewed an earlier draft of the report and provided
valuable suggestions. The study has also benefitted from input from Susan Allin, Ed Hoke, John Mamer, and
Alberto Martini. Daisy Ewed and Royston McNeid provided expert research assistance. Marianne Stevenson
and Parti Rossi input the data from the data collection forms. Daryl Hall edited the report. Report production
support was provided by Bob Skinner, Sharon Clark, and Ann Miles.
iii
BLANK PAGE
W
CONTENTS
Chapter Page
ACKNOWLEDGMENTS iii
GLOSSARY OF ACRONYMS xv
EXECUTIVE SUMMARY xvii
I INTRODUCTION 1
A AN OVERVIEW OF THE IEVS PROCESS 3
1. Matching 5
2. Targeting 8
3. Follow Up 9
B. PERCEPTIONS OF THE IEVS PROCESS 10
C. PRIOR COST-EFFECTIVENESS STUDIES OF IEVS 12
D. AN OVERVIEW OF THE IEVS TARGETING
DEMONSTRATION AND EVALUATION 13
E. AN OVERVIEW OF THE REPORT 15
II THE ENVIRONMENT. ARIZONA AND MICHIGAN
FOOD-STAMP AGENCIES 17
A CHARACTERISTICS OF THE FOOD-STAMP
AGENCIES IN ARIZONA AND MICHIGAN 17
B. IEVS PROCEDURES IN ARIZONA AND MICHIGAN 20
1. Arizona 20
2. Michigan 24
C MATCHES DURING THE DEMONSTRATION 31
III EVALUATION DESIGN 33
A THE GENERAL APPROACH 33
CONTENTS (continued)
Chapter
m
(continued)
IV
P-ge
B. THE RESEARCH SAMPLE 36
1. The Research Sample in Arizona and Michigan 37
2. Demonstration Offices 38
3. Sample Sizes 40
C. THE NEW ffiVS MATCHING AND TARGETING
PROCEDURES 40
1. Arizona 41
2. Michigan 49
D. DATA COLLECTION 52
1. Data Collection Forms 52
2. Monthly Case-Record Extracts 54
3. State Agencies' Accounting Records 54
4. Reports Submitted by State Agencies to Federal Agencies 55
MEASURES OF SAVINGS AND COSTS 57
A. SAVINGS RESULTING FROM THE IEVS PROCESS 58
1. Avoided Benefit Payments 59
2. Avoided Administrative Costs 66
3. Recovered Previous Benefit Overpayments 68
4. Unmeasured Savings • 73
B. COSTS INCURRED DURING THE IEVS PROCESS 73
1. Caseworker Follow-Up Costs 75
2. Claims Establishment and Collection Costs 77
3. Data Processing Costs 82
4. Development Costs 88
ACTION, HIT, AND MATCH RATES 91
A. ACTION RATES 92
vi
CONTENTS (continued)
Chapter Page
V B. REASONS GIVEN FOR FOLLOW UPS THAT DO
(continued) NOT LEAD TO A CHANGE IN BENEFITS 96
1. Income Was Already Recorded in the Casefile 98
2. Income Did Not Affect Benefits or Eligibility 102
3. Case Was Inactive 107
4. Caseworker Could Not Verify Income on the
External Database 108
5. Income on the External D. abase Was Incorrect 109
6. Report Did Not Provide Income Information 109
C CHARACTERISTICS OF CASES AND CLIENTS
FOLLOWED UP AND THOSE THAT WERE ACTED
UPON AS A RESULT OF IEVS 110
1. Characteristics of Cases and Clients in the Research
Sample and Those Targeted for Follow Up 115
2. Characteristics of Cases and Clients Acted Upon
As a Result of IEVS 116
D. MATCH AND HIT RATES 119
VI SAVINGS RESULTING FROM IEVS 127
A. TYPES OF ACTIONS 127
B. SAVINGS FROM CASE CLOSURES, BENEFIT
DENIALS, AND BENEFIT CHANGES 132
1. Savings Per Month , 132
2. Total Savings from Case Closures, Benefit Denials,
and Benefit Changes 139
C. SAVINGS FROM THE DETECTION OF PREVIOUS
BENEFIT OVERPAYMENTS 145
1. Detected Benefit Overpayments 145
2. Recovered Benefit Overpayments 149
D. UNMEASURED SAVINGS 150
1. Savings from Actions in Other Programs 151
2. Qualitative Evidence on Other Potential Savings 154
vii
CONTENTS (continued)
Chapter
VI
(continued)
vn
vm
Page
E SUMMARY OF SAVINGS FROM IEVS 155
COSTS INCURRED BY THE IEVS PROCESS 161
A COST OF CASEWORKERS' FOLLOW-UP 161
1. Tasks Performed by Caseworkers During Follow Up 162
2. Time Taken to Conduct a Follow Up 166
3. Estimates of Follow-Up Costs 171
B. CLAIMS ESTABLISHMENT AND COLLECTION COSTS 173
C. DATA PROCESSING COSTS 178
D. COSTS OF DEVELOPING THE MATCHING AND
TARGETING STRATEGIES 182
E. TOTAL COSTS INCURRED BY THE IEVS PROCESS 183
THE COST-EFFECTIVENESS OF IEVS 189
A ESTIMATES OF THE COST-EFFECTIVENESS OF IEVS 189
1. The Savings-to-Cost Ratios 192
2. Net Savings 195
3. Total Savings 1%
B. SENSmVITY OF THE COST-EFFECTIVENESS
ESTIMATES TO ASSUMPTIONS 197
1. Length of Time Savings Persist 197
2. Recovery of Previous Benefit Overpayments and
Costs of Claims Establishment and Collection 202
3. Hourly Cost of the Caseworkers' Time 204
4. Discounted Future Savings and Costs 204
5. Assumptions Used by Ward and Smucker (1990) 205
6. Assumptions Used by Puma (1989) 206
7. Include Only Savings to the FSP 207
vm
UQ
CONTENTS (continued)
Chapter Page
Vni C. LIMITATIONS OF OUR STUDY 207
(continued)
1. We Included Only the Savings and Costs That
Accrued to the Federal and State Agencies 207
2. The Study Was Conducted in Only Two States
and in Only Some Offices 208
3. The Agency Staff in Both Arizona and Michigan
Knew They Were Participating in the Study 209
4. We Did Not Learn of the Outcome of Some
Follow Ups of Research-Sample Cases 209
5. The Cost-Effectiveness of IEVS Matches May Be
Greater When the Match is First Introduced 210
6. The Number of Follow Ups Was Small 210
7. We Cannot Determine the Precision of Our Estimates 210
D. TARGETING STRATEGIES 211
1. Applicant Targeting 212
2. Suggested Targeting Strategies 214
E- SUMMARY OF FINDINGS AND CONCLUSIONS 215
REFERENCES 217
BLANK PAGE
TABLES
iabie p«r
JJ.1 CHARACTERISTICS OF FOOD STAMP HOUSEHOLDS IN
MICHIGAN, ARIZONA, AND U.S. AS A WHOLE, 1991 19
D.2 SUMMARY OF IEVS PROCEDURES IN ARIZONA 21
D.3 SUMMARY OF IEVS PROCEDURES IN MICHIGAN 25
m.1 OVERVIEW OF EVALUATION DESIGN 34
ffl.2 PROJECT OFFICES IN THE DEMONSTRATION 39
ffl.3 SIZE OF RESEARCH SAMPLE 40
m.4 PREDEMONSTRATION AND DEMONSTRATION
TARGETING STRATEGIES IN ARIZONA 42
ffl.5 PREDEMONSTRATION AND DEMONSTRATION
TARGETING STRA1EGJES IN MICHIGAN 50
IV.l ESTIMATES OF THE LENGTH OF TIME HOUSEHOLDS
WOULD HAVE REMAINED ON FOOD STAMPS IN THE
ABSENCE OF IEVS 65
IV.2 ADMINISTRATIVE COST SAVINGS 67
IV.3 COSTS OF CLAIMS ESTABLISHMENT AND COLLECTION:
ARIZONA 79
IV.4 COSTS OF CLAIMS ESTABLISHMENT AND COLLECTION:
MICHIGAN 80
JV.5 DATA PROCESSING UNIT COSTS: ARIZONA 85
IV.6 DATA PROCESSING UNIT COSTS: MICHIGAN 86
V.I NUMBER OF HITS, ACTIONS, AND ACTION RATES:
ARIZONA 93
V.2 NUMBER OF HITS, ACTIONS, AND ACTION RATES:
MICHIGAN 94
VJ REASONS GIVEN FOR NO CHANGE IN BENEFITS
OR ELIGIBILITY: ARIZONA 99
TABLES (continued)
Table Page
V.4 REASONS GIVEN FOR NO CHANGE IN BENEFITS
OR ELIGIBILITY: MICHIGAN 100
V.5 CHARACTERISTICS OF CASES AND CLIENTS THAT
ARE FOLLOWED UP AND THOSE ACTED UPON AS A
RESULT OF IEVS: ARIZONA Ill
V.6 CHARACTERISTICS OF CASES AND CLIENTS THAT
ARE FOLLOWED UP AND THOSE ACTED UPON AS
A RESULT OF IEVS: MICHIGAN 113
V.7 MATCH AND HIT RATES: ARIZONA 121
V8 MATCH AND HIT RATES: MICHIGAN 122
VI.1 TYPES OF ACTIONS RESULTING FROM IEVS
FOLLOW UPS: ARIZONA 129
VL2 TYPES OF ACTIONS RESULTING FROM IEVS
FOLLOW UPS: MICHIGAN 130
VI.3 SAVINGS PER MONTH FROM CASE CLOSURES AND
BENEFIT REDUCTIONS: ARIZONA 133
VI.4 SAVINGS PER MONTH FROM CASE CLOSURES AND
BENEFIT REDUCTIONS: MICHIGAN , 136
VI.5 TOTAL SAVINGS FROM CASE CLOSURES OR BEN ^FTT
REDUCTIONS: ARIZONA 141
VI.6 TOTAL SAVINGS FROM CASE CLOSURES AND BENEFIT
REDUCTIONS: MICHIGAN 142
VI.7 PREVIOUS BENEFIT OVERPAYMENTS DETECTED BY
IEVS FOLLOW UPS: ARIZONA 146
VL8 PREVIOUS BENEFIT OVERPAYMENTS DETECTED BY
IEVS FOLLOW UPS: MICHIGAN 147
VI.9 ESTIMATES OF SAVINGS FROM MEDICAID: ARIZONA 152
VI.10 ESTIMATES OF SAVINGS FROM MEDICAID: MICHIGAN 153
VI.11 TOTAL SAVINGS FROM IEVS MATCHES: ARIZONA 156
xu
TABLES (continued)
Table P«fe
VI.12 TOTAL SAVINGS FROM ffiVS MATCHES: MICHIGAN 158
Vn.l TASKS INVOLVED IN FOLLOW UPS IN ARIZONA:
BREAKDOWN BY WHETHER ACTION OCCURRED 164
VD.2 TASKS INVOLVED IN FOLLOW UPS IN MICHIGAN:
BREAKDOWN BY WHETHER ACTION OCCURRED 165
VH.3 TASKS INVOLVED IN FOLLOW UPS IN ARIZONA:
BREAKDOWN BY DATABASE 169
Vn.4 TASKS INVOLVED IN FOLLOW UPS IN MICHIGAN:
BREAKDOWN BY DATABASE 170
VH.5 COST OF CASEWORKERS' FOLLOW UPS: ARIZONA 172
VU.6 COST OF CASEWORKERS' FOLLOW UPS: MICHIGAN 174
VE.7 COSTS OF CLAIMS ESTABLISHMENT AND COLLECTION:
ARIZONA 176
VE.8 COSTS OF CLAIMS ESTABLISHMENT AND COLLECTION:
MICHIGAN 177
VD.9 DATA PROCESSING COSTS: ARIZONA 179
Vn.10 DATA PROCESSING COSTS: MICHIGAN 180
VD.11 COSTS OF DEVELOPING IEVS MATCHES: ARIZONA 183
VD.12 TOTAL COSTS OF IEVS MATCHES: ARIZONA 184
VD.13 TOTAL COSTS OF IEVS MATCHES: MICHIGAN 185
Vm.l SAVINGS AND COSTS FROM IEVS: ARIZONA 190
VID.2 SAVINGS AND COSTS FROM IEVS: MICHIGAN 191
Vm.3 ESTIMATES OF SAVINGS-TO-COST RATIOS UNDER
DIFFERENT ASSUMPTIONS: ARIZONA 198
Vm.4 ESTIMATES OF SAVINGS-TO-COST RATIOS UNDER
DIFFERENT ASSUMPTIONS: MICHIGAN 199
xui
TABLES (continued)
Table Page
Vm.5 MINIMUM NUMBER OF MONTHS THAT SAVINGS MUST
PERSIST FOR MATCH TO BE COST-EFFECTIVE:
ARIZONA AND MICHIGAN 201
xiv
GLOSSARY OF ACRONYMS
ADP Automated Data Processing (category of mainframe-computing costs)
AFDC Aid to Families with Dependent Children
AHCCCS Arizona Health Care Cost Containment System (department in Arizona that administers
Medicaid)
ARS Automated Recoupment System (computer system in Michigan that calculates
recoupments)
BEER Beneficiary Earnings Exchange Reports (annual earnings data)
BENDEX Beneficiary Data Exchange (Title II benefits data)
CIS Client Information System (client database in Michigan)
DES Department of Economic Security, Michigan
DSP Designated Staff Person (a caseworker in Michigan who specializes in dealing with
overpayments)
DSS Department of Social Services, Michigan
FAA Family Assistance Administration, Department of Economic Security, Arizona
FCS Food and Consumer Service, U.S. Department of Agriculture
FSP Food Stamp Program
GAO General Accounting Office
HHS U.S. Department of Health and Human Services
IEVS Income and Eligibility Verification System
IRS Internal Revenue Service (also refers to the annual unearned income data maintained
by the Internal Revenue Service)
MESC Michigan Economic Security Commission (agency that maintains state earnings and
Unemployment Insurance data)
MPR Mathematica Policy Research, Inc.
OARC Office of Accounts Receivable and Collections (office in Arizona's Department of
Economic Security that processes claims)
xv
OIG Office of Inspector General (office in Michigan's Department of Social Services that
investigates fraud)
SDX State Data Exchange (data on Supplemental Security Income)
SIPP Survey of Income and Program Participation
SSA Social Security Administration
SSI Supplemental Security Income
SSN Social Security Number
SWICA State Wage Information Collection Agency (collects quarterly state earnings data and
Unemployment Insurance data, also refers to the quarterly earnings data collected by
the agency)
TPQY Third-Party Query (a type of request for information from the Social Security
Administration)
UI Unemployment Insurance (also refers to the Unemployment Insurance data maintained
by the state)
USDA U.S. Department of Agriculture
xvi
EXECUTIVESUMMARY
The Income and Eligibility Verification System (IEVS) was established to verify income
information reported by welfare program applicants and recipients. Misreported income can lead to
errors in eligibility and benefit determination which can divert resources away from the truly needy
and weaken public support for the programs. Minimizing such errors is therefore important.
In 1986, the Food Stamp Program regulations were amended to require states to implement
IEVS. The IEVS regulations require state welfare agencies to compare income reported by
applicants and recipients of food stamps. Aid to Families with Dependent Children (AFDC), and
Medicaid with income reported on six external income databases. For most IEVS matches, the state
agencies create computer tapes listing welfare applicants and recipients, which are then matched to
the external databases. If a match occurs-information on the client is available from the external
database-the caseworker conducts follow-up procedures to investigate whether income has been
misreported. These procedures may include reviewing the client's case, contacting the client, verifying
information on the external database, recomputing eligibility and benefits, investigating fraud, and
recovering benefits paid in error.
The six IEVS external databases are:
1. State Wage Information Collection Agency (SWICA) database, which provides data on
quarterly earnings reported by employers to the state
2. Unemployment Insurance (UI) database, which provides monthly information on UI
receipt
3. Beneficiary Data Exchange (BENDEX) database, which provides monthly information on
receipt of Social Security and other Title II benefits
4. Beneficiary Earnings Exchange Reports (BEER) database, which provides annual earnings
information
5. State Data Exchange (SDX) database, which contains monthly information on receipt of
Supplemental Security Income (SSI)
6. Internal Revenue Service (IRS) database which contains annual information on unearned
income
BACKGROUND TO THE STUDY
After implementing IEVS, some state food-stamp agencies expressed concern that the IEVS
regulations were inflexible and burdensome. While caseworkers followed up many matches with the
external database, only a small proportion of follow ups detected errors in benefits or eligibility. As
these follow ups can be time-consuming, caseworkers perceived that IEVS used a large amount of
resources in relation to the savings it generated and was therefore not cost-effective.
xvii
In response to these concerns, interim regulations were published in 1988 permitting states to
follow up only a subset of recipient matches. The process of selecting a subset of matches to follow
up is known as targeting. The regulations prohibit targeting of applicants.
Despite these regulatory changes, some state agencies argue that, even with targeting, matching
with some databases is not cost-effective. The agencies' concerns with IEVS are largely related to
the external data: some are out-of-date, some are aggregated over too long a period, some duplicate
other IEVS data, and some require third-party verification. Suggested changes to the IEVS
regulations include allowing states to target applicants in addition to recipients and to conduct only
matches they consider cost-effective.
To address the concerns of the state agencies, the Food and Consumer Service of the U.S.
Department of Agriculture contracted with Mathematica Policy Research, Inc. to estimate the cost-effectiveness
of IEVS matches in two demonstration states, Arizona and Michigan. The study
estimates the cost-effectiveness of conducting IEVS matches using a targeting strategy compared to
the situation in which the match is not conducted at all. All but one of the IEVS matches in the
demonstration used a targeting strategy.
THE IEVS DEMONSTRATION
The IEVS demonstration took place in 7 local food-stamp offices in Arizona and 16 local food-stamp
offices in Michigan between July and October 1992. The research sample included only food-stamp
recipients in Arizona and food-stamp applicants in Michigan. (Some of the applicants in
Michigan began to receive benefits during our study.)
Prior to the demonstration, Arizona did not match recipients with the SWICA database or follow
up any match with the BEER and IRS databases. This was because agency staff believed that these
matches were not cost-effective. During the demonstration, Arizona reinstated the SWICA, BEER,
and IRS matches and used a new targeting strategy for each match. In Arizona, we estimated the
cost-effectiveness of the SWICA, BEER, and IRS recipient matches.
Prior to the demonstration, Michigan followed up information from all matches. Staff in
Michigan were concerned that the SWICA and IRS applicant matches were not cost-effective.
During the demonstration, Michigan introduced a new targeting strategy for the IRS match and
continued to conduct the SWICA applicant match with no targeting. In Michigan, we estimated the
cost-effectiveness of the SWICA, UI, BENDEX, SDX, and IRS applicant matches. All except the
SWICA match were targeted during the demonstration.
SAVINGS AND COST MEASURES
Cost-effectiveness is measured as the ratio of program savings from IEVS to the cost of
matching, targeting, and follow up under IEVS. We measure the cost-effectiveness of IEVS from
the perspective of the state and federal agencies that administer the Food Stamp and AFDC
programs. Hence, we do not include savings or costs to the clients, employers and financial
institutions that are required to verify income, or the agencies that maintain the external databases.
xvin
The savings from IEVS fall into four categories:
1. AvoidedBenefitPayments. Benefits may be denied or reduced on the basis of follow-up
information obtained through the IEVS process.
2. Avoided Administrative Costs. If an applicant is denied benefits or a case is closed
because of the IEVS process, the agency will avoid the cost of administering that
case.
3. Recovered Previous Benefit Overpayments. An IEVS follow up may result in the
determination that a client has received incorrect benefits. The savings to the agency
is the portion of the overpayment that is actually recovered from the client.
4. Unmeasured Savings. Savings from IEVS other than those discussed above may be
important but to quantify them were beyond the scope of this study. The most obvious
of these is savings to other programs, such as Medicaid. IEVS may also deter clients
from misreporting income and improve caseworker morale.
The costs of IEVS fall into four categories:
1. Caseworker Follow-Up Costs. These involve primarily the cost of caseworkers' time in
following up IEVS matches. They also include the cost of some supervisor and clerical
staff time, materials and supplies, and overhead.
2. Costs of Claims Establishment and Collection. These include the costs of investigating
fraud, establishing and collecting claims, and conducting hearings and prosecutions.
3. Data Processing Costs. These include payments to the agencies that maintain the
external database, as well as the mainframe computer costs incurred from producing
request tapes or matching extracts from the external databases against the caseload;
processing response tapes and running targeting algorithms; and producing reports of
the matches to be followed up.
4. Development Costs. These are the costs involved in developing and implementing the
matching and targeting strategies. As they are one-time-only costs, they are not
included in our measure of the cost-effectiveness of IEVS.
We were required to make many assumptions in measuring these savings and costs,. Whenever
a range of equally reasonable options was presented, we selected the one that led to the highest
estimate of costs and the lowest estimate of savings. The estimates of the savings-to-cost ratios
presented in this report are therefore low estimates of the cost-effectiveness of IEVS.
xix
ACTION, HIT, AND MATCH RATES
The cost-effectiveness of a match depends on the action rate, the proportion of all follow ups that
lead to a change in benefits, a change in eligibility, or the detection of a previous benefit
overpayment. The central criticism of IEVS is that caseworkers conduct many follow ups that do not
detect misreported income. Our findings support this criticism. The action rates during the
demonstrations were low in both states: 12 percent in Arizona and 6 percent in Michigan. In both
states, the action rate varied by database, from 7 percent for the SWICA match to 16 percent for the
IRS match in Arizona, and from 4 percent for the UI match to 13 percent for the IRS match in
Michigan.
The IEVS regulations require the state agencies to report both the match and hit rates for each
database. The match rate is the number of social security numbers (SSNs) on which information is
available from the external database as a proportion of all SSNs that could potentially be matched.
The hit rate is the number of SSNs that are targeted for follow up as a proportion of all SSNs for
which information is available from the external database. Few states actually do report these rates
(Allin 1991). The IEVS demonstration revealed that it is difficult to calculate these rates because
(1) the components of the match and hit rates are measured in different units (records, SSNs, and
cases), and (2) it may not be possible to observe the number of SSNs that are matched because, for
example, the matching and targeting steps are combined. We were able to estimate the match and
hit rates only for the IRS database in Arizona and for the SWICA, UI, and IRS databases in
Michigan.
Both the match and the hit rates were low. The match rates varied from 8 percent for the IRS
match in Arizona to 44 percent for the UI applicant match in Michigan. The hit rate was "<nly 2
percent for the IRS match in Michigan and exceeded 20 percent only for the SWICA applicant match
in Michigan, which was not targeted. Thus, the targeting strategies used in the demonstration
excluded from follow up many clients on whom information was found on the external database.
SAVINGS FROM IEVS
When a follow up led to a change in benefits, a change in eligibility, or the detection of a
previous overpayment, the resulting savings were large. For every follow up that led to an action,
an average of over $1,000 was saved in Arizona and an average of over $900 was saved in Michigan.
Avoided benefit payments accounted for the largest portion of these savings in both states (52
percent in Arizona and 90 percent in Michigan). Recovered previous benefit overpayments
accounted for 44 percent of all savings in Arizona, but only for 4 percent in Michigan. Overpayments
were more important in Arizona because a recipient case is more likely to have had previous benefit
overpayments than is a case applying for benefits. Avoided administrative costs accounted for only
a small portion of savings in both states (4 percent in Arizona and 6 percent in Michigan).
Savings were larger if the follow up led to a change in benefits or eligibility for the AFDC
program. When an error was detected in a joint food stamps/AFDC case, both avoided benefit
payments and recovered benefit overpayments were, on average, higher for the AFDC program than
for food stamps.
xx
The average savings over all follow ups (including those not resulting in an action) was $122 in
Arizona and $54 in Michigan. The average savings per follow up varied considerably by database.
In Arizona, average savings per follow up were about $62 for the SWICA match, $123 for the BEER
nutch, and $146 for the IRS match. In Michigan, average savings per follow up were $18 for the
SDX match, $63 for the SWICA match, $81 for the UI match, $86 for the BENDEX match, and
$1,129 for the IRS match.
COSTS INCURRED BY IEVS
A follow up cost an average of $40 in Arizona and $16 in Michigan. Caseworker follow ups
accounted for the largest portion of these costs: 80 percent in Arizona and 68 percent in Michigan.
Costs incurred in establishing and collecting claims were also sizable, accounting for 18 percent of
costs in Arizona and 31 percent of costs in Michigan. Data processing costs were small, accounting
for less than 2 percent of costs in both states.
Caseworkers took an average of 50 minutes in Arizona and 13 minutes in Michigan to conduct
a follow up. In Arizona, the time to conduct a follow up varied little by database. However, in
Michigan, it took an average of only 10 minutes to conduct a follow up of the SDX match and 19
minutes to conduct a follow up of the SWICA and IRS matches. The SWICA, BEER, and IRS
matches require more follow-up time because caseworkers must obtain third-party verification of
external income data and compare monthly income reported by the client in a previous quarter or
year with quarterly or annual data on the external database. The UI, BENDEX, and SDX databases
do not require third-party verification and contain current, monthly income information.
In Arizona, the average cost per follow up of IEVS varied little by database: $40 for the SWICA
database, $39 for the BEER database, and $42 for the IRS database. In contrast, the average cost
per follow up in Michigan varied from $12 for the BENDEX database to $106 for the IRS database.
The average cost per follow up for the other databases in Michigan was $13 for UI, $14 for SDX,
and $22 for SWICA.
The costs of developing the three IEVS matches and targeting strategies in Arizona were about
$100,000. In Michigan, we could not estimate the cost of developing the matches as they were
developed prior to our study. Staff in Michigan reported that few resources were used to implement
the new IRS targeting strategy.
COST-EFFECTIVENESS OF IEVS
All matches in the demonstration were found to be cost-effective. For each match, the savings
from the IEVS process exceeded the costs incurred by the process. Table 1 presents the savings-to-cost
ratios and the net savings (savings minus costs) per research-sample case for each IEVS match.
The most cost-effective match in the demonstration was the IRS match in Michigan, with nearly 11
dollars saved for every dollar spent on the match. The least cost-effective match was the SDX match
in Michigan with $1.24 saved for every dollar spent.
Our findings suggest that the net savings that can be realized from IEV£ matches is potentially
very large. If the matches were conducted statewide, net savings per year would be about $355,000
xxi
TABLE 1
THE COST-EFFECTIVENESS OF IEVS MATCHES IN
ARIZONA AND MICHIGAN
State Match Savings-to-Cost Ratio
Net Savings per
Research-Sample Case
(dollars)
Arizona
Michigan
SWICA 1.55
BEER 3.13
IRS 3.53
SWICA 2.82
UI 6.40
BENDEX 7.26
SDX 1.24
IRS 10.66
0.70
2.35
3.00
6.63
5.64
1.56
0.14
2.12
xxii
for the SWICA match and over $1 million for the BEER and IRS matches in Arizona. In Michigan,
net savings per year would be over $2 million for the SWICA and UI matches, over $500,000 for the
IRS and BENDEX matches, and over $50,000 for the SDX match. These estimates far exceed our
estimates of the costs of developing the IEVS matches in Arizona.
All IEVS matches in the demonstration were found to be cost-effective even under a wide range
of alternative assumptions. Even if we ignored any savings from case closures, benefit denials, or
benefit reductions and just included savings from recovered overpayments, all IEVS matches in our
study in Arizona were cost-effective. They were also cost-effective in Arizona even if we assume that
(1) the costs of claims establishment and collection are as high in Arizona as they are in Michigan,
(2) the portion of established claims that is recovered is as low in Arizona as it is in Michigan, or (3)
the hourly cost of a caseworker is as high in Arizona as it is in Michigan. In Michigan, the savings
from case closures, benefit denials, and benefit reductions need persist only for three and a half
months for all the IEVS matches to be cost-effective. If we ignore all savings to the AFDC program,
and attribute all IEVS costs to the Food Stamp Program, all matches in both states, except the SDX
match in Michigan, were still cost-effective.
The SWICA applicant match would be cost-effective if savings persisted for at least two-and-a-third
months. This match ceases to be cost-effective only if we assume that (1) a SWICA recipient
match would have detected all misreported income detected by the applicant match and (2) this
recipient match occurred within two-and-a-tliird months of application.
A study by agency staff in Michigan (Ward and Smucker 1990) found that the SWICA applicant
match is not cost-effective. Their study differs from this study in two important ways: (1) they
estimated that a follow up took an average of over 34 minutes compared to our estimate of 19
minutes, and (2^ they estimated that for applicants savings from case closures, benefit denials, and
benefit reductions persisted for 2.5 months compared to our estimate of 7 months. If we estimate
savings using either of Ward and Smucker's estimates, the SWICA applicant match is still cost-effective.
However, it is not cost-effective if we use both of these estimates.
The results of our study are consistent with the results of a previous study of applicant matching
conducted by Puma (1989). He found that all applicant matches, except the SDX match, were cost-effective.
Although the results are similar, Puma's study differed from ours in five important ways:
(1) it examined only applicant matches, (2) none of its matches were targeted, (3) only offices in
which the caseworker could receive information before certification were included in the study (this
was not the case for Michigan during our demonstration), (4) it included Medicaid savings, and (5)
caseworkers verified external data in only two of the nine offices, while caseworkers in our study
offices verified the SWICA, BEER, and IRS data.
TARGETING
As this study was designed to estimate the cost-effectiveness of the whole IEVS process and not
targeting perse, we cannot draw many firm conclusions about targeting strategies. Because all of the
matches were cost-effective, we can conclude in some sense that all targeting strategies used in this
study were successful. We cannot say whether these matches wouldhave been cost-effective had they
not been targeted. However, the IEVS matches with targeting strategies that excluded many matched
xxm
SSNs from follow up (all of the matches in Arizona and the IRS match in Michigan) had higher than
average action rates and large savings per action.
Our results suggest that there is a trade-off between the cost-effectiveness of a targeting strategy
and total savings from IEVS. For example, the IRS targeting strategy in Michigan targeted only eight
matches for follow up during our study. The match was very cost-effective because one of the follow
ups led to large savings, and as only eight follow ups were conducted, the cost of the match was low.
In contrast, the cost-effectiveness of the SWICA applicant match in Michigan was relatively low
because the match was not targeted and caseworkers consequently followed up all matches. However,
the total savings from the SWICA match were over four times the total savings from the IRS match.
This trade-off occurs because it is nearly impossible to design a targeting strategy that exempts from
follow up only those cases in which there is no misreported income.
The IEVS regulations prohibit applicant targeting. The rationale for this regulation is that it is
less costly to detect misreported income before a case begins to receive benefits. Moreover, the
Puma study is cited as evidence that applicant matches without targeting are cost-effective. However,
many applicants begin to receive benefits before follow up is completed. In Michigan, caseworkers
rarely finished a follow up prior to certification. Although we found that the SWICA applicant match
was cost-effective without targeting, our findings suggest a targeting strategy would increase the cost-effectiveness
of the match. Some targeting strategies that would probably increase the cost-effectiveness
of the match while only marginally reducing savings include: (1) exclude from follow up
inactive cases, (2) exclude from follow up persons under age 18, and (3) exclude from follow up cases
in which the average monthly reported income over the reference quarter is the same as or similar
to the income on the SWICA database.
CONCLUSIONS
All IEVS matches were cost-effective during the demonstrations. We cannot conclude that each
IEVS match is always cost-effective. However, our results suggest that the matches were cost-effective
with the targeting strategy used with each database, with the IEVS procedures used in each
state, and for the types of clients that were in the research sample in each state (recipients in
Arizona, and applicants and new recipients in Michigan).
xxiv
I. INTRODUCTION
To be eligible to receive benefits from the Food Stamp Program (FSP), a household's income
and assets must fall below specified limits. However, if incorrect information is provided at the time
of application or later changes in a household's circumstances are not reported, it is possible for
individuals who are actually ineligible for the program to receive benefits and for eligible individuals
to receive an incorrect amount of benefits. Minimizing such errors in eligibility and benefit levels is
important because it increases the resources available to the truly needy and strengthens public
support for the program.
The Income and Eligibility Verification System (IEVS) was established by Congress under the
1984 Deficit Reduction Act to minimize errors in determining eligibility and benefit levels in the Food
Stamp, Aid to Families with Dependent Children (AFDC), and Medicaid programs. The IEVS
regulations require state agencies to compare income reported by program applicants and recipients
with income reported on six databases containing information on: earnings reported to the state by
in-state employers; earned and unearned income reported to the Internal Revenue Service (IRS);
and the receipt of social security, unemployment insurance, and supplementary security income
benefits. Lists of welfare applicants and recipients are matched, using a computer, to the external
database. If information on the external database is available for the applicant or recipient, the
agency conducts "follow-up"procedures, which may include reviewing the client's case, contacting the
client, verifying information on the external database, recomputing eligibility and benefits,
investigating fraud, and recovering benefits paid in error.
In 1986, the FSP regulations were amended to require states to implement the IEVS
procedures.' These regulations required states to match all applicants and recipients to the six
'The final IEVS regulations are discussed in the February 28, 1986 Federal Register. The
regulations became effective October 1986. The final IEVS regulations pertaining to the FSP are
(continued...)
1
external databases, and to follow up on all cases about which the external database supplied
information.
After implementing IEVS under these regulations, some state food-stamp agencies expressed
concern that the IEVS regulations were inflexible and burdensome. While caseworkers followed up
on many matches with the external database, errors were detected in only a small proportion of follow
ups. As follow ups can be very time-consuming, caseworkers perceived that IEVS used a large
amount of resources in relation to the savings it generated and was not cost-effective. This issue was
especially pertinent as states faced a combination of fiscal contraction and growing caseloads.
The U.S. Department of Agriculture (USDA) and the U.S. Department of Health and Human
Service (HHS) responded to these concerns by publishing interim regulations for comments in
February 1988 permitting states to follow up on only a subset of matches that were most likely to lead
to a benefit savings. The process of selecting a subset of matches to follow up is known as targeting.
The interim amendments to the IEVS regulations gave the states the option to target recipient
matches, although they are still required to follow up all applicant matches.2
While many states have adopted targeting strategies, their efforts to design and implement cost-effective
strategies have been limited by the lack of information with which to evaluate targeting
strategies. In response, the Food and Consumer Service (FCS) of USDA contracted with
Mathematica Policy Research, Inc. (MPR) to conduct a study of targeting under IEVS. The
objectives of this study were to (1) learn about targeting strategies used by state agencies, (2) develop
improved targeting strategies, (3) evaluate the cost-effectiveness of these targeting strategies in two
demonstration states, and (4) disseminate the results of the study to the state agencies.
'(...continued)
contained in 7 CFR, Part? 271-273; the regulations pertaining to the AFDC program are contained
in 45 CFR, Parts 205-206; and the regulations pertaining to the Medicaid program are contained in
42 CFR, Parts 431 and 435.
2The interim IEVS targeting regulations pertaining to the FSP are contained in 7 CFR Part 272.
2
As we learned more about the needs of the state agencies, the focus of the study changed. As
originally planned, the study was to develop and evaluate new or refined targeting strategies. Implicit
in this vision of the study was the assumption that computer matching with an appropriate targeting
strategy was cost-effective. However, during discussions with state agencies in the course of
conducting a census of states' IEVS procedures and recruiting demonstration states, it became clear
that the states questioned this assumption; some argued that some computer matches were not cost-effective
even with targeting. The targeting strategies the states wished to introduce were not fine-tuned
versions of existing targeting strategies, but radically different strategies that involved not
following up on any matches with certain databases. In response, the study evolved into a comparison
of the cost-effectiveness of matching IEVS databases using a targeting strategy with the cost-effectiveness
of not following up any matches with some databases at all.
This report presents the findings from the cost-effectiveness study of IEVS matching and
targeting strategies implemented as part of the IEVS demonstration. In total, five new matching and
targeting strategies were implemented in the two demonstration states, Michigan and Arizona. The
demonstrations were conducted between July and October 1992.
The rest of this chapter is organized as follows. Section A provides an overview of the IEVS
process. Section B discusses the states' perceptions of the IEVS process. Prior research related to
IEVS is discussed in Section C. Section D provides an overview of the demonstration and evaluation,
and Section E presents the structure of the rest of the report.
A. AN OVERVIEWOF THE IEVS PROCESS3
The lEVSprocessis the sequence of procedures involved in verifying client-reported information
under IEVS regulations. It includes preselecting cases and databases to match, as well as computer
matching, targeting, and follow up. While FSP, AFDC, and Medicaid regulations specify that certain
3The content of this section is based on the results of a state census of IEVS procedures
conducted by MPR in 1991 and reported in Allin (1991).
procedures be followed in the IEVS process, the procedures actually implemented vary considerably
from state to state. In this section, we explain the IEVS regulations and procedures, and discuss
briefly how IEVS is implemented by the states.
In discussions of IEVS procedures, technical terms often have different meanings to different
people. To avoid confusion, we define below some of the technical terms "sed in this report.
Screening}* the preselection of a subset of cases to match. Current regulations prohibit most
forms of screening.
Computer matchingis the automated process of identifying information on external databases
that pertains to a welfare case.
A match occurs when information on a welfare case is available from an external database.
Targeting is the selection of a subset of matched cases on which to follow up. Under
current regulations, targeting is permitted for ongoing cases, but not for new applicants.
A Air is a matched case designated for follow up.
Followup refers to any actions taken to (1) check that the client-reported information in the
computer match is valid, (2) verify information on the external database, (3) recompute
eligibility and benefits with n formation from the external database, (4) process claims,
disqualify recipients, and investigate fraud.4
The clientdatabase refers to the state agencies' database of welfare applicants and recipients
that is matched to the external database under IEVS. This database may also contain
information on persons who are neither applicants nor recipients, such as persons who have
become ineligible to receive benefits or persons who live in the home of the applicant or
recipient but are not eligible to receive benefits.
A client is any person listed on the client database.
The three main procedures in the IEVS system-matching, targeting, and follow up-are discussed
below.
4In the regulations, the definition of follow up also includes targeting. For ease of exposition,
we do not include targeting in our definition of follow up, but we do include all actions taken after
a case is targeted.
1. Matching
FSP regulations require states to conduct computer matches of all applicants and recipients with
six external databases:
1. State WagelnformationCollectionAgency(SWlCA) Database. This database, maintained by
a state agency such as the Department of Employment Security or the Department of
Labor, provides quarterly earnings information that employers whose employees are
covered by unemployment insurance must report each quarter. In most states, when the
SWICA match takes place, the data refer to earnings information from the quarter prior
to the previous quarter.
2. Unemploymentlnsurance(UI) Database. This database provides weekly data on UI benefits
received in the previous month. It is often maintained by the same agency that maintains
the SWICA database.
3. BeneficiaryData Exchange(BENDEX) Title II Database.5 This database, also maintained
by the Social Security Administration (SSA), provides monthly information on social
security and other benefits provided under Title II of the Social Security Act such as Black
Lung benefits, Railroad Retirement benefits, and Medicare. At the time of the match, the
data refer to benefits that will be received in the subsequent month.
4. BeneficiaryEarnings ExchangeReports (BEER) Database. This database, maintained by the
SSA, provides annual earnings information compiled from information on the IRS Form
W-2. If the match is made before about April, the data refer to the year prior to the
previous year; if the match occurs after April, the SSA sends data from the previous year
when it becomes available.
5. State Data Exchange(SDX) Database. This database, also maintained by SSA, provides
monthly information on supplemental security income (SSI) benefits. At the time of the
match, the SDX data refer to the subsequent month.
6. Internal Revenue Service (IRS) Database. This database provides annual information on
unearned income, such as interest and dividends, compiled by the IRS from the IRS Form
1099. If the match occurs before July, the data refer to the year prior to the previous
year; if the match occurs in July or later, the data refer to the previous year.
Not all states comply with the IEVS regulations and conducts matches with all six of these databases.
In a state census of IEVS procedures, respondents from three states reported not matching with
BEER, and one respondent reported not matching with BENDEX. Both states in our demonstration
'Strictly speaking, BENDEX refers to the system at SSA through which Title II and Beneficiary
Earnings Exchange Reports information is accessed. However, in this report we use the term
BENDEX to refer to only the Title II information.
did not follow up any cases from some matches. In addition, some states reported that they screened
cases before a match: not all persons on the client database were sent to be matched to the external
database. Most forms of screening are prohibited by the IEVS regulations. However, some forms of
screening are permitted. For example, state agencies are not required to request UI information on all
applicants and recipients.
a. Nonmandated Databases
The IEVS regulations require that state agencies verify social security numbers (SSNs). SSNs can
be verified by matching with the Numident file, which contains a list of SSNs and names, or with another
SSA database, such as BENDEX. Most states conduct a separate match with the Numident database.
However, in nearly all of these states the match is not coordinated with the matches with the IEVS-mandated
databases. A state may not follow up on any discrepancies between the SSN reported by the
client and the SSN on the Numident file until after a match has been conducted with the other databases.
Many states also conduct computer matches with databases other than the six mandated databases, such
as the state's Department of Motor Vehicles.6
b. Timing of Matches
The IEVS regulations require applicant matching at the "next available opportunity." This must be
at least every two weeks with the SWICA and UI databases and every month with IRS and SSA data.
Recipient matching is required less frequently-SWICA daut milt be matched quarterly, IRS data
annually, SSA data by the second month of each certification period. Not all state agencies comply with
these timing regulations.
c. Matching Procedures
The procedures involved in matching with the SWICA and UI databases vary by state. Typically,
one or more of the following procedures is used: (1) the state agency sends a tape to the SWICA, which
*More information about the use of nonmandated databases is available in an FCS report Profiles of States'
Food Stamp Operations (Research and Evaluation Associates 1992).
conducts the match and sends a tape with the matched cases back to the state agency; (2) the SWICA
sends a tape containing earnings or UI benefit information to the state agency, which conducts the match;
(3) a central computer agency conducts the match and sends reports to the state agency (this often occurs
when the state agency and the SWICA are part of the same umbrella organization); or (4) workers at the
local office conduct on-line matches with the databases.
The matches with the BEER and BENDEX databases occur at SSA. Typically, states send a tape
to SSA, which conducts the match and sends a tape with the matched information back to the state
agency. In addition, SSA keeps an "orbit file" of all SSNs that were sent previously by the state agency.7
If BEER or BENDEX information regarding SSNs on the orbit file becomes available or changes, SSA
sends another tape with the new information to the state agency or appends this new information to data
on the match tape.
Each month, SSA sends each state a tape, known as the "Treasury tape," containing data on all SSI
recipients in the state. The state agency then typically matches the data on the tape against its client
database. Eleven states also have direct access to BENDEX and SDX data (but not BEER data) at SSA
via SSA's File Transfer Management System. Through this system, SSA can provide the states with
information on SSI recipients three times a week via electronic file transmission. SSA sends a weekly
tape containing information on changes in BENDEX and SDX data to those states that do not have direct
access to the File Transfer Management System. In addition, all states have access to third-party query
(TPQY) through which they can request a BENDEX or SDX match directly from a computer terminal
at an SSA district office or by completing speciai ds they send to SSA. Information can be obtained
via TPQY in about one or two weeks.
The match with the IRS database occurs at IRS. The state agency sends a tape of client SSNs to
IRS, and IRS matches the tape with its databases and sends a tape containing the matched information
back to the state.
7The SSNs stay on the orbit file until the state explicitly requests their removal.
7
2. Targeting
The 1988 interim amendments to the IEVS regulations permitted targeting of recipients, not
applicants. According to the IEVS state census, most states have implemented a targeting strategy
for at least one database, and over three-quarters of all states have implemented targeting strategies
for at least three of the IEVS-mandated databases. The matches most frequently targeted are those
conducted with the BEER, IRS, and SWICA databases. Only five states have not implemented any
targeting strategies and so follow up on all matches from all IEVS-mandated databases.
The types of targeting strategies used by the state agencies vary considerably by state and by
database. However, the strategies typically consist of a number of rules, most of which fall into one
of four categories:
1. Exempt Casesfrom Follow Up on the Basis ofIndividual or Case Characteristics. This
rule exempts individuals or cases from follow up on the basis of their benefits-related
and/or demographic characteristics. Examples of this type of rule include (a) follow
up only on cases in which a person is currently receiving benefits and (b) do not
follow up on persons under a certain age.
2. Exempt Specified Data Items from Follow Up. Under this rule, certain information
items are not followed up because the information duplicates information available
from other databases.
3. Use of a Tolerance Threshold. Individuals or cases are targeted for follow up if the
income reported on the external database exceeds a given threshold.
4. Use ofa Discrepancy Threshold. Individuals or cases are targeted for follow up if the
difference between income reported on the external database and income on the
client database exceeds a given threshold. A more general variant of this rule is to
follow up if there is any discrepancy between an item of information included in the
two databases-the item may be the dollar amount of income, the receipt of a type of
income, or the employer name.
The IEVS census reported that the most common targeting strategy for the SWICA, UI,
BENDEX, and SDX databases is the use of a discrepancy threshold. The size of the threshold varies
by database and by state. The most common targeting strategy used for the BEER database is to
examine income data that are not also contained on the SWICA database-information on earnings
8
from out-of-state activities, pensions, agricultural work, and self-employment. As the BEER data are
annual and one or two years old at the time of the match, states would require an extensive benefit-history
file in order to compare income from the same periods on the BEER database and the client
database. Hence, most states do not use a discrepancy threshold as part of their targeting strategy
for BEER. The predominant strategy for the IRS database is to follow up on matches only if the
amounts of one or more of the types of unearned income are above a certain tolerance threshold.
States rarely use a discrepancy threshold for the IRS database for the same reason that they do not
use one for the BEER database-the data are annual and out of date.
3. Follow Up
A case designated for follow up is called a hit. There are two main procedures used to send
information on hits to local offices: (1) hard-copy reports (IEVS reports) and (2) messages, or
"alerts,"that appear on the caseworker's computer screen. These procedures may vary by database.
The caseworker is usually responsible for most of the follow up. However, clerical staff,
specialized caseworkers, and fraud investigators may also be involved. Follow-up procedures involve
the activities listed below:
1. Reviewing the Information in the CaseJUe. This involves checking that the client-reported
information used in the computer match is valid and was correctly entered
into the computer; checking that the information had not already been received from
the client or via another match, and whether the computer system has not already
made this check; and checking whether there is a discrepancy between the
information reported by the client and the information provided on the external
database.
2. Contactingthe Client. The caseworker may contact the client to (1) ask for verification
of income, (2) obtain permission for the collateral contact to release information, (3)
inform the client Uiat there is a discrepancy between income he or she reported and
income on the external database, or (4) inform the client that an action is going to
be taken as a result of the IEVS match.
3. Verifyinglnformationfrom the External Database Ifa DiscrepancyExists. This involves
contacting the client and/or making a "collateral contact" with the client's bank or
employer. In some states the computer system produces letters to the collateral
contacts if the caseworker enters into the system the necessary information. Data on
SSI, Title II, and UI benefits are considered already verified and do not require third-party
verification.
4. RecomputingEligibility and Benefits Using Informationfrom the External Database In
many states this can be performed by the computer. After recomputing the eligibility
and benefits, the caseworker inputs the new information into the client database.
5. ProcessingOaims,DisqualifyingRecipients,andInvestigatingFraud. Processing claims
involves computing total overpayments and initiating action to recover the
overpayment. Depending on the state, the caseworker, a specialized worker, or the
state agency's collection division is responsible for processing claims. In most states
a special unit is responsible for investigating fraud.
IEVS regulations require that the states complete follow-up procedures within 45 days of receipt
of the matched information. If follow up is delayed because the state is waiting for information from
collateral contacts, the state is permitted to follow up 20 percent of the cases in more than 45 days.
Estimates of how many cases are followed up within the 45-day limit vary widely from state to state,
but in most states follow up procedures are completed within 45 days for two-thirds to three-quarters
of the cases designated for follow up.
B. PERCEPTIONS OF THE IEVS PROCESS
This section describes some of the states' concerns about IEVS and some changes to the IEVS
regulations suggested by the states. This section draws upon the state census, a report by the
American Public Welfare Association (1989), and discussions with state staff at potential and actual
demonstration sites.7
Most of the agency staff believe that computer matching is useful and generally cost-effective.
However, they perceive a number of problems with some of the information provided by the IEVS-mandated
databases. The three most often cited problems are:
1. Out-of-DateData. BEER and IRS data can be up to 30 months out-of-date. SWICA data
can be up to 6 months out of date. Thus, income on those databases ould refer to
periods of time when the client was not receiving benefits. Moreover, verifying out-of-date
7These concerns were expressed prior to our study.
10
information is more difficult. The benefit data provided by BENDEX, UI, and SDX are
kept up-to-iiate.
2. Data Are Aggregatedover Different Time Periods While clients report monthly income,
income collected by the source agencies are often aggregated over a longer period of time.
BEER and IRS data provide annual income data and SWICA provides quarterly income
data. Thus, it is difficult to make direct comparisons between income reported by the
client and income reported by the external data source.
3. Duplicated Data. Most of the information provided by BEER duplicates data provided
by SWICA. The only income data that are provided by BEER, but not SWICA, are
(1) self-employment income, (2) out-of-state wages, (3) federal and military employees
wages, and (4) agricultural earnings.
Although few states had conducted detailed cost-effectiveness studies, over half of the states
responding to the state census perceived that matches with the SWICA, UI, BENDEX, and SDX
databases were cost-effective. The SWICA match was the most popular-80 percent of the states
viewed the SWICA match as cost-effective. Staff argued that information on earned income was
useful in detecting incorrect benefit and eligibility determinations and easy to use. However, other
respondents were more critical of the match and argued that the SWICA information was out-of-date
and was costly to follow up because it required contact with employers. UI, BENDEX, and SDX
matches were popular because the data are up-to-date, monthly, and do not require third-party
verification.
The least popular matches were those with the IRS and BEER databases. Only 12 percent of
states perceive the BEER match to be cost-effective. Problems with these matches include: (1) the
data are out-of-date, (2) the data are annual, (3) much of the BEER data duplicates data received
from the SWICA match, (4) there is a long turnaround time for receiving the data, and (5) there are
stringent security requirements for using these data.
A common view among state agency staff is that the IEVS regulations do not allow the states
sufficient flexibility to use computer matching in the most cost-effective way. This perception of the
burden of the IEVS regulations is reflected in the evidence we found of noncompliance with the
regulations. Neither of the two demonstration states were in full compliance with the IEVS
11
regulations prior to the demonstration. Changes in the current IEVS regulations that have been
suggested include:
■ Allow states to conduct matches with only those databases viewed as cost-effective
• Allow states to use screening, that is, to send only selected clients to be matched with the
external database
• Allow states to target applicants
C. PRIOR COST-EFFECTIVENESS STUDIES OF IEVS
No previous multi-state study of IEVS has estimated the cost-effectiveness of computer matching
with targeting under IEVS. Greenberg and Wolf (1986) estimated the cost-effectiveness of computer
matching with earnings data, but their study was conducted before IEVS was established. Puma
(1989) examined the cost-effectiveness of using computers to match external data sources with data
reported by applicants, but not recipients. Some states have conducted their own cost-effectiveness
studies, but in general these studies lack sophistication and are difficult to interpret.
Greenberg and Wolf estimated the cost-effectiveness of computer matching with state earnings
databases in four sites: Mercer County, New Jersey; Camden County, New Jersey; San Joaquin,
California; and New Hampshire. These sites were chosen because they had well-functioning wage
matching systems. This study was conducted in 1982, prior to the introduction of IEVS. Greenberg
and Wolf found that wage matching was cost-effective in all four sites, with the cost-benefit ratio
varying from 1.6 to 2.5.
Puma (1989) estimated the cost-effectiveness of computer matching of applicants in nine sites.
Only offices in which information from the matches could be made available to the caseworker before
certification were included in the study. This study was conducted in 1987, before applicant matching
was mandatory. (No site did an IRS applicant match.) Puma found that the cost-benefit ratio,
averaged over all matches, was greater than one in all sites. Overall, the cost-benefit ratio was just
12
under four. The SW1CA match was the most cost-effective. The SDX match was the only match
that, on average over all sites, not cost-effective.
During the census, states were asked to submit to MPR any cost-effectiveness studies of IEVS
matching that they had conducted. Studies were received from 13 states. In general, these studies
were not highly sophisticated, and among those of moderate to high quality, results varied widely.
Although no match was found to be cost-effective by all studies, a majority found the SWICA, UI,
BENDEX, and SDX matches to be cost-effective and the BEER and IRS matches not to be cost-effective.
Michigan-one of our demonstration sites-conducted a cost-effectiveness study of matching with
the SWICA database (Ward and Smucker 1990). This study is of fairly high quality. Ward and
Smucker found that while matching recipients against SWICA data is cost-effective, matching
applicants with SWICA data is not cost-effective. The ratio of savings to cost for the recipients varies
from between 3 and 15 depending on the targeting strategy used-the higher the tolerance threshold
used in the targeting strategy, the higher the savings-to-cost ratio. However, the savings-to-cost ratio
for applicants is less than 0.8. This is contrary to Puma's findings that applicant wage matching is
cost-effective. We discuss Puma's study in more detail in Chapter VIII.
D. AN OVERVIEWOF THE IEVS TARGETING DEMONSTRATION AND EVALUATION
The demonstration involved introducing a total of five new IEVS matching and targeting
strategies in two states, Michigan and Arizona. The criteria used for selecting the states were that
the state was interested in committing the necessary resources to conduct the study, and the state was
as similar as possible to the majority of states in its IEVS system and in demographic and economic
factors. Although states vary considerably in the operation of IEVS, we consider the operation of
IEVS in either Michigan or Arizona to be fairly representative of the other states.
The Arizona demonstration focused on recipients, while the Michigan demonstration focused on
applicants. Arizona introduced three new matching and targeting strategies: (1) matching recipients
13
with the SWICA database and using a new targeting strategy, (2) following up recipient matches with
the BEER database and using a new targeting strategy, and (3) following up recipient matches with
the IRS database and using a new targeting strategy. Michigan introduced two new IEVS strategies:
(1) not following up any match of an applicant with the SWICA database and (2) increasing the
tolerance threshold for the IRS match.
In the evaluation, we estimated the cost-effectiveness of the SWICA recipient match, the BEER
match, and the IRS match in Arizona. In Michigan, we estimated the cost-effectiveness of the
SWICA applicant match (or not conducting the match), and the cost-effectiveness of the IRS match.
We also measured the cost-effectiveness of the UI, BENDEX, and SDX matches in Michigan. No
changes were made to the matching or targeting strategies used with these databases.
Cost-effectiveness is measured as the ratio of program savings from IEVS to the cost of
matching, targeting, and following up under IEVS. Program savings, the numerator of the ratio,
includes FSP and AFDC benefits that would have been erroneously paid to clients, benefits that were
erroneously paid to clients in the past and are recovered, and the cost of administering cases that
would have been opened or maintained on the rolls in the absence of IEVS. The cost of IEVS, the
denominator of the ratio, includes the cost of matching and targeting, the cost of the caseworkers'
follow ups, the cost of investigating fraud, and the costs of establishing and collecting claims. We also
estimated the cost of developing new the new matching and targeting rules. However, as the
development costs are one-time-only costs, we do not include them in the savings-to-cost ratio.
We measure cost-effectiveness from the perspective of the federal and state governments.
Hence, we do not include the costs of IEVS to the clients or to third parties, such as employers and
financial institutions. We also include only directly measurable savings and costs. Hence, we do not
include any measure of the savings that may result because IEVS deters clients from misreporting
their income or because caseworkers' morale has improved.
14
It is important to note that cost-effectiveness is not the only possible criteria by which matching
and targeting under IEVS can be judged. While the savings-to-cost ratio indicates the amount of
program savings that can be expected for every dollar spent on matching and targeting, it does not
provide any information about the percentage of all errors in benefit and eligibility determination that
are detected. The most cost-effective matching and targeting strategy may be one that follows up on
only a few cases that are likely to yield large savings, while allowing many smaller errors go
undetected.
E. AN OVERVIEWOF THE REPORT
This volume of the report (Volume I) provides an overview of the study and discusses the main
results. Volume II consists of a series of appendices that provide more details about the IEVS
procedures used in the two states and the measures of savings and costs used in the study.
Chapter II of this report describes the two demonstration sites, Michigan and Arizona. Chapter
III describes the evaluation design. Chapter IV presents our framework for estimating costs and
savings. It discusses the measurement issues and the assumptions underlying our measures. The last
four chapters present the results of the evaluation. The cost-effectiveness of IEVS depends on three
factors: (1) the proportions of all cases that are matched, targeted for follow up, and lead to savings,
(2) the size of the savings realized as a result of follow up, and (3) the cost of matching, targeting,
and following up cases. Chapters V, VI, and VII examines each of these factors, respectively.
Chapter VIII presents our estimates of the cost-effectiveness of IEVS matches and discusses our
overall findings from the IEVS demonstrations.
15
BLANK PAGE
/I
II. THE ENVIRONMENT: ARIZONA AND MICHIGAN FOOD-STAMP AGENCIES
The IEVS demonstrations were conducted at two food-stamp agencies: Arizona and Michigan.
These agencies were chosen to take part in the demonstration because (1) they were interested in
participating in the study and willing to commit the necessary staff to the demonstration and (2) their
IEVS procedures were fairly representative of IEVS procedures used in other states. In order to
provide some context for the results of the evaluation, this chapter describes the key characteristics
of the two food-stamp agencies. We begin in Section A by describing some general features of the
FSP in the two states. Section B provides a summary of the IEVS procedures in the two states.
Finally, Section C provides a description of the matches that took place during our demonstration.
A. CHARACTERISTICS OF THE FOOD-STAMP AGENCIES IN ARIZONA AND MICHIGAN
In Arizona, the Food Stamp and AFDC programs are fully integrated. Both programs are
administered by the Family Assistance Administration (FAA) within the Department of Economic
Security (DES). The administration of the Medicaid program is the responsibility of the Arizona
Health Care Cost Containment System (AHCCCS) Administration, but Medicaid cases that are also
FSP or AFDC cases are administered by the FAA. In Michigan, all three programs are fully
integrated and administered by the Department of Social Services (DSS). In both states, most
caseworkers work with cases in all three programs.
In July 1992, Arizona's caseload was about 169,000. This is an average-sized state caseload; 20
states had larger caseloads. In comparison, Michigan had a caseload of about 406,000, larger than
that of all but seven other states. Both states, like most others, have experienced a growth in their
caseload over the past three years. Arizona experienced a particularly large growth both in absolute
and relative terms. Between fiscal year 1991 and 1992, the food-stamp caseload in Arizona rose by
nearly 30,000 or 21 percent, while in Michigan the caseload remained fairly steady over this time
period.
17
The average number of cases per caseworker in Arizona is about 152, compared to 231 in
Michigan. The IEVS Census found that the caseload per caseworker varied considerably by state,
ranging from 100 to S23 with an average of 253 cases per caseworker.
The quality control error rates vary considerably by state. In fiscal year 1991, the percentage of
cases with errors detected in a state during the quality control process varied between 4 and 14
percent, and on average was 9.8 percent. Both of the demonstration agencies had error rates that
were about average: in fiscal year 1991, the error rate was 10.9 percent in Arizona and 8.9 percent
in Michigan.
Table II 1 describes some characteristics of the food-stamp households in the two demonstration
states and in the U.S. as a whole estimated from the 1991 Food Stamp Quality Control database.
Important differences between the two states include.
A smaller proportion of the food-stamp households also receive AFDC in Arizona (33
percent) than in Michigan (51 percent) and the U.S. as a whole (41 percent). Also, a
smaller proportion of the food-stamp households in Arizona receive Medicaid than in
Michigan or the U.S. as a whole.
More food-stamp households have earnings in Arizona (29 percent) than in Michigan (15
percent) or the U.S. as a whole (20 percent). The average amount of earnings received
by food-stamp households in Arizona ($196) is larger than in Michigan ($72) and in the
U.S. as a whole ($117).
The proportion of households with elderly persons is slightly higher in Arizona (11
percent) than in Michigan (10 percent), but it is lower in both states than in the U.S. as
a whole (16 percent).
Arizona has a larger proportion of Hispanic and Native American households (28 and 14
percent, respectively) than does Michigan (2 and 1 percent, respectively). Michigan has
a higher proportion of African-American households (46 percent) than does Arizona (8
percent).
The average length of the certification period in Arizona (6.6 months) is much shorter
than in Michigan (12.6 months) and in the U.S. as a whole (9.7 months).
18
TABLE II. 1
CHARACTERISTICS OF FOOD STAMP HOUSEHOLDS IN
MICHIGAN, ARIZONA, AND U.S. AS A WHOLE, 1991
Characteristic Arizona Michigan U.S. as a Whole
Percent of Households wi ;
AFDC Benefits
Medicaid Eligibility
Positive Gross Income
Positive Earnings
Expedited Service
Elderly Persons
Children under 18
No Male Adult Present
Percent Distributions
Income as Percent of Poverty
0
1-50
51-100
101-150
Over 150
Race of Household Head
White, non-Hispanic
Black, non-Hispanic
Hispanic
Native American or Alaskan Native
Other/Unknown
Average Values
Gross Income
Earnings
Household Size
Food Stamp Benefit
Number of Persons with Earnings in Household
Length of Certification Period (months)
Sample Size
33.2 50.9 40.5
42.3 62.5 59.1
86.1 95.4 91.7
29.2 14.6 19.8
8.4 4.7 5.0
10.7 9.8 16.4
63.9 58.4 60.4
56.2 65.0 64.7
13.9 4.6 8.3
45.2 42.4 33.3
31.2 47.0 50.6
9.6 5.9 7.7
0.2 0.0 0.2
45.4 50.9 45.7
7.8 45.5 35.0
27.7 2.1 13.7
13.5 0.6 1.1
5.6 0.8 4.5
$434 $430 $464
$196 $72 $117
2.9 2.4 2.6
$191 $166 $161
0.3 0.2 0.2
6.6 12.6 9.7
1,214 1,644 63,692
SOURCE: 1991 Food Stamp Quality Control database.
19
B. IEVS PROCEDURES IN ARIZONA AND MICHIGAN
The IEVS process is fairly well automated in both states. In both states, the caseworker can use
the system to recompute eligibility and benefits for a defined past period. According to the IEVS
Census, 28 other states have this capability (Allin 1991). As in most states (all but seven), targeting
is completely automated in both states. Twelve states have some automated support for follow up-usual
ly to produce a letter for the client or collateral contact. Arizona has some automated support
for producing letters to the collateral contacts, but follow up is almost completely manual in both
Arizona and Michigan.
As in the majority of states, in Arizona and Michigan the matching process for the food stamp,
AFDC, and Medicaid programs is coordinated. That is, one tape containing SSNs for individuals
participating in one or more of these programs is sent to the external agency.
Below, we describe the matching, targeting, and follow-up procedures used in Arizona and
Michigan. For brevity, we include only a discussion of the procedures used for matches in our
demonstrations. A fuller description of the IEVS procedures in the two states is given in Appendix A
of Volume II.
1. Arizona
The IEVS procedures in Arizona are summarized in Table II.2.
a. Matching
Arizona has conducted matches with all six IEVS-mandated databases. However, prior to our
demonstration, Arizona had discontinued its SWICA quarterly tape match and was conducting only
on-line matches with the SWICA database. And although the state was conducting matches with the
BEER and IRS databases, no matches from these databases were followed up. The BEER and IRS
matches were not followed up because Arizona believed they were not cost-effective.
20
TABLE D.2
SUMMARY OF IEVS PROCEDURES IN ARIZONA
Maich
Match Took
Place Prior to
Demonstration?
Frequency of
Match Which Clients Are Matched? Process Targeting?
Form in Which
Information is Sent
to Caseworkers
SWICA
On-line
Tape
Yes
No
NA Applicants, clients at recertification. Direct on-line access
and clients who report any changes in
circumstances
Quarterly All clients Tape match at FAA
No
NA
Print of screen (on-line
access)
Hard-copy report
K>
BEER Yes Monthly New clients and any clients for whom Tape match at SSA
there is new information on the (response tape returned
system; to FAA via AHCCCS)
SSA sends information on all clients
whose situation has changed
No matches
followed up prior
to demonstration
Hard-copy report
IRS
All Clients
New Clients
Yes
Yes
Annually All clients who are active
Monthly New clients
Tape match at IRS
Tape match at IRS
No matches
followed up prior
to demonstration
No matches
followed up prior
to demonstration
Hard-copy report
Hard-copy report
NA = Not applicable.
o/
During our demonstration, Arizona reinstated the quarterly SWICA recipient match and followed
up matches from the BEER and IRS databases. No matches were followed up from BENDEX.
Only the quarterly SWICA recipient match and the BEER and IRS matches were included in the
demonstration.
The client database in Arizona includes applicants, recipients, and persons who do not receive
benefits but who reside in the households of applicants or recipients. Unless otherwise stated,
Arizona's FAA requests information from the external database on all persons on the client database.
SWICA. Employers in Arizona are required to report their employee's quarterly earnings to the
Administration of Unemployment Insurance, which is a division within the Arizona DES. The
SWICA database, known in Arizona as the "base wage" database, contains information on the SSN
of each employee, the employees' quarterly earnings, and the employers' names and addresses.
Two types of matches can be conducted with the SWICA database, an on-line match and a tape
match. As the SWICA database is in effect "in-house" at the DES, staff can access the database
directly via on-line commands from their computer terminals. The quarterly SWICA recipient match
is a tape match that takes place at the FAA.
BEER. The BEER match is coordinated with the BENDEX match. Arizona sends a tape,
known as the "BENDEX request tape," to the SSA each month. This tape contains the SSNs of all
new clients who have not been sent previously. The SSA matches the SSNs on the BENDEX
request tape with its BEER database in addition to its BENDEX database. The FAA receives two
tapes a month containing BEER information-one consisting largely of matched information on new
clients and the other containing new matched information from the orbit file on clients who were
previously sent to SSA. Depending on when in the month new data on clients on the orbit file is
received by SSA, information on clients who were previously sent to SSA may be included on the first
tape containing matched information on new clients or on the orbit-file tape.
22
IRS. Once a year, Arizona sends a tape to the IRS containing the SSNs of clients on the client
database who are active for either the food stamp or AFDC programs. As not all clients on the
database are sent to be matched, this is a form of screening. After a month or two, Arizona receives
a tape back from the IRS with the matched information. In addition, Arizona sends a tape to the
IRS each month containing the SSNs of all new clients (that is, those clients who have not been
previously matched with the IRS database). About one month later, Arizona receives a tape from
the IRS that contains information on any matches with the IRS database.
b. Targeting
In the past, Arizona conducted targeting on the matches with all six IEVS-mandated databases.
However, prior to the demonstration, Arizona was targeting only its BENDEX match, UI tape match
and SDX match.
c. Follow Up
This section explains the typical follow-up procedures in Arizona. The specific procedures do,
however, vary across local offices.
The manner in which caseworkers are notified of a hit varies by database. Caseworkers are
notified of a hit from the SWICA, BEER, and IRS matches by a hard-copy report. The SWICA
reports are sent from the state office to the local offices, where they are distributed to caseworkers.
For security reasons, BEER and IRS reports are locked in cabinets. Caseworkers must sign for a
report when it is removed from the cabinet and must return the report to the cabinet within a few
hours.
While the IEVS regulations require that caseworkers follow up a hit within 45 days, caseworkers
in Arizona are requested to complete the follow-up procedures in less time. The time allowed to
complete follow up varies by database. The SWICA hits must be followed up within 10 calendar days
after the worker receives the report; the UI, BENDEX, and SDX hits within 14 days; and the BEER
23
and IRS hits within 30 days. Discussions with some field staff in Arizona suggest that these time
limits are not always met, and in some instances, a hit may not be followed up at all.
If the caseworker discovers that the client is currently receiving an incorrect benefit payment,
he or she will recompute die new benefit amount using the new income information. However, if
the caseworker suspects fraud or discovers an overpayment, he or she will complete a referralform
(FA-S26), which is given to an overpayment writer. An overpayment writer is a caseworker who
specializes in calculating overpayments and establishing claims. It may take a couple of weeks for the
overpayment writer to calculate the amount of the overpayment. The overpayment writer completes
a form, FA-529, with information on the case. This form is sent to the Office of Accounts Receivable
and Collections (OARC), which investigates the cause of the overpayment (fraud, client error, or
agency error), proceeds with any legal action, and arranges for the collection of the claim.
2. Michigan
The IEVS procedures in Michigan are summarized in Table II.3.
a. Matching
Michigan conducts matches with all sixIEVS-mandated databases. All matches except the BEER
and the SWICA recipient match are included in the demonstration. During our study period, none
of the research-sample cases were subject to the SWICA recipient match and no BEER match was
processed.1
Before a case is found to be eligible for benefits, only the SSN of the person who applied for
benefits is entered into the client database. That is, applicant matches in Michigan do not include
the SSNs of persons who reside in the same household as the applicant. Unlike Arizona, which
requests information on everyone in the household, Michigan requests information on only one
'All BEER tapes received during the year are processed at one time. The annual processing did
not occur during our study.
24
Match
SW1CA
Applicants
Recipients
UI
Applicants
Recipients
BENDEX
SDX
IRS
Frequency of
Match
TABLE D.3
SUMMARY OF EEVS PROCEDURES IN MICHIGAN
Which Clients Are Matched?
Twice weekly New applicants who have not been active
within the past 105 days
Quarterly All recipients who have received benefits for
the past three months
Twice weekly New applicants who have not been active
within the past 105 days
Monthly Clients who report receiving some unearned
income, clients who have lost employment
within the past three months, and clients who
applied for welfare benefits less than three
months previously
Monthly New recipients and one-third of the caseload
SSA sends information on all clients whose
situations have changed
Weekly All applicants and recipients
Process
Tape match at MESC
Tape match at MESC
Tape match at MESC
Tape match at MESC
Tape match at SSA
Receive tapes from SSA
Monthly Applicants, and recipients due for
redetermination within 3 months
Tape match at IRS
■Notification is sent first to the client. Caseworkers receive a printout listing those clients who were notified.
NA = Not applicable.
Targeting?
No
Yes
Yes
Yes
Yes
Yes
Yes
Form in Which
Information is Sent to
the Caseworkers
Hard-copy report
Hard-copy report
Hard-copy report
Hard-copy report
Hard-copy report
Hard-copy report
NA'
OU"
person in each household for the applicant matches. For recipient cases, information is requested
only on persons who are eligible to receive benefits.
SWICA. Employers are required to report the quarterly earnings of employees who are covered
by Unemployment Insurance to the Michigan Economic Security Commission (MESC). Michigan
performs an applicant match and a recipient match with these SWICA data. Michigan's DSS does
not have direct on-line access to the SWICA database, so both matches involve sending a tape to the
MESC. With the exception of New York, Michigan is the only state required to pay for each match
with the SWICA database.
A tape of the SSNs of "new" applicants is sent about twice a week to the MESC. A "new"
applicant is defined as an applicant who has not previously been active in the previous 105 days. As
a client is defined as active if they have either applied for or receive welfare benefits, this screens out
applicants who previously applied for benefits within the past three months. The rationale for this
screen is that as the SWICA database is updated only on a quarterly basis, it will provide identical
information on a person for three consecutive months. This screening rule could, however, screen
out useful information on a person who applies more than once in a three-month period if these
applications were made in two different quarters.
The MESC conducts the match and returns the matched information on a tape within two or
three weeks. Within a few days of receiving the tape, DSS produces reports of the hits and sends
them to the caseworkers.
UI. The MESC also collects information on UI benefits. Although DSS must pay for the
SWICA match, it does not pay for the UI match. An applicant match and a recipient match are
conducted with the UI database.
The tape of all "new" applicant SSNs sent twice a week to the MESC to be matched to the
SWICA database is also matched to the UI database. The MESC conducts the match and returns
the matched information within about a week. At the beginning of each month, DSS also sends to
26
the MESC a tape containing the SSNs of all clients who report receiving some form of unearned
income, who do not report receiving UI benefits but have lost employment within the past three
months, or who do not report receiving UI benefits but have applied for welfare benefits less than
three months previously.2 The rationale for including persons who report receiving some unearned
income is th»t the client database does not include a field that contains UI benefit information;
hence, any UI benefit data is entered as unearned income.
BENDEX. Michigan sends a BENDEX request tape to ihe SSA during about the third week
of the month. The request tape contains SSNs of all new recipients who have become active that
month and have not already been matched to the BENDEX database. Michigan does not currently
include applicants on the BENDEX request tape. The BENDEX request tape also contains the
SSNs of about one-third of the clients on the client database (chosen by the digits in the case
number).3
The SSA sends two tapes back to Michigan. Both arrive around the middle of the month. The
first tape consists primarily of information on the clients included on the request tape. The second
tape contains any new information on clients who were sent previously and were kept by the SSA
on its orbit file. Within two or three days of receiving the tapes, Michigan produces reports of hits
to be sent to the caseworkers. If, during the application process, a caseworker suspects that a client
is receiving unreported Title II (or SSI) benefits, he or she can send a TPQY card to the SSA.
SDX. At the end of each month, Michigan's DSS receives a tape-the Treasury tape-containing
infotmation on all persons in Michigan who have applied for SSI, receive SSI, or have received SSI
in the past. Because Michigan does not have direct access to the SSA File Transfer Management
System, it does not electronically receive updated information on SSI recipients three times a week.
2As all clients are not sent to be matched, this is a form of screening. However, this form of
screening for the UI match is explicitly permitted by the IEVS regulations.
3As SSA sends a tape from the orbit file containing data on any clients whose benefits have
changed, it is redundant to send SSNs of clients who have previously been sent to SSA. However,
this procedure began before SSA sent data from the orbit file and has not yet been changed.
27
Instead, each week Michigan receives a tape from SSA that contains any new information on persons
on the SDX database.
IRS. Around the second week of each month, Michigan sends the IRS a tape containing SSNs
of all current applicants and SSNs of recipients who are due for redetermination within about three
months. About two or three weeks later, the IRS returns to Michigan a tape with the matched
information.
b. Targeting
Michigan conducts some form of targeting on all of its matches except the SWICA applicant
match. The targeting strategies are implemented at the state office. There is no difference in the
targeting strategies by welfare program. The following explains Michigan's targeting strategies for
each database.
SWICA. No targeting strategy is applied to the applicant match with the SWICA database-all
matches are designated for follow up. However, discussions with agency staff in Michigan suggest
that caseworkers do not currently have time to follow up a substantial proportion of the matches.
UI. Michigan applies the same targeting strategy to its applicant and recipient matches with the
UI database. A match is followed up only if both of the following rules are satisfied:
■ The client is currently active for a program administered by DSS.
The UI database reports that the client has applied for UI benefits in the past 30
days, has received Ui Lenefits in the past 60 days, or has returned to work within the
past 90 days.
The first rule exempts from follow up clients who, at the time the targeting strategy is applied, have
neither applied for nor receive benefits. These clients are not followed up because they cannot lead
to any change in current benefits or eligibility status and because it is difficult to recover
overpayments from clients who are no longer active. The second rule exempts from follow up those
cases for which the receipt of UI benefits is unlikely to affect current benefits or eligibility. However,
28
it does not exempt from follow up clients who have recently stopped receiving Ul benefits because
this may indicate that the client nas recently started work and has earned income.
BENDEX. The targeting strategy for the BENDEX match is to follow up on matches only if
both of the following rules are satisfied:
■ The client is currently active for a program administered by the DSS.
• The client is currently receiving Title II benefits.
The first rule exempts inactive clients from follow up. The second rule exempts from follow up
clients who do not currently receive any Title II income.
SDX. Matches are followed up by the caseworker only if both of the following targeting rules
are satisfied:
The client is currently active for a program administered by DSS.
The client has applied for SSI, is currently receiving SSI benefits, has just had SSI
benefits denied or terminated, or has had a change in address or living arrangements.
These targeting rules exempt from follow up any clients that received SSI benefits in the past,
but are no longer receiving benefits and for which the information consequently cannot lead to a
change in current benefits or eligibility.
IRS. The targeting strategy for the IRS match used prior to the demonstration was is to follow
up only if both of the following targeting rules were satisfied:
The client is currently active in a program administered by DSS.
• The IRS reports levels of unearned income that exceed specified thresholds:
- Interest income exceeds $100 or
- Dividends exceed $100 or
- Agricultural subsidies exceed $100 or
• Capital gains exceed $100 or
- Stock dividends exceed $100 or
- Stock liquidations exceed $100 or
29
Savings bond interest exceeds $100 or
Income from rental properties exceeds $100 or
Royalties exceed $100 or
Bond liquidations exceed $100 or
Prizes and awards exceed $100 or
IRA distributions exceed $100 or
Profit sharing distributions exceed $100 or
Real estate sales exceed $100 or
State income tax refunds exceed $300
c. Follow Up
The follow-up procedures in Mxhigan also vary by local office and by database. For example,
in some offices certain caseworkers process applicant matches, and others process recipient matches;
in other offices caseworkers process both applicant and recipient matches.
Caseworkers are notified of a hit from the SWICA, UI, BENDEX, and SDX matches by a hard-copy
report from the state office. For a hit from the BEER or IRS match, the state office sends a
letter to the client notifying him or her that DSS has been notified of a source of income. The client
is required to schedule an interview with the caseworker within a couple of weeks. If the client fails
to do so, he or she is disqualified from the program and the case is closed. The caseworker receives
a printout that lists those clients who have been sent a letter notifying them of the BEER or IRS
information.
Caseworkers are requested to complete the follow up of all hits within the 45 days specified by
the IEVS regulations. They begin by checking the information in the casefile. If verification is
required, the caseworker sends a letter to a collateral contact. The BEER and IRS matches are not
verified until after the client has given the caseworker the letter from the state office about the
match.
If the estimated amount of overpayment is less than $200 or if fraud is not suspected because,
for example, the agency itself made an error in benefit payments, the caseworker sends the client a
letter about the overpayment. If the client does not dispute the overpayment, the caseworker enters
the amount of the overpayment into a special system on the mainframe computer, the Automated
30
Recoupment System (ARS). The ARS automatically calculates the recoupment, which is the amount
by which the monthly benefit is reduced to recover the overpayment.
If the estimated amount of the overpayment is between $200 and $500 and fraud is suspected,
the caseworker transfers the case materials to a caseworker who is specialized in dealing with
overpayments, a designated staffperson (DSP).4 The DSP checks the amount of the overpayment
and investigates whether there was fraud. If the investigation shows that fraud is a possibility, the
DSP arranges for a hearing. It takes about a month for these procedures to be completed.
If the estimated amount of the overpayment is $500 or more and fraud is suspected, the case is
referred to the Office of the Inspector General (OIG). OIG conducts an investigation and arranges
for any legal proceedings. If the case is referred to the OIG, it can take months or even years before
the exact amount of the overpayment is established.
C. MATCHES DURING THE DEMONSTRATION
This section describes the matches that took place as part of the demonstration. In Arizona, two
quarterly SWICA matches took place. Usually, only one match would take place during our study
period. However, in order to increase the number of hits in our sample, an early SWICA match was
postponed by a few months so it would take place during our study. The first SWICA match, which
took place in August, involved matching the client database against earnings data for the first quarter
of 1992 (January, February, and March). The second SWICA match, which occurred in September,
involved matching the client database against earnings data from the second quarter of 1992 (April,
May, and June). Arizona sent three monthly BEER request tapes to SSA in July, August, and
September, respectively. These tapes were processed (targeted and reports produced) in August,
September, and early November, respectively. Arizona processed two IRS matches during our study.
The first was the annual IRS match, which includes all active clients. The request tape for this match
was sent in early July. The response tape was processed in late October. The second was a monthly
4In some offices where there is no DSP, the caseworker would perform the tasks of the DSP.
31
match including only new clients. The request tape for this match was sent in early October, and the
response was processed in November.
In Michigan, the SW1CA and UI applicant matches took place twice a week throughout the
study. The first reports of hits were produced in August for applicants sent in July. In total, 19
SWICA and UI applicant matches took place. The recipient UI match took place twice for cases in
our sample, in mid-September and mid-October. The BENDEX match took place three times during
our study, in August, September, and October. During the study, Michigan received four Treasury
tapes and 20 weekly SDX tapes. Michigan received IRS response tapes for cases in our sample in
September and October.
32
m. EVALUATIONDESIGN
The evaluation was designed to estimate the cost-effectiveness of new IEVS matching and
targeting procedures. Cost-effectiveness is measured as the ratio of program savings from IEVS to
the costs incurred because of IEVS. Program savings, the numerator of the ratio, includes FSP and
AFDC program benefits that would have been erroneously paid to clients, benefits that were
erroneously paid to clients in the past and are recovered, and the cost of administering cases that
would have been opened or remained on the rolls in the absence of IEVS. The cost of IEVS, the
denominator of the ratio, includes the cost of the time caseworkers devote to follow-up activities; data
processing costs; the cost of investigating fraud, establishing and collecting claims, and conducting
hearings and prosecutions. We also estimated the cost of developing new matching and targeting
rules.
In this chapter, we discuss the design of the evaluation, including the general approach, the
research sample, the new matching and targeting strategies tested in each state, and the data
collection procedures. Table III. 1 provides an overview of the evaluation design in each state.
A. THE GENERAL APPROACH
Each demonstration state, Arizona and Michigan, agreed to develop and operate, for a four-month
period, a total of five new versions of IEVS. In Arizona, the new versions of IEVS involved
conducting three matches with new targeting strategies: (1) the quarterly SWICA recipient tape
match, (2) the BEER match, and (3) the IRS match. Previously, Arizona did not conduct the
SWICA tape match, and did not follow up any hits from the BEER or IRS matches. In Michigan,
the new versions of IEVS were (1) discontinuing the applicant match with the SWICA database, and
(2) increasing the tolerance threshold in the targeting strategy for the match with the IRS database.
The objective of the evaluation is to estimate the cost-effectiveness of the new matching and
targeting strategies in each state. However, it is important to note that we do not estimate the cost-
33
TABLE III.l
OVERVIEW OF EVALUATION DESIGN
Arizona Michigan
New IEVS Procedures 1. Match and target the
quarterly SWICA recipient
match
2. Target and follow up the
BEER match
3. Target and follow up the
IRS match
1. Discontinue the SWICA
applicant match
2. Increase the tolerance
threshold for the IRS
match
Definition of Research
Group
1. May be matched with 1.
SWICA (UI, BENDEX,
and SDX)a
2. May be matched with 2.
BEER (UI, BENDEX, and
SDX)*
3. May be matched with IRS 3.
(UI, BENDEX, and SDX)*
May be matched with
SWICA, UI, BENDEX,
and SDX
May be matched with IRS,
BENDEX, and SDX
May be matched with
BENDEX, and SDX
New Targeting Strategies SWICA, BEER, IRS IRS
Type of Case Food stamp recipients
Number of
Demonstration Offices 7
Food stamp applicants and new
recipients
16
Sample Size (cases) 22,500 13,462
*The matches in parentheses were conducted during our study, but the results of the follow ups were
not recorded.
34
effectiveness of the targeting strategy per se, nor do we estimate the cost-effectiveness of matching
with a database and following up on all hits (no targeting). Instead, the evaluation is designed to
address the question: Is a match against a given external database cost-effective if conducted with
a particular targeting strategy? The policy decision rule implicit in this design is that the state will
implement the new matching and targeting strategy if the ratio of savings to costs is greater than one,
and it will not implement the strategy if the ratio is less than one.
In measuring the cost-effectiveness of the IEVS matching and targeting strategies, the savings
and costs are measured relative to the situation in which the match with the database did not occur,
but the rest of the IEVS procedures used prior to the demonstration are continued. Hence, the
IEVS program is viewed as a set of independent programs rather than a single integrated system.
Implicit in this design is the assumption that the cost-effectiveness of each IEVS database is
independent of every other IEVS database. In other words, matching with a given database will
result in a certain amount of program savings whether or not any other databases are used. One
rationale for this assumption is that many food stamp applicants and recipients have only a single
source of income. Because each database investigates different income sources, it is unlikely that
misreported income would be picked up by more than one database.'
Due to differences in the states' interests and their implementation of IEVS prior to the
demonstration, the demonstration design is fundamentally different in the two states. In Arizona, the
evaluation was designed to estimate the cost-effectiveness of matching recipients with the SWICA,
BEER, and IRS databases. Michigan staff were primarily interested in improving the IEVS matching
and targeting of applicants. Hence, the evaluation in Michigan was designed to estimate the cost-
'The BEER and SWICA databases contain similar income information. However, in Arizona,
the BEER match did not take place prior to the demonstration, and in Michigan, the BEER match
did not occur during the study.
35
effectiveness of matching applicants and "new"recipients2 with the SW1CA, IT.S, UI, BENDEX, and
SDX databases.
In each state a sample of FSP cases were chosen to be in the research sample. We refer to cases
in the research sample as "research-sample cases." The sample was designed to represent the entire
state and to provide enough observations to support statistical estimates of the cost-effectiveness of
IEVS.
Prior to being matched to a database, each case in the research sample was randomly assigned
to one of three groups. In Arizona, cases in the first group were subject to the quarterly SWICA
recipient tape match, cases in the second group were subject to the BEER match, and cases in the
third group were subject to the IRS match. All cases were also subject to the UI, BENDEX, and
SDX matches. In Michigan, all cases were matched to the SDX and BENDEX databases. Cases in
the first group were subject to the SWICA and UI match, as well as the SDX and BENDEX
matches. Cases in the second group were subject to the IRS, SDX, and BENDEX matches, but not
the SWICA or UI match. Cases in the third group were subject only to the SDX and BENDEX
matches.
The follow-up procedures were not affected by the study. However, each caseworker recorded
on a data collection form the outcome of the follow up and the amount of time he or she spent
conducting follow-up activities for the case. In Arizona, data collection forms were completed for
follow ups of the SWICA recipient match, the BEER match, and the IRS match. In Michigan, data
collection forms were completed for follow ups of the SWICA, UI, IRS, BENDEX, and SDX
databases.
"New" recipients are recipients who have recently applied.
36
B. THE RESEARCH SAMPLE
Even though IEVS involves matching client rather than case data, follow ups are conducted on
the whole case. Thus, cases, not clients, were randomly assigned. All clients within a case were
assigned to the same group.
As changes to the IEVS process involved both matching and targeting, random assignment took
place prior to matching. In Arizona, all cases were randomly assigned at the beginning of the study.
In Michigan, a case was randomly assigned once it was registered into the computer system as an
applicant (a few days after the application was submitted). Once a case was assigned to a group, it
remained in that group for the rest of the study. If a research-sample case was denied or closed and
the household later reapplied for benefits within our study period, the case was assigned to the same
research group after the second application.
1. The Research Sample in Arizona and Michigan
The research sample consists of the set of cases that were included in the study. The types of
cases included in the research sample differ between Arizona and Michigan.
Arizona. In Arizona, a case is in the research sample if:
1. The case was open (that is, eligible to receive benefits), suspended (that is, ineligible
to receive benefits for a reason that is temporary), or in the recertification process on
July 1, 1992.
2. The case is open for food stamps. Cases that are open for AFDC and/or Medicaid but
not food stamps are not in the research sample.
3. The case is in a local office included in the demonstration.
Michigan. In Michigan, a case is included in the research sample if:
1. A new application for food stamps was submitted by a person in the case's household
between July 1 and October 2, 1992. We will refer to these cases as "applicant cases".
These cases include households that apply for food stamps for the first time, households
that became ineligible for food stamps but then reapplied, and households who failed
to complete their monthly reporting requirements but then reapplied. A case is not
37
considered an applicant at recertification. However, if a household fails to complete
the recertification procedures in time, it is required to reapply for the program. In this
case, a household would be considered an applicant at recertification.
2. The household applied for food stamps. Households who applied for AFDC and/or
Medicaid but not food stamps are not in the research sample.
3. The case is in a local office included in the demonstration.
Although every case must be an applicant case during the study period to qualify for the research
sample, the application may be approved and begin to receive benefits during our study period. We
refer to cases that are approved during our study as "new"recipient cases. Thus, the sample includes
both applicants and new recipients.
There are two major differences between the research sample in Arizona and Michigan. First,
in Arizona, all the cases are recipient cases, while in Michigan, the research-sample cases are
applicant cases at some time during the study period. Second, in Michigan the cases flowed into the
research sample during the study period, while in Arizona, the research sample consisted of a stock
of cases and did not vary in size during the study period.
2. Demonstration Offices
The offices that participated in the demonstration were chosen by state staff with input from
MPR. The objective was to choose offices that were representative of the state and were large
enough to meet the targeted sample size. Table III.2 lists the demonstration offices in Arizona and
Michigan and presents a brief description of each office.
In Arizona, there were seven demonstration offices in seven counties. The offices were chosen
to include both rural and urban counties and to include only offices that were under the supervision
of a key member of the Arizona IEVS team. An office on an Indian Reservation was also
included.
In Michigan, there were 16 demonstration offices in 12 counties. The offices were chosen to
include both rural and urban offices, offices serving Detroit (Wayne County), and offices that were
38
TABLE HI.2
PROJECT OFFICES IN THE DEMONSTRATION
Office
Arizona
Phoenix
Mesa
Buckeye
Tucson
Flagstaff
Winslow
Window Rock
Michigan
Bay
Muskegon
Crawford
Saginaw
Eaton
Sanilac
Genesee
Ionia
Wexford
Jackson
Midland
Wayne - Fullerton
Wayne - Greydale
Wayne - Hamtramck
Wayne - Maddelein
Wayne - Oakman
Description/Location
Urban
Urban
Rural
Urban
Urban
Rural
Indian Reservation
Small urban
Midsize urban
Upper northern rural
Large urban
Southern rural
Midstate rural
Large urban (Flint)
Midstate rural
Lower northern rural
Small urban
Midstate rural
West Detroit
West Detroit
East Detroit
East Detroit
West Detroit
39
under the supervision of members of the Michigan IEVS team. The rural offices were chosen to be
geographically representative, including offices in upper northern Michigan, lower northern Michigan,
mid-state Michigan, and southern Michigan. The urban offices include sma'I urban areas, mid-size
urban areas, and large urban areas. Five offices are located in Wayne County, which accounts for
over 40 percent of the total caseload. Three of these offices are located in West Detroit and two
are located in East Detroit.
3. Sample Sizes
The sample sizes were chosen to be large enough to estimate the cost-effectiveness of each
database, but not so large as to waste project resources and caseworker time on unnecessary
observations. Table III.3 shows the realized sample sizes in Arizona and Michigan. The random
assignment algorithm was designed so that different numbers of cases were assigned to each group.
The likelihood of a case being assigned to a group was set so that there would be enough hits in each
group to measure the cost-effectiveness of each match with a similar level of statistical significance.
TABLE III.3
SIZE OF RESEARCH SAMPLE
Research Group Number of Cases
Arizona
1. SWICA
2. BEER
3. IRS
3.856
8.507
10.137
Total 22,500
Michigan
1. SWICA, UI, BENDEX. SDX 2.460
2. IRS, BENDEX. SDX 3 86i
3. BENDEX, SDX 7j41
Totel . 13,462
40
C. THE NEW IEVS MATCHING AND TARGETING PROCEDURES
This section describes in detail the new IEVS procedures used in Arizona and Michigan during
the demonstration. In both states, some IEVS procedures used in the demonstration were radically
different from the matching and targeting strategies that were used prior to the demonstration. In
Arizona, matches that were not conducted prior to the demonstration were conducted during the
demonstration. In Michigan, an applicant match was discontinued. In both states, the matching and
targeting strategies do not distinguish between programs-the targeting strategy for food-stamp-only
cases is the same as that for food-stamp/AFDC cases.
1. Arizona
Table III.4 provides a summary of the targeting strategies used in Arizona during the
demonstration and those strategies used prior to the demonstration. Most elements of the new
targeting strategies used in Arizona have been used in other states. However, the targeting strategies
are innovative in that they combine many elements of targeting strategies previously used separately.
a. Match and Target the SWICA Database
In the first new IEVS procedure, Arizona continued to conduct its SWICA on-line match and
reinstated the quarterly SWICA recipient tape match with a new targeting strategy. Under the new
targeting strategy, caseworkers followed up on matches only if all of the following rules were satisfied:
1. Use ofindividual and case characteristics
The person is active in either the current month or was active in one of
the two previous months
The person is 16 years of age or older as reported on the client database
in the current month3
The case received a benefit during at least one month over the reference
quarter
3Current month refers to the month in which the computer job is run.
41
TABLE III.4
PREDEMONSTRATION AND DEMONSTRATION TARGETING STRATEGIES IN ARIZONA
Match Predemonstration Targeting Strategy Demonstration Targeting Strategy
SWICA
Tape Match did not take place Follow up if all of the following are satisfied:
INDIVIDUAL AND CASE CHARACTERISTICS
• Person is active for food stamps in current month or was active in one of the two previous
months
• Person is 16 or older in the current month
• Case received a benefit during at least one month over the reference quarter
TOLERANCE THRESHOLD
• The person's total quarterly earnings reported on the SWICA database from all employers
are $3,600 or more
DISCREPANCY THRESHOLD
• The difference between the total prorated earnings on the SWICA database and the total
earnings reported on the client database over the same quarter is 20 percent or more of the
total prorated earnings on the SWICA database
BEER No matches were followed up Follow up if all of the following are satisfied:
INDIVIDUAL AND CASE CHARACTERISTICS
• Employer identification code on the BEER database is different from the employer
identification code on the SWICA database
• Person was active for at least six months during the reference year
• Information from the BEER database for the same employer during the same reference
period has not already been received
• Person is active for food sumps in current month or was active in one of the two
previous months
• Person is 16 or older in the current month
• Case received a benefit during at least one month of the reference year
•V*2
TABLE IH.4 (continued)
Match
IRS
Predcmonstration Targeting Strategy
No matches were followed up
Demonstration Targeting Strategy
Follow up if all of the following are satisfied:
iNDtVrDUAL AND CASE CHARACTERISTICS
• Person was active for at least six months during the reference year
• Person is active for food stamps in current month or was active in one of the two
previous months
• Person is 16 or older in the current month
• Case received a benefit during at least one month of the reference year
TOLERANCE THRESHOLD
• Total unearned income for the case, excluding UI income and prior year tax refunds,
exceeds $100
¥3
2. Use ofa tolerance threshold
The person's total quarterly earnings from all employers as reported on
the SWICA database are $3,600 or more
3. Use ofa discrepancy threshold
The difference between the total prorated earnings on the SWICA
database and the total earnings reported on the client database over the
same quarter is 20 percent or more of the total prorated earnings on the
SWICA database. The prorated earnings on the SWICA database are
calculated by dividing the person's quarterly earnings by three to find the
average monthly earnings, and multiplying the average monthly earnings
by the number of months in which the person was active in either the
Food Stamp or AFDC programs.
The first set of rules for this targeting strategy exempts from follow up persons for whom changes
in earnings will probably not affect benefits and persons from whom it will be difficult to recover
overpayments. Earnings of persons younger than age 16 are not counted toward income under the
FSP. If a person did not receive benefits during the reference quarter, errors in reported earnings
could not have caused an overpayment. Persons who have not been active for three months are not
followed up because they will typically remain inactive over the period in which benefits could be
recovered. It is extremely difficult to recover overpayments from persons who are not receiving
benefits.
The second rule exempts from follow up some persons whose earnings are low enough that they
will not affect eligibility for the FSP, although unreported earnings could still affect the level of
benefits. The gross monthly income eligibility threshold for a household of three is currently $1,207.
Hence, if a person was the single earner in a three-person household, he or she could earn up to
$3,621 each quarter and still be eligible for benefits. As the typical working FSP household contains
just under three persons and only one earner, a tolerance threshold of $3,600 a quarter will exempt
from follow up a high proportion of persons whose earnings are low enough for them to be eligible
for food stamps. However, it will also exempt from follow up some persons whose earnings are too
44
high for them to be eligible for food stamps. For example, a person in a one-person household would
not be eligible for food stamps if he or she earned more than $718 per month, or $2,154 per quarter.
Persons with earnings of less than $3,600 are excluded from the external database prior to the
match. Excluding persons prior to the match, rather than after the match, conserves computer
resources but has no impact on which clients are followed up. However, strictly speaking this is a
screening rule rather than a targeting rule. As such, it is not in compliance with the IEVS
regulations.
The third rule compares client-reported income with income on the external database. When
Arizona previously matched with the SWICA database, the state followed up only if the difference
between the total quarterly earnings reported by the client and the total quarterly earnings reported
on the SWICA database exceeded 20 percent of the total quarterly earnings on the SWICA database.
Under this rule, a high proportion of matches were targeted for follow up, many of which did not
lead to changes in benefits or eligibility. The problem with this targeting strategy was that it did not
take into account that some persons were not active during some of the reference quarter because,
for example, they had not yet been accepted into the program. In these months, their earnings were
not recorded on the client database and were treated as zero earnings in the calculation of total
quarterly earnings. Hence, in these cases the total quarterly earnings reported on the SWICA
database were much higher than the total quarterly earnings on the client database even though the
client may have correctly reported earnings while active on the program.
Ideally, Arizona would like to compare earnings as reported by the client with earnings reported
on the SWICA database during only those months in which the client is active. However, the
SWICA database reports total quarterly earnings, not monthly earnings. The new targeting strategy
addresses this problem by prorating the quarterly earnings on the SWICA database over the months
in which the person was active. For example, if the client earned $2,500 over the quarter but was
active for only two months, his total prorated earnings would be $1,670(2,500 + 3x2). However,
45
prorating does not completely solve the problem. If earnings were higher in the months in which the
client was not active, the prorated earnings would still be higher than the client's true earnings. For
example, suppose a person earned $1,000 in the first month of the quarter and $750 in the
subsequent two months, and became active in the second month of the quarter. The total prorated
earnings on the SW1CA database would be $1,670 [(1,000 + 750 + 750) + 3 x 2], but the total
quarterly earnings as reported on the client database would be only $1,500(0 + 750 + 750). So, to
further reduce the number of matches to be followed up, the targeting rule requires a 20 percent
difference between the prorated earnings on the SWICA database and the earnings on the client
database.
b. Match and Target the BEER Database
Under the second new IEVS procedure, Arizona introduced a new targeting strategy for the
BEER database and followed up some hits. Under the new targeting strategy, caseworkers followed
up on matches if all of the following rules were satisfied:
/. Use ofindividual and case characteristics
The employer identification code on the BEER database is different from
the employer identification code on the SWICA database
The person was active for at least six months during the reference year
Information from the BEER database for the same person and the same
reference period has not already been received
The person is 16 years of age or older as reported on the client database
in the current month
The person is active either in the current month or was active in one of
the two previous months
The case received a benefit during at least one month of the reference
year
Arizona opted not to use a tolerance threshold as part of this targeting strategy.
46
Both the BEER and SWICA databases provide earnings data. However, the BEER database
provides information on some forms of earnings, such as out-of-state earnings, not provided by the
SWICA database. When earnings data from the same employer are provided by both databases, the
data from the SWICA database are viewed as the more useful because they are more recent and are
aggregated by quarter instead of by year. Hence, the first targeting rule restricts the matches to be
followed up to those in which the employer reported on the BEER database differs from the
employer reported on the SWICA database.4
As discussed above, there is a problem in comparing client-reported earnings with earnings from
an external database when the client is not active during all of the reference period. This is an even
more pertinent problem in the BEER match because the earnings reported on the BEER database
cover 12 months. To prevent following up matches in which the client was not active for a substantial
period of time, the second targeting rule requires the client to have been active for at least six
months during the reference year.
A match is made with the BEER database for all new clients and all clients who have reported
a change in circumstances. This may be an actual change in name or address or a change because
information provided previously, such as the person's date of birth or SSN, was incorrect on the client
database. Hence, it is possible for the same person to be matched more than once even though there
is no new earnings information from the BEER database for this person. Consequently, follow-up
efforts on essentially the same information may result. By excluding all information that duplicates
information already received, the third targeting rule prevents duplicate follow-up efforts.
4A technical difficulty with this element of the targeting rule is that the employer identification
code on the BEER database is a federal employer identification code, while the employer
identification code on the SWICA database is a state employer identification code. The state
employer identification code was "translated'' to the federal employer identification code using a
mapping between the two codes provided by the Administration of Unemployment Insurance.
Unfortunately, there was not always a correct translation between the two identification codes.
47
The remaining three elements of the first rule of the targeting strategy were part of the SWICA
targeting strategy and have the same purpose.
c. Match and Target the IRS Database
Under the third new IEVS procedure, Arizona introduced a new targeting strategy for the IRS
match and followed up on the hits. The same targeting strategy was used for both the IRS annual
and monthly matches. Under the new targeting strategy, caseworkers followed up on matches only
if all of the following rules were satisfied:
1. Use ofindividual and case characteristics
The person was active for at least six months during the reference year
The person is 16 years of age or older as reported on the client database
in the current month
The person is active in either the current month or was active in one of
the two previous months
A person in the case received a benefit during at least one month of the
reference year
2. Use ofa tolerancethreshold
Unearned income, excluding UI income and prior year tax refunds,
summed over all persons in the case is more than $100
The first rule in this targeting strategy-the exemptions based on individual and case
characteristics-was also part of the BEER targeting strategy and were used in this match for the same
reasons. The rationale for the second rule is that to be eligible for the FSP, a household must have
total liquid assets in each month of less than either $2,000 (if the household does not contain an
elderly FSP participant) or $3,000(if the household does contain an elderly FSP participant). Hence,
the targeting rule specifies that only persons for whom the total unearned income for the case
exceeds a threshold should be followed up. The amount of interest income that corresponds to asset
48
holdings of $2,000 will vary with the interest rate. However, with a 5 percent rate of interest a
household with assets of $2,000 held all year would earn $100 in interest income.
Two types of unearned income are excluded from the calculation of total unearned income: (1)
UI income and (2) prior year tax refunds. UI income is excluded because the match with the UI
database can provide more recent and less aggregated information about the receipt of UI benefits.
Prior year tax refunds are excluded because they are disregarded when determining income eligibility
for the FSP.
2. Michigan
Table III.5 provides a summary of the matching and targeting strategies used in Michigan during
the demonstration and those strategies used prior to the demonstration.
a. No SWICA Applicant Match
Under the first new IEVS procedure, Michigan discontinued its applicant match with the SWICA
database. No matches with the SWICA database in the second and third research group were
followed up. In the first research group, all SWICA matches were followed up.
The rationale for not conducting the SWICA applicant match is that a cost-benefit study of the
SWICA match conducted by Michigan staff (Ward and Smucker 1990) found that following up on
SWICA applicant matches was not cost-effective. This was because no action was required, and
hence no savings realized, for over 95 percent of the applicant matches. (An action was defined by
Ward and Smucker as an application denied, a case closed, or a change in benefits.)
The IEVS regulations explicitly state that all applicants must be matched and followed up~they
prohibit the targeting of applicants. There are two rationales for this regulation. First, savings can
be achieved at a lower cost prior to certification. Prior to certification, benefits can be denied, but
after certification, previous benefits overpayments must be recovered. Second, Puma (1989) found
that following up on all applicant matches was cost-effective.
49
TABLE III.5
PREDEMONSTRATION AND DEMONSTRATION TARGETING STRATEGIES IN MICHIGAN
o
Match Predemonstration Targeting Strategy Demonstration Targeting Strategy
SWICA
Applicants None All matches are followed up in first research group, no
matches are followed up in second and third re