State Automation Systems Study

              ■*
,■?££*. United Stote.
(UK) Dec
V
apartment of
Agriculture
Food and
Consumer
Service
Office of
Analysis and
Evaluation
COMPLETED
State Automation
Systems Sudy
Volume II
December 1995
United States
Department of
Agriculture
Food and 3101 Park Center Drive
Consumer Second Roor
Service Alexandria. VA 22302
State Automation Systems Study
Final Report
Volume II
Analytical Presentation of State Data
Jack Slocum
Elizabeth Wenchel
Carolyn Uchtenstein
Chris Fortune
Julie Neafach
Ann Perper
A product of:
The Orkand Corporation
8484 Georgia Avenue
Silver Spring, MD 20910-5695
December 1995
This study was conducted under contract number 53-3109-2-007 with the
Food and Consumer Service, U.S. Department of Agriculture. Points of
view or opinions stated in this report do not necessarily represent the
official position of the Food and Consumer Service.
ACKNOWLEDGMENTS
The authors would like to thank the many people who made important contributions to this
report. The following individuals deserve special recognition.
Diana Perez. Christopher Beavers, and Shelia Little of the Food and Consumer
Service provided overall leadership, direction, and support throughout the study.
The study could not have been completed without the cooperation of clients and
staff from the Food Stamp Program and the individual State agencies.
i\
TABLE OF CONTENTS
Page
EXECUTIVE SUMMARY 1-1
I. INTRODUCTION 1-4
A. Background 1-4
II. CURRENT DEGREE OF AUTOMATION AND STATE OF
DEVELOPMENT II-1
A. Background II-1
A. 1 Degree of Automation II-1
A.2 Stage of Development II-2
B. Automated Features 11-3
B.l Applicant Check-In 11-3
B.2 Applicant Interview 11-7
B.3 Eligibility Determination/Benefit Calculation 11-9
B.4 System Verification 11-10
B.5 Computer Matching 11-12
B.6 Notice Generation 11-15
B.7 Monthly Reporting 11-17
B.8 Program Management 11-18
B.9 Issuance 11-19
B. 10 Claims Collections 11-21
C. Level of Integration 11-23
D. Degree of Automation 11-25
E. Stage of Development 11-27
F. Relationship of Degree of Automation to State 11-28
of Development
III. STATE SYSTEM DEVELOPMENT PROCESSES Ill-1
A. Background Ill-I
B. Project Management Factors 111-2
C. Use of System Development Life Cycle Methodologies II1-5
Volume II Page i
9 , t
D. Hardware/Software Platforms 1II-6
E. Observations and Conclusions III-7
IV. SYSTEM TRANSFERS IV-I
A. Background IV-1
B. Frequently Seen Characteristics of Successful Transfers IV-I
C. Factors Affecting Transfer Success IV-3
D. Observations and Conclusions IV-8
V. STATE AUTOMATION COSTS/COST ALLOCATION METHODOLOGIES V-l
A. Background V-l
B. Cost Allocation Methodologies V-2
C. State APD Funding Requests V-5
D. State Cost Accounting and Cost Controls for ADP V-7
E. Guidelines for Determining Reasonableness of State
ADP Funding Requests V-8
F. Recommendations V-IO
F. 1 APD Cost Recommendations V-IO
F.2 Cost Allocation Improvements V-IO
F.3 Development and Operations Cost Reporting V-IO
VI. STATE IMPLEMENTATION OF REGULATORY CHANGES VI-1
A. Background VI-1
B. Approach VI-2
C. Findings VI-3
C. 1 Performance - Timeliness in Implementing Regulatory Changes VI-3
C.2 Problems Encountered in Making Changes in a Timely Manner VI-6
C.3 Organizational Structure for Implementing Changes VI-8
C.4 Availability of Resources for Implementing Regulatory Changes VI-9
C.5 Other Constraints in Implementing Timely Regulatory Changes VI-10
Volume II Page ii
W
D. Analysis VI-11
D. 1 Relationship of Degree of Automation to
Implementing Regulatory Changes VI-11
D.2 Relationship of System Age to Timeliness of
Regulatory Change Implementation VI-12
D.3 Relationship of State of System Development to
Timeliness of Regulatory Change VI-12
D.4 Relationship of Utilization of Change Control Committee and Other
Formalized Procedures to the Ability of a State to Implement Timely
Rerjulatory Changes VI-13
Vn. LEVEL OF AUTOMATION AND FSP NEEDS VIII
A. Background VII-1
B. Analysis VII-2
B.l Caseloads VII-2
B.2 FSP Error Rates VII-3
B.3 Claims Collected VII-4
B.4 FSP Administrative Costs VII-5
B.5 Regulatory Changes VII-5
B.6 Costs/Benefits VII-5
B.7 Fraud and Abuse VII-6
B.8 User Satisfaction VII-6
C. Observations and Conclusions VII-6
APPENDICES
A. CURRENT DEGREE OF AUTOMATION AND STATE
OF DEVELOPMENT TABLES A-l
B. STATE SYSTEM DEVELOPMENT PROCESS TABLES B-l
C. SYSTEM TRANSFER TABLES C-l
D. STATE AUTOMATION COSTS AND COST ALLOCATION
METHODOLOGIES D-l
E. STATE IMPLEMENTATION OF REGULATORY CHANGES TABLES . . El
F. LEVEL OF AUTOMATION AND FSP NEEDS TABLES F-l
G. STATE SYSTEM PROFILES G-l
Volume II Page iii
EXECUTIVE SUMMARY
The review of State development processes and functionality of the public assistance systems
supporting the Food Stamp Program was conducted to:
Identify the level of automation, as determine by each State, to support the needs
of the Food Stamp Program (FSP);
Review the effectiveness of the system to meet the FSP needs; and,
Ascertain the general level of eligibility worker and supervisor satisfaction with
the capabilities, reliability, and accuracy of the automated systems.
The statistical results and findings contained in this volume of the report of the State Automation
Systems Study reflect the ability of each State and the District of Columbia to provide a sound
technical system that contains the ability to capture and verify client information, calculate
eligibility and benefits for registrants, and provide a reasonable method to track and reconcile
benefits paid.
We evaluated and rated on an arbitrary scale the ability to perform each requisite function to
indicate a high, medium or low level of automated functionality. The scale was established to
be able to compare one State's system against the relative capability of another State. The
summary, below, contains the results cf this rating approach. The numbers in the Low, Medium,
and High columns represent the number of States receiving the rating for that specific functional
area.
Function Rating
Low Medium Hip
Registration 15 20 16
Applicant Interview 18 19 14
Eligibility Determination/ 17 13 21
Benefit Calculation
Verification 15 12 24
Computer Matching 18 16 17
Notices 15 16 20
Monthly Reporting* 5 6 15
Worker Statistics 26 10 15
Issuance 15 23 13
Claims Collection 15 10 26
* Every State is not required to perform monthly reporting.
Volume II Page 1-1
Two additional rating categories were established to provide a view of the overall level of
automation and a composite picture of the functional and programmatic integration of the public
assistance requirements. The rating findings axe:
Low Medium High
Level of Automation 9
Level of Integration 17
18
6
23
23
Note: Due to the age of the system or lack of a Statewide system, not every State is
represented in the above statistics.
A number of significant findings were reached at the conclusion of the State visits regarding
system functionality, system transfers, deve opment costs, the cost allocation process, and
regulatory changes. A detailed report of the f ndings is contained in the following chapters. A
summary of the more important finding is contained in Table 1.1 below:
Table 1.1 Summary of Findings
Applicant
Registration
1. Duplicate entry of the same information should be
eliminated.
2. Workers need access to historical participation
information when processing client applications.
Verification/
Computer Matching
1. Many checks are performed as part of a batch update
cycle process with data that is less than current from
outside data sources.
| Level of Integration/
Degree of Automation
1. Twenty-nine (29) States are moderately to highly
integrated.
2. Forty-one (41) States have a moderate to high degree
of automation.
Development
Process
1. Participation of both State programmatic and systems
areas in the planning, development, and
implementation phases of the project are extremely
important in helping ensure a successful development
process.
2. Many States are currently using or beginning to use
standard developmem lifecycle methodologies to plan
and execute system development projects.
Volume II Page 1-2
Systems Transfers Most States prefer to use a transfer system since it
allows them to have a starting point from which they
can plan and implement their own changes.
Creation of a centralized electronic database of State
systems information on the status of public assistance
systems would be a major benefit to those States
undertaking evaluations of new system solutions.
System Cost/
Cost Allocation
Every State would like the cost allocation process to be
more consistent among Federal agencies and easier to
complete and gain Federal approvals than is currently
the case.
Regulatory
Changes
Delays in implementation are more likely to be caused
by lack of timely dissemination of information than by
systems issues.
An estimate of the technical difficulty of implementing
a change would be a valuable asset in determining the
timeframe for national implementation of a change.
Level of Automation/
Food Stamp Program
Needs
1. No relationships were found between the degree of
system automation and the following:
• cost per household;
• error rates;, and
• percentage of claims collected.
2. Eligibility workers tend to be more satisfied with
newly created system and those with less sophisticated
features since they reduce their job-related stress levels.
3. Highly-integrated systems that allow the client to
receive full service with the least amount of
bureaucratic delays and additional trips to State offices
are viewed as the most beneficial.
Volume II Page 1-3
I. INTRODUCTION
A. Background
This volume of the report addresses detailed findings and suggested potential guidelines for FCS
review of system development efforts for the Food Stamp Program (FSP). These guidelines
focus on FCS efforts to provided effective and efficient oversight and monitoring of the States
system automation efforts, as well as determining the reasonableness of State funding requests
for these projects. FCS can use the study findings to reevaluate the current standards and
procedures related to State automation efforts to increase the efficiency and effectiveness of State
automated systems.
To develop the guidelines and standards for State FSP automation, information was collected
from States to identify those factors that affected the following areas:
• Success of system transfers
• Success of system development efforts
• Development costs
• Operational costs
• Ability to meet FSP needs
• Degree of automation
• Level of integration
• FCS monitoring and oversight
Data were collected from five data sources - Food and Consumer Service (FCS) headquarters
monthly and quarterly reoorts. questionnaires sent to State personnel, State personnel interviews
conducted in all 50 States and the District of Columbia, State Advanced Planning Document
(APD) documentation, and survey forms completed by randomly-selected eligibility workers and
eligibility worker supervisors; within each State.
Data collection for the State Automation Systems Study began in June 1992 and continued
through December 1993. Historical information was obtained from APDs and correspondence
provided by State staff. State personnel working in the Food Stamp Program, automated data
processing (ADP) or management information systems (MIS) groups, and State data centers were
interviewed during the visit to each State.
Volume II addresses the technical findings of our study of Siate automated systems in support
of the Food Stamp Program. It is organized to address each of the seven research objectives
identified at the beginning of the State Automation Systems Study:
• Current degree and state of systems development
• State system development processes
• System transfers
• Level of automation and FSP needs
• State funding requests for automation
• Operational cost accounting and cost control measures
Volume II Page 1-4
• Implementation of regulatory changes
• Level of automation and FSP needs
The remainder of Volume II contains six chapters that address all of the above items.
Discussions about State funding requests and operational costs are combined into a single chapter,
Chapter V - S ite Automation Costs and Cost Allocation Methodologies.
Volume II Page 1-5
II. CURRENT DEGREE OF AUTOMATION AND STATE OF DEVELOPMENT
A. BACKGROUND
This chapter discusses the degree of Food Stamp Program automation and stage of system
development for each State. The information was collected during a 16-month period, from
August 1992, when the first pretest site visit occurred, through December 1993.
A.1 Degree of Automation
For this analysis the degree of automation was determined based on (a) the level of functionality
in each State's system and (b) the level of system and program integration.
The systems reviews focused on those system features that seemed to have the greatest potential
for improving caseworker effectiveness and efficiency. A review of system functionality in terms
of compliance with FSP Model Plan Requirements was not a part of these reviews.
System demonstrations were conducted in the State agency central offices on either a test
database or in the production system. Examination of the system in a test environment enabled
the reviewer to assess some aspects of system functionality that could not have been viewed in
a production environment. In many cases, the demonstrators were only able to describe how a
function worked, but could not show how the function worked due to built-in system security.
Information on the level of automated functionality, therefore, had to be supplemented through
staff discussions and the pre-visit questionnaires.
In adapting, transferring, or developing systems that meet FSP requirements, States have
implemented a wide variety of automated systems and features to support their workers. As a
result, some State systems may have more automated features than other States. For instance,
when a client submits en application for assistance to the State office, one system may
immediately perform a check for duplicate participation based on the name and Social Security
number (SSN) of the applicant before any other application information has been entered.
Another system may perform the firsi check for duplicate participation only after all application
information has been entered into the system. While the FSP regulations only require that a State
check for duplicate participation before a client is certified as being eligible to receive benefits,
the system that is able to identify already existing clients before the new application has been
entered into the system, is considered to be "more" automated because it performs the check
before the worker has entered all of the application information.
Within each State, the automated features for major FSP functions were identified. To compare
the level of automated functionality across all States, a scoring method was developed that would
reflect the presence or absence of the feature and its relative importance to other features. For
instance, a system that automatically mails all notices would be considered to be more automated
than a system that automatically mails notices requested by the worker and both would score
higher than a system that has no automated notices. This permitted the comparison of State
systems for each major functional area, such as eligibility determination. For instance, a weight
of "1" would be given if a function was performed on-line versus a "0.5" if the function was
Volume II if Page II-1
performed in a batch mode. This provided a mechanism for analyzing the relative level of
automated functionality among many States within each functional area and for the overall
system.
The weights for the individual components of a functional area were added to get a summary
score. The scores for each functional area were standardized through the use of mean and
standard deviation techniques to make the scores of the different functional areas comparable. The
standardized scores were assigned to one of three levels of functionality: high, medium, or low.
The three levels of functionality were determined to be an acceptable categorization given that
there were, at most, only 51 scores for any functional area. The designation of high, medium,
and low was based on the assumption that the standardized functionality scores follow a standard
normal distribution.
The second type of information needed to assess the degree of automation is level of integration.
This relates to the number of separate systems needed to support the Food Stamp Program as well
as the number of assistance programs that are served by the system or systems. As an example,
a State that has one automated system that determines eligibility, processes claims, sends notices,
and issues benefits for the Food Stamp Program, Aid to Families with Dependent Children
(AFDC), and Medicaid Programs is considered more integrated than a State that utilizes multiple
systems for all programs, or a State that utilizes one system for each program. A score is
assigned to each State for the degree of integration.
The scores for level of automated functionality and level of integration are then summed to
reflect one total score to provide a mechanism for a comparative analysis of all States in terms
of degree of automation.
A.2 Stage of Development
ADP development methodologies generally recognize the following stages of system
development:
• Planning Stage - usually includes a feasibility study, alternatives analysis, requirements
analysis, cost benefit analysis, conceptual design, and plans for system development and
implementation. For State system development efforts, the planning stage may also
include preparation of the Implementation APD and the request for proposal (RFP),
proposal review, and selection of a contractor.
• Development Stage - preparation of a detailed system design, a detailed system
architecture to include hardware and software specifications, coding, testing, and
conversion.
• Implementation Stage - includes all of the activities discussed in the plans prepared
during the development stage including conversion, pilot installation, and full installation.
Volume II ^ Page 11-2
• Operational Stage - Statewide processing, ongoing enhancements, hardware expansion,
and system maintenance activities continue; accommodate changes in caseloads, system
capacities and improvements in operational performance and efficiency.
Because there may be multiple systems within a State that support the Food Stamp Program, a
single stage of development may not adequately describe the system status.
B. AUTOMATED FEATURES
We examined automated features of systems that support the FSP and, in the case of integrated
systems, AFDC and Medicaid. To a lesser extent, information was also gathered on the issuance
systems when they were a part of the eligibility determination and benefit calculation (ED/BC)
system. During the system demonstrations, the evaluation team reviewed the automated features
checked off by program staff in the preliminary questionnaire. We examined automated features
for the following major functions: application receipt, processing, verification, interviewing,
sending notices, computer matching, monthly reporting (no longer required by FCS but continued
by some States), eligibility determination, benefit calculation, claims collections, notices and
alerts, issuance, and reporting.
In this chapter, we describe the relevance of the automated features that potentially reduce worker
time spent on FSP tasks through increased efficiency and effectiveness. The actual findings
associated with the automation review for each State can be found in Appendix A. Throughout
the remainder of this chapter, reference is made to relevant tables found in Appendix A. Rating
categories of high, medium, and low will be governed by different scores in each of the
functional areas described. The value range for the categories in each functional area will be
listed.
B.l Applicant Check In
Overview
Registration - The 30-day application processing standard is initiated when the application for
food stamp benefits is filed with the appropriate food stamp office. An application can be filed
as long as it contains the applicant's name and address and the signature of a responsible
household member. Most States provide a pre-screening form that is used to determine the need
for expedited benefits. States enter the name, address, and date of filing into the system to
monitor the application processing timeframe required for completing the application,
interviewing the applicant, and verifying the necessary information prior to certification. Many
States refer to the automated support for filing an application as "registration." Registration can
include a variety of activities:
• Registering the applicant and appropriate household members for work on the system.
• Entering the available information on household members into the system.
Volume II (l Page II-3
Performing social security number (SSN) enumeration for household members who do not
have SSNs.
Scheduling an interview date.
Generating notices of scheduled interviews, required verifications, or notices for
rescheduling interviews.
Identifying the need for expedited service.
Performing duplicate participation cross checks for FSP participants within the appropriate
jurisdiction.
Monitoring the application processing standard.
The full application may be entered before, during, or after the client interview is conducted.
Registration of the application causes a number of system functions to occur in systems that are
highly automated.
Duplicate Participation - FCS regulations require that automated systems should "crosscheck
for duplicate cases for all household members by means of a comparison with food stamp records
within the relevant jurisdiction."1 FSP duplicate participation checks must be performed at
certification, recertification, annually, and when a new household member is added. A': a
minimum, the check is to be performed on the name and SSN for each household member. If
the SSN is not available, the State must do SSN enumeration. The date of birth and address are
optional.
As duplicate participation checks are performed for Aid for Families and Dependen* Children
(AFDC) and Medicaid, the check need not be limited to one assistance program or one system
although it is more efficient if the worker is not required to access multiple systems to perform
the check for all assistance programs. In determining the level of automated functionality, the
breadth and depth of the search and whether the results are available on-line or off-line were
considered. Table A-l (Part A) in Appendix A, Application Log in Functionality - Check for
Duplication Participation, shows the availability of the automated features in each State system
that supports the Food Stamp Program.
When an FSP application is filed, only the applicant's name and address and the signature of a
responsible member of the household or an authorized representative is required. Most States,
however, will request and receive additional information from an applicant that will facilitate
logging the application onto the system and conducting the duplicate participation search, since
a name is usually not sufficient to perform a search (see Table A-l (Part B), Data Elements, Used
in Duplicate Participation Search).
Section 272 10 (bXIMv) ADP/CIS Model Plan. Certification
Volume II v Page 11-4
Many States have come to rely on the SSN as the primary element to log the application into the
system and perform the initial duplicate participation search. This is especially the case if the
SSN is also used as the client identification number. Since the SSN is also used for other
searches of State and Federal databases, the use of the SSN during the duplicate participation
search was given more weight than the other data elements used by States, which were all given
equal weights of less value than features in Table A-l - Part A.
Many States prefer to obtain as much information as they can at the time an application is filed
and perform any searches, whether for duplicate participation or for Income and Eligibility
Verification System (IEVS) or other database matches, early in the applicant process. Any
information that is available to the State can then be reviewed by the caseworker either before
or at the time of the interview with the applicant. States, however, are prepared to process any
applicants that are filed with just a name and address.
Findings
In designing an efficient and effective system, the following features are important:
• Duplicate entry of the same information should be avoided. The system should
provide for one-time entry of any client information used for the duplicate participation
check regardless of the number of separate systems that are checked at the time. For
instance, client/applicant name, date of birth, and social security number could be entered
once for a search of client cross-reference; FSP, AFDC, and/or Medicaid databases, if
they are separate; and other State agency databases containing information on
employment, unemployment benefit receipt, motor vehicle registration, etc. This is
especially important for States that still have separate systems (or subsystems) that support
FSP, AFDC, and Medicaid.
• Access to historical participation records at the time an application has been filed
can save a worker considerable time. During application filing, States access historical
participation records to determine whether an individual (or household) has participated
in the Food Stamp Program previously and, if so, how recently. If the system is
integrated, information on prior participation in AFDC, Medicaid, and other assistance
programs are also checked. If the historical record is still available on-line to the worker,
the worker can either view the historical records or can transfer the information from the
old record to the new applicant record. If the information is up to date, this will save the
worker time and will provide useful information for determining the applicant's status or
the potential for applicant fraud.
• The usefulness of on-line access to recent historical records declines with age. Access
to the historical records can be either on-line, off-line, or a combination of both. States
with smaller caseloads may be able to maintain all historical records in an on-line mode
for a longer period of time than States with larger caseloads, which often keep only the
most recently inactive cases on-line, moving older inactive cases off-line. The off-line
search may be performed either through an on-line request to conduct a batch search or
through paper-based requests for the older records. The older the record, the less current
Volume II (& Page II-5
the information will be and the less useful during application processing. The older
records mrst be maintained, however, and made available upon request in response to
claims, fair hearings, and other potential legal liabilities (e.g., class action suits).
• Carefully select and limit the information that is archived. For instance, caseworker
notes could be purged after a short time, but the payment history and case information
may be indefinitely archived. The number of records, type of records, and accessibility
(on-line, off-line, or archived) can have an impact on the system architecture in terms of
mainframe capacity, response time, the amount of direct access storage, etc.
• Archived data is of value only when accessible to the worker. For data that is
archived or remains on-line, the current system must be able to access the information and
make it available to workers upon request. This may be difficult for States that have
implemented new systems that are considerably different from their prior systems,
sometimes requiring the State to maintain some version of the older system in order to
access the older records.
Summary
Registration is not a required FSP function. Although an efficient registration function is
beneficial to the smooth functioning of the application process, it is only a small component in
the overall efficiency of FSP.
As shown in Figure 2.1, page II-7, when all automated features for Application Log-In
Functionality are considered for all States in terms of high, medium, and low levels of
functionality, there is an almost equal distribution among the three categories, with 20 States
having a moderate level of automated functionality (a total score of 10.5 to 12.5), 16 with a high
level of automated functionality (a total score of 13 or above), and 15 that have a low level of
automated functionality (a total score of 10 or below).
Most States (45) log the application into the terminal when the application is submitted, with 26
States entering some additional application information into the terminal. Twenty-seven State
systems automatically assign the case number when the case is put into the system. Beyond these
relatively basic features, there is only a small subset of State systems that provide additional
helpful application log-in features.
All States used some automated features associated with duplicate participation check at the time
of registration, but few offered the full range of automated duplicate participation features. In
summary, 42 States utilize the full name to perform the search. The SSN for all household
members is the second most frequently used search element, used by 39 States. Nineteen States
continue to use a client ID number that is not the SSN.
Volume II Page II-6
Figure 2.1 Application Log-In Functionality Summary Scores
M«dli»n (20)
B.2 Applicant Interview
Overview
Completing the application form and entering the application information into the automated
system is the first of a series of functions required to determine eligibility. The application may
be completed by the client prior to the interview or it may be completed at the time of the
interview. Information from the completed application may be completed at the time of receipt
or after eligibility has been determined. Table A-2, Application Completion and Input of
Application Information, in Appendix A, describes system features that perform these functions.
Findings
The optimal procedure for the applicant interview is to have it take place while the client
application is completed interactively. This procedure eliminates the separate steps of the
applicant llling out the application form, the form being entered into the ADP system, and the
eligibility worker interviewing the applicant. The fewer steps an application has to go through,
and the less paperwork involved, the more efficient the process. In this regard, electronic forms
are more effective than paper forms as they require less processing time and fewer steps in the
process.
The following actions can increase the efficiency and effectiveness of the interview process:
• Elimination of unnecessary paper to the degree possible. A svstem should eliminate the
need for interim worksheets or turnaround documents.
Volume II , Page II-7
Most States still require that an applicant complete a detailed paper application form.
Many States that have interactive interviewing have a short form which the applicant
signs, but others still require completion of the full application. Some local jurisdictions
are experimenting with the use of multimedia technologies for applicants to enter the
information directly into the system. The information is, of course, reviewed by a
caseworker prior to determining eligibility and calculating benefits.
• Elimination of the printed case file. Some States are looking at electronic imaging
possibilities to increase accessibility to the case file by other offices and reduce file
storage requirements (personnel, space, and equipment).
• Automated budgeting module for calculating monthly budgets based on format of original
source data.
• Ability to make changes to active case files quickly without exiting from current work.
For instance, if a worker receives notice of a change of mailing address for an existing
case, the worker should be able to update the case file on-line without exiting from
current work.
• Create one client record format that is used by all programs so that any changes to client
data need be changed only one time, instead of making the change for every assistance
program in which the individual is participating. This ensures that consistent changes and
updates are made across all programs.
Summary
The level of automated functionality for systems supporting FSP related to completing the
application information and entering the information into the system reflects a generally equal
distribution of States that fall into the high, medium, and low categories, as described on page
II-2, of level of functionality (see Figure 2.2, page II-9). Eighteen States have a low level of
automated functionality in this area (as indicated by a score of 3.9 or lower), indicating that there
is potential for increasing working efficiency in this area. Fourteen States are highly automated
(5.5 or higher scoring range) and nineteen reflect a moderate level (4.0 to 5.4 scoring range) of
automated functionality.
Specifically, caseworkers enter application information during the interview in only 9 States, in
27 States the casev/orker enters application information after the interview, and in 9 States clerks
enter the application information after the interview. Most State systems (47) have the ability
to copy information from historical records into the current record; however, fewer than half the
States have systems with other useful features, such as allowing the worker to skip screens that
are not necessary for a particular application.
Volume II Page II-8
. 'V
/
Figure 2.2 Application Completion and Input of Information Summary
High (14)
Low (IS)
M»dium (19)
B.3 Eligibility Determination/Benefit Calculation
Overview
Once the caseworker has obtained the necessary applicant information, verified the accuracy of
the information provided, and determined household composition, the next step is to calculate the
net income and assets of the household, determine whether the applicant is eligible to receive
food stamp benefits, and calculate the amount of the benefits.
State systems offer a variety of automation features to assist the worker in performing these tasks
for the Food Stamp Program, and, if integrated, for the AFDC and Medicaid Programs. The
distribution of these automation features by State is provided in Table A-3, System Functionality
During Eligibility Determination and Benefit/System Calculations, in Appendix A.
Findings
Some systems determine eligibility based on the information entered into the system; other
systems validate a worker-determined eligibility. Some systems can also perform non-urgent
background processing which allows caseworkers to work more efficiently.
Most systems perform the required benefit calculations in a reasonable and accurate manner. The
level of this functionality varies from systems that calculate benefits from raw income, resource,
and expense data entered by the caseworker to systems that only calculate the benefit based on
the calculation of the monthly budget by the caseworker. Some systems also calculate monthly
income. Whenever caseworker calculations can be eliminated by an automated system,
calculation errors are reduced.
Volume II Page II-9
Summary
The overall level of automated functionality related to determining eligibility and calculating
benefits in terms of high, medium, and low level of automation is reflected in Figure 2.3 below.
Twenty-one States show a high level of automated functionality in this area (scores of six and
seven). 13 show a moderate degree (scores of four and five), and 17 show a low level (scores of
one to three) of automated functionality. This is supported by Table A-3 in Appendix A. A
higher number of systems support automated calculations than support eligibility determination.
Specifically. 44 States used an automated system to calculate monthly income, 41 States used it
to calculate benefits, and 37 States used it to determine eligibility. Only five systems determine
people within the household who comprise the assistance group.
Figure 2.3 Eligibility Determination
and Benefit Calculation Summary
«gh (2 0
Low (17)
Medium (13)
B.4 System Verification
Overview
Caseworkers are required to verify certain applicant information such as residence, birth date,
income, etc. Verification is performed to certify an applicant as eligible for food stamp benefits
and determine the proper amount of benefits. Applicants are required to provide the information
that is requested. If an applicant does not provide the necessary documentation, then food stamp
benefits can be denied. Automated systems that document the request and receipt of verification
information are necessary in some States if benefits are to be denied for inadequate
documentation. Clients have successfully brought suits against some States when the
documentation of verifications requested and received have been inadequate. Paper trails are
dependent on caseworker handwriting and consistent documentation of notices sent requesting the
Volume II .^ Page 11-10
documentation. The majority of States do not encounter adversarial relationships with welfare
advocates.
Verification of application information occurs throughout the application processing period --
from the time the application is logged into the system until eligibility is determined, at
recertification, and no less frequently than annually. Verification can take several different forms,
including review of paper documents and data in automated systems that validates information
provided by the applicant. Some systems require that an entry be made into the system indicating
that each mandatory verification has been performed. Five automated features that assist the
worker in performing and tracking verifications are: SSN verification, tracking outstanding
verifications, missing verification screen alerts, alert printouts, and enforced verification
requirements. These features are detailed in Table A-4, System Verification Features, in
Appendix A.
Some systems provide an automated listing of verifications for the applicant to provide to the
State to process the application. The worker is not required to fill out a form to provide to the
applicant. The verification listing clearly documents (usually in the appropriate language) the
required verifications for the applicant and provides an audit trail and documentation for the
State. This feature can be very helpful in States with numerous client fair-hearing requests.
Findings
Automation of the verification process allows for more on-line verification and results in
improved timeliness of application processing. The most effective form of automatic verification
results from a system that tracks outstanding verifications and provides screen alerts to
caseworkers of missing verifications.
Summary
The distribution of high, medium, and low scores for the levels of system verification
functionality that support the FSP worker are reflected in Figure 2.4, page 11-12. A total of 24
States scored between 3.0 and 4.5 (high), 12 scored 2.0 (medium), and 15 scored between 0.0
and 1.5 (low).
Most States (39) use their automated system to verify SSNs. About half of the States (29) use
their automated system to track outstanding verifications; most of these States use system screen
alerts to notify the caseworker of missing verifications. In addition, about half of the Staes (26)
use their system to enforce verification requirements.
Volume II Page 11-11
Figure 2.4 System Verification Features Summary
High (24)
Low (15)
(12)
B.5 Computer Matching
Overview
In determining eligibility and calculating an applicant's benefit amount, States perform computer
matches on a variety of State and Federal databases to verify client participation, income,
resources, or assets. States are required to use an IEVS to obtain wage and benefit information
for all household members from State and Federal databases, such as State wages, retirement
income from the Social Security Administration (SSA), benefit information from SSA,
unemployment insurance benefits, etc. Members of an applicant household are matched against
the various databases to verify eligibility and determine the amount of benefits to which they are
entitled.
The productivity of a caseworker, however, can be greatly affected by the method of presenting
the match information to the worker. For instance, the paperwork burden can be considerable
if the worker has to review paper printouts reflecting the matching results of all household
members (whether there was a match), then re-enter information from the printout and match
results into the system. Some States set tolerances levels for differences in dollar amounts
beyond which the workers resolve the match and enter information into the system. Other
systems have fully automated matching capabilities so that the worker need only enter a code in
a screen, resulting in calculation or denial of eligibility.
We collected information on the system's automated features associated with computer matching
as well as information about the databases against which States match and whether the match was
performed on-line or off-line. The tables reflecting this information are presented in Appendix
Volume II
vA Page 11-12
A, Table A-5, Computer Matching Functionality (Parts A through D). We were able to develop
an automation score for Part A and Part B reflecting automation features. Part C and Part D are
descriptive in that they show the Federal and non-Federal databases that are utilized in the
matching process. The scoring approach and the features and databases are described for each
table.
Computer Matching Automation Features - As shown in the Table A-5 (Part A), Appendix
A, half of the States perform computer matching at the time an application is logged into the
system.
Computer Matching - System Alerts - System alerts for computer matching are screen messages
to alert the worker about discrepancies or matches that have been identified for applicants and
recipients. Table A-5 (Part B), in Appendix A, shows the variety of system alerts intended to
inform the worker of discrepancies.
Computer Matching - Non-Federal Databases - Table A-5 (Part C) in Appendix A shows the
non-Federal databases that are used by the States for computer matching. The databases required
for IEVS matches are indicated with an asterisk.
This descriptive table shows the various databases a State may match against as well as the
frequency of the matches. Some questions about computer matching could not be answered by
State staff Both Food Stamp Program and MIS staff were asked questions about computer
matching. For this reason, both tables on databases and frequency of matching were not given
automation scores for inclusion in the level of functionality scoring.
Computer Matching - Federal Databases - Table A-5, Part D reflects the Federal databases and
frequency of matches for each State which responded to the questionnaires and/or interview
questions. Most matches with Federal databases are performed on a monthly basis with the
exception of State Data Exchange (SDX) and Beneficiary Data Exchange (BENDEX) databases
which are performed more frequently.
Findings
There appears to be a fine line between too many system alerts and just enough to help a worker
manage his/her workload. The absence of system alerts for computer matching means that a
worker must review paper printouts to identify matches on applicants or recipients.
Some systems perform computer matching more frequently than is required. Depending on the
design of the user interface with the system, increased frequency can result in increased
caseworker workload. Each State must decide whether the increased workload is justified by the
reduced costs associated with reductions in benefits.
Some States perform on-line computer matching with outside databases while others perform
batch matches with on-line access to the results of the match by the worker. In terms of worker
productivity, on-line searches of outside databases did not appear to be more efficient or effective
than on-line access to the results of batch computer matching. On-line access to outside databases
Volume II Page 11-13
can be time consuming to the worker, interrupting the work-flow. On-line access to -ither
assistance files appears to be very helpful.
A review of the benefits achieved from each matching source should be done to determine if the
source provides enough validation to be cost effective.
Summary
Figure 2.5, page 11-15 summarizes the automation scores for Tables A-5 (Parts A and B),
omitting the descriptive tables showing hederal and non-Federal database matching. A score of
5.5 and above shows a high degree of automation, a score between 4.0 and 5.0 shows a medium
degree of automation, and a score between 0 and 3.5 shows a low degree of automation.
Seventeen States show a high level of automated functionality in this area, 16 show a moderate
degree, and 18 show a low level of automated functionality.
The ability of a system to report the discrepancies on-line, prioritize the matches, or indicate
discrepancies that exceed a certain threshold has a greater impact on the efficiency of the
caseworker than the other features.
Only 20 States perform computer matching before the interview is conducted. The majority of
States perform computer matching after the interview, i.e., during the initial certification period,
and at the time of recertification. Thirty-eight State systems perform a complete search of the
databases. Overall, less than half of all States (23) provide on-line alerts to workers about
computer matching discrepancies. Twenty-two systems permit the worker to review the
matching detail on-line. Twenty-five systems indicate only those discrepancies that exceed
specified thresholds.
Volume II ,<A Page II-14
Figure 2.5 Computer Matching Summary
"f^M W*$%P& -°-"8»
Utdum(l6)
B.6 Notice Generation
Overview
Client notices must be prepared and sent in response to a number of circumstances that occur
during application registration, eligibility determination and recertification, benefit calculation,
and case closure. Notices may be completed either manually (with copies maintained in the case
file) or by the system; the notices can be maintained in the system and/or case folder. There have
been a number of court cases throughout the country regarding the clarity of the notices and
whether they are understandable by the recipient and timely. Notice documentation becomes very
important during any fair hearing.
States that have been able to implement notice systems that maintain a historical record of the
notice content and date it was sent or provided to the recipient are in better positions to avoid fair
hearings or provide evidence that the notice was timely and clear.
The are several potential problems associated with manually-prepared notices. For non-English
speaking recipients, translations have to be provided (in some States, the number of languages
for which notices must be prepared are numerous). Copies have to be readable and filed in th^
case folder, creating bulky folders and the potential for misfiling. Caseworker handwriting may
not be clear. And, caseworkers not totally familiar with the policies and procedures of all the
programs may not consistently apply program policies for all recipients.
The paperwork, especially in some locales requiring many notices, can be especially burdensome
on workers. An automated system for producing notices can reduce the paperwork, the paper,
the space required for storing the paper, and State-caused errors, as well as the number of fair
hearings requested by clients.
Volume II . Page 11-15
In at least one State, the State hesitated to close cases due to client failure to appear for a
recertification interview because there was an inadequate record of notification to the client of
the interviews scheduled, action to be taken for failure to appear, and notice of the adverse action.
The level of automated functionality may range from no automated notices to fully automated
noticing systems. Some systems require that a worker select the type of notice required and enter
into the notice system dates and other information. The level of automated functionality is
measured by the amount of input that is required by a worker to generate the notice.
Findings
Table A-6, Notice Generati m Functionality, in Appendix A, reflects the array of automation
features that States use to support the generation of notices.
Systems that generate at least some notices automatically, such as notices about benefit changes
resulting from mass system changes, and also have the ability to generate worker-initiated notices
are more effective than other types of systems. Combining AFDC and FSP notices was found
to be efficient and reduce costs.
States with high participation rates and high worker caseloads have come to rely on automated
notice capabilities to protect the State during court suits and fair hearings.
Summary
Figure 2.6, Notice Generation Summary, page II-17 shows the distribution of States falling into
the high, medium, and low ranges for level of automated functionality related to generating
notices. A score between 0.0 and 3.5 indicates a low automation level, between 4.0 and 5.5
indicates a moderate degree of automation, and 6.0 to 8.0 indicates a high degree of automation.
Twenty States show a high level of automated functionality in this area, 16 show a moderate
degree, and 15 show a low level of automated functionality. Most (44) of the States generate
notices automatically and 32 of these also generate notices when the worker initiates them.
However, only half (26) of the States have systems with combined FSP and AFDC notices.
Volume II Page 11-16
-
Figure 2.6 Notice Generation Summary
Low (15)
Ugh (20)
UtOfcj* (16)
B.7 Monthly Reporting
Overview
While monthly reporting is no longer an FSP requirement for all FSP recipients, a subset of
recipients, such as those who receive income and/or those whose status changes during the month,
are required to report. The purpose of the reporting is to adjust eligibility and/or benefit levels
as needed. Some States limit the reporting to a quarterly basis, others require monthly reports
from all households, regardless of any change in status.
The level of automated functionality is measured by the amount of worker input required to mail
the monthly reports, generate related client notices, and enter the receipt of the report and any
changes that were reported by the clients.
Findings
Monthly reporting is a function that can be made significantly less time consuming by means of
automated features. The automated features that are most effective are: the system determines
cases which are required to report, the system produces monthly reports for mailing, and the
system generates warning notices for those clients who report late.
Summary
Table A-7 in Appendix A, Monthly Reporting Functionality, presents system features for seven
monthly reporting characteristics. More than half the States (26) require monthly reporting and
most have developed a variety of automated features to assist the worker.
Volume II Page 11-17
Figure 2.7. page 11-18 shows the distribution of States requiring monthly reporting that fall into
the high, medium, and low ranges for the level of automated functionality associated with
monthly reporting. Fifteen States show a high level of automated functionality in this area
(scores of 3.0 and above), only six show a moderate degree (score of 2.5), and only five show
a low level of automated functionality (score of 2.0 and below). This figure indicates that those
States that perform monthly reporting have automated the proce« to a great degree.
Figure 2.7 Monthly Repo '" S
low (5)
H gh (1 5
B.8 Program Management
Overview
The automated features that support program management provide State FSP management staff
with reports on caseworker performance, backlog statistics, and client service measurements. The
ability of managers to obtain management reports upon request is not a widespread feature of the
automated systems. Generally, the eligibility determination/benefit calculation systems have been
developed to support program functionality at the caseworker level, with management-level ad
hoc reporting functionality developed and implemented after implementation, if at all. Most
managers indicated that the system support for ad hoc reporting was minimal, whether from an
automated perspective or from the management information systems group supporting the system
and program staff. Table A-8. Program Management Functionality, in Appendix A, gives a score
for each State's level of program management automation.
Volume II Page 11-18
Findings
A variety of automated features have been developed by States to support program management.
Some of these features are integral to the management of the programs supported by the system;
others are add-on features not considered necessary for program operations.
Summary
Figure 2.8, page 11-19 reflects the distribution of States that have a high, medium, and low level
of automated functionality associated with program management. The majority of States (26)
have a low level of automated functionality in this area (scores of 1.5 and below) and only ten
have a moderate level (scores of 2.0 to 3.0). The number of States with a high level of
automated functionality (scores of 3.5 and above) is only slightly less (15) than has been the case
with other automation features. The most popular automated feature is E-mail for sending
messages and memos. This feature is included in 33 State systems. Other widespread features,
included in the systems of about one third of the States, are daily reports of work needing
attention and on-line case narratives.
Figure 2.8 Program Management Summary
High (i:
Low (26)
B.9 Issuance
Overview
The primary focus of the data collection teams was on the eligibility determination and be 'efit
calculation systems that support the FSP. Food Stamp Program staff familiar with the systems
were interviewed and either they or information systems support staff or caseworkers provided
demonstrations of the systems. Staff responsible for managing issuance systems, since they were
usually located in other organizational units or agencies, often did not participate in the
Volume II Page 11-19
discussions. FSP staff answered questions about the issuance systems to the extent of their
knowledge, i.e.. from the perspective of the caseworker and the degree to which the issuance
systems had an impact on FSP program effectiveness.
Findings
Table A-9 (Part A) reflects the types of issuance utilized within the State. Fewer than half the
States (18) mail the majority of their coupons, and seventeen of these also issue authorization-to-participate
cards and/or provide direct access systems or other issuance methods, such as
electronic benefit transfer (EBT). Over 30 States are undertaking an EBT effort, or are in various
stages of investigating EBT.
The majority of States have the same basic system featuies:
System links document numbers of original and replacement issuances
System creates monthly issuance Files for ongoing cases
System creates daily issuance files for new and other special issuances
System check for duplicate issuance is automated
System provides on-line display of entire issuance history
Automated features tend to be in areas that make mail issuance more efficient, such as zip code
edits and techniques that facilitate stuffing coupons into envelopes.
Although most of the systems check for duplicate issuance, create a monthly issuance file for
ongoing cases, create a daily issuance file for new or special issuances, and prevent issuance until
all application data are complete, many States provide no other autumated issuance features
(Table A-9. Parts B and C). In States that have decentralized issuance methods, the preparation
of consolidated monthly reports representing all of the issuance locations and/or counties can be
quite burdensome.
Summary
Only fifteen States reflect a low level of automated functionality associated with food stamp
issuance (scores of 4.0 and lower), a number in keeping with the general distribution of low
automation States. What appears different in this chart is the lower proportion of States with a
high level of automation (scores of 6.5 and above), with only 13 States falling into the highly-automated
sector, a number somewhat under the norm for highly-automated systems. A medium
level of functionality corresponds to scores of 4.5 to 6.0.
Volume II N> Page 11-20
Figure 2.9 Issuance Summary
High (13)
Low (15)
Medium (23)
B.10 Claims Collections
Overview
The claims and collections functionalities are often not integrated with the automated systems.
When system transfers were at their peak and the Alaska/North Dakota models were being
implemented in a number of States, the original models did not contain an integrated claims and
collection component. States that have subsequently automated claims and collections have
usually done so in association with their accounting systems. Recoveries in the form of
recoupments for active cases are often handled separately and as a part of the issuance system.
Table A-10 (Parts A and B), Claims and Collections Functionality, in Appendix A shows the
automation level of each State in regard to claims collection.
Findings
Table A-10, Automated Claims and Collections Functionality (Parts A and B), in Appendix A
rates the levels of functionality for all States in regard to automated claims and collections
processes. When the claim system is integrated with the FSP system, there is greater pressure
on the line worker to identify potential claims and enter information into the system that refers
the case to an investigator, at which point it is out of the hands of the worker. Eligibility
workers operating in an environment that is not well supported by automation tend not to
perceive the identification of potential fraud, abuse, or errors as a high priority. The review of
historical case records to extract information needed to calculate the amount of a claim or
recovery can be very burdensome on the caseworker.
Volume II , Page 11-21
Staff responsible for investigations need information to pursue this task; access io historical
records can be very helpful in this process. Some States also have designated collections staff
responsible for tracking the status of outstanding claims and recoveries. If these are tracked in
the accounting system and not linked in some manner to issuance systems, the burden on the
worker can be considerable. The separation of duties between caseworkers, investigators, and
accounting staff that is needed has led to fragmented systems supporting each of the groups,
sometimes resulting in poor performance in identifying potential cases for investigation and
collecting or recovering funds due to the State.
The review of automated claims and collection systems was difficult in that personnel
demonstrating the principal FSP system did not have access to claims and collections components
and/or were not familiar with the functionality of any automation supporting these areas. The
review identified the following features:
Claims systems that were integrated with the principal FSP system
Data exchanges between FSP and collection systems
Ability to track claims status
Automated generation of notices regarding overpayments and underpayments
On-line entry by caseworker of cause of overpayments and underpayments
On-line entry by caseworker of suspected fraud
Automated creation of collection record
Automated calculation of correct benefits
Automated calculation of monthly recoupment amounts
Automated subtraction of recoupment amounts from issuance
Automation collection method determination
Ability of worker to view complete collection record
On-line record of outstanding claims and claims collected
Summary
The distribution of States into high, medium, and low categories of automation reflected a smaller
number of States in the medium category. The number of States that fall into the high level of
automated functionality (scores of 10.0 and above) is slightly more than in other functional areas.
The number in the low level of automated functionality category (scores of 7.0 and lower) is
about the same. However, there are fewer States falling into the middle category (scores of 7.5
to 9.6) than has appeared for other functions.
Only 31 States have their claims systems integrated with their FSP system. The feature included
in the most State systems (40) is tracking of claims status. Other features included in about half
of the systems are the generation of notices of overpayments and underpayments and the entering
on-line of overpayment and underpayment cause and if fraud is suspected. The automated
collection features used by the most States are calculating the recoupment amount and subtracting
it from the monthly allotment, maintaining an on-line record of outstanding claims and claims
collected, and creating a claims collection record after a claim has been established.
Volume II \\ Page 11-22
Figure 2.10 Claims and Collections Summary
High (26)
C. LEVEL OF INTEGRATION
This automation study focused on the level of integration of the automated systems that support
the FSP. Automated systems are critical tools used by States to deliver services and benefits and
the level of integration can have a considerable effect on the effectiveness of the State's program
administration. But automation is only a tool. The types of integration can include application
integration and organization integration.
Application Integration - The level of system integration is based on the number of programs
served by a system as well as the number of systems required to support the FSP. Whether the
system is a Statewide system is also factored into the analysis. Table A-ll, Level of System
Integration, in Appendix A indicates separate systems existing within each State, the programs
supported by the systems, whether it is a Statewide system, and an indicator of the integration
level. The integration level was assigned by each evaluation team according to the information
reflected in this table as well as the team's own subjective perception of integration from the
perspective of a line worker. Although there are many types of line workers (e.g., caseworker,
clerical staff, supervisors, investigators, claims collectors, issuance staff, etc.N 'He greatest weight
was given to the level of integration at the level of the caseworker (i.e., income maintenance
worker), who is responsible for determining the eligibility of an individual or household for
benefits as well as for the calculation of benefits for delivery. Since caseworkers comprise the
largest group of line workers, the potential for increased efficiency and effectiveness was felt to
be greatest at this level.
The fact that a State may have many different systems supporting the FSP as well as other
programs does not necessarily indicate that the level of integration is low. For example, if a
caseworker is able to seamlessly access, update, and exchange information with other systems
without exiting one system to go to another or using another terminal or microcomputer, the team
Volume II - (,• Page 11-23
could have assigned a higher level of integration to the State's systems than would otherwise be
apparent from the table. Information in the table, however, will explain why a particular State
may have received a low score for level of integration. Nebraska has three separate systems
supporting FSP, and the primary FSP system does not support AFDC. Medicaid, or General
Assistance. This State received a very low integration level rating.
Organizational Levels of Integration - There are many different levels of organizational
integration within a State which may have an impact on a program's effectiveness and
performance. The more organizational units that are involved in the maintenance of on-going
systems or the development of new systems, the more communication and coordination and
staffing resources are needed to accomplish the system objectives. Some examples include:
• Departmental Integration - A single automated system may support Medicaid eligibility,
food stamps, and AFDC for two or more departments within a State. If an automated
system supports programs that are located within one department, communications and
coordinations between program policy staff and MIS staff are facilitated. As the number
of departments that serve one client increases, the requirements for information exchange
(both automated and non-automated) and coordination increases.
• Divisional-level Integration • Integration of public assistance and food stamp programs
within one division seems to facilitate the ease with which changes and enhancements in
the existing system can be made as well as the ease of system development efforts. For
instance, a Department of Social Services (DSS) may have one division that is responsible
for "income maintenance" that includes both FSP and AFDC (and perhaps other
programs). Or DSS may have two separate divisions for AFDC and FSP.
• Statewide Integration - Some State Data Centers serve all State agencies and are
organizationally in a separate department. Some States have data centers that are devoted
to the social service and/or public assistance programs. Caseload size is a major factor
determining the organization of the data center and the ability of the State Data Center
to handle the business of the health, social services, nutrition, and income maintenance
programs. Some State agencies responsible for administering FSP, AFDC, and Medicaid
have their own data centers and/or mainframes for their systems.
Integration at the Worker Level - The level of integration at the worker level
determines training approaches, dissemination of program policy changes, and on-going
training for systems. Integration at the caseworker level enables States to provide a single
point of entry for social and health service programs, which many believe to be necessary
for certain clients ultimately to become self sufficient. Program integration at the worker
level is difficult if the systems that support the workers are not integrated and if those
systems do not support the worker in determining eligibility, making referrals, and
identifying the totality of services that are available for a client.
Program Integration at the Worker Level The level of program integration at
the field office level tends to vary according to the State and characteristics of that
State, county, or region (i.e., urban/rural), and is generally left to the discretion of
Volume II r -\ Page 11-24
county supervisors or district managers. Most States have generic workers, some
of whom are specialized for certain programs, such as Medicaid eligibility. In
some States, generic workers utilize different automated systems for the programs
they serve.
System Integration at the Worker Level This varies greatly among States.
Some systems appear fully integrated at the worker level, but are separate systems.
In other States, the systems are completely separate, requiring duplicate data entry
from the same application form into two separate systems. A generic worker
could be using two separate systems.
Summary
Table A-l 1 in Appendix A provides specific information as to the integration level of each State,
including the number of systems and number of programs served. States with a score of 5.0 (the
maximum score) are judged to have a high level of integration. A total of 13 States fall in this
category. States with a score of 4.0 to 4.9 have moderately high level of integration; 12 States
are in this category. Scores between 3.0 to 3.9 indicate a moderate level of integration; only 7
States fall into this category. A score between 1.1 and 2.9 indicates a low level of integration;
8 Stares have a low level of integration. States with an integration level score of 1.0 or lower
have a very low level of integration; 10 States are in this category. (Some States, such as
California, did not receive any integration level score due to the structure of the State's automated
systems.)
D. DEGREE OF AUTOMATION
Overview
The degree of automation of a State system is determined by a combination of factors. These
include the number of automated features, the amount of duplicate steps in the process, and the
amount of unusual or non-routine effort in the process.
Findings
Table A-12, Degree of Automation/Stage of Development, in Appendix A summarizes the
findings presented above related to level of automated functionality and level of integration. The
first column of the exhibit, level of functionality, comes from computations of the multiple tables
and scores given to the various automated features. The second column shows the level of
integration scores taken from Table A-l 1. Although the scores in the first and second columns
were derived through different methods, when the first and second columns are added, a score
for the degree of automation is created.
The level of functionality score in column one was arrived at by averaging the scores for the
different functions (after standardizing each function's set of scores because the score for the
different functions have different maximum values) and assigning five levels based on the normal
distribution probability covered by the averages of all 51 States. The level of integration score
Volume II rc Page 11-25
in column two is derived from Table A-l 1 which factors the number of separate systems existing
within a State, the programs supported by the systems, and the comprehensiveness of the system
into a relative rating on a scale of 1.0 to 5.0.
The degree of automation score can range from 1.0 to 10.0. Twenty-three States were found to
have a high degree of automation (indicated by a score of 6.6 to 10.0). Eighteen States have a
moderate degree of automation (a score between 3.6 and 6.5). Nine States have low degree of
automation (a score between 1.0 and 3.5) is nine (see Figure 2.11, page 11-27).
Summary
Given the distribution of the degree of automation scores, no specific conclusions can be drawn.
It seems that the more automated systems are more effective and efficient but other factors, such
as the age of the system, make it difficult to make generalizations. Each State has specific client
needs and a unique automated data processing environment that dictates the most appropriate
level of automation to meet its needs with maximum effectiveness.
Figure 2.11 Degree of Automation Summary
High (?3)
Mediun (IS)
Volume II M Page 11-26
E. STAGE OF DEVELOPMENT
Overview
Table A-12, Degree of Automation/Stage of Development, in Appendix A indicates the stage of
development of each State system. This summary of the development status of all State's
systems is based on information gathered during the site visits which occurred from August 1992
to December 1993. The last two columns, the numbers of years since the primary system was
completed and the status of any replacement systems, respectively, show how old the existing
system is and the stage of the replacement system, if there is a replacement system.
Findings
Most of the States with older systems (nine years or older) are actively engaged in developing
another system. Table A-13, Degree of Automation/Stage of Development, is an arrangement
of the stage of development information according to the age of the system, ordered from oldest
to newest system. The older systems with the lowest degree of automation are almost all in some
stage of system design, development, or implementation.
The status of replacing system is defined as one of the following stages:
Investigating -
Planning -
A pre-planning or investigation stage. This phase can include activities
such as observing other State systems, attending Agency for Children and
Families (ACF) transfer conferences, American Public Welfare Association
(APWA) conferences, and viewing vendor demonstrations.
The planning stage. This phase includes gathering information, deciding
on the most appropriate type of system, and producing Advanced Planning
Documents.
Developing -
Implementing -
The development stage. This phase is the initial part of implementation,
in which requirements, system specifications, software development occurs.
The implementation stage. In this phase, the system has been tested,
training is usually taking place, conversion may be occurring, and
implementation of hardware and software may be occurring in local
offices.
• Development -
Halted
In some instances development has been halted due to factors such as
change in scope, request by the Federal government, or contractor protests.
Operational - Operational system stage indicates that an operational system is in place
and no plans exist to replace it or make major changes.
A more detailed breakdown of the current status of system development efforts is presented in
Table A-14, Current Status of System Development Efforts. This table summarizes the current
Volume II ?>± Page 11-27
status of system development efforts for the ED/BC system. Data were collected on the stage
of development of a system and whether it is operational or in the implementation, development,
or planning phases.
If the system is operational, the age of the basic eligibility determination system is also noted as
it provides an indicator of potential timeframe for system replacement or major enhancement.
Table A-15, Development Status of Primary System Supporting the Food Stamp Program, in
Appendix A shows the development status and other data collected for operational Food Stamp
Program systems including whether major enhancements are planned or underway and the nature
of the enhancements.
Summary
A total of 28 States had an operational system in place with no plans for changing it. Of the 23
States which had systems under development, 2 were in the pre-planning phase, 11 were in the
planning phase. 6 had systems actually under development, 3 were in the process of implementing
a newly-developed system, and 1 was in a development stoppage phase. Of the 23 systems under
development, 17, or 74 percent, were in States where the existing system was nine years old or
older.
F. RELATIONSHIP OF DEGREE OF AUTOMATION TO STAGE OF
DEVELOPMENT
Overview
If a State rates low as to degree of automation (see Table A-12 in Appendix A), this indicates
a lack of advanced features, such as electronic application capability or automatic verification.
States with a low degree of automation rating should have a new system in the planning,
development, or implementation stage.
Findings
A review of the States with a low or medium degree of automation demonstrates that:
• Of those States with a low degree of automation, 55 percent are planning a new system,
11 percent (one State) are in the implementation or development stage, 11 percent (one
State) has halted development, and 11 percent (one State) has no development plans.
• Of those States with a medium degree of automation, 16 percent are planning a new
system, 28 percent are in the development stage, 5 percent (one State) are in the pre-planning
or implementation stage, and 44 percent have no development plans.
Not surprisingly, of those States with a high degree of automation, 78 percent have no
development plans at this time.
Summary
Volume II V, Page H-28
Of the States that were rated as having a low or medium degree of automation, 66 percent have
recognized this deficiency and are in one stage or another of developing a replacement system.
One-third of these States, or a total of nine, do not have any plans at this time to upgrade or
replace their existing systems. These States in particular need further attention to determine the
reasons for the effectiveness or lack of effectiveness of their current systems and encourage the
development of replacement systems as warranted.
Volume II > ^ Page H-29
III. STATE SYSTEM DEVELOPMENT PROCESSES
A. BACKGROUND
This chapter discusses the current approaches used by the States in the design and development
of information systems. This activity has been changing rapidly; the use of computer-aided
systems engineering (CASE) tools has become widespread. It has been shown that, during
development, adherence to an industry-accepted system development life cycle (SDLC) standard
is a necessary component of a successful system implementation. The use of industry standards,
combined with strong project management skills and cost controls, makes the success of the
project more likely.
FCS is seeking new approaches for reviewing and approving State APD requests. One approach
is to evaluate how closely a proposed State solution parallels accepted industry standards. The
latest industry approach for the development of efficient and cost-effective systems is both
mission- and business-oriented. Systems must be cost-effective as well as serve the stated goals
and objectives of the organization. Systems that support the Food Stamp Program should be
moving in that direction.
Some of the major characteristics associated with industry standards for software development
include:
• A recognized, commercial SDLC methodology is used to plan and track the planning,
development, and implementation of a software project.
• Users and systems and management staff participate in all phases of the project planning
and development cycle. This includes using field staff to validate requirements and
functionality and participate in conversion and implementation activity.
• There are periodic reviews of project progress to include timeliness and quality of
deliverables and cost-effective progress toward the projected goals of the development
task.
• Standard hardware and software platforms are used to process the finished system product.
In Federal systems, a number of design philosophies have become norms in the creation of
acceptable application systems:
• Interoperability - the ability to interact with other system architectures through open
system interfaces or standard hardware/software design techniques.
• Portability - the ability to transfer a software application from one hardware platform to
another without re-engineering.
• Expandability - the capability to expand the hardware and/or software platform without
re-engineering or major hardware restructuring.
Volume II «• Page III-1
• Transferability - the ability to migrate the application to another hardware system or
installation without major disruptions to the client's expected level of service.
In most projects, success can be measured by a number of different factors. These factors enable
an oversight organization to evaluate the level of achievement of many aspects of the project
before the actual experience is gained from the users of the system. Factors evaluated in this area
include:
• The project provides regulatory and design criteria for functionality and performance that
meet planned expectations.
• The original cost and scheduled development estimates are met within accepted variances.
Modifications due to changes in regulations, project priorities, project funding and the like
should be taken into consideration when evaluating the achievement of project estimates.
• Appropriate levels and areas of the organization participate in the system development and
the participation is appropriate to the development task at hand. For example, the use of
field staff to test screen layouts and functionality is more appropriate than having them
review programming documentation.
• A senior-management oversight group is used to evaluate progress, provide directional
guidance, and provide support and encouragement during the planning and development
process.
• A proactive post-implementation process is undertaken to evaluate and document the
actual benefits achieved.
The use of formal development techniques assist in the creation of effective and efficient systems,
but do not guarantee success. Success can only be achieved by creating a well-defined plan,
effective execution of the plan, and support of all agencies involved in the financial, resource and
regulatory aspects of the project.
B. PROJECT MANAGEMENT FACTORS
Each State has its own preferred method of managing system projects. In reviewing the project
management methods and the outcome of a variety of system efforts, we conclude that the
following factors have a significant impact on the success of the project effort.
Organization - Every organization uses formal or informal project staff to manage the design
and implementation effort of a major systems project. One of the keys to a project's success
depends on the thoroughness and effectiveness of this staff to acquire, utilize, and manage the
resources necessary to staff and execute the project plan.
In our reviews of the State Food Stamp systems projects, we found that this project organization
was used consistently in every State's project process. Factors such as when resources were used,
Volume II ^ ^ Page III-2
involvement of a formal senior management oversight group, and level of commitment of the
staff, often play a significant role in the overall effectiveness of the management process.
Project managers were assigned from within the State organization to lead the projects. In 63
percent of the projects, the manager was assigned to the task full-time or almost full-time with
little or no additional functional responsibilities (Table B-l, Project Staffing Chart, in Appendix
B). In addition, in 76 percent of these same projects, the project manager came from within the
State's public assistance or systems staff (Table B-l). The remainder of the full-time project staff
was usually composed of several program, MIS or contractor staff. When necessary, many States
used additional personnel to staff specific project tasks, using program field staff, internal MIS
technicians, or contractor personnel to staff the requirements.
Another aspect of the project organization that was reviewed was the consistency of staffing of
key management positions during the entire project cycle. Projects vhose key staff members
change more frequently would seem to be less effective than those whose management team
remains intact for the duration of the task. While no direct correlation can be made between
consistent staffing and project success, special attention should be paid during FCS oversight of
those projects where such turnover is found, to ensure that the project does not suffer.
Table B-l, Project Staffing Chart, contains information on the level of staffing consistency for
most States. Some information is missing for States whose system development projects were
too new to have staffing experience or whose projects ended long ago and no meaningful
infoimation was available. In 67 percent of the projects the project manager remained throughout
the project; in 31 percent of the projects there w<is one change in the project manager. There was
a problem with the consistency of the project manager on only one project. Although there was
more turnover in other types of project staff, th; problem was not acute (e.g., only one State had
a high turnover in key FSP staff and only four States had a high turnover in key MIS staff). The
project staffing score in Table B-l was computed based on the project manager's background,
the amount of time he or she committed to the project, and whether the project manager was in
charge throughout the project (the amount of time devoted to the project is weighted); the
maximum score possible is 4.
Many States utilized executive oversight committees whose role was to monitor the overall
direction and progress of the project and establish guidelines and priorities for project resources.
In several situations, the oversight committee played a more active role and was involved in
nearly every project decision. The more common practice was to deal with directional, staffing
resource and policy decisions so that the project would not be unduly burdened with these
extraneous issues. The use of this type of committee should be encouraged in future projects
since it binds senior management support directly to the task and helps ensure that the appropriate
level of attention and resources are provided.
State Staff and Contractor Participation/Roles - A second important aspect of the project
management process is to determine what organizational areas are represented in the design and
management of the task and at what point in the project process does the involvement occur. For
example, avoiding the use of program field staff in the requirements definition phase could create
a void in the definition that would need to be corrected later in the project cycle. These types
Volume II , , Page III-3
of time-consuming and expensive revisions could be avoided if the right players are involved at
the right time.
Table B-2, Programmatic User Participation, in Appendix B presents programmatic involvement
and Table B-3. MIS Participation, in Appendix B, presents State MIS participation in the
planning, design, and implementation phases of the project. In addition, the type of role
undertaken is depicted. Most projects involved a user group (only four States did not use one)
and almost all States involved MIS staff (only two States did not). User groups were involved
more in the planning and design phases of the project than in the implementation phase (89 and
85 percent of the States involving user groups used them in the planning and design phases,
respectively, whereas only 78 percent of the States involving user groups used them in the
implementation phase). Both user groups and MIS staff were involved most heavily in the role
of making recommendations to the project. User groups made recommendations and
reviewed/approved project plans in 85 percent of the projects involving user groups, but they
established requirements for the system in only 74 percent of these projects. Similarly, MIS staff
made recommendations in 85 percent of the projects involving MIS staff, but established
requirements in only 76 percent of these projects and reviewed/approved plans in only 70 percent.
An overall participation rating is also provided in these tables. The rating is an accumulated
score that represents the level of participation rather than the level of success of the participation
process. More weight is given to those groups which were actively involved in all three aspects
of the project process than if they were only involved with one or two phases. For the
programmatic staff, establishing requirements is rated as the most important role and providing
recommendations as the least important. Since MIS staff are more valuable in reviewing the
project design and performance aspects of a project, review and approval was rated high and
making recommendations w..s rated the lowest. The maximum score possible for user
participation is 11 and the maximum score possible for MIS participation is 6.
We feel that the more meaningful the involvement of both State programmatic and MIS staff, the
more effective the resulting project planning and design efforts. Without the input from both of
these groups, starting with the initial planning aspects of the project, significant omissions of
requirements; design features; and system performance characteristics may arise to delay project
completion and add to project costs.
Table B-4, Contractor Roles - Project Planning, in Appendix B presents the involvement of
contractors in each State's planning effort and Table B-5, Contractor Roles - Project
Development/Implementation, in Appendix B, presents contractor involvement in the design and
implementation stages. Contractors were involved more in the design and implementation stages
than in the planning stage. Most of the States (82 percent) used a contractor for at least one step
of the design and implementation stages, whereas only 61 percent of the States used a contractor
for some step of the planning stage.
Contractors play a major role in the development and implementation of public assistance systems
and appear to continue as support staff long after project completion. State staffs have been
severely impacted by reductions-in-force and hiring freezes the past several years and find
themselves unable to support these types of systems. Each State is assigned a rating of contractor
Volume II Page III—4
involvement to indicate its level of dependence on external resources to complete the project.
The rating for project planning was computed so that States with less contractor involvement
received a higher score than States with more contractor involvement; the maximum rating
possible is 15. The rating for the project's development and implementation phases was
computed so that a State with a moderate amount of contractor involvement received a higher
score than a State with none/little or a great deal of contractor involvement; the maximum
possible rating is 27. Especially with current projects, this dependence is increasing and may
have a significant impact on project costs for future projects. Emphasis should be placed on the
use of as many internal resources as possible to reduce the contractor requirements and enable
State staff to assume the full system support roles soon after project completion.
C. USE OF SYSTEM DEVELOPMENT LIFE CYCLE METHODOLOGIES
An SDLC methodology represents an established, proven set of tools, approaches, and steps
which are undertaken during the planning, design, and implementation of a systems project. Its
purpose is to ensure a consistent and uniform approach to the development of a useful and cost-effective
product. The key to the benefit of using an established standard is that the results can
be predicted based on the quality of information utilized. It is differentiated from "State standard
methods" in that the process can be traced and the results tracked against the standard. If the
standard is unique to a specific organization, there is no uniform way to evaluate the effectiveness
of the result.
The importance of using an SDLC methodology is that FCS can review any project, determine
where it is in the life cycle, and determine how well the State has progressed without spending
an inordinate amount of time researching the background of the project. The existence of
checkpoints, reviews, and documentation facilitate improved project tracking. This should enable
problem situations to be identified earlier, assuming regular FCS site visits and reviews occur
during the project. With early detection of problem areas, corrective action can be initiated by
the appropriate agency to correct the deficiencies.
States that were using an accepted SDLC methodology were also using the technique when
maintaining the application. Based on the size and sccpe of the enhancement, some or all of the
steps were being followed. For relatively simple SDLC tasks, steps such as requirements
definition and prototyping were not used; however, alternatives analysis, general and detailed
designs, and unit/systems testing steps were utilized.
Table B-6, System Development Life Cycle Steps, in Appendix B lists the identifiable steps that
were used to evaluate how each State used the SDLC method. Table B-7, State Usage of System
Development Life Cycle Methodology, in Appendix B depicts the number of steps each State
used during its most recent project and whether the methodology was used for the duration of the
project. The SDLC score was computed as a combination of the consistency with which the
SDLC methodology was used (based on the number of steps used) and whether the SDLC
methodology was used throughout the project; the maximum score possible is 5. Eighteen States
were rated as not having used any steps or having used less than 10 SDLC steps. Of the States
that used 10 or more SDLC steps, only 64 percent used the methodology throughout the project.
With 39 percent of the States not following a recognized SDLC methodology and another 22
Volume II *\1 Pace III-5
percent not following an SDLC methodology throughout the project, there is a significant chance
for inefficiencies to enter the project process, costing the States and FCS time and money.
D. HARDWARE/SOFTWARE PLATFORMS
Industry hardware and software standards are well-defined and followed by virtually all of the
States. This section compares State public assistance system platforms to "industry standards".
To begin with, hardware and software industry standards are not fixed, rigid specifications. There
are several generations of IBM mainframe hardware that run the same software systems and
provide efficient processing capability. In turn, earlier release levels of system or applications
software are not, necessarily, less efficient than the current release level.
Tables B-8, Central Processing Unit (CPU) Inventory Table, B-9, Software Inventory Table, and
B-10, Network Inventory Table, in Appendix B depict the installed hardware and software
systems used to support food stamp systems at the time of the State visits. I ty-one of the
States use IBM or IBM-compatible mainframe systems under MVS/ESA (32), MVS/XA (8), or
VM/DOS/VSE (1). CICS (40), ADABAS (14), and IMS (15) are also well represented.
The currency of the hardware generation or software release level is less important if the State's
configuration provides appropriate functionality and processing power and is within the vendor's
maintenance support umbrella. For instance, if a State is using a mainframe system that is one
generation behind the current offering (i.e., IBM 3090/200E) under a -1 generation operating
system (i.e., MVS/XA), then the configuration has the capability to grow into larger processors
as the workload expands. In addition, the functionality of the MVS/XA operating system
supports all hardware and software functions, and the cost of additional equipment on the used
market is 40 to 90 percent less than the cost of comparable new equipment. This situation may
be much more cost effective than if the State had acquired the current generation of hardware and
software.
More important to the overall view of a State's configuration adequacy is the amount of product
expansion available to meet workload growth. This is especially true in those States that provide
support to multiple agencies in a common State data center. Since all agency workloads are
growing, system performance, reliability, and software restrictions are based on platform
constraints.
In our visits, systems capacity, reliability, expandability, and software constraints did not appear
to be areas of concern. Some States have specific shortcomings (i.e., floor space limitations to
equipment growth, inadequate telecommunications network capacity, etc.), but there were no
overall problem areas. A number of States were using a form of distributed processing
capability, but, for the most part, this approach has not yet found its way into the mainstream of
public assistance processing.
Volume II U Page III-6
E. OBSERVATIONS AND CONCLUSIONS
Attention to the project management process is an important factor in the overall success of any
systems development project. While good management practices does not guarantee success,
ineffective management will add to the time and cost of developing efficient system solutions.
Based on the observations made during our visits to the States, the following
observations/conclusions can be made:
• The more successful project management teams have been composed of staff from all
departments with a vested interest in the design and functionality of the system.
Involvement normally begins with the initial planning stage and continues through project
implementation.
• Use of an executive oversight committee to establish direction and resolve priority and
resource conflicts should be strongly encouraged. This group will tie the State senior
management more closely to the project and ensure that all State organizations, as much
as possible, support the project effort.
FCS should ensure that project checkpoints are included in every project plan, reflecting
the deliverables to be provided and the cost expended at each point in the project process.
This information will enable FCS to more closely track the progress of the project and
determine delays and problem areas before they become major stumbling blocks.
• FCS should strongly encourage the use of an accepted SDLC methodology for use by the
States throughout the entire project process. This will help ensure that the project can be
effectively tracked and adequate planning and resources have been assigned.
• States use accepted industry standard hardware and software to support the public
assistance systems. Issues of compatibility, reliability, and expandability are being
adequately addressed.
Volume II 4 Page III-7
IV. SYSTEM TRANSFERS
A. BACKGROUND
FCS policies regarding the transfer of existing systems were intended to reduce development costs
and allow operational systems meeting FSP and State needs to be implemented quickly.
Although regulations do not require that States transfer systems if they can justify new
development efforts, many States have interpreted the regulations as requiring them to transfer
existing systems from other States. States have complied with this requirement to varying
degrees; some States transfer the design concept, then develop a customized system, while others
transfer the existing system, dropping and adding functionality to meet their specific requirements.
The intent of the Federal requirement was to reduce the time it takes a State to implement an
automated system, the cost associated with implementation, and the risk of failure. In reality,
costs have continued to grow; proposed development and implementation time estimates have,
generally, been exceeded; and some transferred systems have failed to meet all FSP and State
automation requirements.
There are no guidelines for evaluating a transfer candidate's efficiency and effectiveness in its
existing State or for estimating the performance of the transferred system in the new State. The
level of sophistication and functional capability of the transferred system must be compared
against the new processing environment. The performance of the existing systems may not
compare favorably to the performance possibilities of newer, State-of-the-art technologies. For
instance, newer hardware, software, and telecommunications architectures may provide faster
response times, make it easier to implement software changes, and be easily expandable to
accommodate fluctuations in caseload sizes. While most of the characteristics and circumstances
noted in the regulations are easily compared among systems, determining the efficiency and
effectiveness of systems operating in two different States is more difficult.
B. FREQUENTLY SEEN CHARACTERISTICS OF SUCCESSFUL TRANSFERS
In identifying factors that contribute to successful system transfers, the degree of transfer must
be identified before defining a successful transfer. A transfer may range from a conceptual
transfer to a complete transfer of all existing application code. For the purposes of this study,
a system is considered to be a transfer if a State indicated that it transferred a system and the
Federal government approved the transfer.
The following characteristics can be used to judge the relative success of a system transfer:
Ratio of actual to estimated development time and cost figures - There are a variety of factors
that can impact the development time and cost of a major application development project. It
is expected, however, that many of these factors should be accounted for in the initial time and
cost estimates and that the final statistics should be within an acceptable range.
User satisfaction - End users of the system should feel that the system helps them perform their
work more efficiently and effectively and does not create additional stress within the work
Volume II L] I Page IV-1
environment. Stress can be caused by poor performance, inadequate functionality, or system
interface design problems.
Department of Health and Human Services (DHHS) certification and FCS approval -
Formal acceptance by the regulatory agency indicates that functionality is appropriate and
accurate.
Expected benefits and improvements in operational support achieved by the system - A
formal review of improvements, such as staff reductions, error rate improvements, and enhanced
collections documents the cost/operational benefits achieved.
Table C-l, Survey of State Transfer Satisfaction, depicts the sources of information that relate
to the success factors mentioned above. Twenty-nine States either transferred their current system
or are in the process of transferring a system. The ratios of actual to projected cost cover a wide
range of fiscal performance. Some projects finished well under budget, while a number of others
spent much more than originally anticipated (some eight or nine times more than the original
estimate) In some cases, the financial figures are not well documented, due to the age and
accuracy of some of the financial data. In a number of cases, all the financial information was
not available or found, and, since 17 of the States were still in the development process, full costs
of their systems had not yet been determined.
User Satisfaction data in Table C-l were gathered from surveys conducted as part of the State
Automation Systems Study for all 50 States and the District of Columbia. The numbers in the
table represent averages of the responses of all the eligibility workers or supervisors in a
particular State. The averages below 2.00 for any of the five categories that address how satisfied
eligibility workers and supervisors are with their respective systems show that 6 States have
systems that are not providing an adequate level of support to their work requirements. One
State, Kentucky averaged below 2.00 in 3 of the 6 categories surveyed; Rhode Island was below
average in 2 categories. Other States appear to be providing above average levels of support.
Overall, the results of the User Satisfaction Surveys did not provide clear indications of States
with specific and consistently-reported strengths and weaknesses. Most responses were in the
middle of the range and very few offered narrative comments to augment the multiple choice
questions. Using a similar survey effort during transfer candidate analysis or during new system
pilot testing might provide valuable input to the State's evaluation process and FCS' project
review process, although more subjective responses might generate more insightful feedback.
Since FCS no longer requires a formal post-implementation review, there is no current vehicle
to determine how well the system meets the regulatory, functionality, and performance criteria
established for the project. It is strongly recommended that FCS reinstate the post-implementation
review and formalize it in a manner similar to the Family Assistance Management
Information System (FAMIS) certification of the Department of Health and Human Services.
This type of review is the best vehicle to capture the capability, performance, and benefits of the
new system.
States have not formally tracked and reported on the achievement of quantifiable benefits.
Without such information, it is impossible to ascribe any supportable cost or performance savings
Volume II ^ ' Page IV-2
to the system. Efforts to review such data resulted in no meaningful information being found in
any State.
C. FACTORS AFFECTING TRANSFER SUCCESS
A number of factors were discovered during our visits to the States that had a direct bearing on
the ability of the State to achieve its developmental objectives within the originally projected time
and cost parameters. The type of system needed to support the Food Stamp Program normally
requires a multi-year effort to develop and implement. During the development period, changes
to the economy, the State political environment, and regulatory requirements can drastically alter
the initial plans and estimates. Technological changes resulted in some of the older, established
transfer candidates being less effective in the newer systems environment, since newly available
features could not be utilized without major rework. Finally, priorities, staffing resources, budget
reductions, and similar State-oriented factors can change during a long development cycle and
can impact the State's ability to complete the project on time and under budget.
Transfer Selection Criteria - Each State developed its own matrix of elements that were
considered important in the evaluation of candidates for transfer. The factors chosen were not
selected based on any regulation or standard format, but were based on what resources the State
needed to staff and administer the Food Stamp Program. While each review and evaluation can
be taken as a unique process, there were several common criteria that were shared by many of
the States. Table C-2, States Transfer Selection Criteria, in Appendix C shows the most
important transfer selection criteria for each State that has transferred or is in the process of
transferring a system. These 29 States selected the following criteria most frequently as being
important in the selection decision:
• System functionality (22 States)
• Similar caseload and/or FSP organization (19 States)
• F/VMIS certification of the existing system (17 States)
• Similar hardware/software platform (15 States)
Overall, the States used similar criteria to examine which systems were the best candidates for
transfer. One key criterion was DHHS FAM1S certification. This would be another reason for
FCS to create its own formal certification process or work with DHHS, in some manner, to share
in DHHS' certification reviews and provide an FCS approval to the finished system.
Other factors that were mentioned as criteria for system transfers during State visit interviews
included:
• Urban/rural State environment
• County versus State program administration
• Geographic size and characteristics of the State
• Caseworker roles and responsibilities
• State ADP development and operational expertise
• Centralized versus distributed systems
• Historical impact of State advocacy groups
Volume II ^ lj Page IV-3
• State employee unions
Use of certain criteria was associated with greater user satisfaction with the operational system.
These criteria included similar hardware/software platform and system functionality.
In addition to the factors above, a number of issues regarding specific system characteristics and
the ability of the individual State to manage a $20 to $50 million implementation project were
mentioned. These factors did not all have a direct effect on the system selection process, but they
did have an impact on a State's ability to successfully complete a project of this magnitude. The
major areas influencing whether a State can successfully complete a project of this type are:
Age of Transferred System - The age of the transferred system potentially affects its
performance and efficiency in the receiving State's environment. Older systems also can
require more modifications than originally anticipated to improve functionality and/or
efficiency.
• Project Management - The development approach used SDLC methodology, the
effectiveness of the system development management process control, and the
effectiveness of contractor support played significant roles in the overall effectiveness of
the transfer effort.
User Involvement in the Transfer Selection Process - The State headquarters and field
staff food stamp operations staff usually have a thorough understanding of the
requirements of an automated system to support the Food Stamp Program. The
effectiveness of the system will ultimately be determined by the user satisfaction level,
as well as the operational efficiency of the system. User involvement is considered to be
a critical factor in an effective selection process for a transfer system candidate.
• State Management and Oversight Capabilities - There are two areas that impact total
project success, as well as transfer success. One is a high level of State management
oversight in the system development and implementation process. Such oversight can
help reduce directional and priority conflicts. The second is the extent and quality of this
involvement. Detailed management involvement can encourage State and contractor staff
to meet target dates and deadlines, ensure that the system meets user objectives and
requirements, and verify that the benefits associated with implementation can be achieved.
• Effectiveness of Consulting Efforts - States with inadequate ADP expertise and project
management capabilities can utilize knowledgeable and experienced consultants to greatly
enhance the chances for a successful transfer. The success achieved can be related to the
degree of involvement of both State and consultant staff in the planning and development
of the new system, as well as the State's ability to maintain the system after the consultant
has finished the project.
• State Procurement Policies and Practices - States are now required to comply with
Federal procurement requirements for competition. States that have not historically
operated in this manner have had to change their procurement practices to comply. If
Volume II ,j c Page IV-4
contractors are required as consultants or for system transfer, development, or
implementation, the type of procurement options that are available can affect the project
management approach. For instance, the type of contract vehicle used during various
project phases may influence the quality of the work effort. The procurement strategy
also can influence the availability and selection of competent contractors. Types of
procurement strategies include: firm fixed price, time and materials, cost plus fixed fee,
incentive fees, award fees, subcontracting, sole-sourcing, purchase orders, and others.
• State Budgetary Constraints - If the public assistance program budget is reduced or
limited in any way during the development process, the system design effort may be
impacted. Reduction in system functionality, fewer technical staff to commit to the
project, or selection of low-cost solutions to meet reduced budgets may limit the
effectiveness of the final system.
• Regulatory Environment - Developing a system during a period when many regulatory
changes in the program take place can negatively impact the timeframe for system
development.
Thoroughness and Proficiency of the Selection Process - To transfer an appropriate system,
States must be able to obtain pertinent information about other systems. Table C-3, States
Methods for Obtaining Transfer System Information, in Appendix C presents the sources used
by the transfer States to gather evaluation information and includes the number of States which
used each type of source. System demonstrations, State inquiries and visits, and reviews of
system documentation are the most consistently used methods. Discussions with system vendors
and contractors and the two regulating Federal agencies, DHHS and FCS, were used less
frequently. Demonstrations by vendors and discussions with FCS were methods used by States
whose system users ultimately were more satisfied with the system.
Table C-3 also presents the number of systems reviewed for transfer and ultimately considered
feasible. Most States reviewed more than two systems and many reviewed more than five. Most
States found one or two systems to be feasible and often based the final decision on cost or
convenience.
In most cases, a State would assemble a number of staff from diverse areas, including food stamp
operations, MIS, and management, to conduct the review. We feel the makeup of the review
team and the approach used may have significant influence on the selection process. Program
users, for instance, would not be in a position to understand potential technical problems in
transferring a particular system and technical personnel would not understand the degree of
functionality or automation needed for their State.
Volume II . Page IV-5
The transfer process itself does not include a number of required tasks performed in a specific
order; however, there are certain activities that should be included in every evaluation:
• Compare the similarity of functions, caseload volume, system interactions, and
hardware/software technologies of the potential transfer candidate to what is used in the
receiving State. Major discrepancies should not exclude a candidate, but a detailed plan
should be developed to address how the differences will be corrected during development.
• Determine whether the bidding contractor has experience with the recommended system.
If the contractor and contractor staff have had previous experience with the system, one
would expect that the system would be implemented more quickly than with a contractor
who has not had prior experience. In addition, the proposed development plan should be
more thorough in addressing those areas where changes must be made.
• Identify the operational problems of the system, in terms of poor performance or missing
functionality that will need to be added.
• Decide what other programs need to be added and what functional modifications will be
necessary to make the system practical for the receiving State. The addition of assistance
programs, such as Medicaid eligibility or Child Support Enforcement, is probably a more
significant change than the addition of a claims tracking module.
• Determine if changes in system architecture, hardware, or software are needed. Items
such as workstation functionality, distributed versus centralized, or changes in database
platforms will require extensive rework. All systems require some change to meet user
needs and State requirements. Users and technical staff may have different perceptions
about the amount of change that is required and the perception is very subjective.
All States appear to have done a reasonable job in selecting the system they needed to support
their development effort. The subsequent level of success achieved by any particular State was
not unduly affected by the platform it chose. Other factors in the development cycle seem to
have had more of an impact.
Nearly every State mentioned that the lack of a centralized database of transfer information had
a negative impact on the system transfer process. The existence of a centralized, national
clearinghouse of information addressing the current status of each State's automated system or
development effort would have made the selection process easier to undertake and eliminated a
great deal of duplicate effort. If, for example, FCS maintained an up-to-date database with
information about each State's food stamp system, States would have a source of information that
could be used to determine which candidates best met their needs, what problems had been
encountered, what corrections had been made, and what results had been achieved. In turn, this
would make the selection process faster and more meaningful. An additional benefit of a
centralized, national database would be that FCS would have a more accurate and complete
picture of every State system that it funds and monitors.
Volume II Page IV-6
Project Management Team - The membership in the project management team should be
representative of the State's senior management perspective and have appropriate representation
from both the programmatic and MIS areas to cover functional and technical requirements.
Senior management's goals and expectations regarding the timeframes and cost parameters that
are acceptable to the State must be inputs in the development of the new system. Functional and
workflow considerations from the programmatic areas are an integral consideration and should
be reviewed by all areas that will be supported by the new automation vehicle. In an integrated
solution, every area which will be supported by the system should participate in the selection to
ensure that its functional requirements are taken into consideration.
The management roles and involvement in the transfer process should be the same as in the
management of the planning and development aspects of the project. Goals, functionality, and
technical platform issues should be jointly developed prior to the selection evaluation. Review
criteria and candidate rating should include all participants so that all adjustments and
compromises can be arrived at jointly. This type of partnership will help resolve conflicts that
often arise during the developmental phases of the project.
FCS should take an active role in the transfer process by providing system transfer information
and observing the selection process for every State. The level of success in this, the first stage
of a development effort, may be an accurate predictor of how successful the full project may be.
Trouble in the system selection stage may be symptomatic of management problems and should
serve as a warning to FCS to increase its oversight of the specific project.
Adequacy of State/Contractor Resources/Skills to Complete the Project - Success or failure
of a project, whether a new State system development effort or a transfer from another State, will
depend more on the project management and technical resources available than any other factor.
If State funding or priorities change, an adequately staffed project will be able to modify the plan;
however, if State staff or funding for contractor support is cut, there may be no way to
reasonably salvage the project.
!n today's environment, very few States have adequate technical or programmatic staff to
develop/modify a new public assistance system without extensive external contractor support and,
in many cases, yearly ongoing maintenance support. In the majority of the States, internal MIS
staffing levels have been frozen or reduced over the last several years. Many of the States do
not have sufficient staff to develop new systems or work closely with contractor staff to learn the
design characteristics of their efforts to thus be able to effectively support the new system after
the development is completed. It has become incumbent on the contractor community to be the
major source of new system implementation staffing. With this comes higher costs. It costs a
State substantially more to have a contractor develop and modify a system than it would if the
work is done by State staff The average development effort now costs between $20 and $40
million for a two- to four-year effort. It may be very difficult to deal with the financial
requirements of higher State staffing levels today, but it is debatable whether this course would
be more expensive than using external contractors. In the long run, however, the avoidance of
paying additional personnel payroll and benefits expenses may cost substantially more in system
development costs.
Volume II ^ I Page IV-7
Degree of Transfer and Customization - The degree of transfer is based upon what was
transferred. A low degree of transfer would occur when only the system concepts are used and
actual coding, files, and formats reworked. A high degree would entail the use of the system
code, screen layouts, and report formats as they were used in the original system. In the first
column of Table C-4 in Appendix C, each transfer State is given a rating for degree of transfer
on a scale of 1 (conceptual only) to 10 (entire system transferred, as is). Some States, that were
in the early stages of the selection process, did not have information to answer this question.
The second column in Table C-4 measures the degree of customization. This relates to the
amount of modification required for the transferred system to meet the State's functional and
technical requirements. A rating of 20 means that 20 percent of the transferred system needed
to be modified to some extent. A rating of 100 means that every aspect was changed to some
extent. The degree of customization does not directly coincide with the degree of transfer. For
example, as shown in Table C-4, Rhode Island transferred its entire system, but then modified
75 percent of it, while North Dakota also transferred its entire system, but only customized 30
percent. For all transfer States, the lowest modification percentage was 20 percent (North
Carolina and Tennessee). This level still represented a significant amount of extra effort to
customize the application. There was insufficient data to attempt to correlate the cost of
modifying transfer systems versus new development, but it appears that the two costs are not
appreciably different. More detail on cost is provided in Chapter V.
D. OBSERVATIONS AND CONCLUSIONS
Overall, the majority of the States view transferring as the preferred method for developing new
public assistance systems. Even those States that internally developed their current systems feel
that the benefits of using a transfer system as the basis for a new system development effort now
is preferable to a custom design and development effort.
The primary benefit of transferring a system relates to the presence of a proven foundation with
specific functionality. One of the most difficult aspects of developing a new system is to
determine what it is supposed to do and how it will do it. With no starting point, it normally
takes a long and difficult planning process to design the basic structure and gain agreement on
the basic functionality of a new system. Even in the era of joint application development (JAD),
it takes a great deal of effort and compromise to reach consensus on such features. With a
transfer, the effort is confined to defining what to add, delete, or modify; this is much easier to
accomplish than starting from scratch.
The advantages and disadvantages of system transfers, as indicated by each State, are presented
in Table C-5 in Appendix C. The advantages most frequently cited were reductions in risk (30),
development time savings (29), and cost savings (28). The area considered to be the biggest
disadvantage was the need to customize the transferred system (24).
Other observations and conclusions related to system transfers arc as follows:
• A centralized database of information on the status of public assistance systems in each
State should be created and maintained by FCS. This data will be useful for State
Volume II >x Page IV-8
referrals when new projects begin, can provide FCS with current information on the
development and/or operational status of each State system, and will help ensure that only
solid, proven systems are used as transfer candidates.
State transfer evaluation teams should be composed of staff from all affected departments
to ensure that all functional and technical issues are fully addressed and understood by the
evaluation team.
The FCS post-implementation review process should be reinstated to validate the accuracy
and functionality of the final system and ensure that the actual benefits achieved are
quantified and compared to the projected benefits in the APD. Without this effort today,
there is virtually no formal review of benefits achieved and no way to determine the cost-effectiveness,
if any, of the overall development effort.
Volume II C^O page IV-9
V. STATE AUTOMATION COSTS AND COST ALLOCATION METHODOLOGIES
A. BACKGROUND
States receive funding from three sources for system development efforts: DHHS for AFDC,
Medicaid, and other DHHS programs if included in the integrated system; USDA for food
stamps, and the State itself. The rate at which the Federal agencies fund the system development
effort varies. In general, DHHS provides 90 percent funding for development in support of
DHHS programs and FCS provides 50 percent funding (formerly 75 percent and 63 percent) for
the portion allocable to food stamps.
When States decide to improve or replace their automated systems, they must justify their
decisions not only to FCS and DHHS, but also to their State legislatures or budget officers.
States must present their business case to the legislature, in some cases utilizing the cost benefit
analysis prepared for FCS and DHHS. In many cases, however, the justification is basic, such
as the need to produce timely, accurate benefits to the needy and avoid sanctions resulting from
a high error rate. ^Tien State budgets are tight, as they have been in recent years, the availability
of Federal funding may be one of the predominant incentives for a major system effort, without
which the Stale could not proceed with its effort to automate.
The States request approval and funding from FCS during the planning and development process
by means of the Advanced Planning Document. Although FCS may approve the total system cost
at the time of the first APD submitta!, this funding amount may be modified through an
Advanced Planning Document Update (APDU) over the course of the project. Each modification
resulting in changes in system functionality and design, contract modifications, and costs must
receive FCS approval. The original system budget is, therefore, modifiable as long as sufficient
justification exists for the changes. Because of reasonable funding requests, the eventual cost of
the project may far exceed that which was originally approved.
The basis for the allocation of costs varies from State to State and sometimes during the course
of the development effort in the same State. A project may be conditionally approved until an
allocation approach has been agreed to by all