Evaluation of the nutrient standard menu planning demonstration final report

              Ab« Associates Inc.
Evaluation of the
Nutrient Standard
Menu Planning
Demonstration
55 Wheeler Street
Cambridge, Massachusetts
02138-1168
617 492-7100 telephone
617 492-5219 ftaimiU
Final Report
August 1998
Contract No. 53-3198-4-001
Hampden Square, Suite 500
4800 Montgomery Lane
Bethesda, Maryland
20814-5341
301 913-0500 telephone
301 652-1618fusmik
640 North LaSalle Street
Suite 400
Chicago, Illinois
60610-3781
312 867-4000 telephone
312 867 -4200 ftatmtU
Prepared/or:
JohnEndahl
U.S. Dept. of Agriculture
Food and Nutrition Service
Office of Analysis and Evaluation
3101 Park Center Drive
Alexandria, VA 22302
Prepared by.
Mary Kay Fox
Mary Kay Crepinsek
Nancy Buratein
Patty Connor
Frederic Glantz
Abt Associates Inc.
55 Wheeler Street
Cambridge. MA 02138
Table of Contents
Page
List of Exhibits iii
Acknowledgments xiv
Chapter 1 Introduction 1-1
The National School Lunch and School Breakfast Programs 1-1
The NSMP Demonstration 1-5
The School Meals Initiative for Healthy Children 1-5
Organization of this Report 1-7
Chapter 2 Evaluation Design , 2-1
Research Objectives 2-1
Evaluation Design 2-2
Standards Used in Evaluating Nutrient Content 2-6
Estimation of Program Impact 2-8
Chapter 3 Implementation ofNSMP in Demonstration SFAs 3-1
Selection of Demonstration SFAs 3-1
Early Implementation Challenges 3-4
Status of Implementation in Spring 1997 3-4
Implementation ofNSMP in Demonstration SFAs 3-8
Characteristics That May Predict Successful NSMP Implementation 3-18
Chapter 4 Nutrient Content of Lunches Before and After NSMP 4-1
Overview of the Analysis 4-1
Frequency of Foods Offered in Lunch Menus 4-2
Nutrient Profile of Lunches Served Before and After NSMP 4-5
Differences Between Weighted and Unweighted Analyses 4-11
Chapter 5 Nutrient Content of Breakfasts Before and After NSMP 5-1
Frequency of Foods Offered in Breakfast Menus 5-1
Nutrient Profile of Breakfasts Served Before and After NSMP 5-3
Differences Between Weighted and Unweighted Analyses 5-10
Chapter 6 Impact ofNSMP on Program Operations and Costs 6-1
Program Participation 6-1
Meal Acceptability 6-3
Meal Costs 6-4
A la Carte Revenues 6-12
Use of USDA-Donated Commodities 6-12
Menu Planning Practices 6-16
Abt Associates Inc. U Table of Contents i
Table of Contents
(Continued)
Page
Chapter 7 Burden and Acceptability of the NSMP System 7-1
Start-up Costs 7-1
Maintenance Costs 7-6
SFA Directors' Perceptions About Burden of NSMP Tasks 7-9
Perceptions of School Food Service Staff 7-11
SFA Directors' Attitudes Toward NSMP 7-13
State Directors' Attitudes Toward NSMP 7-16
Chapter 8 State Monitoring of SFAs Using the NSMP Option 8-1
State Monitoring Requirements 8-1
State Agency Experiences with Demonstration SFAs 8-3
Changes in Monitoring Procedures to Accommodate SMI Reviews 8-6
Perceptions about SMI Monitoring Requirements 8-11
Chapter9 Accuracy of SFAs'Nutrient Analyses 9-1
Problems Detected in Nutrient Analyses Conducted by SFA Staff 9-1
Findings from On-Site Meal Observations 9-12
References
Appendix A Supplementary Exhibits: Nutrient Content of Lunches A-1
Appendix B Supplementary Exhibits: Nutrient Content of Breakfasts B-1
Appendix C Study Implementation C-l
Appendix D Methodology Used in Nutrient Analysis D-l
Appendix E Methodology Used in Study of Meal Costs E-l
Appendix F Methodology Used in Sampling and Weighting Surveys of
School Food Service Staff F-l
AM Associates Inc. Table of Contents ii 3
List of Exhibits
Page
Exhibit 1.1 Traditional Meal Pattern Requirements for the National School Lunch Program . 1 -3
Exhibit 1.2 Traditional Meal Pattern Requirements for the School Breakfast Program 1-4
Exhibit 2.1 Evaluation Design 2-3
Exhibit 2.2 Data Collected at Each Measurement Point 2-4
Exhibit 2.3 Nutrient Standards Used in the Evaluation 2-7
Exhibit 3.1 Characteristics of Demonstration SFAs 3-2
Exhibit 3.2 Final Implementation Status of Demonstration SFAs 3-5
Exhibit 3.3 Reasons for Withdrawal from the Demonstration 3-7
Exhibit 3.4 Software Systems Used by Demonstration SFAs 3-9
Exhibit 3.S Disclosure of Nutrition Information by Demonstration SFAs 3-12
Exhibit 3.6 Cafeteria Promotions Conducted by Demonstration SFAs 3-13
Exhibit 3.7 Use of Fortified Foods by Demonstration SFAs 3-14
Exhibit 3.8 Reported Successes and Problems in Meeting Nutrient Standards 3-16
Exhibit 3.9 Time Required by Demonstration SFAs to Achieve Full Implementation 3-17
Exhibit 3.10 Characteristics of Demonstration SFAs by Final Implementation Status 3-19
Exhibit 4.1 Foods Offered More and Less Often in NSMP Lunch Menus Compared to
Pre-NSMP Lunch Menus 4-4
Exhibit 4.2 With the Exception of Calories in Secondary School, Lunches Served
Before and After NSMP Provided More than One-third of the RDA 4-6
Exhibit 4.3 Lunches Served After NSMP Provided Fewer Calories from Fat and
Saturated Fat than Lunches Served Before NSMP 4-8
Exhibit 4.4 More SFAs Met Standards for Calories from Fat and Saturated Fat in
Lunches After Implementing NSMP 4-10
AM Associates Inc. List of Exhibits Si f
List of Exhibits
(Continued)
Page
Exhibit 4.5 Lunches Served Before and After NSMP Met the NRC Recommendation
for Cholesterol but Not for Sodium 4-12
Exhibit 4.6 Weighted and Unweighted Analyses of 1997 Lunches Yield Slightly Different
Mean Values for Calories and RDA Nutrients, but Conclusions About
Whether Lunches Met NSMP Standards are Similar 4-14
Exhibit 4.7 Weighted and Unweighted Analyses of Lunches Yiel \ Similar Results
for Calories from Fat, Saturated Fat, Protein, and Carbohydrate 4-16
Exhibit 4.8 Weighted and Unweighted Analyses of 1997 Lunches Yield Slightly Different
Mean Values for Cholesterol, Sodium, and Fiber, but Conclusions About
Whether Lunches Met NRC Recommendations are Similar 4-17
Exhibit S. 1 Foods Offered More and Less Often in NSMP Breakfast Menus Compared
to Pre-NSMP Breakfast Menus 5-2
Exhibit 5.2 With the Exception of Calories, Breakfasts Served Before and After NSMP
Provided More than One-Fourth of the RDA 5-4
Exhibit 5.3 Breakfasts Served After NSMP Provided Significantly Fewer Calories from
Fat and Saturated Fat than Breakfasts Offered Before NSMP 5-7
Exhibit 5.4 More SFAs Met Standards for Calories from Fat and Saturated Fat in
Breakfasts After Implementing NSMP 5-8
Exhibit 5.5 With the Exception of Sodium in Secondary Schools, Breakfasts Served
Before and After NSMP Met the NRC Recommendations for Cholesterol
and Sodium 5-9
Exhibit 5.6 Weighted and Unweighted Analyses of Breakfasts Yield Similar Mean
Values for Calories and RDA Nutrients 5-11
Exhibit 5.7 Weighted and Unweighted Analyses of Breakfasts Yield Similar Results
for Calories from Fat, Saturated Fat, Protein, and Carbohydrate 5-13
Exhibit 5.8 Weighted and Unweighted Analyses of 1997 Breakfast Menus Yield
Comparable Results for Cholesterol. Sodium and Fiber Content 5-15
Exhibit 6.1 Average Daily NSLP Participation Was Essentially Constant Over the Course
of the Demonstration 6-2
Abt Associates Inc. „ List of ExnlWts Iv r
List of Exhibits
(Continued)
Page
Exhibit 6.2 Mean SBP Participation Rates Before and After NSMP 6-3
Exhibit 6.3 Kitchen Managers' Perceptions About Changes in Lunch Plate Waste
Since Implementation of NSMP 6-5
Exhibit 6.4 Kitchen Managers' Perceptions About Changes in Breakfast Plate Waste
Since Implementation of NSMP 6-7
Exhibit 6.5 School Food Service Staff Perceptions About Students' Acceptance of School
Meals Since NSMP 6-8
Exhibit 6.6 On Average, the Cost of Producing a Reimbursable Lunch Under NSMP
Was Comparable to or Lower than the Cost Before NSMP 6-10
Exhibit 6.7 Costs of Producing a Reimbursable Lunch Before and After Implementation
of NSMP 6-11
Exhibit 6.8 On Average, the Cost of Producing a Reimbursable Breakfast Under NSMP
Was Comparable to or Lower than the Cost Before NSMP 6-13
Exhibit 6.9 Costs of Producing a Reimbursable Breakfast Before and After Implementation
of NSMP 6-14
Exhibit 6.10 A la Carte Revenues Before and After Implementation of NSMP 6-15
Exhibit 6.11 Value of USDA-Donated Commodities Used Before and After Implementation
of NSMP 6-16
Exhibit 6.12 On Average, Lunches Offered After NSMP Included as Many Options as
Lunches Offered Before NSMP 6-18
Exhibit 6.13 The Amount of Variety in Weekly Lunch Menus Planned After NSMP Was
Comparable to Menus Planned Before NSMP 6-19
Exhibit 6.14 On Average, Breakfasts Offered After NSMP Included as Many Options as
Breakfasts Offered Before NSMP 6-21
Exhibit 6.15 The Amount of Variety in Weekly Breakfast Menus Planned After NSMP Was
Comparable to Menus Planned Before NSMP 6-22
AM Associates Inc.
\
List of Exhibits v
List of Exhibits
(Continued)
Page
Exhibit 7.1 Non-Labor Start-up Costs Reported by Demonstration SFAs 7-2
Exhibit 7.2 Labor Hours Devoted to NSMP Implementation, by Task 7-5
Exhibit 7.3 Most of the Time Devoted to NSMP Implementation Was Spent on Data
Entry and Staff Training 7-7
Exhibit 7.4 Time Required for NSMP Maintenance Compared to Time Spent on Previous
Menu Planning Practices 7-8
Exhibit 7.5 SFA Directors Found Obtaining Menu Production Data to be the Most
Burdensome NSMP Task 7-10
Exhibit 7.6 School Food Service Staff Perceptions About Impact of NSMP on Overall
Paperwork Burden 7-12
Exhibit 7.7 School Food Service Staff Perceptions About Effect of NSMP on Job 7-13
Exhibit 7.8 SFA Directors' Opinions About NSMP 7-14
Exhibit 7.9 SFA Directors' Perceptions About Advantages and Disadvantages of NSMP ... 7-15
Exhibit 7.10 State Directors'Opinions About NSMP 7-17
Exhibit 7.11 State Directors' Estimates of Non-Demonstration SFAs' Menu Planning
Selections 7-18
Exhibit 7.12 State Directors' Perceptions About Why SFAs WiU Not Select NSMP 7-19
Exhibit 7.13 Recommendations of SFA and State Directors to Increase Likelihood of
Successful NSMP Implementation 7-21
Exhibit 8.1 Reasons for State Visits to Demonstration SFAs 8-3
Exhibit 8.2 NSMP Topics Covered During State Visits to Demonstration SFAs 8-4
Exhibit 8.3 Methods Used by State Agency Staff to Assess Accuracy of SFAs' Nutrient
Analyses 8-5
Exhibit 8.4 Summary of Problems Identified by State Monitoring Staff 8-7
Inc. U«t of Exhibits vi n
List of Exhibits
(Continued)
Page
Exhibit 8.5 Changes in State Schedules and Procedures to Accommodate SMI Monitoring .. 8-9
Exhibit 8.6 State Directors' Opinions About SMI Monitoring Requirements 8-12
Exhibit 9.1 Errors Found in Weekly Menu Analyses Completed by SFA Staff 9-3
Exhibit 9.2 Errors Found in Recipes Entered and Analyzed by SFA Staff 9-6
Exhibit 9.3 Data Entry Errors in Foods Added to the Nutrient Database by SFA Staff 9-8
Exhibit 9.4 Prevalence of Missing Data in Information Provided by Manufacturers and
Data Entry Decisions Made by SFAs 9-9
Exhibit 9.5 Age Groups Analyzed by Predominant Grade Configuration 9-11
Exhibit 9.6 Adherence to Planned NSMP Menus 9-13
Exhibit 9.7 Items Omitted and Not Substituted in Observed Lunches 9-M
Exhibit 9.8 Unanalyzed Items Offered During Observed Lunches 9-15
Exhibit 9.9 Adherence to Planned Portion Sizes 9-16
Exhibit A.1 Mean Calorie and Nutrient Content of Lunches Served Before and After
NSMP A-1
ExhibitA.2 Mean Nutrient Profile of Lunches Served Before and After NSMP Compared
to NSMP Standards and NRC Recommendations A-2
Exhibit A.3 Number of SFAs That Satisfied Nutrient Standards and Recommendations for
Lunch Before and After NSMP A-3
Exhibit A.4 Distribution of Percentage of Calories from Fat and Saturated Fat in Lunches
Served Before and After NSMP A-4
Exhibit A.5 Mean Calorie and Nutrient Content of Lunches Served Before and After
NSMP: Unweighted Analysis A-6
Exhibit A.6 Mean Nutrient Profile of Spring 1997 Lunches: Comparison of Weighted and
Unweighted Analyses A-7
Inc. I1 LMofExhMts vN
List of Exhibit*
(Continued)
Pay*
Exhibit A.7 Mean Nutrient Profile of Spring 1994 Lunches: Comparison of Weighted and
Unweighted Analyses A-8
Exhibit A.8 Mean Nutrient Profile of Lunches Served Before and After NSMP Compared
to NSMP Standards and NRC Recommendations: Unweighted Nutrient
Analysis A-9
Exhibit A.9 Number of SFAs That Satisfied Nutrient Standards and Recommendations for
Lunch Before and After NSMP: Unweighted Nutrient Analysis A-10
Exhibit A. 10 Distribution of Percentage of Calories from Fat and Saturated Fat in Lunches
Served Before and After NSMP: Unweighted Nutrient Analysis A-l 1
Exhibit A. 11 Mean Calorie and Nutrient Content of Lunches Served Before and After NSMP:
Middle Schools and High Schools A-12
Exhibit A.12 Mean Nutrient Profile of Lunches Served Before and After NSMP Compared to
NSMP Standards arid NRC Recommendations: Middle Schools and
High Schools A-13
Exhibit A. 13 Number of SFAs That Satisfied Nutrient Standards and Recommendations for
Lunches Before and After NSMP: Middle Schools and High Schools A-14
Exhibit A. 14 Mean Calorie and Nutrient Content of Lunches Served Before and After NSMP:
Middle Schools and High Schools Unweighted Analysis A-13
Exhibit A.15 Mean Nutrient Profile of Lunches Served Before a.id After NSMP Compared
to NSMP Standards and NRC Recomrnendaticns: Middle Schools and High
Schools Unweighted Nutrient Analysis A-16
Exhibit A. 16 Number of SFAs That Satisfied Nutrient Standards and Recommendatior
for Lunches Before and After NSMP: Middle Cchools and High Schooi-
Unweighted Nutrient Analysis A-17
Exhibit A.17 Mean Calorie and Nutrient Content of Lunches Served Before and After NSMP
Elementary Schools in Early-Implementing SFAs A-19
Exhibit A. 18 Mean Nutrient Profile of Lunches Served Before and After NSMP Compart 1
to NSMP Standards and NRC Recommendations: Elementary Schools
in Early-Implementing SFAs A-20
Inc. ^ List of Exhibits vW 7
List of Exhibits
(Continued)
Exhibit A. 19 Number of SFAs That Satisfied Nutrient Standards and Recommendations for
Lunches Before and After NSMP: Elementary Schools in Eariy-Implementing
SFAs A-21
Exhibit A.20 Mean Calorie and Nutrient Content of Lunches Served Before and After NSMP:
Elementary Schools in Early-Implementing SFAs
Unweighted Nutrient Analysis A-22
Exhibit A.21 Mean Nutrient Profile of Lunches Served Before and After NSMP Compared
to NSMP Standards and NRC Recommendations: Elementary Schools in
Eariy-Implementing SFAs Unweighted Nutrient Analysis A-23
Exhibit A.22 Number of SFAs That Satisfied Nutrient Standards and Recommendations for
Lunches Before and After NSMP. Elementary Schools in Early-Implementing
SFAs Unweighted Nutrient Analysis A-24
Exhibit A.23 Mean Calorie and Nutrient Content of Lunches Served Before and After NSMP:
Secondary Schools in Early-Implementing SFAs A-25
Exhibit A. 24 Mean Nutrient Profile of Lunches Served Before and After NSMP Compared to
NSMP Standards and NRC Recommendations: Secondary Schools in
Eariy-Implementing SFAs . A-26
Exhibit A.25 Number of SFAs That Satis*rd Nutrient Standards and Recommendations for
Lunches Before and After NSMP: Secondary Schools in Early Implementing
SFAs A-27
Exhibit A.26 Mean Calorie and Nutrient Content of Lunci.es Served Before and After NSMP:
Secondary Schools in Early-Implementing SFAs
Unweighted Nutrient Analysis A-28
Exhibit A.27 Mean Nutrient Profile of Lunches Served Before and After NSMP Compared to
NSMP Standards and NRC Recommendations: Secondary Schools in
Eariy-Implementing SFAs Unweighted Nutrient Analysis A-29
Exhibit A.28 Number of SFAs That Satisfied Nutrient Standards and Recommendations for
Lunches Before and After NSMP Secondary Schools in Eariy-Implementing
SFAs Unweighted Nutrient Analysis A-30
Inc. List of Exhibits ix
/0
List of Exhibits
(Continued)
Page
Exhibit A.29 Grade Groups Used by Demonstration SFAs and Associated Nutrient Standards
for Lunches A-32
Exhibit A.30 Mean Percent FDA Provided by Lunches Served Before and After NSMP
Compared to Minimum Nutrient Standards Defined in Regulations A-33
Exhibit A.31 Mean Percent RDA Provided by Lunches Served Before and After NSMP
Compared to Minimum Nutrient Standards Defined in Program Regulations:
Unweighted Nutrient Analysis A-34
Exhibit B. 1 Mean Calorie and Nutrient Content of Breakfasts Served Before and After
NSMP B-l
Exhibit B.2 Mean Nutrient Profile of Breakfasts Served Before and After NSMP Compared
to NSMP Standards and NRC Recommendations B-2
Exhibit B.3 Number of SFAs That Satisfied Nutrient Standards and Recommendations for
Breakfast Before and After NSMP B-3
Exhibit B.4 Distribution of Percent of Calcries from Fat and Saturated Fat in Breakfasts
Served Before and After NSMP B-4
Exhibit B.5 Mean Calorie and Nutrient Content of Breakfasts Served Before and After
NSMP: Unweighted Analysis B-5
Exhibit B.6 Mean Nutrient Profile of Spring 1997 Breakfasts: Comparison of Weighted
and Unweighted Analyses B-6
Exhibit B.7 Mean Nutrient Profile of Spring 1994 Breakfasts: Comparison of Weighted
and Unweighted Analyses B-7
Exhibit B.8 Mean Nutrient Profile of Breakfasts Served Before and After NSMP Compared
to NSMP Standards and NRC Recommendations: Unweighted Nutrient
Analysis B-8
Exhibit B.9 Number of SFAs That Satisfied Nutrient Standards and Recommendations for
Breakfast Before and After NSMP: Unweighted Nutrient Analysis B-9
Exhibit B.10 Distribution of Percent of Calories from Fat and Saturated Fat in Breakfasts
Served Before and After NSMP: Unweighted Nutrient Analysis B -10
tac UatofExMM* a
//
List of Exhibits
(Continued)
Page
Exhibit B. 11 Mean Calorie and Nutrient Content of Breakfasts Served Before and After NSMP:
Middle Schools and High Schools B-12
Exhibit B. 12 Mean Nutrient Profile of Breakfasts Served Before and After NSMP Compared to
NSMP Standards and NRC Recommendations: Middle Schools and
High Schools B-13
Exhibit B. 13 Number of SFAs That Satisfied Nutrient Standards and Recommendations for
Breakfast Before and After NSMP: Middle Schools and High Schools B-14
Exhibit B. 14 Mean Calorie and Nutrient Content of Breakfasts Served Before and After NSMP:
Middle Schools and High Schools Unweighted Analysis B-15
Exhibit B. 15 Mean Nutrient Profile of Middle and High School Breakfasts Served Before and
After NSMP Compared to NSMP Standards and NRC Recommendations:
Middle Schocls and High Schools Unweighted Nutrient Analysis B-16
Exhibit B. 16 Number of SFAs That Satisfied Nutrient Standards and Recommendations
for Breakfast Before and After NSMP Middle Schools and High Schools:
Unweighted Nutrient Analysis B-17
Exhibit B. 17 Mean Calorie and Nutrient Content of Breakfasts Served Before and After
NSMP: Elementary Schocls in Early-Implementing SFAs B-19
Exhibit B. 18 Mean Nutrient Profile of Breakfasts Served Before and After NSMP
Compared to NSMP Standards and NRC Recommendations: Elementary
Schools in Early-Implementing SFAs B-20
Exhibit B. 19 Number of SFAs That Satisfied Nutrient Standards and Recommendations
for Breakfast Before and After NSMP: Elementary Schools in Early-
Implementing SFAs , B-21
Exhibit B.20 Mean Calorie and Nutrient Content of Breakfasts Served Before and After
NSMP: Elementary Schools in Early-Implementing SFAs Unweighted
Nutrient Analysis B-22
Exhibit B.21 Mean Nutrient Profile of Breakfasts Served Before and After NSMP Compared
to NSMP Standards and NRC Recommendations: Elementary Schools in Early-
Implementing SFAs Unweighted Nutrient Analysis B-23
Abt Associates Inc. List of Exhibits xi
^
List of Exhibits
(Continued)
Page
Exhibit B.22 Number of SFAs That Satisfied Nutrient Standards and Recommendations for
Breakfast Before and After NSMP: Elementary Schools in Early-Implementing
SFAs Unweighted Nutrient Analysis B-24
Exhibit B.23 Mean Calorie and Nutrient Content of Breakfasts Served Before and After NSMP:
Secondary Schools in Early-Implementing SFAs B-2S
Exhibit B.24 Mean Nutrient Profile of Breakfasts Served Before and After NSMP Compared
to NSMP Standards and NRC Recommendations: Secondary Schools in Early-
Implementing SFAs B-26
Exhibit B.25 Number of SFAs That Satisfied Nutrient Standards and Recommendations for
Breakfast Before and After NSMP: Secondary Schools in Early-Implementing
SFAs B-27
Exhibit B.26 Mean Calorie and Nutrient Content of Breakfasts Served Before and After NSMP:
Secondary Schools in Early-Implementing SFAs Unweighted Nutrient
Analysis B-28
Exhibit B.27 Mean Nutrient Profile of Breakfasts Served Before and After NSMP Compared
to NSMP Standards and NRC Recommendations: Secondary Schools in Early-
Implementing SFAs Unweighted Nutrient Analysis B-29
Exhibit B.28 Number of SFAs That Satisfied Nutrient Standards and Recommendations for
Breakfast Before and After NSMP: Secondary Schools in Early-Implementing
SFAs Unweighted Nutrient Analysis B-30
Exhibit B.29 Grade Groups Used by Demonstration SFAs and Associated Nutrient Standards
for Breakfasts B-32
Exhibit B.30 Mean Percent RDA Provided by Breakfasts Served Before and After NSMP
Compared to Minimum Nutrient Standards Defined in Regulations B-33
Exhibit B.31 Mean Percent RDA Provided by Breakfasts Served Before and After NSMP
Compared to Minimum Nutrient Standards Defined in Program Regulations:
Unweighted Nutrient Analysis B-34
Exhibit D.l Weighted and Unweighted Coding for Sample Menu D-4
ExnibitEl Overview of Meal Coit Methodology Framework E-2
AW Associate* Inc. .1st of Exhibits xll
/J
List of Exhibits
(Continued)
Pag*
Exhibit E.2 Calculation of the Total Food Costs Attributable to Breakfast and Lunch E-4
Exhibit E.3 Calculation of the Labor Costs Attributable to Breakfast and Lunch E-6
Exhibit E.4 Calculation of the Total Reported Cost of Producing Breakfast and Lunch E-7
Exhibit E.5 Estimation of the Percentage of Food Costs That Arc Reimbursable E-8
Exhibit F. 1 Number of Schools in Universe and Number of Schools Sampled by Stratum
and Across All School Types: Spring 1996 and Spring 1997 F-l
Exhibit F.2 Response Rates for Surveys of School Food Service Staff F-3
Abt Associates Inc. List of Exhibits xiii
H
Acknowledgments
The NSMP Demonstration and its attendant evaluation are indebted to the staff in the 34 school
districts who participated in the demonstration and supported the evaluation through its three-year
time line. Special thanks are due to staff in the 23 districts that remained in the demonstration all
three years. Their unfailing cooperation made this evaluation possible.
Thanks are also due to staff in the 17 State offices who provided information on experiences with
demonstration SFAs and on reactions to new regulations governing meal planning practices in Child
Nutrition programs.
Staff of the Office of Analysis and Evaluation, Food and Nutrition Service, of the U.S. Department
of Agriculture had responsibility for overseeing the project. The Project Officer, John Endahl,
guided all stages of project planning and implementation as well as development of study reports.
Leslie Cristovich also provided valuable insights and direction throughout the project and in
preparation of the final report.
Finally, several staff members at Ab* Associates played important roles on the project including
Patty Connor (Manager of On-site Data Collection and Nutrition Coding); Diane Greene (Survey
Director); Gail Langeloh and Mary Kay Crepinsek (Senior Nutritionists); Nancy Burstein (Director
of Analysis); Frederic Glantz (Task Manager for the Cost Analysis); K.P. Srinath (Sampling
Statistician); Ellie Lee and Don Laliberty (Programmers); and Susan Altman (Editor). Eileen Fahey
served as the contract secretary, responsible for production of all study reports, and was ably assisted
by Eileen McEnaney and Judi Groves.
AMAssociates Inc.
ti
Acknowledgments xiv
Chapter 1
Introduction
The National School Lunch Program (NSLP), operated by the Food and Nutrition Service (FNS) of the
U.S. Department of Agriculture (USDA), has been providing meals to the Nation's school children since
1946. The School Breakfast Program (SBP) has been in operation since the early 1970s. Over the years,
research has shown that meals offered in both the NSLP and SBP have provided children with the
calories, vitamins, and minerals needed to sustain health and promote normal growth. More recent
research has shown, however, that in comparison to current public health recommendations, school
lunches are high in fat, saturated fat, and sodium, and low in carbohydrate.
Since 1993, when this situation was first identified, FNS has been working on many fronts to enhance the
nutritional quality of meals offered in the NSLP and SBP. As part of this ongoing initiative, FNS
sponsored a demonstration project to evaluate the acceptability and impact of an alternative system for
planning lunch and breakfast menus. This system, known as Nutrient Standard Menu Planning (NSMP),
utilizes computerized nutrient analysis to monitor the nutrient content of meals served to students. Abt
Associates Inc. of Cambridge, Massachusetts was awarded a contract to conduct an independent
evaluation of the three-year demonstration project. This report presents results of that evaluation.
The National School Lunch and School Breakfast Programs
All public and private nonprofit schools are eligible to participate in both the NSLP and the SBP. The
NSLP was established in 1946 "to safeguard the health and well-being of the Nation's children and to
encourage the domestic consumption of nutritious agricultural commodities and other foods."1 Today the
NSLP operates in over 94,000 schools and institutions. More than 26 million children receive meals
through the program on any given day; about half of these meals are provided free of charge.
The SBP began as a pilot project in the mid-1960s to support the provision of breakfast to children living
in "poor areas and areas where children [had] to travel a great distance to school."2 The SBP was
officially authorized as a permanent program in 1975, and the target population was expanded to include
"all schools where [the program] is needed to provide adequate nutrition for all children in attendance."1
Currently, the SBP operates in approximately two-thirds of the schools and institutions that offer the
NSLP, most commonly in schools that serve large numbers of economically disad vantaged children. On
an average day, roughly seven million children receive breakfast through the SBP. The vast majority of
these meals are provided free of charge.
School Food Authorities (SFAs) participating in the NSLP and SBP receive two types of federal
assistance: donated surplus commodities and cash reimbursements. Entitlement to surplus commodities
1 N«uon«l School Lunch Act oflM6. PL 79-396.
2 Child Nutrition Act of 1996. PL 19-642.
3 1975 Amendments to theChikJ Nutrition Act, PL 94-105
Abt Associates Inc.
tf
Introduction 1-1
is based on an established per-meal flat rate ($0,145 per meal in 1996-97) applied to the number of
reimbursable lunches served the previous year. In addition, SFAs may receive bonus commodities which
do not count against entitlements. Subject to availability, SFAs may request bonus commodities in
amounts that can be used without waste. The type and amount of bonus commodities available varies
from year to year depending on purchasing decisions made by USDA.
Cash reimbursements for NSLP and SBP meals are based on the number of meals served, established
per-meal reimbursement rates, and the poverty level of participating students. SFAs receive a base
payment for each meal served, with substantially higher rates paid for meals served free or at a reduced
price to income-eligible students. Childrens' household size and income determine their eligibility for
free and reduced-price meal benefits.
Nutrition Standards for School Meals
Program regulations require that meals served in the NSLP and SBP meet defined nutrition standards in
order to be eligible for Federal subsidies. Program regulations for the NSLP stipulate that "the nutrients
of the lunch, averaged over a period of time, approximate one-third of the National Academy of Science's
Recommended Dietary Allowances (RDAs) for children in specific age/grade groups."4 To ensure that
these standards are met, program regulations have historically included food-based menu planning
guidelines. These guidelines, originally known as the Type A meal pattern," define specific types of
food iO be offered as well as minimum acceptable portion sizes (Exhibit 1.1). Although Lie authorizing
legislation for the SBP did not include a specific RDA goal for breakfast meals, a meal pattern, modeled
after the NSLP meal pattern, was developed to ensure that breakfasts would provide approximately one-fourth
of the RDA (Exhibit 1.2).
Over the years, research has shown that, with few exceptions, the NSLP has provided students the
opportunity to satisfy one-third of their daily nutritional requirements (Wellisch, J., et al., 1983; St.
Pierre, R., et al., 1992; and Burghardt, J., et al., 1993). Similarly, meals offered in the SBP have
provided, on average, 20-25 percent of the RDAs (St. Pierre, R., et al., 1992; and Burghardt, J., et al.,
1993).
The most recent nationally representative study of school meals, the School Nutrition Dietary
Assessment Study (SNDA), published in 1993, focused attention on another aspect of nutritional quality
(Burghardt, J., et al., 1993). The SNDA study found that NSLP meals were high in fat, saturated fat, and
sodium, and low in carbohydrate, in comparison to recommendations included in the Dietary Guidelines
for Americans (U.S. Departments of Health and Human Services and Agriculture, 1995) and in the
National Research Council's (NRC) Diet and Health report (National Research Council, 1989). At the
time, programs were not required to meet these guidelines.
In response to the SNDA findings, and as part of the national Healthy People 2000 initiative, USDA
made a commitment to implement the Dietary Guidelines in the NSLP and SBP. The Department
pledged to "increase to at least 90 percent [by the year 2000] the proportion of school lunch and
breakfast services ... that are consistent with the nutrition principles in the Dietary Guidelinesfor
Americans'' (U.S. Department of Health and Human Services, 1991).
4 7 CTR 245. Pwt 210
Abt Associate* Inc. Introduction 1-2 n
Exhibit 1.1
Traditional Meal Pattern Requirements for
the National School Lunch Program1
Meat or meat alternate
Lean meat, poultry, or fish
Cheese
Large egg(s)
Cooked dry beans or peas
Peanut butter
Peanuts, soy nuts, tree nuts, or seeds
1 serving per meal
2 OK.
2oz.
1 serving
1/2 cup
4 tbsp.
1 oz. s 1/2 the requirement
Vegetables, fruits and/or full-strength jukes2 2 or more servings per meal, 3/4 cup
total portion
Bread or bread alternate
Enriched or whole-grain bread
Enriched or whole-grain biscuit, muffin, roll or
equivalent
Cooked enriched or whole grain rice, macaroni,
noodles, or other cereal grains such as bulgur or
corn grits
1 or more servings per meal/
8 servings per week
1 slice
1 serving
1/2 cup
Milk
Fluid milk (whole milk and low fat milk must
be offered daily)
1 serving per meal
1/2 pint (8 fluid oz.)
1 Requirement* were changed in June 1995 when the School Meals Initiative for Healthy Children wai
implemented (tee text).
2 No more than one-half of the total requirement may be met with full-strength fruit or vegetable juice.
a Introduction 1-3
Exhibit 12
Traditional Meal Pattern Requirements for
the School Breakfast Program1
HMHRHHHHHBHBI
4 components must be offered:
1 serving of fluid milk
1 serving of vegetables, fruit, or juice
2 servings of bread/bread alternate, meat/meat alternate, or one serving of each
Meat or meat alternate
Lean meat, poultry, or fish
Cheese
Large egg
Cooked dry beans or peas
Peanut butter
Peanuts, soy nuts, tree nuts, or seeds
loz.
loz.
1/2
4 tbsp. ■
2 tbsp.
loz.
Vegetables, fruits, and/or full-strength juices 1/2 cup
Bread or bread alternate
Enriched or whole-grain bread
Enriched or whole-grain biscuit, muffin, roll or
equivalent
Enriched, whole-grain, or fortified cereal
1 slice
1 serving
3/4 cup or 1 oz.
Milk
Fluid milk (whole milk and low fat milk must be
offered daily)
1/2 pint (8 fluid oz.)
Requirements were changed in June 1995 when the School Meals Initiative for Healthy Children was
implemented (see text).
AM Associate* Inc. n Introduction 1-4
The NSMP Demonstration
As part of an ongoing initiative to enhance the nutritional quality of meals offered in the NSLP and SBP,
FNS sponsored a demonstration project to evaluate the acceptability and impact of an alternative system
for planning lunch and breakfast menus. This system, known as Nutrient Standard Menu Planning
(NSMP), uses computerized nutrient analysis to plan menus rather than food-based menu planning
guidelines. Under NSMP, SFAs are free to use whatever portions and combinations of food they wish in
order to meet specific nutrient-based standards. Thus, in theory, NSMP provides more flexibility in
menu planning while, at the same time, providing a greater degree of assurance that meals served to
students meet nutrition standards.
FNS selected 35 SFAs to participate in the demonstration, from a pool of 127 applicants. SFAs were
purposefully selected to provide diversity in geographic location, district size, student participation rates,
food service program characteristics, and staff experience with computerized nutrient analysis. One SFA
dropped out shortly after selection because school board approval for the project was not obtained, and
the district was not replaced. Thus, the demonstration began with 34 SFAs.
Participating SFAs were expected to implement NSMP for both lunch and breakfast in all schools.
Menus were to be planned using newly developed nutrient analysis software systems approved by FNS.
Approval was granted only to systems that met detailed specifications set forth by the Agency. A key
requirement was that the nutrient analysis be based on a weighted average of all foods served in
reimbursable meals. A weighted analysis incorporates information about students' food selection
patterns and gives more weight to foods that are selected more often.
Approved NSMP software systems also incorporated the Child Nutrition Database (CN Database),
which was developed by USDA specifically for use in the Child Nutrition programs. The CN Database
includes nutrient information from the USDA standard nutrient database for foods routinely used in
school food service programs. Initially, USDA planned to augment the main database with nutrient
information for an extensive number of commercially prepared food items. USDA staff requested
nutrition information from major food manufacturers that serve the school food service industry. As
discussed later in this report, however, food manufacturers were slow to respond to this request.
Menus planned under NSMP were expected to meet a defined set of nutrient standards. Lunches were
expected to provide one-third of the RDA and breakfasts were expected to provide one-fourth of the
RDA. Both meals were expected to be consistent with Dietary Guidelines goals for the percentage of
calories from total fat and saturated fat.
Chapter 2 includes detailed descriptions of the demonstration time line and implementation protocol as
well as the research design used in the associated evaluation.
The School Meals Initiative for Healthy Children
USDA's commitment to implementing the Dietary Guidelines in school meals was formalized in the
School Meals Initiative for Healthy Children (SMI),' which promoted expansive changes in nutrition
S Federal RefM»er 60 113. 31188 31222. Jwt 13. 1995
to
Introduction 1-6
standards for school meals. The original SMI proposal, issued in June 1994, required that all SFAs use
NSMP (referred to as NuMenus) or an alternative known as Assisted NuMenus (ANSMP) to plan NSLP
and SBP menus. (ANSMP allows SFAs to arrange or contract for menu development and nutrient
analysis by an outside source such as a State agency, a consortium of SFAs, or a consultant.) Further, the
initial SMI proposal required that school meals be consistent with the Dietary Guidelines by the start of
SY 1998-99. The extended time line was built into the proposed regulation because comments received
during public hearings and in response to an initial Federal Register notice indicated that some SFAs
would need a considerable amount of time to implement NSMP or ANSMP and to develop menus
consistent with the Dietary Guidelines.
In November 1994, as part of the reauthorization of the Child Nutrition programs. Congress enacted The
Healthy Meals for Healthy Americans Act.6 This law precipitated two important changes in the
Department's initial SMI proposal. Fust, it required the Department to develop a modified food-based
menu planning system, similar to those shown in Exhibits 1.1 and 1.2, that schools could use in lieu of
NSMP or ANSMP. Second, the law shortened the time line for implcmc nation of the Dietary
Guidelines, requiring that all SFAs be in compliance by tl * first day of the 1996-97 schooi year (two
years earlier than the Department had suggested), unless a waiver was granted by the cognizant State
agency.
The final SMI rule, issued in June 1995, provided three options for menu planning: NSMP, ANSMP,
and a modified food-based meal pattern (one for breakfasts and one for lunches).7 Menu planning
options were further expanded in May 1996, when The Healthy Meals for Children Act allowed use of
the traditional NSLP and SBP meal patterns, i.e., those in effect before the modified pattern defined in
the SMI rule, or "any reasonable approach'' to satisfy the nutrient standards defined under SMI. SFAs
that elect to use either of the food-based systems (the modified meal patterns defined in the SMI rule or
the traditional meal patterns) or another approach approved by a cognizant State agency do not have to
analyze the nutrient content of planned menus. However, every five years State tgency staff must
analyze a representative weekly menu as part of a mandated SMI re view and compare results of the
analysis to the same nutrient standards used in NSMP and ANSMP.
The regulatory changes described above, which were implemented after the inception of the NSMP
demonstration, have already incorporated NSMP as a menu planting option fcr the NSLP and SBP.
Nonetheless, findings from the demonstration are important in understanding the challenges and benefits
associated with use of NSMP. The collective experience of SFAs that participated in the demonstration
will contribute to meaningful dialogue about the current NSMP system and may inform future decisions
about program requirements, training, technical assistance, and monitoring.
6 PL I03-44S.
7 M. 104-149.
Abt Aaaoclrt—Inc. Introduction- 1-6 *l
Organization of this Report
Eight additional chapters in this report present the following information:
• Chapter 2 provides an overview of the design and objectives of the evaluation.
• Chapter 3 describes the demonstration SFAs and the process of NSMP implementation.
• Chapters 4 and 5 present information about the impact of NSMP on the nutrient content
of lunches and breakfasts. These chapters also include comparisons of weighted and
unweighted analyses of the same menus.
• Chapter 6 reports on the impacts cf NSMP on food service program costs and
operations.
• Chapter 7 describes start-up costs associated with implementing NSMP It also
summarizes the opinions and recommendations of key stakeholders, including SFA
directors, school food service staff, and State agency staff.
• Chapter 8 discusses monitoring and accountability under the new SMI regulations.
Problems encountered by State agency staff who visited demonstration SFAs or
completed SMI reviews in other SFAs are described.
• Chapter 9 reports on problems encountered in the nutrient analyses completed by SCA
staff. Findings from on-site observations of meal service are also included.
Appendices provide supplementary exhibits as well as additional information about data collection
methodologies, the development of sample weights for surveys of school food service staff, and
methodologies used in assessing nutrient content and meal costs.
Inc. u Introduction 1-7
Chapter 2
Evaluation Design
This chapter outlines the research objectives defined for the evaluation and describes the research design.
The sequence and timing of data collection are also discussed and resoondents and data sources are
identified. Finally, the chapter identifies the nutrients that were examined in the analysis and the
standards 'hat were usrd to evaluate the average nutrient content of school meals.
Research Objectives
The study included both process and impact evaluations, with the following research objectives:
Process Evaluation
To identify difficulties and successes experienced during NSMP implementation.
To assess start-up and maintenance costs associated with NSMP.
To determine which SFA characteristics are associated with successful implementation
ofNoMP.
To assess monitoring and accountability under NSMP, including the accuracy of nutrient
analyses completed by SFAs.
To assess the user-friendliness and efficiency of NSMP software systems.
"o assess the opinions and recommendations of key stakeholders (SFA directors, school
food service staff, and State agency staff) regarding the NSMP system.
hacMCt Evahttfioa
To compare the average nutrient content of meals served under NSMP to defined
nutrient standards.
To compare the average nutrient content of meals served before and after
implementation of NSMP.
To compare results of weighted and unweightr) nutrient analyses of menus planned
under NSMP.
To compare costs (food, labor, and other) associated with producing school meals before
and after implementation of NSMP.
To compare school food service operations, including participation rates and commod. ty
usage, before and after implementation of NSMP.
to
a-1
Evaluation Design
The demonstration spanned three academic years, from SY 1994-95 through SY 1996-97. The sequence
and timing of major milestones in both the demonstration and its associated evaluation are summarized in
Exhibit 2.1. FNS selected SFAs in January 1994. Shortly after that, while traditional meal patterns were
still in use and before SFA staff received any training on NSMP, baseline data for the impact -valuation
were collected.
Key staff from demonstration SFAs attended a three-day training session in June 1994. Staff were
expected to begin serving meals planned using NSMP the following fall, with the goal of achieving full
implementation (i.e., NSMP-planned menus served for both breakfast and lunch in all schools in the
district) by Fall 1995.
The original design called for collection of initial process evaluation data in Spring 199S, when SFAs
were expected to be approximately six months into NSMP implementation. As discussed in Chapter 3,
however, work on NSMP implementation was substantially delayed in most SFAs for reasons that were
beyond SFA control. Consequently, initial process evaluation data were collected about six months later,
in the Fall of 1995
Follow-up data were collected in Spring 1996 and Spring 1997. Comparable data were collected each
year, with the exception of data on meal costs, which were collected only in the Spring of 1997 and only
from SFAs that h~d fully implemented NSMP.
Data Collection
This section provides an overview of the data collection methodologies used. Exhibit 2.2 provides a
summary of the types of data collected at each measurement point. An expanded discussion of study
implementation is provided in Appendix C.
Interviews with SFA Directors
SFA directors were interviewed at three points during the demonstration. In Fall 1995. directors were
interviewed via telephone for purposes of the process evaluation. These interviews focused on
experience with NSMP implementation to date, including progress made, difficulties encountered, and
solutions employed. SFA directors were asked to assess advantages and disadvantages of the NSMP
system and to offer recommendations for improving the demonstration as well as the NSMP system in
general.
SFA directors were also interviewed in person in Spring 19% and 1997. in conjunction with follow-up
data collections.1 Interviews included questions about resources devoted to NSMP implementation.
experiences with NSMP software, and perceptions about the burden and acceptability of NSMP. In
districts where someone other than the SFA director had primary responsibility for actually working with
NSMP software, this staff member (referred to as the data manager) was also interviewed.
toe. Evaluation OMign 2-2
tn
BLANK
PAGE
Exhibit 2.1 Evaluation Design
Jan 1994 Jure 1994 Fall 1994 Fall 1995
NSMP
ImplctTtcntotion FNS selects
SFAs
SFAs receive
NSMP training
SFAs expected
to begin NSMP
implementation
SFAs expected
to be fully
implemented
I
i I I ■
1 i
SY 93-94 SY 94-95 SY 95-96 SY 96-97
Spring 1994 FaU 1995
Fo
oring 1996 Soring 1997
Data
Collection Baseline data
collected
Initial process
evaluation data
collected
How-up data
collected
Follow-up data
collected
Notes: Process evaluation data were originally scheduled to be collected in Spring 1995.
See Exhibit 2.2 for a summary of data collected at each measurement point.
Inc. Evaluation Design
Jf'
Exhibit 2.2
Data Collected at Each Measurement Point
1 ! , —1
Measurement Point
Data Codec
Interviews
wfthSFA
Directors,:
SFA
Menus and
Recipes
Data on
Meal
Costs .
-vcyscf
On-S d Service
Observation* | S|aff
Interviews [
fdthStat
Directors j
Baseline Data: Spring 1994 / / / /3
Initial Process Evaluation Data: Fall I99S / / /2
Follow-up Data: Spring 1996 / / / / /2
Follow- «p Data: Spring 1997 / / / / / /' /2 /3
1 Interviews actually completed in Summer 1997 after the school year had ended.
2 SFAs were asked to submit start-up logs every six months beginning in January 1995.
' Administrative records were collected by mail in Fall 1994 (for SY 1993-94) and Fall 1997 (for SY 1996-97) after SFA staff had prepared year-end summaries for the preceding school year.
Notes: Interviews were also conducted with FNS Regional office liaisons in Fall I99S. These interviews were not repeatcl in subsequent years, however, because it was determined that
Regional office staff were not very involved with demonstration sites and were therefore unable to contribute new information.
SFA directors who dropped out of the demonstration were interviewed by phone shortly after withdrawal. The interview focused on factors contributing to the decision to terminate
participation in the demonstration.
AM Associate* Inc. PZ£ Evaluation Design 2-4
SFA Menus and Recipes
Detailed information on menus and recipes used in demonstration SFAs was collected in Spring 1994,
1996, and 1997. Each time, data were collected on the menus offered during a specified five-day period
(referred to as the target week).2 Target weeks were spread between mid-April and mid-May. Data were
collected for lunches and breakfasts served in elementary schools, middle schools, and high schools.
The following information was collected for all reimbursable menu items: full name; type of food (e.g.,
white vs. whole wheat bread; fat content of milk or ground meat); form of food (e.g., fresh, frozen,
canned; raw or cooked; convenience entree or prepared from scratch); brand name, if appropriate;
preparation method; and planned portion sizc(s). Recipes were collected for all items prepared by
combining two or more foods or ingredients. In addition, SFA staff were asked to provide, for each menu
item, the total number ofservings anticipated for the district as a whole, i.e., the number of individual
servings of each menu item expected to be served, exclusive of a la carte servings and servings sold to
teachers or other adults. This information was needed to support a weighted nutrient analysis, as
required under NSMP (see Chapter 3).
In Spring 1997, when SFAs were using NSMP, additional data were collected to permit an assessment of
the accuracy and completeness of nutrient analyses conducted by SFA staff. This included printouts of
recipe and menu analyses for the target week (to assess the accuracy of data entered and analyzed);
vendor-supplied nutrient information sheets (to assess the accuracy of data entry for foods added to the
nutrient data base at the local level); and back-up menu production records (to assess the accuracy of
serving projections used in weighted nutrient analyses).
Data on Meat Costs
To assess the costs of producing reimbursable meals before and after NSMP, a substudy of meal costs
was conducted in three randomly selected schools (one elementary school, one middle school, and one
high school) in each district. In Spring 1994 and again in Spring 1997, school food service staff in these
schools were trained by Abt Associates staff to collect data for the cost substudy. Data were collected
during the same target week covered in the menu analysis (see above) and included detailed menu
production records and time studies. Menu production records included, for each food served during the
target week (including a la carte foods and foods sold to adults), information on the number of servings
prepared, the number of servings left over, and the number of servings actually served or sold. Daily
time records were completed by all employees in the sampled schools for the full five-day period. The
time records were used to allocate time spent on breakfast and time spent on lunch. A time allocation
grid was also obtained for central office staff to identify time devoted to breakfast and lunch production.
Data from administrative records (bid/inventory lists and salary rosters) were used to translate food
production information and time re.ords into food and labor costs, respectively. These data were used to
compute the average cost of producing a reimbursable lunch and breakfast, if offered, in each school. A
detailed description of the methodology used in the study of meal costs is provided in Appendix E.
2 To obtain a reasonable aiataamrnr of nutrient content, it is necessary to examine meals offered over a period of time rather than a single
meal. The National Research Council (SRC) recommends that group feeding programs plan menus so that nutrient standards are met over a
five- lo ten-day period. A sample five-day period, equivalent to a full week in most school districts, is routinely used in USDA-sponsored
evaluations of Child Nutrition programs. NSMP requirements specify that analyses be based on a typical school week, ranginc from three to
seven days.
AM Associate* Inc. h Evaluation Design 2-5
On-Site Observations
On-site observations of meal preparation and service assessed compliance with planned menus, recipes,
and portion sizes. One-day observations were completed during the target week in each of the three
schools selected for the cost substudy. Observers documented whether items on the planned menu were
actually offered and collected descriptive information on any additional items offered. They also
observed food preparation and documented the nature and extent of deviations from planned recipes. In
addition, observers weighed five random portions of all foods offered to children to determine average
portion sizes.3 Finally, for purposes of the substudy of meal costs, staff observed students as they passed
through servings lines and recorded which items students included in their reimbursable meals.
Surveys ofSchool Food Service Staff
In Spring 1996 and 1997, school food service staff in a random sample of schools in each SFA
completed brief mail surveys. Surveys assessed staff perceptions about NSMP, particularly perceptions
about burden and about the impact of NSMP on the acceptability of school meals. The methodology
used in selecting schools and in developing weights for use in the analysis is described in Appendix F.
Interviews with State Agency Staff
Brief telephone interviews were completed with directors of State agencies and other relevant staff in Fall
1995 and Summer 1997. Interviews included questions about problems encountered in demonstration
SFAs or other SFAs using NSMP as well as State agency directors' perceptions about monitoring under
the new SMI guidelines (see Chapter 1).
Start-Lip Logs
SFA directors were asked to maintain detailed logs during the implementation period, documenting the
number of staff hours devoted to a variety of different NSMP tasks. Directors were instructed to record
only hours associated with work that would not have been done in the absence of NSMP. Respondents
also provided information on the hourly wages of individual staff members so that costs associated with
the reported labor could be calculated. SFAs were asked to submit start-up logs every six months,
beginning in January 1995 and continuing until full implementation was achieved.
Administrative Records
Data on program participation, food service income and expenses, a la carte revenues, and commodity
utilization (total dollar value of commodities received) were collected by mail in Fall 1994 (for SY 1993-
94) and Fall 1997 (for SY 1996-97), after year-end summaries had been prepared by SFA staff.
Standards Used in Evaluating Nutrient Content
Two sets of nutrient standards weie used in evaluating the nutrient content of NSLP and SBP meals
(Exhibit 2.3). The first set is comprised of standards specified in the NSMP demonstration protocol
(and, under SMI, carried over into current NSLP and SBP regulations). These include standards for
calories and target nutrients for which RDAs have been established (protein, vitamin A, vitamin C,
calcium, and iron) as well as for the percentage of calories from fat and saturated fat.
3 Commercially pre-park«|rrl item* such as cartons of mult and |wtas^ of cookie*, crackers, or chip* were excluded became previous
experience has thown minor, if any. deviations front expectations due to quality control in commercial pack aftng processes
AM Associate* Inc. Evaluation Design 24
&
Exhibit 23
Nutrient Standards Used in the Evaluation
Nutrient Standard
Standards Defined for NSMP
Nutrients with established RDAs:'
Breakfast: One-fourth of the RDA
Calories, protein, vitamin A, vitamin C, *
calcium, and iron Lunch: One-third of the RDA
Nutrients included in the Dietary Guidelines
for Americans'}
Total fat S 30% of total calories
Saturated fat < 10% of total calories
NRC Recommendations3
Carbohydrate > 55% of total calories
Protein < 15 % of total calories4
Cholesterol Breakfast: £ 75 mg'
Lunch: slOOmg6
Sodium Breakfast: soOOmg5
Lunch: sSOOmg6
1
National Research Council (1989). Recommended Dietary Allowances, 10th edition. Washington. DC: National Academy
Press.
1 U.S. Departments of Health and Human Services and Agriculture (1995). Nutrition and Your Health: Dietary Guidelines
for Americans, 4th edition. Washington, DC: U.S. Government Printing Office.
1 National Research Council (1989). Diet and Health: Implicationsfor Reducing Chronic Disease Risk. Washington DC:
National Academy Press.
4 Diet and Health recommends a maximum protein intake equivalent to leu than twice the RDA. To achieve recommended
levels of calories from fat and carbohydrate, the percentage of total calories from protein needs to be in this range.
' One-fourth of suggested maximum intake of cholesterol (300 mg) and sodium (2,400 mg).
6 One-third of suggested maximum intake of cholesterol (300 mg) and sodium (2,400 mg).
Abt Associates Inc.
*
Evaluation Design 2-7
A second set of standards, based on the National Research Council (NRC) recommendations, was
defined for nutrients and food components that are analyzed by NSMP software but not included in
standards defined for NSMP. These include the percentage of energy from carbohydrate and protein as
well as total cholesterol and sodium content. Neither demonstration SFAs, nor current NSMP users
outside the demonstration, are required to meet these additional standards. They are used in this report
solely for the purpose of facilitating interpretation of the data.
Data on fiber content are also presented in this report. However, these data are not compared to a
standard because there is no scientific consensus about an appropriate benchmark for daily consumption
of fiber.
Estimation of Program Impact
The impact evaluation examined the effect of NSMP on nutrient content, meal costs, and key aspects of
food service operations (e.g., participation rates, a la carte revenues, use of donated commodities,
number of daily choices offered, and variety of foods offered over the course of a week). Because
relatively few SFAs had implemented NSMP by Spring 1996, impact analyses focus on comparisons of
baseline (Spring 1994) and final follow-up (Spring 1997) data. Throughout the text the terms 'pre-
NSMP' or 'before NSMP* are used to refer to Spring 1994 data and 'after NSMP' or 'NSMP' are used
to refer to Spring 1997 data.
The basic measure of program effect is the difference between estimates made prior to NSMP
implementation and those made afterward. Simple /-tests are used to test the statistical significance of
differences in means or proportions. It is important to realize that impacts that are detected cannot
necessarily be ascribed to NSMP. In the absence ot a control group, it is impossible to rule out the
possibility that changes over time were influenced by factors external to the demonstration. Thus,
information on program impacts should be considered suggestive rather than conclusive, and is not
necessarily representative of what might happen in programs nationwide.
Impact analyses are further limited by small sample sizes. Some of the SFAs selected to participate in
the demonstration dropped out or never implemented NSMP. Others implemented NSMP only in
elementary schools. Sample sizes vary from one analysis to the next depending on the number of SFAs
included. The maximum sample size for any SFA-lcvel analysis is 19 (analysis of elementary lunch
menus). These small samples provide adequate statistical power to detect only very large effects.
Consequently, the absence of statistical significance cannot be interpreted with confidence as evidence
that a meaningful difference does not exist. Rather, the conclusion must be that very large differences do
not exist.
AM AMoeif■ Inc. Evaluation Daaign 2-f
Jd
Chapter 3
Implementation of NSMP in Demonstration SFAs
This chapter describes the SFAs that participated in the demonstration, the extent to which SFAs were
successful in implementing NSMP, and difficulties and successes experienced by SFA staff. The first
section of the chapter describes the SFAs selected to participate in the demonstration. The next two
sections summarize challenges faced by SFAs during early stages of implementation and the final
implementation status of all SFAs at the end of the demonstration (Spring 1997). The next section
describes expectations of the original NSMP protocol and the extent to which SFAs were able to meet
these expectations. The final section of the chapter identifies characteristics that may predict successful
NSMP implementation.
Selection of Demonstration SFAs
SFAs were invited to volunteer for the NSMP demonstration in the Summer of 1993. Thirty-five SFAs
were ultimately selected by FNS staff from a pool of 127 applicants. One SFA dropped out shortly after
selection because school board approval for the project was not obtained. Thus, the demonstration began
with a total of 34 SFAs.
SFA selection was structured to ensure that the demonstration would provide an opportunity to assess the
feasibility and impacts of NSMP in a variety of different school districts. As summarized in Exhibit 3.1,
FNS staff purposefully structured the sample of SFAs to ensure diversity in geographic location,
community type, school district size, and program participation rates.
The nutrition and food service experience of the SFA director and the availability of nutrition expertise
among other SFA staff were also considered in selecting sites. About two-thirds of the selected SFAs
had a registered dietitian on staff. In about half of die SFAs, the director and/or NSMP coordinator was a
registered dietitian. SFAs selected for the demonstration were managed both by veteran school food
service professionals (up to 38 years of experience) and by relative newcomers (1 -2 years of experience).
Characteristics of existing food service programs were also taken into account. Sites were selected to
ensure variation in centralization of menu planning, use of cycle menus, the extent of centralized food
production, use of standardized recipes, and prevalence of scratch food preparation (as opposed to use of
pre-prepared convenience items).
A final factor that influenced site selection was prior experience with computerized nutrient analysis.
Because ready access to a computer is a prerequisite for NSMP implementation, all of the districts
selected had some pre-existing computer capability. However, only 13 of the 34 selected districts had
actually analyzed the nutrient content of their menus by computer prior to the demonstration. In about
half of the selected districts, the SFA director and/or NSMP coordinator had prior experience with
computerized nutrient analysis.
Abt Associate* Inc. lmp4«m»nUtk>n of NSMP in Demonstration SFA» 3-1
Exhibit 3.1
Characteristics of Demonstration SFAs
FNS Region
Mid-Atlantic
Mountain Plains
Midwest
Northeast
Southeast
Southwest
Western
Community Type
Urban
Suburban
Rural
Enrollment
Under 2400
2400-9,999
10,000-24,999
25.000 or more
Mean enrollment
Average Daly NSLP Participation Rate
40 percent or less
41-55 percent
56-70 percent
71 percent or more
Mean participation rate
Percent of Lunches Served Free
25 percent or less
26-50 percent
51-74 percent
75 percent or more
Mean percentage
4
4
5
5
5
6
5
9
16
9
6
9
t
11
33.984
8
9
13
4
53%
10
10
8
6
44%
AM A«wocia«M Inc. ■pMMMM of NSMP In Demonstration SFAs 3-2
$SL
Exhibit 3.1
(continued)
Nutrition/Food Service Experience of Staff
Registered dietitian or nutritionist on stiff
SFA director or NSMP coordinator is registered dietitian
SFA directors' total food service experience (years)
Minimum
Maximum
Mean
SFA directors' experience as director (years)
Minimum
Maximum
Mean
Characteristics of Food Service Program Prior to NSMP
Fully centralized menu planning
Cycle meniKs) used
Some centralized food production1
Percentage of entree items prepared from scratch
25% or less
26-30%
More than 50%
Mean percentage
Prior Experience with Computerized Nutrient Analysis
District menus previously analyzed by computer
SFA director or NSMP coordinator had prior experience
with computerized nutrient analysis
23
It
1
38
15
1
19
8
21
10
17
7
15
12
51%
13
17
1 50% or rnore of schools in distrkt are served by base or central kitchens.
Note: Some data may differ from application forms or from previous reports (Few. MK, 1997) because mformation for SFAs
was clarified/updated in subsequent interviews.
Source: Application forms for NSMP demonstration (Fall 1993).
Abt Associates Inc. Imptemontetion of NSMP In Domooolintioo SI As 3-3
33
Early Implementation Challenges
From the outset, NSMP implementation in all SFAs was complicated by a number of factors over which
SFAs had no control. Most important was a significant delay in the approval of NSMP software
systems. Software vendors were slow to submit products for USDA review and approval, and many of
the programs submitted did not satisfy the functional criteria defined by USDA. The delay in the
availability of approved software systems effectively postponed concerted implementation efforts in most
SFAs. While most SFAs began working on NSMP at some level shortly after receiving training in June
1994, the majority did not begin to work in earnest until sometime afterward.
By Jaruary 199S, six months after SFA staff were trained and the point at which process study data were
originally scheduled to be collected, only two NSMP software packages had been approved. At that tune,
FNS contacted all demonstration SFAs and asked that they select one of the two available systems, move
forward with NSMP implementation during the remainder of SY 1994-95, and continue to work toward
full implementation by Fall 1995. Several SFAs lobbied for approval of other softwait systems, and a
third system was ultimately approved.1
While all but two of the SFAs that completed the demonstration eventually selected one of the three
available software systems, many SFAs did not really begin working with NSMP software until well into
the latter half of SY 1994-95 or the subsequent summer months. This delayed start-up had a substantial
impact on the progress SFAs were able to make during the first year of the demonstration.
Another problem that complicated implementation was the poor response USDA received from food
manufacturers contacted to supply nutrient information for the CN Database. Very few manufacturers
responded to the request, and much of the data submitted was found to be incomplete, inaccurate, or
otherwise questionable. As a result, the database included in the approved software systems did not
include many of the commercially purchased products used in school food service. This situation placed
an unanticipated burden on demonstration SFAs. SFA staff had to assume responsibility for obtaining
nutrient information tor nationally-available commercial products and for entering these data into the
their own databases.
Status of Implementation in Spring 1997
By Spring 1997, when final follow-up data collection began, less than half (16) of the original 34 SFAs
were considered fully implemented (Exhibit 3.2). Ten of these SFAs had implemented NSMP in
accordance with expectations—NSMP menus were being served in all schools and menus had been
developed using approved NSMP software and district-wide menu production information.
The other six SFAs were serving NSMP menus in all schools but used a modified approach to weighted
nutrient analysis. Modifications included basing saving projections used in weighted analyses on data
from a subset of schools rather than all schools in the district or on staff predictions rather than actual
menu production data. One of these SFAs also used an unapproved software system one that had been
in use prior to the demonstration.
I Sawe January 1995. 1! other software sysetns have been approved
AM Associates lr>c. ■nptewiHtJun or NSMP In DwwonsUaHon SFAs 3-4
Final Implementation Status of Demonstration SFAs
Fully implemented
Implemented without modifications
Weighted analysis completed using approved software
and district wide menu production data
Implemented with modifications
Weighted analysis completed using approved software
but used menu production data from a subsample of
schools1
Weighted analysis completed using approved software
but used staff predictions rather than district-wide
menu production data
Weighted analysis completed using unapproved
software and staff predictions rather than district-wide
menu production data
Partially implemented
Implemented for elementary lunch; no analysis done for
lunch at other levels or for breakfast at any level
Implemented for elementary breakfast and lunch; no
analysis done for other levels
Implemented for breakfast at all levels; no analysis
done for lunch
Not implemented
No menus analyzed fully
Lunch menus analyzed, but with unapproved software
and without weighting; breakfast menus not analyzed
Breakfast and lunch menus analyzed but without
weighting; analysis based on one "set" menu each day,
without consideration of numerous other available
choices
Dropped out
10
i
2
4
2
1
1
3
1
1
11
l
One SFA used district-wide menu production data for 'inch menus but based breakfast analysis on data from one elementary
school one middle school and one high school
Inc. hnphimentation of NSMP in Demonstration SFAs J-5
3^
Because the unapproved software did not have the ability to conduct a weighted analysis, SFA staff
developed a companion spreadsheet program to compute weighted weekly averages (based on staff
projections).
Four SFAs achieved only partial implementation and three SFAs never implemented NSMP. Three of
the partially implemented SFAs implemented NSMP only in elementary schools, and the other SFA
implemented NSMP only for breakfast In all four of these SFAs, directors reported that they had not
achieved full implementation because they were unable to dedicate the required number of staff labor
hours.
Among the three SFAs that had not implemented NSMP by Spring 1997, one was still in the process of
developing and analyzing menus. The director was doing all the work herself and had experienced
several false starts and multiple software problems. The other two non-implemented SFAs had analyzed
breakfast and lunch menus, but the analyses deviated substantially from the NSMP protocol. One of
these SFAs used unapproved software and used unweighted nutrient analysis. The other SFA based
analyses on a single defined meal each day rather than incorporating all of the menu options that were
actually available. For example, daily lunch analyses included only one entree, ignoring a second entree
offered in elementary schools and numerous other entrees (more than ten on some dayv) offered in middle
and high schools.
Finally, a total of 11 SFAs withdrew from the demonstration sometime before Spring 1997.
In most cases, several different factors contributed to the decision to terminate participation. Concern
about the use of weighted nutrient analysis was the primary reason for withdrawal, cited by eight of 11
SFA directors (Exhibit 3.3). Most often, the concern centered around the amount of staff labor required
to obtain district-wide menu production data. Some SFA directors were also concerned that weighted
nutrient analysis would limit flexibility in menu planning and decrease the ability of individual schools to
cater to students' preferences. A few directors were worried that a weighted analysis could only meet
nutrient standards if popular high-fat food items were eliminated or offered much less frequently. These
SFA directors believed that such changes would have a negative impact on lunch participation in middle
and high schools.
Problems with NSMP software contributed to the withdrawal of five SFAs. Specific problems included
incompatibility of available NSMP software with other software currently in use for other aspects of
program operations; the need for upgraded hardware in order to run NSMP software; and numerous
problems with the prototype NSMP software received at the June 1994 training and/or with early
releases) of approved systems.
Five of the SFA directors who dropped out of the demonstration were concerned about the time line for
implementation. Specifically, they were unhappy that they were expected to implement NSMP by the
start of the 1993-% school year despite the lengthy delay in the software approval process. Directors
reported that they could not dedicate the amount of labor required to implement NSMP within the
condensed time frame.
X
Exhibit 33
Reason for Withdrawal from the Demonstration
Concerns about weighted nutrient analysis
Labor requirements
Potential negative impact on flexibility in menu planning
Potential negative impact on secondary school
participation
NSMP software problems
Approved NSMP software systems not compatible with
existing system
Could not afforJ necessary hardware upgrade
Numerous problems with NSMP software
Demonstration time line too constrained
Change in SFA director
Concerns about NSLP
SMI regulations issued prematurely
NSLP may be block granted
Note: Respondents could report more than one reason for withdrawal.
Source: Interviews with directors of drop-out SFAs. November 1995-January 1997.
Two SFAs terminated participation because there had been a change in SFA directors. The new directors
were not prepared to assume responsibility for continuing with NSMP implementation. Finally, two SFA
directors were influenced by situations that were not directly related to NSMP or to the demonstration.
One director believed that issuance of the SMI regulations (see Chapter 1) prior to the conclusion of the
evaluation undermined the intent of the demonstration. He was therefore unwilling to devote the time and
resources necessary to continue with the project. The second SFA director was concerned that the NSLP
was going to be block granted (an idea that was being debated by the Congress at the time), and that, as a
consequence, her district would cease to participate in the program.
AM of NSMP in Demonstration SFAs 3-7
il
Implementation of NSMP in Demonstration SFAs
The sections that follow describe key components of NSMP and summarize the difficulties and successes
experienced by SFA staff in attempting to follow implementation guidelines in each area. All 23 of
the SFAs that remained in the demonstration through Spring 1997 are included in these analyses.
Additional information related to implementation is provided in Chapter 7 which reports on start-up
and maintenance costs incurred by demonstration SFAs as well as SFA staff perceptions about the
burden and acceptability of the NSMP system.
Staff Responsibility
To increase the probability of success, SFAs were required to identify one individual who would assume
responsibility for overseeing NSMP implementation. In 17 of the 23 SFAs that remained in the
demonstration through Spring 1997, SFA directors assumed direct responsibility for NSMP
implementation. Ten of these directors did virtually everything themselves, including data preparation
and entry. The other seven directors delegated data preparation and entry tasks or received substantial
assistance from other staff or volunteer assistants, including dietetic interns and State office staff. For
one SFA, the State office essentially took over these tasks because neither the SFA director nor his
limited staff had time to do it. All of the SFAs in which responsibility for NSMP implementation was
delegated were either very large (enrollments in excess of 25,000) or large (enrollments between 10,000
and 24,999).
NSMP Software Systems
As noted above, demonstration SFAs had three approved NSMP software systems from which to choose.
These were School Nutrition Accountability Program (SNAP); Computer-Assisted Food Service
(CAP'S); and NUTRIKIDS (developed and marketed by Lundifiyte Systems, Inc.). By the time the
demonstration was over, many other software systems had been approved.
Aoout half of the 23 SFAs that remained in the demonstration through Spring 1997 used NUTRIKIDS
(Exhibit 3.4). Five of these SFAs originally started with SNAP, the prototype software distributed at
training, but switched to NUTRIKIDS during the course of the demonstration. SNAP was the next most
common software package, used by five SFAs.
CAFS was used by three SFAs. Two of these SFAs already owned the CAFS food service management
software system and simply purchased the NSMP software as an enhancement. The other SFA that
elected to purchase CAFS did so because of the system's other features, i.e., modules that support a
variety of food service administration and operations tasks.
One SFA elected to use one of the NSMP systems that received approval later in the demonstration—
PCS, a multi-module school food service management system offered hy PCS Revenue Control Systems,
Two SFAs continued to use a software system that was in place prior to Lie demonstration (Practorcare),
even though the software never received USDA approval. One of diese SFAs was in the fully-implemented-
with-modifications group and the other SFA never implemented NSMP.
Some SFAs experienced problems with NSMP software during the initial stages of implementation.
Most of these problems were related to glitches in initial versions of the software, inexperience on the
AM *IIOCIHM Inc. Impiomerrtaoon of NSMP in Demonstration SFAs 3-f
3?
Exhibit 3.4
Software Systems Used by Demonstration SFAs
Approved NSMP Systems
NUTRIKIDS
SNAP
CAFS
PCS
Unspproved Software Systems
Practorcarc
12
5
3
1
Note: Excludes 11 SFA» that dropped out of the demonstration.
Source: SFA director interviews, Spring 1997.
part of SFA staff, or problems with hardware. The frequency of software problems decreased as the
demonstration continued and both software manufacturers and SFA staff gained more experience.
Overall, SFAs using approved NSMP software systems were pleased with their selections. The majority
found the software to be both easy to use and time efficient.
For most SFAs, the only major problem with NSMP software was the fact that the nutrient database did
not include information for brand name food products. This shortcoming placed the burden of acquiring
and entering these data on the shoulders of SFA staff. In many SFAs the task of collecting, reviewing
and entering data for literally hundreds of food items consumed a sizeable amount of SFA resources.
Data for Weighted Nutrient Analyses
To support development of serving projections for weighted nutrient analyses, SFAs were required to
maintain menu production records which documented the number of individual portions actually served
in reimbursable meals. A la carte and adult servings were not to be included in these counts. Unless an
SFA planned to conduct separate analyses for individual schools within the district, weighted analyses
were to be based on district-wide data. A sample menu production record was provided in the NSMP
training manual, but SFAs were free to adapt the form for their own use.
As noted in the preceding discussion of reasons for SFA withdrawal, many SFAs found it difficult to
comply with this requirement. The amount of labor involved in collecting and synthesizing data from all
schools in the district was the major problem Although virtually all SFAs maintained some form of
menu production records, these records were maintained only at individual schools. Few SFAs dealt with
Abt Associates Inc. Imptomsntatlon of NSMP in Demonstration SFAs 3-9
*
these records at the district level. Most SFAs used purchasing records, ordering requisitions and
inventory reports, rathet than detailed menu production records to track district-wide food usage.
Consequently, the need to gather and synthesize menu production data required most SFAs to implement
and manage an extensive stream of paperwork.
SFA staff also identified more specific difficulties involved in obtaining these data. These included the
need to convert food production information traditionally reported in volume or weight (e.g., ten gallons
of gravy) to servings, the unit of measure needed for NSMP; separating foods sold a la carte from those
sold in reimbursable meals; maintaining separate counts for similar items that were traditionally counted
together, e.g., cookies, sandwiches, chips; and obtaining production information for self-serve foods,
including, but not limited to, salad bars and other specialty food bars.
While the sentiment regarding the requirement for district-wide menu production data was largely
negative, seven SFA directors found this requirement, in combination with the requirement that all
recipes be standardized, to be beneficial. These directors reported that the increase in accountability and
standardization allowed them to identify and rectify inefficiencies and weaknesses in their overall
operations.
Standardized Recipes
SFAs were expected to have a standardized recipe for every menu item that included more than one
ingredient or involved preparation other than simple heating. A standardized recipe was defined as "one
that has been tested to provide an established yield and quality through the use of ingredients that remain
constant in both measurement and preparation methods." Districts were free to use standardized USDA
recipes or to use/develop their own recipes. Approved NSMP software included modules to develop,
analyze, and modify standardised recipes.
SFAs that used salad bars or other theme bars were encouraged to develop recipes based on menu
production information, i.e., the number of reimbursable meals served from the theme bar and the total
quantity of food used. Several versions of a recipe might be needed if different types of schools offered
different items or, for any pre-portioned foods, different portions. SFA staff were expected to
reanalyze/update theme bar recipes if there were changes in the food items offered.
This requirement proved to be more challenging than some SFA directors expected. When initial process
evaluation data were collected, few SFA directors expected standardized recipes to be a problem. As
implementation progressed, however, and SFAs gained experience in this area, difficulties increased.
The level of effort required to modify and test existing recipes was greater than many SFA directors
anticipated. Recipes often needed to be tested several times before a product was deemed acceptable.
SFA directors complained that there were not enough "healthy" quantity recipes available. Although
USDA issued an expanded set of recipes during the course of the demonstration, many SFAs
incorporated only a few of these items, reporting that many of them were "inappropriate" for their
populations (i.e., "my students will not eat that").
Resistance among SFA staff, particularly cooks, was another problem faced by SFA directors striving to
standardize food production. Many cooks were unhappy with the increased emphasis on standardization,
which required that they abandon some of their favorite recipes and refrain from making individual
decisions about how a recipe might be improved. The need for a standardized approach to recipe
preparation and food portioning was a recurrent theme in training sessions nek) for SFA staff.
Abt Aaeoclatee Inc. " Implementation of NSMP In Demonstration SFA* 3-10
Y0
Age/Grade Groups Used In Analyzing Menus
The demonstration protocol defined four different age/grade groupings for use in menu analysis:
preschool and early elementary grades K-l (ages 3-6); upper elementary grades 2-5 (ages 7-10); middle
school grades 6-8 (ages 11-13); and high school grades 9-12 (ages 14-17). SFAs were expected to plan
and analyze separate menus for these four age groups. Because all school districts were not expected to
have exactly identical grade configurations, the protocol allowed menu planners to incorporate one
additional age group at either end of the defined span (e.g., 6-10 or 10-14).
If the disparity in age/grade groupings included two or more years at either end of the spectrum, menu
planners were required to develop customized standards using a specialized function available in
approved software. Technical guidance issued after the training session stipulated that the calorie
requirement in any customized standard be within ten percent of the calorie requirement for the oldest
and youngest children in the group. This requirement effectively limited the number of ages that could be
included in developing a customized standard and required that two separate menus be analyzed for most
schools with wide grade spans, e.g., elementary schools covering grades K-6 or K-8 or combination
schools covering grades K-l2.
From the beginning, many SFAs had difficulty de.ermining which age groups to use in their nutrient
analyses. This was particularly true for SFAs that had a number of different grade configurations for one
type of school (e.g., elementary schools that covered grades K-6,1 -3, and K-12). SFAs' initial confusion
about this issue was exacerbated by the issuance of the SMI regulations because the requirements
specified in the regulations differed from those defined for the demonstration. First, although the SMI
regulations encourage use of the four basic age/grade groups defined in the NSMP protocol, these are
presented as an option rather than a requirement An alternative scheme presents minimum, less
stringent, age/grade groupings. Second, the SMI regulations permit calculation of a combined
middle/high school menu. Finally, although the -egulations allow for development of customized
standards to meet individual district needs, no restriction is placed on the number of age groups that can
be included in a combined, customized standard.
In the end, the age/grade groups used by many SFAs were not consistent with expectations of the initial
protocol. Only two of the 20 SFAs that planned elementary school menus (16 fully implemented SFAs
and four partially implemented SFAs) planned two separate menus for lower and upper elementary
grades. Six of the 16 SFAs that planned middle and high school menus (all of the fully implemented
SFAs) used a combined middle and high school analysis rather than separate analyses for each school
level. Finally, as discussed in Chapter 9, the age spans chosen for analysis, particularly for elementary
menus, were not always appropriate.
Alternative Offer-versus-Serve Rule
NSMP menus, whether for breakfast or lunch, must include a minimum of three menu items, and one of
the items must be milk. In addition, one of the three items offered in lunch menus must be an entree.
Menu items are defined as any single food or combination of foods served together, e.g., a sandwich is
considered one menu item.
Demonstration SFAs were required to implement offer-versus-serve (OVS), a regulatory provision that
allows students in secondary schools and, at the discretion of SFAs, elementary schools, to refuse some
of the food items offered in reimbursable lunches. The OVS rule defined for the demonstration required
that students take a minimum of two menu items. No restriction was placed on the items that could be
refused.
AbtAeaoclaOalnc. Implementation of BBS In Pemonetretlon SFAe i£TT
HI
Confusion about the OVS rule increased sharply when final SMI regulations were published in June
1995. The SNA regulations require that students select a minimum of two menu items and one of the
selected items must be an entree. Most SFA directors incorrectly believed that the SMI regulations
superseded the guidelines provided during the June 1994 NSMP training. Consequently, only five of the
20 fully or partially implemented SFAs followed the OVS rule defined for the demonstration. The
remaining 15 SFAs followed the SMI version of the OVS rule or maintained the OVS rule used under the
traditional meal pattern (students must take 3 of 5 meal components). This situation made it impossible
for the evaluation to provide FNS with information about the acceptability or impact of the "any two
items" OVS rule.
Disclosure of Nutrition Information
SFAs were not required to disclose nutrient information, but they were encouraged to do so as a means of
increasing awareness of both NSMP and the SFA's efforts to improve the nutritional quality of school
meals. Half of the SFAs that fully or partially implemented NSMP disseminated information about the
nutrient content of school meals (Exhibit 3.5); some SFAs used more than one method. Eight SFAs used
printed menus that were sent home to parents, three used posted information in cafeterias, and two
published information in newspapers or local cable television bulletin boards.
ExMbtt 3.5
Disclosure of Nutrition Information by Demonstration SFAs
Disseminate information about nutrient content of meals
Yes, on printed menus sent home
Yes, in newspapers/on local TV
Yes, post on cafeteria line
No
Reasons for not disclosing nutrition information
Don't have time
Don't want to do so until NSMP is implemented in all
schools
Software only gives monthly (rather than weekly) average;
monthly summary is not useful
Not completely comfortable with accuracy of information yet
Believe it will confuse parents more than help them
No money to print/send home menus
Note: Respondents could report more than one form of disclosure »nd more thin one reason for non-disclosure
Source: SFA director interviews. Spring 1997.
AM AttoclatM Inc. ■ttpwentnurtion of NSMP In D#monsti Alton SFAs 3-12
¥&-
SFA directors who did not disclose nutrition information had several reasons for not doing so. Four
directors said they simply did not have the time to prepare and disseminate this information. Two
directors did not want to disclose nutrition information until NSMP was implemented in all schools. Two
directors reported that they could only get monthly printed summaries from their software analysis
system and did not believe that a monthly summary was useful. One director was not yet comfortable
enough with the results of her NSMP analyses to disseminate the results. Another believed that
disclosure of information would be more confusing than helpful. Finally, one director indicated that the
SFA did not have enough money to support printing and distribution of menus for parents, the vehicle
she saw as the most viable one for disseminating information on the nutrient content of planned menus.
Promotional Activities in School Cafeterias
Another activity that was encouraged, but not required, was use of cafeteria promotions to increase
awareness of NSMP and to encourage children to accept new/modified menu items. Three-quarters of
the fully or partially implemented SFAs conducted at least one special cafeteria promotion before Spring
1997 (Exhibit 3.6). Most, but not all, promotions took place in elementary schools and most involved
taste testing activities. SFAs also reportedly assisted with classroom nutrition/health education activities
and held special 5 A Day promotions to encourage fruit and vegetable consumption.
Exhibit 3.6
Cafeteria Promotions Conducted by Demonstration SFAs
Conducted special promotions in school cafeterias
Yes 15
No 5
Activities conducted
Taste testing new products/recipes 10
Coordinated activity with classroom nutrition/health 8
education
Held S A Day or other promotion to increase fruit and 5
vegetable consumption
Special nutrition awareness week (including National 2
School Lunch Week)
Note: Respondents could report more than one activity.
Source: SFA director interviews. Spring 1997.
Abt Associates Inc. Imptem«ntetton of NSMP in Demonstration SFAs 3-13
«3
UM of Fortified Foods
While formulating plans for the demonstration, FNS staff were concerned that some SFAs might satisfy
nutrient standards by offering highly-fortified foods rather than planning an appropriately balanced menu
using foods from the basic food groups. Program planners considered establishing restrictions en the use
of fortified foods. Although such restrictions were never implemented, with the exception of limitations
on the use of vegetable protein products (VPP), SFA staff were discouraged from using such foods.
Staff in two-thirds of the SFAs that remained in the demonstration through Spring 1997 were approached
by vendors offering fortified foods (Exhibit 3.7). The most common items offered
Exhibit 3.7
Use of Fortified Foods by Demonstration SFAs
Approached by vendor offering fortified foods
Yes
No
Food offered by vendors
"Super" donuts/s veet rolls
Other fortified bread products
Milk-based drinks
Dessert items
Vegetarian/soy entrees
"Energy" bars
Fruit drinks
Fruit rollups
Yogurt
Other
Incorporated fortified foods not used prior to NSMP
Yes
No 23
Note: Fortified foods have been specially fortified with one or more
Source: SFA director interviews. Spring 1997.
vitamins or minerals
Inc. ImpfOffi+otsffOfi of NSMP wt rvfTiontlfMlon SFAs 3-14
&L
were "super" donuts and sweet rolls similar to those already used in many SBP programs across the
country. Less commonly encountered fortified foods included other fortified bread products such as
bagels and muffins, milk-based drinks, desserts, vegetarian entrees., "energy" bars, fruit drinks, fruit
rollups, and yogurt.
Although two-thirds of the SFAs that did not withdraw from the demonstration were solicited by vendors
offering specially fortified foods, none or these SFAs incorporated a fortified food that was not already
being offered before NSMP (e.g., several SFAs used "super" donuts before joining the demonstration).
Meeting Nutrient Standards
Directors of the 16 fully implemented SFAs were asked about their problems and successes in meeting
nutrient standards, based on the menus being offered in Spring 1997. Two SFAs reported that all of their
planned menus satisfied all of the relevant NSMP standards. The remaining 14 SFAs reported that one
or more of their planned menus failed to meet at least one of the nutrient standards.
Exhibit 3.8 summarizes SFA directors' reports for breakfast and lunch menus. With the exception of
calories and the percent of calories from saturated fat, more than three-quarters of the SFAs that offered
breakfasts reportedly satisfied individual nutrient standards for breakfasts. The calorie standard was the
most problematic. Seven of the 15 fully implemented SFAs that offered breakfast in Spring 1997
reported that one or more of their planned menus did not meet this standard. Four SFAs were unable to
consistently meet the standard for saturated fat. Other standards that were problematic for one or more
SFAs included the percent of calories from fat (3 SFAs); vitamin A (3 SFAs); iron (1 SFA); and vitamin
C (1 SFA).
Many of the same nutrients were problematic for lunch menus. Only eight of the 16 fully implemented
SFAs reported meeting the standards for calories from fat and saturated fat on a consistent basis. Four
SFAs reported difficulty meeting the standard for calories, specifically in secondary school menus. A
comparable number reported similar difficulty with the standard for iron. Finally, two SFAs reported
difficulty with the standard for vitamin A and one SFA had difficulty with the standard for vitamin C.
SFAs were encouraged to monitor levels of carbohydrate, cholesterol, sodium, and dietary fiber in
planned menus, although specific standards were not established for these nutrients. Only two of the 16
fully implemented SFAs reported monitoring these nutrients. SFA directors who were not monitoring
these nutrients consistently reported that they (and their staff) did not have time to do anything that was
not required.
Calendar Time Required for Full Imptomentatlon
Most SFAs required substantially more time to implement NSMP than originally anticipated. Among the
16 SFAs that fully implemented NSMP, an average of 19 months elapsed between the time SFA staff
started actively working on NSMP implementation (which in many cases was several months after
receiving training) and the time NSMP menus were served in all schools (Exhibit 3.9). The range was
very broad, with a minimum of three months and a maximum of 33 months. In the great majority of the
SFAs that achieved full implementation, active work on NSMP spanned more than one calendar year.
Only three SFAs achieved full implementation in less than 12 months, while six SFAs required more than
two years.
Inc. of NSMP in Demonstration SFAs 3-1S
if
Exhibit 3.8
Reported Successes and Probkms in Meeting Nutrient Standards
Breakfast
Number of Fuify Implemented SFAs
(■-IS)
Calories 8 7
Protein 15 0
Calcium IS 0
Iron 14 1
Vitamin A 12 3
Vitamin C 14 1
Percent of calories from fat 12 3
Percent of calories from saturated fat 11 - 4
Lunch
Calories 12
Protein 16
Calcium 16
Iron 12
Vitamin A 14
Vitamin C IS
Percent of calories from fat 8
Percent of calories from saturated fat 8
Number of Fully Implemented SFAs
(■-16)
4
0
0
4
2
1
8
8
Note: One fully implemented SFA does not offer breakfut.
Source: SFA director interviews, Spring 1997.
Inc. of NSMP In Demonstration SFA*
-#
3-16
MMII
Time Required by Demonstration SFAs to Achieve Full Implementation
Month! elapsed between initiation of active work on
NSMP and full implementation
Less than 12 months 3
13 -24 months 7
More than 24 months 6
Mean (months) 19
Minimum (months) 3
Maximum (months) 33
SFA directors' estimates of time required under more
favorable circumstances
Less than 12 months 9
13-24 months 6
More than 24 months 1
Mean (months) 16
Minimum (months) 3
Maximum (months) 30
Source: SFA director interviews. Spring 1997
Although SFAs were able to work on implementation tasks during the period when approved software
was not available (e.g., gathering menu production data, standardizing and/or modifying recipes,
evaluating potential new commercial products, etc), delays in software approval may have introduced
some inefficiencies and led to a more protracted implementation period than would have been
experienced under more optimal conditions. Likewise, the fact that SFA staff had to acquire and enter
nutrition information for many commercially-prepared foods (because the version of the nutrient database
included in the NSMP software did not include this information) may have contributed to an extended
implementation period.
To get some idea of the time period required for NSMP implementation in the absence of these
complications, directors of the 16 fully implemented SFAs were asked to estimate the amount of time
AM AsaociatM Inc. knptomontation of NSMP hi Demonstration SFAs 3-17
XL
they would need to implement NSMP if they were to do it again under more optimal circumstances (i.e.,
with readily available and more complete software) and with the knowledge gained during their initial
experience. Responses are summarized in Exhibit 3.9. As shown, the time line for NSMP
implementation projected by most SFA directors was shorter than their actual experience in the
demonstration, but not dramatically so. On average, projected implementation time lines were about
three months shorter than actual demonstration experience. Nine of the 16 SFA directors estimated that
they could implement NSMP u. 12 months or less. Six directors estimated that they would need one to
two years to implement Only one SFA director indicated that he would need more than two years to
implement NSMP.
Characteristics That May Predict Successful NSMP Implementation
One of the research objectives defined for the evaluation was to determine whether particular SFA
characteristics influence the likelihood of successful NSMP implementation. In view of the
implementation rates described in preceding sections, this question seems especially important
However, the study's ability to draw conclusions about relationships between SFA characteristics and
successful implementation is limited by small sample sizes. Group sizes for "implementers" and "non-implementers"
are so small that only extremely large differences between the two groups can be deemed
statistically significant It is therefore more informative to examine the data and identify patterns that ire
suggestive of meaningful associations.
Exhibit 3.10 shows tabulations of final implementation status by selected SFA characteristics. Included
are baseline characteristics considered in selecting demonstration SFAs as well as several other variables
thought to be potentially important to NSMP implementation. The latter group includes whether the SFA
director assumed direct responsibility for NSMP implementation and whether there was a change in SFA
directors during the course of the demonstration. All 34 of the SFAs selected for the demonstration are
included in these tabulations.
It is important to note that several of the characteristics that could potentially be influential are highly
correlated with each oth s. For example, urban SFAs tend to include more schools, have larger total
enrollments, serve more reimbursable meals on an average day, and include a greater proportion of
children in poverty than suburban and rural SFAs. The small available sample size does not support
multivanate estimates of the separate effects of each characteristic.
In comparing SFAs that achieved full implementation, with and without modifications, to SFAs that did
not achieve full implementation, several interesting patterns emerge, as summarized below.
Urban SFAs were ranch less Mkefy to impfcmeat NSMP than either suburban or
rural SFAs. Only two of nine urban SFAs achieved full implementation, and one of
these SFAs used a modified approach to weighted analysis. In contrast, eight of 16
suburban SFAs achieved full implementation (four SFAs used a modified approach), as
did six of nine rural sites (one SFA used a modified approach).
?r
Exhibit 3.10
Characteristics of Demoastratioa SFA* by Ffaul ImpknieaULoa Statin
Coaaaaity Type
Urban 1
Suburban
Rural
Enrollment
Under 2,500
2^00-9,999
10,000-24,999
25,000 or more
Perceat of Laacaes Served Free
25 percent or less
26-50 percent
51 percent or more
Characteristics of Food Service Program Prior to NSMP
Some centralized food production1
Yes
No
Fully centralized menu planning
Yes
No
Cycle mcnu(s) used
Yes
No
NatrMaa Experience of Staff
Registered dietitian or nutritionist on staff
Yes
No
7 9
1 16
3 9
4 6
4 9
2 1
t 11
5 10
4 10
9 14
1 17
10 17
11 21
7 13
5 10
13 24
13
5
23
II
AM Inc. of KSMP in Demonstration SFAs
W
3-19
Exhibit 3.10
(c
Natritioa Experience of Staff (continued)
SFA director or NSMP coordinator is
registered dietitian
Yes
No
Prior with Computerized Nutrient
District menus previously analyzed by
computer
Yes
No
SFA director or NSMP coordinator had
prior experience with computerized
nutrient analysis
Other Factors that May
hai
SFA director personally managed NSMP
iMaaaaaaaaal
Yes
Chaaajs in SFA directors during
demonstration
Yes
No
10
I
7
11
10
t
11
7
4
14
18
16
13
21
17
17
24
10
7
27
1 50% or n.ore of schools in district ire served by base or central kitchens.
Note: Some data may differ from application forms or from previous reports (Fox. MK. 1997) because information for
SFAs was clarified/updated in subsequent interviews
Source: Application forms for NSMP demoiisuanon (Fall 1993) and Spring 1997 implementation status.
AM Inc. of NSMP in SFAa
Jt>
Although most of the urban SFAs were also either large or very large, exploratory
analyses suggest that being urban had an influence on implementation that was
independent of size. Potential reasons that urban SFAs may have more trouble with
NSMP implementation than their suburban or rural counterparts include financial
constraints (several urban SFA directors were in the midst of or facing a hiring freeze)
and a more transient or less reliable work force (several urban SFA directors mentioned
long-standing vacant positions and frequent staff turnover). It is also possible that
urban SFAs are dealing with food preparation facilities that are aging or poorly
equipped. In short, urban SFAs may be less able to devote the time and energy required
for NSMP implementation because of numerous competing priorities.
Very large SFAs and, to a lesser extent, small SFAs were less Hkery to implement
NSMP than medium or large SFAs. Only one of 11 very large SFAs (enrollment of
25,000 or more) implemented NSMP without modifications. An additional two very
large SFAs implemented NSMP using a modified approach to weighted nutrient
analysis.
Most very large SFAs that did not implement NSMP (7 of 8 SFAs) withdrew from the
demonstration. As discussed earlier in this chapter, the primary reason for termination
was concern about the use of weighted analysis, specifically the level of effort involved
in collecting and organizing district-wide menu production data. The burden associated
with complying with this requirement clearly escalates as district size increases, unless a
district is fully computerized or has highly centralized food production. In fact, the data
suggest that SFAs with enrollments between 10,000 and 25.000 may also have trouble
implementing the weighted analysis requirement Only three of eight SFAs in this group
were able to fully implement NSMP without use of a modified approach to weighted
analysis.
At the opposite end of the spectrum, small SFAs (less than 2.500 students) also
appeared to have some difficulty with implementation. Four of the six small SFAs
selected for the demonstration never implemented NSMP. Three of these SFAs dropped
out. Feedback from SFA directors indicates that while the issue of weighted analysis
played a role in the decision to drop out, a larger issue was the fact that these directors
were personally responsible for multiple aspects of day-to-day operations. Those who
dropped out found the amount of work involved in NSMP implementation, particularly
within the constrained time frame in which they were ultimately asked to work, to be
more than they could handle.
SFAs that had aoaae ceatratatd food production prior to NSMP were aaore andy
to achtrre tnl napli mi nUHna, wanont modaVanon, than SFAs that had nttte or
no centrahaed food production. Seven of the 17 SFAs that had a moderate degree of
centralized food production prior to NSMP, defined in this analysis as 50 percent or
more of schools serviced by a base or central kitchen, achieved full implementation
without using a modified approach to weighted analysis. The same was true for only
three of the 17 SFAs that had less centralized food production.
i SFA* Ml
61
The presence of centralized food production may benefit NSMP implementation by
reducing the amount of work required to document district-wide food usage. In addition,
SFAs with some centralized food production may face fewer problems in identifying
standardized recipes and ensuring that those recipes are used in all food production
locations.
SFAs that had fully centralized menu planning prior to NSMP were more likely to
achieve foil implementation, without modification, than SFAs that had fully or
partially decentralized menu planning. Most SFAs (21 of 34) had fully centralized
menu planning systems in place prior to the demonstration. Among the 13 SFAs that
entered the demonstration with some degree of decentralized menu planning, only two
achieved full implementation without using a modified approach to weighted analysis.
To implement NSMP in situations where individual schools have limited or total
autonomy in menu planning, SFAs must either reduce the level of local control—i.e.,
move to a fully centralized system—or analyze menus for each individual school. The
latter approach significantly increases the time and labor required to implement NSMP.
fete. fmvtmmMaben o* NSMP In OwiHXtliaOon SFA« 3-22
SI
Chapter 4
Nutrient Content of Lunches Before and After NSMP
This chapter presents information on the nutrient content of lunches served in demonstration SFAs before
and after implementation of NSMP. The following research questions are addressed:
• How do lunches served after implementation of NSMP compare to established NSMP
standards and NRC recommendations?
• Does the nutrient content of lunches served after implementation of NSMP differ
significantly from lunches served prior to the use of NSMP?
• Does the nutrient profile of NSMP lunches differ significantly if the nutrient analysis is
unweighted? Do conclusions about the impact of NSMP differ if the analysis is
unweighted?
The chapter is divided into four sections. The first section describes the analysis that was conducted and
how the data are presented in this report. The second section provides a context for the analysis of
nutrient content by summarizing differences in the types of foods included in lunch menus before and
after NSMP. The third section describes and compares the nutrient content of lunches served before and
after the implementation of NSMP. Results are based on weighted analyses, the analytic approach that is
the foundation of the current NSMP system (The difference between weighted and unweighted analyses
is described later in this chapter.) The fourth and final section of the chapter compares results of
weighted and unweighted analyses of Spring 1997 lunch menus. The objective of this comparison is to
determine whether the two analytic techniques lead to similar conclusions about the extent to which
NSMP lunches satisfy nutrient standards and recommendations.
Overview of the Analysis
The analysis includes 19 SFAs that implemented NSMP, either district-wide (16 SFAs) or for elementary
school lunches only (3 SFAs), prior to Spring 1997. Data are presented for elementary school lunches
and secondary school lunches. In cases where SFAs planned two elementary school menus (n=6) or
separate middle and high school menus (n=10), data were combined in a single, appropriately weighted,
analysis. All analyses were completed using NUTRIKIDS software.1 A detailed description of the
methodology' used in conducting all nutrient analyses is provided in Appendix D.
Appendix A includes supplementary exhibits that summarize data for SFAs that implemented NSMP by
Spring 19%, referred to as "early-implementing SFAs," as well as data for middle school lunches and
high school lunches for the ten SFAs that analyzed separate menus.
LunchByte Systems. Inc Version 7.0.1997
Abt Assocl ites Inc. Nutrient Content of Lunches Before and After NSMP 4-1
/J
Reference RDA Standards
The RDAs specify allowances for four different groups of school-age children (4-6 year olds, 7-10 year
olds, 11-14 year olds, and 15-18 year olds), as well as for 1-3 year olds, the oldest of whom may attend
school-district-operated preschools. For purposes of the evaluation, RDA standards were assigned to
each SFA based on the grade spans included in the districts' schools. In some cases, a defined RDA
standard was used, for example, RDAs for 7-10 year olds for an elementary school that included grades
two through five, or RDAs for 11-14 year olds for a middle school that included grades six through eight.
When the grade span for a particular type of school encompassed more than one established RDA age
group, for example, an elementary school with kindergarten through grade six (ages 5-11), the NSMP
software was used to develop a customized RDA standard (see Appendix D). Abt Associates' coding
staff ensured consistency in RDA standards across SFAs for schools with comparable grade spans as
well as a consistent standard for Spring 1994 and Spring 1997 analyses.
In addition to an analysis tailored to reflect the actual makeup of SFA schools, a separate analysis was
conducted to compare lunches served before and after NSMP to the minimum nutrient (RDA) standards
defined in current program regulations. Results of this analysis are summarized in Appendix A.
Weighted and Unweighted Analy-ta
The principal analysis presented in this chapter describes the average nutrient content of lunches served
in participating SFAs before and after NSMP, based on a weighted analysis of lunch menus. Weighted
analysis is the standard defined for the NSMP demonstration and in current program regulations. A
weighted analysis takes into account the number and types of foods actually taken by students, giving
greater weight to foods selected more frequently. As such, results of a weighted nutrient analysis provide
a picture of the average lunch served to students.
An unweighted analysis, in contrast, does not incorporate information on student selection patterns and
constitutes a simple average of all foods made available to students. An unweighted analysis includes an
average serving of every type of food offered (i.e., milk, entree, fruits and vegetables, and desserts), and
thereby reflects the average lunch offered to students.
To assess similarities and differences between the two analytic approaches, lunch menus were analyzed
both ways. Results for Spring 1997 menus are discussed in this chapter. Results for Spring 1994 menus
and a comparison of pre-NSMP and post-NSMP menus using an unweighted analysis are summarized in
Appendix A.
Frequency of Foods Offered in Lunch Menus
Because nutrient content is driven by the mix of foods included in a menu, it is helpful to understand
changes SFAs' may have made over the course of the demonstration in the mix of foods offered. Foods
included in lunch menus offered before and after NSMP were assigned one of 52 different food/food
group codes. Data were tabulated to produce frequency distributions showing the percentage of daily
lunch menus in which each food/food group was included. For example, a frequency of 67 percent for
pizza indicated that 67 percent of the daily lunch menus reported by demonstration SFAs included pizza.
Atrt AaaoclatM Inc. Nutrient Content of LuncheeMMand After NSMP 4-2
J-f
Data for pre-NSMP and NSMP menus were compared, and foods or food groups that increased or
decreased over time by fiv; or more percentage points were identified. Results are summarized in
Exhibit 4.1. Foods listed in the Offered More Often section of the exhibit appeared more often in NSMP
menus than in menus planned before NSMP. Likewise, foods listed in the Offered Less Often section of
the exhibit appeared less often in NSMP menus than in pre-NSMP menus.
Changes noted in planned menus were generally consistent with recommendations made during NSMP
training and with changes reported by SFA directors. Elementary school lunch menus planned after
NSMP included:
• more skim milk and flavored lowfat milk and less whole milk;
• more unbreaded meat, poultry, and fish, and fewer breaded versions of these items;
• more raw vegetables (other than salad); and
• more "extra" bread/grain choices.
The number of changes noted in middle and/or high school menus was greater than elementary school
menus, but the pattern of change was similar. Compared to pre-NSMP lunch menus for middle and/or
high schools, NSMP lunch menus included:
• more skim milk and less whole milk;
• more fresh fruit;
• more raw vegetables and salad and fewer french fries and other types of potato
products;
• more pasta-based entrees, salad bars, and Chefs salads, and fewer entrees that tend to
be high in fat, including breaded meat, poultry, and fish, burgers, nachos, hot dogs, and
specialty food bars (e.g., taco ban, potato bars);
• more rice;
• more desserts; and
• fewer snack chips.
In addition, NSMP menus for all types of schools included fewer cooked vegetables (other than
potatoes); elementary and middle school menus included less flavored skim milk; and middle and high
school menus included less fruit juice. These findings are somewhat counter-intuitive but explainable.
Comments from SFA directors indicate that use of cooked vegetables was decreased and counterbalanced
by an increase in raw vegetables and both side-dish and entree salads. According to SFA staff, student
acceptance of raw vegetables and salads is substantially higher than cooked vegetables (other than french
fries and other types of potatoes).
The decrease in the frequency of flavored skim milk can be traced to three SFAs that were using this type
of milk prior to NSMP but discontinued it because of poor student acceptance.
Finally, the change in the frequency of juice is attributable to changes made in four SFAs 'hat offered
juice every day or almost every day prior to NSMP. After NSMP, menu planners in three of these SFAs
offered fresh fruit rather than juice. In the fourth SFA, the frequency of juice dropped because of the
elimination of a sack lunch, previously offered every day, in which the beverage was juice.
AM Associate* Inc. Nutrient Content of Lunches Before and After NSMP 4-3
ft
Exhibit 4.1
Foods Offered More and Less Often in NSMP Lunch Menus
Compared to Pre-NSMP Loach Menus
Offered More Often'
Skim milk / / /
Flavored lowfat milk /
Fresh fruit / /
Raw vegetables (other than salad) / /
Salad /
Unbreaded meat, poultry, or fish /
Pasta-based entrees •
Salad bar (entree) /
Chefs salad /
Rice /
Extra bread/grain3 /
Desserts / /
Offered Less Often2
Whole milk / / /
Flavored skim milk / /
Fruit juice / /
crench fries / /
Potatoes other than french fries / •
Cooked vegetables other than potatoes / / /
Breaded meat, poultry or fish / /
Hamburgers/cheeseburgers / /
Nachos /
Hot dogs /
Specialty food bars /
Chips /
1 Foods were offered in mor# daily menus after NSMP than before NSMP. Difference of more than S percentage points
2 Foods were offered in fewer dairy menus after NSMP than before NSMP. Difference of more than 5 percentage point..
1 Bread or grain item available to all students, regardless of entree selected
Nutrient Content of LunchM Before and After NSMP
#
Nutrient Profile of Lunches Served Before and After NSMP
This section presents data on the average nutrient content of lunches served before and after NSMP,
relative to NSMP standards and NRC recommendations. All results are based on weighted analyses.
The section includes three separate discussions as outlined below.
• Nutrient Content Relative to RDAs. Mean contribution to RDAs for calories, protein,
vitamin A, vitamin C, calcium, and iron is evaluated in light of the NSMP standard for
lunch (33% of the RDA).
• Percentage of Calories from Total Fat, Saturated Fat, Carbohydrate, and Protein.
The mean percentage of calories provided by each macronutrient is compared to NSMP
standards (total fat and saturated fat) or NRC recommendations (carbohydrate and
protein).
• Cholesterol, Sodium, and Fiber Content Mean cholesterol and sodium content are
compared to NRC recommendations. Information on mean fiber content is presented
but not compared to a standard because no scientific consensus exists about a
recommended daily intake of fiber.
Discussions are organized around two research questions identified at the beginning of the chapter
(1) the extent to which lunches served after implementation of NSMP (NSMP lunches) met defined
nutrient standards and NRC recommendations, and (2) significant differences in the nutrient content of
lunches offered before and after implementation of NSMP. Comments about non-significant differences
between pre-NSMP and NSMP lunches are limited to situations where conclusions about meeting NSMP
standards or NRC recommendations differ, (e.g., when the mean for NSMP lunches satisfies an NSMP
standard or NRC recommendation but the mean for pre-NSMP lunches does not, or vice versa).
Unless otherwise noted, findings for the supplementary analyses summarized in Appendix A (early
implementers; middle school lunches and high school lunches; and minimum nutrient standards defined
in the regulations) a*e comparable to those reported here.
Mean Nutrient Content Relative to RDAs
In general, lunches served both before and after imp' ^mentation of NSMP met NSMP standards for
calories and all key RDA nutrients. Indeed, as shown in Exhibit 4.2, lunches served at both points in
time provided more than one-third of the RDA, with the exception of calories in secondary school
lunches.2
Elementary school lunches served after NSMP provided significantly fewer calories and less protein than
lunches served prior to NSMP. These differences have little substantive importance, however, because
lunches served at both points in time exceeded the NSMP standard of one-third of the RDA.
Secondary school lunches served after NSMP were somewhat lower in calories than lunches served
before NSMP. Although this difference was not statistically significant, the mean for NSMP lunches fell
2 DMI on actual mnacacrtyi^mitrwnicaaicrt of hiKJMKrvedafcpfCKMedmExMM A 1 A more detailed summary of the (
!«Exla*att42ll»out^4 5upfovtdedinExhibttsA2-A4
Abt Associates Inc. Nutrient Content of Lunches Before and After NSMP
^7
Exhibit 4.2 With the Exception of Calorie* in Secondary Schools. Lunches Served
Before and After NSMP Provided More than On. -third of the RDA
Elementary School Lunches
110%
100%
Calories Protein Vitamin A Vitamin C Calcium Ion
Secondary School Lunches
100%
Calories Protein Vitamin A Vitamin C Calcium Iron
Before NSMP After NSMP
'Difference is statistically significant at the 5 percent level.
•Difference is statistically significant at the 1 percent level.
AM Associate* Inc. Nutrient Content of Lunches Before and After NSMP 4-6
JT
just below the one-third RDA standard. (Pre-NSMP lunches provided, on average, exactly one-third of
the RDA.)
The only nutrient which differed significantly for secondary school lunches was vitamin A. Lunches
served after NSMP provided significantly more vitamin A than lunches served before NSMP. Again,
however, the relative importance of this difference is minimal in terms of this evaluation because the
vitamin A content of lunches served at both points in time more than satisfied the one-third RDA
standard.
In summary, the data indicate that the contribution of NSMP lunches to students' daily nutrient needs was
comparable to the contribution of lunches planned before NSMP. In other words, improvements in
NSMP lunches, as described in subsequent sections, were achieved without a negative impact on the
relative contribution of lunches to students' daily needs for calories and nutrients.
Number ofIndividual SFAs Meeting RDA Standards
In addition to examining mean nutrient content across all SFAs, the analysis assessed whether
implementation of NSMP affected the number of individual SFAs that met NSMP standards for calories
and RDA nutrients. With the exception of calories in secondary schools, a majority of SFAs provided
one-third of the RDA for each target nutrient both before and after NSMP (Exhibit A.3).
Calories in secondary school lunches were problematic both before and after implementation of NSMP.
As noted in the preceding section, the pre-NSMP mean, across all SFAs, just met the one-third RDA
standard. On an individual basis, only eight of the 16 SFAs met the standard before NSMP. After
NSMP, the number of SFAs meeting the standard dropped to six and thr mean calorie level was slightly
lower, falling just below tlte one-third benchmark. This pattern is consistent with SFA directors' reports
that some SFAs found it difficult to meet the standard for calories.
Percentage of Calories from Total Fat, Saturated Fat, Carbohydrate, and Protein
On average, neither NSMP lunches nor lunches served prior to NSMP met the NSMP standards for
percent of calories from total fat and saturated fat or the NRC recommendations for percent of calories
from carbohydrate and protein (Exhibit 4.3). Failure to meet NSMP standards for calories from fat and
saturated fat is consistent with SFA director reports that these standards were the most difficult to meet
at lunch, particularly for secondary schools.
Nonetheless, the nutrient profile of NSMP lunches was significantly better than pre-NSMP lunches.
Specifically, NSMP lunches derivedfewer caloriesfrom fat and saturatedfat and more calories from
carbohydrate than lunches served before NSMP. This is true for both elementary and secondary school
lunches and, in all cases, differences between pre-NSMP and NSMP lunches were statistically significant.
The mean percentage of calories from fat in elementary school NSMP lunches was five percentage points
lower than the mean for pre-NSMP lunches. This is a substantial difference, representing an overall
decrease in calories from fat of 19 percent.3 As a result, elementary school NSMP lunches came close to
meeting the NSMP standard of no more than 30 percent of calories from fat. For secondary school
3 SMEINMA.1 Ota!* Iran fci.pw of hi* 9 oOonei
Abt Associate* Inc. Nutrtent Content of Lunch** B*for* and After NSMP 4-7
4\I
Exhibit 4.3 Lunches Served After NSMP Provided Fewer Calories from Fat and
Saturated Fat than Lunches Served Before NSMP
Elementary School Lunches
Total Fat Saturated Fat Carbohydrate Protein
Secondary School Lunches
70%
60%
^50%
I
"§ 40%
Zo 30%
| 20%
£ 10%
0%
52%*
NSMP Standard orNRC Recommendation
Before NSMP After NSMP
'Difterence is statistically significant «; the 5 percent level.
"Difference is statistically significant at the 1 percent level.
Nutrient Content of Lunch*. Bofor* and After NSMP 44 ±
lunches, the mean percentage of calories from fat in NSMP lunches was three percentage points lower
than pre-NSMP lunches, equivalent to a 12 percent decrease in actual fat calories.
In addition, the percentage of calories from saturated fat in elementary school lunches dropped three
percentage points, representing an overall decrease in calories from saturated fat of 25 percent.4 The
equivalent for secondary sdnol lunches was a decline of one percentage point and a 16 percent drop in
calories from saturated faL
Finally, NSMP lunches provided significantly more calories from carbohydrate than pre-NSMP lunches
and, particularly in the case of elementary school lunches, came close to meeting the NRC
recommendation of rrore than S3 percent of calories from carbohydrate.
Thus, although, on average, NSMP lunches continued to exceed the standards for calories from fat and
saturated fat and to fall short of the recommendation for calories from carbohydrate, the data suggest that
implementation of NSMP had a significant impact in moving school lunches closer to these goals.
Overall, the magnitude of the change was larger for elementary school lunches than for secondary school
lunches.
Number of Individual SFAs Meeting Standardsfor Fat and Saturated Fat
Before NSMP, only one of the 19 SFAs included in this analysis met the NSMP standard of no more than
30 percent of calories from fat in elementary school lunches (Exhibit 4.4). Further, elementary school
lunches served in almost three-quarters of SFAs (14 of 19) exceeded this standard by a substantial
margin, providing more than 35 percent of calories from fat. After NSMP, the number of SFAs meeting
the standard for percent of calories from fat increased to more than one-third (7 of 19), a statistically
significant difference, and only three SFAs provided more than 35 percent of calories from fat.
The number of SFAs meeting the NSMP standard for percent of calories from saturated fat in elementary
school lunches also shifted dramatically over the course of the demonstration (Exhibit 4.4). Before
NSMP, none of the 19 SFAs met die NSMP standard of less than 10 percent of calories from saturated
faL After NSMP, about one-quarter of the SFAs (5 of 19) met the standard (a statistically significant
difference).
Similar shifts were noted in the number of SFAs meeting fat and saturated fat standards in secondary
school lunches, however, neither change was statistically significant.
Finally, the number of SFAs meeting the NRC recommendation for percent of calories from carbohydrate
was significantly higher, for both elementary school lunches and secondary school lunches, a