The story of Team Nutrition pilot study outcome report : final report

              0 oi4-A-°\ /H£^vfJ<?r/l|
Tfie SIory of Team Nutrition
Pilot Study
Outcome Report
Final Report
\w-^
4-500- ^o^Tol
0
United States Department of Agriculture . olCl .
Food and Nutrition Service i »*
Office of Analysis, Nutrition and Evaluation ''<•••
^f"
The Sfory of Team Nutrition
Pilot Study
Outcome Report
Final Report
*rt
United Stun Department of Agriculture .
Food and Nutrition Service *J
Office of Analysis, Nutrition and Evaluation 'n» •
ACKNOWLEDGMENTS
This report is the result of collaborative efforts from many skilled researchers. Technical
guidance was provided by Craig Lefebvre at Prospect Associates and David Maklan at Westat,
Inc. Data collection was directed by Danny McGoldrick at Prospect, and was conducted in the
field by Laura Biesiadecki, Elyse Levine, Pam Cusick, Erin Dabbs, Stephanie Kipnis, and Amy
Buehler, all of Prospect. Telephone interviews with parents were supervised by Pat Licodo at
Westat. Leslie Lytle, Sarah E. Samuels, and Jean Goldberg were consulted for developing the
survey instruments.
The analyses were led by Brad Chaney and Paul Zador of Westat. Brad Chaney developed
bivariate and multivariate models from surveys of students, teachers, and parents. Multivariate
analyses based on measures of observed behavior were designed and reported by Paul Zador.
Preliminary analyses were conducted by David Lowe, Tom Milke, and Patricia Stouffer with
programming assistance from Bill Hartley and Fauzia Tirmazi, and survey support by Kirstin
Moesmger, Ellen Herbold, Suzanne McNutt, Janice Hall, Caren Celebuski, Sharon Hammond,
and Laura Flicker, all of Westat.
The complexities of the data and analyses presented by this evaluation required extensive
review by staff at the U.S. Department of Agriculture, Office of Analysis and Evaluation (OAE),
and by external reviewers. At OAE, Carol Olander, Jill Randell, Mike Fishman, Jay Hirschman,
Pat McKinney, and John Endahl reviewed progress on the data collection, analyses, and
preliminary reports. External reviewers Tom Baranowski of the University of Texas, Leslie Lytle
of University of Minnesota, Howell Wechsler at the Center for Disease Control and Prevention,
and Howard Bloom at New York University provided insightful comments to preliminary reports.
We would like to thank especially the Team Nutrition Coordinators at the seven pilot
implementation sites, whose efforts and enthusiasm paved the way for this comprehensive
evaluation. They are:
Julia L.M. Thorius, Julie A. Rosin, and Teresa Nece at Des Moines Independent
Community Schools;
Delores D. Coffey at Hamblen County Schools;
Lisa Griffin, at Tulsa Public Schools;
Brenda Padilla and Hal Bush, at Vacaville Unified School District;
Mary Ann Weber and Esther Monclova Butler at Cleveland Public Schools
Margaret Burton and Anne Marie Sheehan at Lawrence Public Schools; and
Samuel S. Petracca at Passaic Board of Education.
Finally, we thank the principals, teachers, food service staff, and custodial staff of the schools
that participated in the study. Their talent and drive lay the groundwork for the research
presented in this report.
i
CONTENTS
EXECUTIVE SUMMARY
TEAM NUTRITION—AN OVERVIEW
1.1 Background To Team Nutrition 1-1
1.2 The Team Nutrition Concept 1-2
1.3 Team Nutrition Pilot Implementation Project 1-6
1.3.1 District, School, and Classroom Selection 1-8
1.3.2 Implementation Requirements 1-10
1.4 Structure ofthe Outcome Measures Report 1-14
EVALUATION OF TEAM NUTRITION PILOT COMMUNITIES
2.1 Objectives of the Team Nutrition Outcome Evaluation 2-1
2.2 Research Questions 2-2
2.3 Outcome Evaluation Approach 2-4
2.3.1 Study Design and Scope 2-4
2.3.2 Data Collection 2-7
2.3.3 Overview of Analytic Approach 2-11
2.4 Other Key Evaluation Issues 2-12
2.4.1 The Use of Multiple Data Sources 2-12
2.4.2 Scaling Student Self-Reports 2-13
2.4.3 Scaling Parent Perceptions of Student Change 2-17
2.4.4 Cafeteria Observations of Student Food
Selection and Consumption 2-17
2.4.5 The Choice of a Mixed Models Repeated
Measures Approach 2-18
2.4.6 Aggregating Data Across Districts Versus Performing
Separate Analyses for Each District 2-19
2.4.7 Tests of Statistical Significance 2-20
2.4.8 Exploratory Analysis of Survey Data 2-20
2.5 Summary 2-21
i
Contents (continued)
3 THE STARTING POINT—BASELINE SNAPSHOT
3.1 Site Profiles 3-1
3.1.1 Characteristics of Schools 3-2
3.1.2 Demographic Characteristics of Students and
Their Families 3-3
3.2 Nutrition Environment 3-6
3.2.1 Nutrition Education Activity 3-6
3 2.2 Opinions Toward Nutrition Education 3-7
3.2.3 Food Service Practices 3-8
3.3 Students' Skill-Based Knowledge, Skills, and Behavior
at Baseline 3-13
3.3.1 Similarities Between Implementation and Comparison
Students 3-13
3.3.2 Ceiling Effects: Room for Improved Scores 3-14
3.4 Summary 3-15
4 THE INTERVENTION—DESCRIPTION OF THE TEAM NUTRITION PILOT
IMPLEMENTATION PROJECT IN THE FOUR OUTCOME COMMUNITIES
4.1 Requirements of Classroom Curriculum 4-1
4.1.1 Number of Lessons Taught 4-1
4.1.2 Lesson Planning and Duration 4-2
4.1.3 Classroom Activities Conducted 4-2
4.1.4 Materials Used 4-4
4.2 Cafeteria, School, and Community Activities 4-4
4.2.1 Core Activities Conducted 4-4
4.2.2 Core Event Planning Time and Duration 4-5
4.2.3 Media Coverage 4-8
4.2.4 Number of TN Channels 4-8
4.3 Food Service Activities and Changes 4-9
4.4 Summary 4-10
Contents (continued)
STUDENT IMPACTS—SKILL-BASED NUTRITION KNOWLEDGE AND MOTIVATION
5.1 Methodology 5-2
5.1.1 Measuring Change in Skill-Based Knowledge and Motivation
Over Time 5-2
5.1.2 Tests of TN Impact 5-3
5.2 Estimated Effects of Team Nutrition on Students'
Skill-Based Knowledge 5-4
5.3 Estimated Effects of Team Nutrition on Students' Motivation 5-6
5.4 Summary 5-8
STUDENT IMPACTS—PARENT PERCEPTIONS
6.1 Methodology 6-1
6.1.1 Measures of Parent Perceptions and Hypotheses 6-1
6.1.2 Analytic Approach 6-2
6.2 Parents* Perceptions of Changes in Students 6-3
6.2.1 Retrospective Assessment of Changes in Students' Nutrition
Awareness and Behavior 6-3
6.2.2 Parent Perceptions of Students' Eating Behavior Over Timt 6-4
6.3 Summary 6-5
STUDENT IMPACTS—SCHOOL LUNCH SELECTION AND CONSUMPTION
7.1 Research Design: Overview 7-1
7.1.1 Research Hypotheses 7-1
7.2 Methodology 7-3
7.2.1 Measuring Change Over Time 7-3
7.2.2 Statistical Methods 7-4
7.2.3 Sample Size 7-5
7.3 Effects of Team Nutrition on Selection and Consumption of Foods 7-7
7.3.1 Effect of Team Nutrition on Grain Selection and
Consumption 7-7
7.3.2 Effect of Team Nutrition on Fruit Selection and
Consumption 7-8
7.3.3 Effect of Team Nutrition on Vegetable Selection and
Consumption 7-9
Contents (continued)
7.3.4 Effect of Team Nutrition on Milk Fat Selection and
Consumption 7-10
7.3.5 Effect of Team Nutrition on Diversity of Food Choices 7-11
7.4 Overview of Team Nutrition Effects 7-12
7.4.1 Using Meta-Analysis to Combine District Level
Evaluations 7-12
7.4.2 Overall Effect Estimates for Team Nutrition 7-13
7.5 Summary 7-14
STUDENT IMPACTS—SELF-REPORTED BEHAVIOR
8.1 Initial Test of Team Nutrition's Impact on Eating Behavior 8-2
8.2 Development of More Complete Muttivariate Models of
Eating Behavior 8-4
8.2.1 Description of Explanatory Variables 8-5
8.2.2 Theoretical Approach to Explaining Students' Eating Behavior 8-10
8.2.3 Analytic Model of Variable Relationships 8-10
8.2.4 Selection of Explanatory Variables 8-12
8.2.5 Additional Analytic Issues > 8-13
8.2.6 Three Muttivariate Approaches 8-16
8.2.7 Interpretation of Results 8-17
8-3 Results of the Muttivariate Analysis 8-17
8.3.1 Uniform Treatment Model 8-17
8.3.2 Discrete Components Model 8-19
8.3.3 Level of Exposure Model 8-22
8.4 Summary 8-24
SUMMARY AND IMPLICATIONS
9.1 What Do We Know About Team Nutrition Impacts From
This Evaluation? 9-1
9.2 What Questions Remain? 9-4
9.3 How Do Team Nutrition Impacts Compare to Other Nutrition
Education Efforts? 9-7
Contents (continued)
Appendix A Data Collection Methodology
Appendix B Measures and Analysis Models Based on Survey Questionnaires
Appendix C Measures and Analysis Models for Students' Observed Behavior
Appendix D Complete Regressions
Appendix E Instruments and Instructions
Appendix F References Cited
Table
2.1
2.2
3.1
3.2
3.3
3.4
3.5
3.6
3.7a
3.7b
3.8
List of Tables
Number of Treatment and Comparison Schools and
Fourth-Grade Classes in Each District 2-5
Reliability of Student Survey Measures of Knowledge, Motivation, and Behavior,
Using Cronbach's Alpha 2-15
Characteristics of School Districts Participating in Outcome Study 3-1
Characteristics of Outcome Study Schools by Treatment Condition
at Phase I Pretest 3-3
Phase I Baseline: Characteristics (in percentages) of Students
and Their Families by Treatment Group and District 3-4
Phase II Baseline: Characteristics (in percentages) of Students
and Their Families by Treatment Group and District 3-5
Teacher Opinions (in percentages) About Nutrition Education and
Promotion by District and Phase at Baseline 3-7
Menu and Food Service Procedures at Pilot Study Schools 3-11
Average National School Lunch Program (NSLP) Participation by
District, Study Phase and Certification Status 3-12
Average National School Lunch Program (NSLP) Participation by
District, Study Condition and Certification Status 3-13
Mean Baseline Scores and Ranges for Implementation and
Comparison Students 3-14
vii
List of Tables (c »ntinued)
4.1 Team Nutrition Classroom Implementation for Fourth-Grade Students by
District and Phase 4-3
4.2 Average Number of Times Activities Were Conducted Per Class
(Fourth-Grade) 4-3
4.3 Average Number of Times Materials Were Used Per Class, by District
and Phase 4-4
4.4 Sample of School and Community Events by District, Intended Audience,
and Phase 4-6
4.5 School-Wide and Community Events 4-7
4.6 Student Exposure to Team Nutrition Channels 4-9
5.1 Overall Team Nutrition Impact on Number of Correct Answers to Nutrition
Skill-Based Questions (Regression Coefficients) 5-4
5.2 Overall Team Nutrition Impact on Nutrition Motivation (Regression
Coefficients) 5-7
6.1 Percentage of Parents Who Retrospectively Reported "A Lot More" Change in
Children's Nutrition Awareness and Behavior by Area 6-4
7.1 Number of Students Participating in Cafeteria Observations, by District 7-6
7.2 Effect of Team Nutrition by District on Grain Selection and Consumption 7-8
7.3 Effect of Team Nutrition by District on Fruit Selection and Consumption 7-9
7.4 Effect of Team Nutrition by District on Vegetable Selection and Consumption 7-10
7.5 Effect of Team Nutrition by District on Milk Fat Selection and Consumption 7-11
7.6 Effect of Team Nutrition by District on Diversity of Food Choices Tasted .7-11
7.7 Overall Team Nutrition Effects 7-14
8.1 Overall Team Nutrition Impact on Self-Reported Eating Behavior
(Univariate Regression Coefficients) 8-3
8.2 List of Explanatory Variables Considered for Muttivariate Analysis, and
Definitions 8-6
8.3 List of Non-TN Variables That Were Retained in the Final Muttivariate Models... 8-14
8.4 Uniform reatment Model: Average Impact of Participation on Students'
Self-Reported Eating Behavior (Muttivariate Regression Coefficients) 8-18
8.5 Uniform Treatment Model: Average Impact by District of Participation on
Students' Self-Reported E ting Behavior (Regression Coefficients) 8-20
8.6 Discrete Components Model: Effect of Specific Types of TN Participation on
Students' Self-Reported Eating Behavior (Regression Coefficients) 8-20
8.7 Level of Exposure Model: Mean Scores for Students' Self-Reported
Eating Behavior, by Number of Channels of Participation 8-23
viii
List of Tables (continued)
8.8 Effect Per Channel of TN Participation on Students' Self-Reported
Eating Behavior (Regression Coefficients) 8-23
List of Figures
Figure
1.1 Team Nutrition Approach to Effective Nutrition Education 1-6
1.2 Districts Participating in the Team Nutrition Pilot Implementation 1-9
5.1 Average Number of Correct Answers to Nutrition Skill-Based Questions by TN
Students Before and After Participating in Team Nutrition 5-6
5.2 Average Scores on Nutrition Motivation Before and After Participating in
Team Nutrition 5-8
6.1 Students' Improvement in General Nutrition Awareness and Behaviors After 3
Months as Reported by Their Parents 6-3
6.2 Team Nutrition Impact on the Average Number of Students' Healthy Eating
Behaviors as Perceived by Their Parents 6-5
8.1 Average Scores on Self-Reported Eating Behavior Measures Before and After
Participating in Team Nutrition 8-4
8.2 Theoretical Relationship of Variables That May Affect Students' Eating
Behavior 8-10
8.3 Relationship of Variables as Tested in the Multivariate Models 8-11
The USDA's Team Nutrition is a school-based initiative to encourage children to make healthy
food choices. Team Nutrition has been adopted by over 30,000 schocls throughout the country
for implementation in the 1998-99 school year. To evaluate whether Team Nutrition is
successful in changing students' nutrition skills, motivations, and, ultimately, behaviors, USDA
initiated the Team Nutrition Pilot Implementation Project and conducted process and outcome
evaluations. Phases I and II of this pilot project were conducted in seven school districts in the
Spring and Fall of 1996.
A summary of the pilot implementation, evaluation methods, analysis, bud key findings follows.
WHAT IS TEAM NUTRITION?
Team Nutrition is a nationwide initiative designed to help schools plan and serve healthier meals
and to encourage students to select and eat healthier foods.
The mission of Team Nutrition is:
To improve the health and education of children by creating innovative
public and private partnerships that promote food choices for a healthful
diet through the media, schools, families, and the community.
The campaign consists of two interrelated components to help implement the School Meals
Initiative for Healthy Children:
• Multi-faceted nutation education is delivered through multiple channels to build skills and
motivate children to make healthy food choices. The cornerstone of the nutrition
education is a set of classroom modules developed by Scholastic, Inc.
• Technical assistance includes the dissemination of training standards and materials,
grants to states to develop training programs, and a resource system to enable food
service personnel to access education and training programs.
Team Nutrition messages focus on specific, healthy behaviors: eating less fat; eating more
fruits, vegetables, and grains; and eating a variety of foods. Based on social learning theory
(SLT), and social marketing tenets, Team Nutrition assumes that behavior is influenced by a
variety of factors including intrapersonal, interpersonal, institutional, and community variables,
as well as public policy. Thus, the program is designed to reach children through multiple
sources—teachers, peers, parents, food service staff, the media, and the community.
Rfi
WHAT IS THE TEAM NUTRITION PILOT IMPLEMENTATION PROJECT?
To demonstrate the Team Nutrition initiative and to evaluate its effects on children, USDA
implemented the Team Nutrition Pilot Implementation Project in seven school districts. Four of
these districts—Des Moines, IA, HambJen County, TN, Tulsa, OK, and Vacaville, CA—were
selected to serve as the focus for the outcome evaluation. With guidance from USDA, these
districts implemented Team Nutrition activities in selected local elementary schools.
The pilot project was intended to demonstrate the optimal implementation of Team Nutrition.
This consisted of teaching all of the classroom lessons in each of the relevant grades and
constructing a set of core activities designed by the individual districts and approved by USDA.
The classroom lessons were based on modules developed by Scholastic, Inc. for administration
at three grade levels (Pre-K and K, 1 and 2, 3 through 5). Each module consists of eight to ten
lessons that include activities to involve students, peers, parents, teachers, and cafeteria staff.
Core activities included cafeteria and school events such as chef visits, community events like
nutrition fairs, media coverage, and parent events. The pilot implementation was conducted in
the Spring semester of the 1995-96 school year (Phase I) and replicated in the Fall semester of
the 1996-97 school year (Phase II).
OVERVIEW OF THE OUTCOME EVALUATION STUDY METHODOLOGY
To evaluate the effects of the pilot project, a pre-post test and comparison group design was
implemented in each of the four outcome evaluation communities. Each district nominated at
least two matched pairs of elementary schools, one of which would be later randomly assigned
to the treatment condition (implementation schools) and the other to the control condition
(comparison schools). In the comparison schools, neither Team Nutrition nor other nutrition
education programs were implemented during the course of the pilot study.1
The basic evaluation question is whether students in the implementation schools exhibit
changes in skill-based nutrition knowledge, motivation and, most importantly, healthful food
choice behaviors that are not found among the comparison school students and can thus be
attributed to the intervention. The focus of the evaluation is fourth-grade students. Surveys
were conducted among all fourth-grade students, as well as among their parents and teachers,
in the implementation and comparison schools immediately prior to and upon completion of the
11n the accompanying report, The Story of Team Nutrition: Case Studies of the Pilot Implementation Communities.
(USDA. 1996), nutrition lessons were reported in some comparison schools However, none were reported among
fourth-grade classes, the focus of the outcome evaluation.
Team Nutrition activities. In addition, pre- and post-implementation cafeteria observations were
conducted of fourth-grade students to assess changes in food choices and consumption.
The primary study sample consisted of a total of 3,323 students (about evenly divided across
phases) from 12 implementation and 12 matched comparison schools. Between 86 percent and
91 percent of the students completed surveys in class under the direction of their teachers.
Over 80 percent of the students from Phase I were surveyed again 6 months later, when they
were in the fifth grade. In telephone surveys, parents of implementation and comparison school
students assessed their children's nutrition awareness and eating behaviors. The 15-minute
surveys yielded response rates between 72 percent to 87 percent. Teachers' self-administered
surveys assessed their nutrition knowledge, attitudes toward teaching nutrition, and satisfaction
with the Team Nutrition classroom curricula. Response rates for the teachers' surveys ranged
between 79 percent and 86 percent across the four data collection periods.
In addition to the surveys described above, students eating lunch prepared by their school's
cafeteria were observed to obtain measures of their actual food choices and consumption.
Three days of observation were conducted with matching menus both before and after the
Team Nutrition implementation period. Between 3,300 and 3,500 lunch trays were observed at
each data collection period. Measures from the cafeteria observations focus on changes in
students' selection and consumption of fruits, vegetables, and grains; changes in the percent of
fat in milk choices and consumption; and overall diversity of foods that were selected and
tasted.
Information from classroom activity logs completed by teachers, and school/community event
logs completed by Team Nutrition Coordinators in each district, provided estimates on students'
exposure to Team Nutrition. Other sources of data included extant information on schools' size,
racial/ethnic mix of students, and the proportion of students certified for free or reduced school
lunches.
WHAT ARE THE KEY FINDINGS?
Study results are based primarily on findings from the student surveys and observations, with
some corroboration and explanation from parent and teacher data. Analysis of the baseline
data revealed that implementation and comparison school students were similar in their nutrition
skills, motivations, and behaviors prior to the Team Nutrition Pilot and that there was indeed
room for improvement in each of these areas.
X.I
Team Nutrition Raised Students1 Skill-Baaed Knowledge
On three different measures of nutrition skills developed from the student survey, Team
Nutrition students showed increases greater than the changes evidenced by comparison school
students. Two of these gains were statistically significant, and remained so at the 6-month
followup.
• Team Nutrition students showed small, significant increases in their ability to identify
healthier choices from pairs of foods (e.g., trench fries versus baked potato).
• Students exposed to Team Nutrition improved modestly but significantly in their ability to
correctly classify foods using principles from the Food Guide Pyramid.
• Team Nutrition students showed smaller, non-significant improvements in applying the
balanced diet concept to food choices.
Team Nutrition Motivated Students to Eat Healthier
Team Nutrition students showed a statistically significant increase, relative to comparison
students, in all three measures of motivation. These results also remained significant at the 6-
month followup.
• Team Nutrition students increased their positive responses on general nutrition-related
attitudes such as "I like to taste new foods."
• Team Nutrition students exhibited increased positive attitudes about the effects of eating
more fruits, grains, and vegetables every day.
• Students from Team Nutrition implementation schools chose more positive responses in
cognitive rules for healthy choices (i.e., their willingness and ability to follow different
strategies for healthy eating).
All of these changes while statistically significant were of small magnitude.
Team Nutrition Encouraged Students to Eat Healthier
On some but not all measures of food consumption, Team Nutrition produced small, positive
changes among implementation school students. These include:
• Implementation students indicated more healthful choices in their survey answers for
"usual" food choices, food choices made in the past 2 weeks, and the variety of foods
eaten "yesterday." The differences between implementation and comparison students
were significant immediately following the activities, but no longer significant at the 6-
month followup.
xiii
• Cafeteria observations also found that students at implementation schools significantly
increased the diversity of foods selected and tasted from the school lunch menus, and
significantly increased their consumption of grains. Observations for fruit, vegetables
and milk fat consumption did not show statistically significant impacts from Team
Nutrition participants.
• Parents of TN students reported more improvement in their children's nutrition
awareness and greater frequency of nutrition conversations, but not healthier eating
behavior than parents of students in the comparison group.
Modeling Potential Explanations for Team Nutrition Effects
Multivariate modeling controlling for demographic characteristics of students and their
households was utilized to identify what factors influence the students' self-reported eating
behaviors. Models predicting behavior change included students' knowledge and motivation at
baseline and environmental influences (such as the presence of role models). Students' eating
behavior was found to be positively and significantly related to having higher nutrition
knowledge and motivation at baseline, being female, having a higher household income, having
a greater number of positive role models, and having a parent who attempted to influence their
eating behavior. When individual and family characteristics were examined for interaction with
TN participation, none of the effects was statistically significant.
To get a better understanding of how TN impacts occur, three different multivariate models of
Team Nutrition's effects on eating behavior were examined. They were a "uniform treatment"
model that defines the Team Nutrition initiative as a single homogeneous intervention; a
"discrete components" model that looks at the effectiveness of individual Team Nutrition
intervention components, and a "level of exposure" model that treats the initiative as an
accumulation of students' exposures to Team Nutrition messages through a variety of channels.
All three models supported the general overall finding that:
Team Nutrition has a small, but statistically significant, positive impact on students' self-reported
eating behavior even after adjusting for other explanatory factors.
The analyses demonstrate that the "level of exposure" was the strongest prediction model and
was also theoretically plausible within both an SLT and social marketing framework. This
suggests that future Team Nutrition initiatives, as well as other nutrition education efforts, focus
more clearly on maximizing the exposure of their target audiences to multiple reinforcing
channels of communication.
xiv
Overall, the results of the pilot study show that Team Nutrition results in healthful changes in
students' skill-based nutrition knowledge and motivations when it is implemented as
recommended. The effects of Team Nutrition on healthy eating behaviors are generally
positive, but the changes are small and not always significant, especially for observed
behaviors. The findings are comparable to those from evaluations of similar nutrition education
programs. Given where students started and the difficulty that school-based programs typically
encounter when seeking to change behavior, dramatic changes were not expected.
The lessons learned from this pilot study should prove invaluable to the thousands of schools
currently enrolled in Team Nutrition, and validates other such comprehensive approaches to
nutrition education.
xv
1.1 BACKGROUND TO TEAM NUTRITION
The United States Department of Agriculture (USDA) School Meals Initiative for Healthy
Children is a comprehensive plan to ensure that children have healthy meals at school. A major
part of this plan is an update of nutrition standards so that school meals meet the Dietary
Guidelines for Americans. Recognizing that simply publishing a regulation is not likely to
change what children eat at school, USDA established Team Nutrition (TN) to ensure that
schools are able to plan and serve healthier meals and that students select and eat healthier
foods.
The mission of Team Nutrition is ...
To improve the health and education of children by creating innovative
public and private partnerships that promote food choices for a healthful
diet through the media, schools, families, and the community.
Team Nutrition is a nationwide, integrated program that consists of two interrelated initiatives to
help implement the School Meals Initiative for Healthy Children.
Murtifaceted Nutrition Education is delivered through the media, in schools, and at home to
build skill-based knowledge and motivate children to make food choices for a healthful diet. The
initiative is built with an emphasis on the school setting to reinforce/support nutrition policy
changes in school meals. In-school education is provided by classroom modules designed by
Scholastic, Inc. in partnership with the USDA. The materials bring focused, science-based
nutrition messages to children in a manner that they will understand, while strengthening social
support for healthy food choices among parents, educators, and food service professionals.
Training and Technical Assistance is the second focus of Team Nutrition. This initiative is
designed to ensure that school nutrition and food service personnel have the education,
motivation, training, and skill-based knowledge necessary to plan, purchase, prepare, and serve
healthy meals that appeal to children and meet the Dietary Guidelines for Americans. It is also
intended to provide personnel with a clear vision of their roles in the school community and as
integral team members of comprehensive school nutrition programs. This assistance includes
the dissemination of training standards and materials, grants to states to develop training
programs, and a resource system to enable instructors and food service personnel to access
information for education and training programs.
1-1
/
While Team Nutrition presents a complete approach to nutrition education in schools, its use
with existing nutrition curricula is also encouraged. To date, Team Nutrition has been adopted
by more than 30,000 local schools throughout the country. These TN schools demonstrate their
commitment to meeting the Dietary Guidelines for Americans by distributing TN materials to
teachers, children, and parents; by involving school food service staff, teachers, children,
families, and administrators in lively and entertaining nutrition activities; by sharing successful
strategies and programs with other schools; and by engaging private and other public-sector
partners in the community to support TN activities.
1.2 THE TEAM NUTRITION CONCEPT
The concept for Team Nutrition is grounded in a theoretical framework that provideJ an
explanation of how individuals make behavior choices—Social Learning Theory (SLT) (Bandura
1986). Social Learning Theory is the theoretical framework most often utilized in research
studies focused on changing specific eating behaviors (Contento et al., 1995).
Social Learning Theory stands in contrast to theories based on the premise that increased skill-based
knowledge creates more favorable attitudes, which in turn, is reflected in behavior.
Interventions using the latter framework typically are information-based approaches to nutrition
education, wherein knowledge and attitudes are the primary outcomes of interest. In most SLT
empirical studies, the primary outcome is a change in a targeted behavior such as those
promoted through Team Nutrition, including: eating less fat; eating more fruits, vegetables, and
grains; and eating a variety of foods. Skill-based knowledge and attitudes toward the specific
behavior being targeted for change, motivation, self-efficacy, and the ability to perform the new
behavior are conceptualized as intervening variables.
The premise of SLT is that personal characteristics, environmental factors, and behaviors are all
engaged in "reciprocal determinism," or dynamic interaction. In other words, behavior can affect
and/or be affected by personal characteristics and environmental factors. As noted by Contento
et al., (1995). SLT-based nutrition education programs are developed to address multiple
influences on children's behavior, including:
• Personal factors: health-related skill-based knowledge and beliefs, belief that
one can engage in specific behaviors (self-efficacy).
• Behavioral factors: current behaviors, intentions to act, existence of incentives
and/or reinforcers.
J^ 1-2
• Environmental factors: parental and peer influences and support, cultural norms
and expectations, opportunities and barriers to engaging in new behaviors, and
the availability of adult and peer role models.
One of the personal characteristics that has been explored in the nutrition literature is self-efficacy,
which is a person's confidence in performing a particular behavior (Sheeshka et al.,
1993; Parcel et al., 1995). Findings from studies focused on improving self-efficacy suggest
that changes in eating behavior are more readily achieved if the desired change is broken into
well-defined steps. For example, if the desired change is to have children go from eating only
two servings of fruits and vegetables a day to eating five a day, interim goals should aim at
increasing their consumption by one more serving per day. Team Nutrition activities introduce
students to fruits, vegetables, and grains to encourage them to try new foods. Hands-on
activities show students how to increase their consumption of these healthful foods in order to
build confidence and promote incremental changes that, ultimately, will lead to habitual healthy
choices.
The TN initiative also addresses environmental factors that have been shown in previous
studies to affect eating behaviors, including availability of food items at home or school (Kirby et
al., 1995; Simons-Morton, et al., 1991), and prompts by parents to eat foods (lanotti. et al.,
1994). Training for food service staff complements the School Meals Initiative for Healthy
Children to ensure that healthful foods are prepared using methods that optimize their freshness
and appeal. Parent involvement is encouraged in school and community-based activities and in
activities suggested for the home or supermarket.
The TN initiative captures effective components of previous SLT-based programs. In their
review of nutrition education intervention in school settings, Contento et al., (1995) identified 23
studies that employed a SLT-based theoretical approach. Examples include the Know Your
Body Program developed by the American Health Foundation (Walter 1989); multiple school-based
interventions developed by the Minnesota Heart Health Program (Perry et al., 1985,
1987,1988); the Children and Adolescent Trial for Cardiovascular Health (CATCH) (Perry
1990); the Heart Smart Program (Arbeit et al., 1992); and Gimme 5 (Domel et al., 1993) among
others. In each of these cases, the investigators designed programs that addressed the three
factors listed above. They also included numerous program components such as curricula,
changes in school lunches, school-wide events, family involvement activities, messages
delivered through mass media, point-of-choice labeling of healthier food choices in grocery
stores and restaurants, and increasing the time children spent in physical education classes.
1-3 3
This latter activity is particularly important as many behaviorally focused nutrition interventions
are incorporated into comprehensive school health education initiatives. CATCH, for instance,
targeted delaying the onset of smoking behavior and increasing students' level of physical
activity along with encouraging better eating behaviors (Perry et al., 1990).
In developing and implementing the TN initiative, USDA relied on social marketing concepts and
methods that have been employed in numerous nutrition education and public health programs
(Andreasen 1995; Lefebvre and Flora, 1988). Social marketing is a planning process that
incorporates psychological theories that focus on factors leading to behavior choices, such as
SLT, and then folds that understanding into program development. It has an ecological
perspective that assumes behavior is influenced by a variety of factors, including intrapersonal,
interpersonal, institutional, and community variables, as well as public policy.
Social marketing is characterized by its focus on creating programs that meet the needs and
concerns of specific groups of people—for this pilot project, these are the students, their
parents, their teachers, food service staff, district food service coordinators, and school
administrators. Each of these groups of people have differing needs and concerns relevant to
the project. Students want to learn in an active and fun environment and have lunches that
taste good. Parents want their children to learn and practice skill-based knowledge that will
prepare them for a healthy and productive life. Teachers want to learn basic nutrition concepts
so they are comfortable teaching the curriculum while also balancing the other curricular
demands on their time. Food service staff need to learn new preparation methods and also
want to be viewed as contributors to the education environment in the school. District food
service coordinators need to learn how to implement the various TN school activities while
balancing this responsibility with their many other demands; and finally, administrators want to
expose their students (and teachers) to new and innovative curricula while also having to
conserve and allocate precious resources. In developing the materials for the pilot project, and
in planning and implementing the project itself, attention was paid to each of these groups.
The cornerstone of the social marketing approach is the development of a "marketing mix" for
each discrete group of people. The four P's of this marketing mix - product, price, place, and
promotion—are described on the following page.
t 1-4
Product. Social marketing "products" are typically behaviors rather than tangible items
that characterize commercial marketing practice. The product of Team Nutrition is
healthier eating behaviors among students. For teachers, food service staff,
coordinators, and administrators in the pilot, it is implementing the program and
participating in the evaluation activities.
Pricing. Pricing represents the costs of the product. Social marketing also recognizes
that adopting new behaviors, like acquiring new products, has a number of associated
"costs"—though not necessarily fiscal ones. TN "prices1' from the students' perspective
include investments in time to apply learned skill-based knowledge, outside the
classroom, such as reading Nutrition Facts labels. On a community level, costs
associated with TN include additional curricula, classroom activities, community
activities, and food service training, each of which represents an investment of time and
resources by the individual teachers, parents, schools, and school districts, including the
food service staff and community partners. Initiatives such as Team Nutrition are
structured to provide classroom activities that can be integrated into existing curricula
without relinquishing time spent on core subjects. In addition, teacher training and
technical assistance for food service staff help to minimize resource expenditures and
maximize the opportunities for leveraging resources through community partner
organizations. Additional resources were given to the pilot schools to offset the costs
associated with participating in the evaluation.
Placement Placement involves making the resources available to the consumer to
facilitate behavior change. How information and materials are distributed and then
implemented by TN schools is thus the third area of planning a social marketing
program. Previous trials such as CATCH have demonstrated that school-based
educational programs involving multiple channels such as classrooms, food service,
community, and families can be successful at changing student eating patterns in
different areas of the country. The TN materials were distributed to all schools in the
pilot project as they are, albeit in smaller quantities, in the larger TN initiative. The
school representatives and teachers use these materials to disseminate the messages
through the classroom, the cafeteria, and take-home activities.
Promotion. Promotion is required to encourage adoption and maintenance of the
desired behavior. Therefore, the final task of planning a social marketing program is
developing the communication tools to promote adoption of the program message and
behavior change.
In the pilot project, promotion (or communication) tools included curriculum, school and
community-based activities, and public service announcements (PSAs) using familiar
Disney characters aimed at the children to get them to adopt the behavior. To
encourage adoption of the program, teachers attended two training sessions to learn the
curriculum and also received lesson plans to aid implementation. Coordinators received
orientation materials and had biweekly telephone calls with evaluation staff. School food
service staff received the technical assistance materials developed by the USDA and
training provided by the districts; administrators received orientation materials.
1-5
A graphic model of the TN approach to nutrition education, showing its major components, is
provided in Figure 1.1.
Figure 1.1. Team Nutrition Approach to Effective Nutrition Education
1.3 TEAM NUTRITION PILOT IMPLEMENTATION PROJECT
The TN Pilot Implementation Project was designed to evaluate whether the TN approach results
in healthier food choices by students, as wen as to inform future decisions about TN policy and
technical assistance. The pilot project was implemented in two nhases—once in the Spring of
1906 and again in the Fall of 1996. The Fall implementation was essentially a replication of the
Spring implementation with a new set of students; however, participating districts made changes
in some of the activities based on their experiences in Phase I. The two-phase design served at
least two purposes—to allow replication of any effects observed in Phase I and to evaluate the
I* 1-6
effects of a somewhat more "mature" program when it is implemented for the second time.1 In
addition, students who participated in the Phase I pilot were surveyed again during Phase II to
ascertain whether any changes in skill-based knowledge, motivation and/or behavior are
sustained over time.
The TN Pilot Implementation Project is designed as an efficacy evaluation conducted in a field
setting. It addresses the following question: can Team Nutrition have a positive impact when
implemented as intended? Because it is designed as a model intervention, the pilot project
differs from the overall TN effort in several ways:
• The pilot schools received multiple copies of the Scholastic classroom materials-one for
each teacher participating in the intervention.
• Representatives from each of the pilot communities were oriented to Team Nutrition, the
pilot project, and the evaluation during a 2-day planning meeting at USDA.
• Each of the participating school districts was provided with two teacher training sessions
(4 hours each) for all implementation school teachers in Phase I. The first session took
place just prior to the start of the intervention, while the second was conducted 2 to 4
weeks after the intervention began to review and incorporate teacher experiences.
• Each district submitted an implementation plan for the pilot and received feedback and
technical assistance from USDA in formulating the plan.
• The coordinators in each of the pilot communities had regular contact with USDA staff,
as well as evaluation contractor staff.
• Each pilot community received a stipend to support its involvement in the evaluation of
the initiative. Although none of this money was to be spent on program implementation,
it was a useful resource for the districts, provided motivation for participation, and
compensated the school districts for the time they devoted to participating in the rigorous
evaluation protocol.
Schools participating in Team Nutrition that were not in the pilot study receive a start-up kit, but
no formal orientation, training or other resources.
While the pilot communities benefited from some support that other TN schools will not receive,
they also operated under some constraints by virtue of their participation in the pilot
implementation and evaluation. These included:
• Limited teacher preparation time after receipt of the Scholastic materials in order to
accommodate the implementation and evaluation schedule in the Spring semester.
1 Although teachers and staff in Phase II were more experienced with Team Nutrition, students may have been at a
somewhat different level devetopmentalry because Phase II took place at the start of the school year.
1-7 7
• Limited time (roughly 8 weeks) to teach the Scholastic lessons and conduct the
corollary TN activities, again to accommodate the evaluation schedule.
• Constraints on their ability to enlist the media to promote events due to the need to
balance publicity against contamination of the comparison schools.
• Time that might have been spent on implementation was instead used to support
the evaluation effort (i.e., teacher, parent, and student surveys; activity logs;
cafeteria observations; site visits; etc.).
Although somewhat distinguished by the aforementioned supports and constraints, the TN Pilot
Implementation Project was conducted in real world settings—not a laboratory environment.
The pilot communities faced obstacles in implementing the initiative that many schools might
encounter, including competition for class time from other subjects and preparation for
standardizing tests, food service contract changes, and a potential teacher strike in one site.
Thus, the lessons learned from the pilot should prove valuable to schools around the nation that
choose to become TN schools.
The next section of this report describes the district and school selection for the TN Pilot
Implementation Project as well as the requirements for participating districts.
1.3.1 District School, and Classroom Salaction
In July 1995, the USDA issued announcements through its seven regional offices to recruit
school districts to participate in the Pilot Implementation Project. Public school districts that
participated in the National School Lunch Program were eligible to apply. The goal of the
recruitment was to select at least one school district from each of the seven USDA regions;
however, participation in the pilot was completely voluntary.
The application for becoming a TN Pilot community included answering a set of questions about
the district's ability to carry out the project and a resume for the district's proposed Team
Nutrition Coordinator (TNC). Project requirements included district implementation of an
intensive form of Team Nutrition and participation in the evaluation protocol. As part of their
application, districts were asked to nominate at least two matched pairs of elementary schools.
Each matched pair of schools were to be as similar as possible with respect to the following:
number of stuck, .its; the percentage of students eligible for free or reduced-price lunches; the
racial and ethnic composition of the schools; the extent of existing nutrition education efforts;
and the type of cafeteria service provided (e.g., menu choices available and type of kitchen).
/
1-8
Seven TN Pilot districts were competitively chosen from the applications received. The
selection criteria included an assessment of the district's ability to carry out the project, a desire
to have a district in each USDA region, and the need to have cost-efficient access to the
communities for evaluation purposes. Applications were approved from one district in each of
the seven USDA regions (Figure 1.2).
Figure 1.2. Districts Participating in the Team Nutrition Pilot Implementation
Each district selected three grades to implement the appropriate Scholastic modules. Three of
the seven school districts (Lawrence, MA; Passaic, NJ; and Cleveland, OH) were assigned to a
limited process evaluation (basic process) and did not participate in the outcome evaluation
protocol. Findings from the process evaluation conducted in the seven districts are presented in
an accompanying report, "The Story of Team Nutrition: Case Studies of the Dilot Implementation
Communities" (USDA-FNS, 1998).2
2
For the remainder of this report. The Story of Team Nutrition: Case Studies of the Pilot Implementation
Communities" will be referred to as the Case Studies Report.
1-9 f
The focus of this report is on the four districts that participated in both an intensive process and
outcome evaluation of the TN Pilot Implementation Project—Oes Moines, IA; Hamblen County,
TN; Tulsa, OK; and Vacaville, CA. These districts were chosen for the intensive evaluation
based on their ability to meet the application deadline and a demonstrated strong capacity for
both implementation and evaluation of the TN initiative
The Des Moines and Tulsa school districts each nominated four pairs of schools for the pilot
project, while Hamblen County and Vacaville nominated two pairs of schools. One-half of the
pairs in each outcome evaluation district were randomly assigned to the treatment condition and
the other half to the comparison condition. Thus, the outcome evaluation of the TN Pilot
Implementation Project includes twelve elementary schools in which the initiative was
implemented, aod twelve comparison schools. The comparison schools delayed nutrition
education curricula until the conclusion of the TN Pilot Implementation Project.
While the TN Pilot was implemented at three different grade levels in each school, the focus of
the outcome evaluation is on the fourth-grade students. Children at this age (8 through 10
years) are capable of completing survey instruments that were integral to the current evaluation
(Domel et al., 1994). Also, resources were not available to design and implement a multiple-grade
evaluation protocol.
1.3.2 Implinrntation Rtquinnrnts
Upon being selected for the TN Pilot Implementation Project, each district was required to
submit an implementation plan to the USDA detailing their plans for the initiative. To facilitate
development of these plans, as well as to educate the districts on their role in the eva> jatfon
protocol, a 2-day planning meeting was held in Washington, DC, on November 6 and 7,1995.
The meeting included key personnel from USDA, representatives from the seven selected pilot
evaluate . communities (including the TNC), Scholastic, Inc., and evaluation contractor staff.
The meeting included presentations by each of the seven participating school districts, an
overview of the TN components, a description of the evaluation activities to be conducted, and a
review of the implementation requirements.
/
0 1-10
As participants in the TN Pilot Implementation Project, each of the seven districts agreed to
conduct teacher and food service staff training, as well as a set of core school and community
activities, to reflect the TN concept described above. These requirements are described below.
Classroom Components
As noted earlier, the cornerstone of the TN initiative is a set of classroom materials developed
by Scholastic, Inc. The TN classroom materials consist of three modules, one for each of the
following sets of grades:
• Module 1: Pre-kindergarten and kindergarten (Pre-K and K).
• Module 2: First and Second Grades (1 and 2).
• Module 3: Third through Fifth Grades (3 to 5).
While based in the classroom, the Scholastic modules are designed to involve other students in
the school as well as the school cafeteria staff and parents. Each of the modules consists of a
set of eight to nine lessons and contains teacher guides, classroom activities, videos, posters,
student magazines, and parent take-home pieces. The lessons, designed to involve students in
making decisions about nutrition, include a number of activities that can be integrated into
existing curricula Each lesson in Modules 2 and 3 also includes an activity that links the lesson
to the school cafeteria ("Lunchroom Link").
The nine lessons in Module I each include several suggested activities that can be related to a
subject area such as language, music, science, or social studies. For exanu-ie, the lesson on
how our senses learn about foods includes a science activity in which the students smell foods
while blindfolded, and an art activity in which students plan menus involving foods of a particular
color for each day. Modules 2 and 3 are more structured, with similar activities for each lesson.
For example, each lesson includes a "Getting Started" session, two to three discrete activities, a
"Lunchroom Link", and a few other activities. The activities in these lessons include a fruit and
vegetable tasting party, setting up an imaginary restaurant and creating menus, and planning
and holding s food fair. Appendix A in the Case Studies Report provides a brief description of
each of the Scholastic modules.
1-11
//
The outcome evaluation focused on Module 3. This module entitled "Food Works," is designed
for grades 3 through 5. The teaching kit includes the following items:
• "Dear Teacher" tips—Overview of how the kit is organized.
• What Do You Know About Food—Pretest quiz can also be used as a posttest.
(Optional).
• Lesson Planner Matrix—References to lesson components, teaching objectives, and
specific activities.
• Nutrition and Physical Activity Information—Basic information about the Dietary
Guidelines, Food Guide Pyramid, and the link with physical activity.
• Resource Materials—A list of materials and other organizations that teachers can use to
supplement the curriculum.
• Lesson Plans—8 lesson plans that contain a variety of engaging activities.
• Classroom Reproducibles—To copy for students.
• Parent Reproducibles—Eight one-page handouts coordinated with the classroom
lessons to send home with students. Each provided brief overviews of classroom
lessons and suggested reinforcing activities at home, in the supermarket, or on outings.
For example, "Gardening Together" provides instructions on growing vegetables in
gardens or pots and creating an indoor compost pile.
• Take Out—Family newsletter.
• "Food Works" Video—-Highlights activities that will be recreated in the lessons.
• "Food Works* Magazine—A set of magazines for students.
• Posters—2 colorful motivational posters.
The "Food Works" module is designed to be integrated into the regular classroom curriculum. It
includes eight lessons that are described below. Every lesson includes two or three classroom
activities, a "Lunchroom Link" and reproducibles for students and parents.
• Lesson 1: Food Grows—Students discover where food comes from. This lesson
involves Language Arts and Science.
• Lesson 2: Bodies Grow—Students learn how the Food Guide Pyramid can help them to
eat healthier. This lesson involves Arts and Math.
• Lesson 3: Read All About It—Students analyze their diets and set goals. This lesson
involves Language Arts and Art.
• Lesson 4: Label Lowdown—Students are introduced to the Nutrition Facts food label.
This lesson involves Language Arts and Math.
• Lesson 5: Fat Facts Feature—Students investigate the fat content of various foods.
This lesson involves Language Arts, Art, and Math.
• Lesson 6: Sense-ational Food—Students examine the influence of sensory information
on food choices. This lesson involves Language Arts and Science.
\>-
1-12
• Lesson 7: The World on a Plate—Students expand the range of food choices and try
foods from various cultures. This lesson involves Social Studies, Language Arts, and
Art.
• Lesson 8: The Great Nutrition Adventure—Students recall *hat they have learned about
healthy eating and share it with family and friends. This lesson involves Language Arts
and Art.
In addition to teaching every lesson in each Scholastic module to ev^ry class in the appropriate
grade, all the pilot schools (except comparison sites) also agreed to conduct teacher and food
service staff training and to implement a set of core school and community activities. These
latter requirements are described below:
Teacher Training and Classroom Implementation
• In Phase I only, all teachers involved in teaching any of the lessons from the TN
classroom modules were required to participate in a training program. This training
was conducted onsite in each district by Prospect Associates and included one
session prior to implementation and one session 2 to 4 weeks later. It should be
noted that the final Scholastic modules were not available at the time of the initial
training session and draft materials were used. Phase II training was required only
for new teachers and was conducted by TNCs.
• All eight of the Scholastic lessons for Modules 2 and 3—and at least eight of the
nine lessons in Module 1—were to be implemented for the selected grade in each
phase (Spring and Fall). Because the lessons for Module 1 are less structured, and
include various numbers of suggested activities, there were no instructions for
specific activities beyond completing three activities in each lesson. For each lesson
in Modules 2 and 3, the teachers were required to conduct, at a minimum, the
"Getting Started" session and each of the activities for the lesson. They were also
required to complete the "Lunchroom Link" for at least four of the lessons, and to use
the parent and student reproducibles and video as specified by the Scholastic
module.
School and Community Core Activities
In each phase, implementation schools were expected to:
• Conduct at least two school-wide cafeteria events.
• Implement at least three parent contact activities.
• Host at least two chef activities.
• Hold at least one district-wide TN community event.
• Conduct at least one district-wide media event.
1-13 v\
School Food Service gtgff Training
During the course of the pilot project, each implementation school was required to:
• Conduct at least 10 hours of food service staff training.
• Institute menu changes to make menus consistent with the Dietary Guidelines.
Prior to implementation, each district submitted their TN activity plan to the USDA for approval.
Because some of the core activities could be structured to meet multiple requirements (e.g., a
chef activity could also be a school-wide cafeteria event with media coverage), each school was
required to complete at least five activities that represented each of the types of core activities
presented above. In addition, no single event could be counted toward more than two of the
core activities. The Case Studies Report documents in detail how each of the seven pilot
districts implemented Team Nutrition in both Spring and Fall phases.
1.4 STRUCTURE OF THE OUTCOME MEASURES REPORT
This chapter has reviewed the objectives and conceptual underpinnings of USDA's TN initiative.
The discussion included a presentation of the pilot program's overall design along with brief
descriptions of pilot school selection and implementation requirements. Chapter 2 provides a
description of the evaluation design, outcome measures and anal/sis approach. Chapters 3
and 4 set the stage for evaluation by describing the baseline characteristics of the pilot
communities as well as the extent of TN implementation in each district. The impact of Team
Nutrition on students is explored in Chapters 5 through 8. Chapter 5 describes effects on
student's skill-based nutrition knowledge and motivation to choose healthy foods. A measure of
change in students' eating behaviors based on their parents' perceptions is presented in
Chapter 6. Observed changes in their school lunch food choices and consumption are
described in Chapter 7. In Chapter 8, TN effects are modeled to explore changes in students'
eating behavior based on self-reports. Multivariate models for eating behavior, skill-based
knowledge and motivation address the extent to which TN effects can be explained by student
characteristics, (e.g., demographics) and features of the implementation (e.g., the intensity of
the classroom and core activities). A comparison of TN effects to those of other school-based
nutrition programs is presented in Chapter 9. Details of the data collection and analyses are
provided in the appendices.
IV 1-14
CHAPTER 2: EVALUATION OF TEAM NUTRITION PILOT COMMUNITIES
15
2.1 OBJECTIVES OF THE TEAM NUTRITION OUTCOME EVALUATION
The primary behavioral objectives of Team Nutrition (TN) are for students to:
• Eat more fruits, grains, and vegetables,
• Eat a greater variety of foods, and
• Eat less fat.
As described earlier, Team Nutrition uses a multi-faceted approach to influence children's skill-based
knowledge, motivations, and, ultimately, behaviors in these areas. Team Nutrition also
uses a long-term approach: It is intended to provide an intensive classroom curriculum at least
three times from kindergarten through the fifth grade supplemented with school and community
events.
The primary objective of the TN evaluation is to determine the degree to which the pilot
implementation brings about changes in students' skill-based knowledge, motivation, and
behavior as they relate to healthy eating. To meet this primary objective, a combination
pretest/posttest and comparison group design was implemented in each of the four outcome
communities. As described in Section 2.3.1, data were collected through surveys of students,
parents, and teachers, and by lunchroom observations, to detect changes in students. This
study focuses on students in the fourth grade and compares data collected at the beginning and
end of each implementation phase of the pilot (in the Spring and Fall semesters of 1996) for
both TN and comparison school students. In addition, for Phase I only, followup data were
collected from students during the next school year (i.e., Fall, 1996, when they were in the fifth
grade). By focusing on a single grade, this study is able to provide information more cost-effectively
than if the entire kindergarten through fifth grade exposure to Team Nutrition was
examined. As a consequence, however, this study does not address the long-term cumulative
impact of the multiple classroom, community, and school experiences.
For the purpose of this evaluation, TN impacts are defined as student changes that can be
attributed to the intervention, independent of any changes that may be occurring without Team
Nutrition. Consequently, impacts may be different from the absolute amount of change that is
observed for, or reported by, students. For example, cross-sectional data from the Continuing
Survey of Food Intake by Individuals (Munoz et al., 1997) indicate maturation affects on
children's eating behavior. These data show that a smaller proportion of male and female teens
compared to pre-teens consume the recommended amounts of fruits. Similarly, fewer teenage
2-1
/£
girls consume the recommended amounts of vegetables and dairy foods compared to girts ages
6 to 11. It seems plausible that motivation to eat healthfully also decreases as pre-teens move
into adolescence.1 In such cases, students exposed to Team Nutrition who maintain the same
level of consumption or motivation as they mature may be considered to have been positively
affected by the program, even though they show no absolute change. Alternatively, due to
maturation, students might improve their nutrition knowledge over time regardless of their
participation in Team Nutrition. In these cases, the absolute amount of change would overstate
the effect of Team Nutrition, and the appropriate measure is whether the improvement was
greater among students who participated in Team Nutrition than among students who did not
participate. The study design is well suited for separating TN impacts from changes. The
availability of both pretest and posttest data for the comparison group allows absolute change
over time to be estimated, and the difference between the change observed among the
comparison group and the change observed among TN participants allows the TN impact to be
estimated (also referred to as the "double difference" method).
2.2 RESEARCH QUESTIONS
The outcome evaluation's primary objective can be operationalized by constructing a number of
explicit research questions:
Change in Student Skill-Based Nutrition Knowledge
• Is there more improvement in the level of healthy food choice knowledge and skill
among students attending intervention schools than among students attending the
comparison schools?
• Does intervention school students' knowledge of the Food Guide Pyramid improve
more than comparison school students' knowledge?
• Does the TN intervention enhance students' ability to apply the balanced diet
concept?
• Do any differences in knowledge that are related to TN participation persist into the
next academic year?
1 No comparable data are available that assessed nutrition motivation among pre-teens and teens Graves et al.,
(1862), found that responses to questions about nutrition attitudes were similar for students between grades four to
six, however, the Hems differed significantly from those used in this evaluation.
2-2 n
Chanoe in Student Nutrition Motivation and Attitude
• Is there a relatively greater increase in general motivation to eat healthy among
students attending treatment schools when contrasted with students attending the
comparison schools?
• Is there relatively more positive change in attitude towards the consequences of
eating fruits, vegetables, and grains among treatment than comparison school
students?
• Does the TN intervention enhance the cognitive rules by which students make
healthy food choice decisions? Do any differences in motivation that are related to
TN participation persist into the next academic year?
Relative Chanoe in Student Food Consumption Behavior
• Do parents of students in treatment schools report relatively more positive changes
in their children's eating behaviors than do the parents of comparison school
students?
• Does participation in Team Nutrition result in relatively more healthy eating behaviors
of students when eating in school cafeterias? Did these students increase their
selection and consumption of grains, fruits, and vegetables? Increase the
consumption of lower fat milks? Increase the variety of foods selected and
consumed?
• Is there more relative improvement in self-reported food choices for students
attending treatment schools when contrasted with students attending the comparison
schools? Were their usual food choices moro healthy? Were their choices in the
last two weeks more healthy, especially with regard to fruits and vegetables? Did
they report eating a greater variety of foods? Are any differences that are found
maintained several months later.
Other Factors Affecting Eating Behavior and TN Impact
Because the TN initiative is based on the premise that behavior change occurs through the
influence of many personal and environmental factors, the evaluation also addresses the
relationship between such factors and eating behavior. This aspect of the evaluation is
addressed by the following research questions:
• Is there a relationship between the students' initial level of nutrition knowledge and/or
motivation and their self-reported food choices? Are changes in knowledge and
motivation associated with corresponding changes in food behavior?
• How significant are nutrition-related role models such as parents, siblings, teachers,
and friends for students changing their eating behaviors?
2-3
) f
• How does the parent's involvement in TN activities and events relate to changes in
their child's healthy food choices?
• Is the experience, motivation, and involvement of TN teachers related to changes in
their students' food choices?
• Does the intensity of students' exposure to TN activities and events determine the
degree of change in their food choice behaviors?
• Does the number of channels through which students receive nutrition messages
affect the likelihood that students change their food consumption behavior?
• Are there any other differences among the four school districts that are related to
changes in student's eating behavior?
2.3 OUTCOME EVALUATION APPROACH
2.3.1 Study Design and Scope
As discussed in Chapter 1, four school districts—Des Moines, Tulsa, HamWen County, and
Vacaville—were selected to participate in the outcome and intensive process evaluation of the
TN Pilot Implementation Project.
From the nominated matched pairs of schools, one school was randomly assigned to the
treatment condition, and the other school was assigned to the comparison condition. This
resulted in four treatment and comparison schools in Des Moines and Tulsa and two treatment
and comparison schools in Vacaville and Hamblen County. Thus, for each phase of the pilot,
there were twelve treatment and twelve comparison schools. The comparison schools agreed
to delay nutrition education in the relevant grades until after the implementation and evaluation
of the TN Pilot. Table 2.1 summarizes the distribution of treatment and comparison schools
across districts, and the number of fourth-grade classes involved in the evaluation.
The TN Pilot was implemented and evaluated in two phases—once in the Spring of 1996, and
again in the Fall of 1996. Each phase included a ore-post design to asaeas whether student
changes occur in the treatment schools that do not occur in the comparison schools In
addition, Phase I students were administered the student questionnaire a third time during the
next school year in order to determine whether any changes found immediately after the
intervention persisted into the next school year.
2-4
If
Table 2.1. Number of Treatment and Comparison Schools and Fourth-Grade Classes in
Each District
School District
Treatment
Schools
Comparison
Schools Total
NumbT of School*
DesMoines 4 4 8
Hambien County 2 2 4
TUN 4 4 8
Vacaville 2 2 4
TOTAL 12 12 24
Number of Classes
(Phms* 1 and II)
DesMoines 21 22 43
Hambien County 12 15* 27
Tulsa 20** 23* 43
Vacaville 17* 14 31
TOTAL 70 74 144
Number of classrooms increased by one in Phase I
** Number of classrooms decreased by two in Pha*e il
f Number of classrooms decreased by one in Phase II
This research design may be evaluated with regard to internal and external validity. Internal
validity concerns whether some non-experimental factors (i.e., not related to Team Nutrition)
affected the results of the study. For example, students' eating behavior could be influenced by
the knowledge that they were being observed in the cafeteria, by the surveys they completed,
by maturation over time, or by historical events (e.g., the presentation of a new diet plan on a
popular television show). The pretest/posttesi comparison group design largely controls for
threats to internal validity because the comparison group generally had experiences, other than
Team Nutrition, that were similar to the intervention group. That is, they were observed, took
the same surveys, were studied over the same time period, and would have been exposed to
the same events.
While the study design provides strong controls for threats to internal validity, it does not
eliminate all of them. For exampJe, it is possible that there was an interaction between the
observation process and the implementation, so students who are exposed to the evaluation
process might respond to the implementation differently. One such potential scenario is that
2-5
«&>
students tried to give socially desirable answers, and that TN participants became more capable
of identifying which answers were most socially desirable. In this regard, the use of multiple
perspectives (i.e., lunchroom observations and parental observations in addition to the student
self-reports) helps to verify whether self-reported changes are consistent with the changes
observed by others.
Another possibility is that, because of the small number of schools involved, the matching and
randomization procedures may not have been sufficient to guarantee that the two sets of
schools/students were equivalent. To examine this possibility, the two groups were compared
across a variety of student characteristics, and only small differences were found (see Chapter
3). Generally, the repeated measures approach that is used in the analyses minimizes the
importance of such differences, because change is measured on a student-by-student basis
rather than by comparing the overall distributions for TN participants and the comparison
students (see Section 2.4.5). To further limit the effect of such differences, variables measuring
student characteristics were included in the multivariate models in Chapter 8 and in Section
6.2.1 (which does not use a repeated measures approach) to statistically adjust for any
differences.
External validity concerns the degree to which the findings can be generalized to a larger
population beyond the students, schools and districts in the study. To the extent possible,
steps were taken to increase the external validity of the study. The selection of school districts
from different geographic regions adds some generalizability. In addition, districts were asked
to nominate school pairs that varied from each other with respect to student and community
characteristics.
However, there remain important limitations to the generalizability of study results. First, the
small number of districts and schools limits the degree to which they can provide reliable,
nationally representative data. Second, as discussed in Chapter 1, the districts were chosen in
part by their wiingnoss to participate and their perceived ability to carry out the project, so that
districts that are less interested or seemingly less able might experience different impacts.
Finally, each participating district was provided training in how to implement the program,
technical assistance in planning their program, and a monetary support to offset its participation
in the evaluation. In actual practice, school districts would not receive these resources, nd it is
possible that such extraneous supports do have an effect on the impact of the program.
One other important design component that does not affect the validity of the study, but does
have implications for its interpretation, was the decision to make the comparison between
2-6 a
schools that implemented Team Nutrition and schools that did not provide any nutrition
education. Because of this choice, this study is not designed to provide data on how Team
Nutrition compares to other nutrition education programs, including those that might have been
implemented at participating schools before the study began. The extent to which treatment
and comparison schools provided nutrition education before the study began is discussed in
Chapter 3.
In sum, the pretest/posttest comparison group design is a strong design that controls for many
of the factors that threaten internal validity. Though the study is not designed to provide
nationally representative data, the sampled districts provide considerable diversity. As in every
study, issues of cost and practicality lead to some compromises from an "idear design, and
while the weaknesses listed above should be considered when interpreting the data, they
should not be construed as invalidating them. What this evaluation does best is to address the
question: When Team Nutrition is implemented as intended, does it have a positive effect on
students?
2.3.2 Data Collection
2.3.2.1 Overview
To measure impacts, five types of data collection were conducted:
• Surveys conducted in the classroom to measure students' skill-based knowledge,
motivations, and self-reported behavior relevant to the TN objectives.
• Cafeteria observations of student food choices and food consumption during the
pretest and posttest to characterize TN's impact on students' eating behaviors.
• Surveys administered to the parents of the sampled students to measure their
perceptions of changes in their children's nutrition motivation and behavior.
• Surveys administered to the teachers of the sampled students to measure their
motivation to teach nutrition, previous training in nutrition, previous teaching
experience, and satisfaction with TN training.
• Activity logs completed by teachers and district Team Nutrition Coordinators to
estimate students' exposure to classroom, school-wide, and community activities.
Student surveys and observations at treatment and comparison schools took place in each
phase before and after the TN intervention. Parent and teacher surveys were conducted during
the same timeframes. In addition, the student surveys were repeated a third time among the
2-7
Phase I students during late Fall, 1996 (when they were in the fifth grade). All surveys were
approved by the Office of Management and Budget prior to their use.
The student, parent, and teacher surveys also served two other purposes:
• To document the exposure of students, parents, and teachers to the TN treatment,
and
• To offer an explanatory context for any TN impacts detected in treatment school
students (e.g., are they reflective of parents' and/or teachers' nutrition-related beliefs
or behaviors, their involvement in TN activities, etc.?).
These five data collection efforts form the basis of the outcome evaluation. Each is described
briefly below. Additional detail can be found in Appendix A of this report
2.3.2.2 Student Surveys
The student questionnaire was administered in the classroom as a written questionnaire to ail
fourth-grade students before and after the TN intervention in both treatment and comparison
schools. There were 1,657 students in Phase I and 1,666 students in Phase II; the response
rates across districts ranged from 86 percent to 91 percent. Because most of the analyses in
this report focus on changes over time, data from each student at two points in time were
required. A total of 1,509 students in Phase I and 1,441 students in Phase II completed both
the pretest and posttest. Additional information about the number of students and response
rates is provided in Appendix A
The questionnaire contained a variety of items to assess students' skill-based nutrition
knowledge, motivation, and behavior. Many of these items were based on research instruments
used in other nutrition studies, but modified to fit the goals of this study or to correct problems
that were observed when the survey items were pre-tested. Additional information about the
sources of the items and their use to create scales is provided in Appendix B.
2.3.2.3 Observations of Student Food Choices and Consumption
To assess actual eating behavior among students, an observation protocol was used to
measure food choices and consumption among all fourth graders eating school lunch.
Observations were made during each phase in the treatment and comparison schools on three
selected days prior to and after the TN intervention. Within each phase, schools were asked to
provide the same menus at baseline and posttest so that any changes in student behavior could
2-8
not be attributed to the foods that were available. Only observations of students who
participated in the National School Lunch Program (NSLP) on matching days of pre- and
posttest (i.e., days when the same menu was offered) were included in the analyses.
Depending on the student's participation, between one and three pairs of observations were
available.
In Phase I, trained observers recorded the food choices made by each student as (s)he came
through the cafeteria line and then visually estimated the portion of each item that was
remaining when the student had finished eating. In Phase II, the methodology was changed in
order to increase the accuracy of the student observations. This change involved weighing the
amount of each food Kern remaining after the child had finished eating. Because of this change
in methodology, one might expect that the results for Phase II would show decreased standard
errors. However, there is little reason to think that the TN impact was significantly affected by
this methodological change (i.e., based on the regression coefficients). While the visual
estimates are presumably less accurate than measurements based on weight, the training
process for Phase I was sufficiently rigorous to be confident that no systematic bias (i.e., making
consistent under- or over-estimates) was introduced by the observers.
A full description of the protocols used for food selection and consumption measures is found in
Appendix A. The protocol was adopted from standard procedures recommended by USDA
(Comstock, et al., 1979), procedures reported in published studies (Comstock, et al., 1980;
Graves and Shannon, 1983; Kirk and Wolff, 1985), and methodology recommended by
researchers currently conducting plate waste studies (Fox, personal communication, November,
1995; Nicklas, 1995). Adaptations were made in response to the pre/posttest design in this
study, and the need to carry out data collection simultaneously at multiple sites. These
circumstances required substantial training and monitoring to assure reliability among all
observers (reliability is reported in Appendix C). The observations were tracked so that the
calculations could be made on a student-by-student basis. This is the first USDA study to
observe the same students at multiple points in time.
In addition, the observation protocol depended on consistent availability and presentation of
menu items at the two test times in each phase; the cooperation of teachers and administrators
who distributed identifying stickers; and the cooperation of students. Appendix C.4 notes
inconsistencies across these aspects. Most of the menu variations occurred on isolated days,
and no obvious pattern of differences between implementation or comparison schools emerged.
The foods for which menu variations appeared to present the most serious problems were
entrees and the availability of side dishes. As described in Appendix C 4.2, exploratory
2-9
{
analyses found only very slight differences when the inconsistent data were discarded.
Therefore, the full data set was used in the analyses.
In Appendix A, Tables A.6 (Phase I) and A.7 (Phase II) detail the number of students observed,
the number of meals they ate in the cafeteria per observation period, and the total number of
meals assessed for implementation and comparison schools within districts. More than 3,500
meals were assessed at Phase I baseline and again at the Phase I posttest. In Phase II, the
number of meals assessed at baseline and posttest was over 3,300 at each data point.
2.3.2.4 Parent Surveys
The parents of fourth-grade students were surveyed to assess home nutrition practices, relevant
attitudes and knowledge, perceptions of children's nutrition habits, and parent involvement in TN
activities. The main objectives of this survey were to provide an independent measure of
students' nutrition awareness and behavior, and to identify those factors within the home setting
that might serve as mediating/intervening influences on change in the students' eating
behaviors.
As with the students, the parents were surveyed both before and after the TN intervention. The
parent surveys were conducted primarily by telephone. However, those parents without
telephones and those for whom telephone numbers were unavailable were sent questionnaires
by mail. The response rates to the four parent surveys ranged from 72 percent to 87 percent.
2.3.2.5 Teacher Surveys
The teachers implementing Team Nutrition in their classrooms, as wed as their counterparts in
the comparison schools, answered a variety of questions focusing on their nutrition knowledge,
attitudes, and behavior. They also provided information on their attitudes toward teaching
nutrition generally and their TN curriculum experience specifically These self-administered
surveys were conducted in supervised group settings before and after the intervention For the
purposes of analysis, only the data from fourth-grade teachers are examined here. The
response rates for the four teacher surveys ranged from 79 percent to 86 percent.
2.3.2.6 Other Data Sources
To document the implementation of the TN initiative in the pilot communities, as well as to
inform the outcome evaluation, a process evaluation was also undertaken. This evaluation was
2-10
3^
conducted in an intensive fashion in the four outcome communities and in a limited fashion in
the other three districts. The process data included: TN teacher activity logs for each lesson
taught; TN core activity logs for each school-wide or district-wide activity; personal interviews
with teachers, principals, and food service managers; and observations of meal preparations.
The results of the process evaluation are reported in detail in the Case Studies Report (USDA-FNS,
1998). That report documents the extent to which TN lessons and activities were
conducted at each site, and synthesizes information from interviews with Team Nutrition
Coordinators (TNCs), principals, teachers, food service staff, and community partners. A
summary of TN implementation in each of the four outcome communities is presented in
Chapter 4 of this report as well. The findings of the process evaluation broaden the explanatory
context for observed student impacts.
2.3.3 Overview ofAnalytic Approach
The primary focus of this study is on whether students who participated in Team Nutrition
changed in a way that was different from students who were in the comparison group.
Generally, change is measured by comparing responses immediately prior to the intervention
(i.e., the pretest) with afterward (i.e., the posttest) for Team Nutrition and comparison groups.
Additionally, Phase I students were administered the student questionnaire a third time roughly
6 months later, when they were in the fifth grade. These data from multiple points in time are
analyzed using a repeated measures approach (described in Section 2.4) to compare the
direction and amount of change for both the treatment and comparison students; net differences
between the two groups are attributed to participation in Team Nutrition. The one exception to
this analytic approach occurs for one of the parent measures of student change. For this
measure, parents were asked at the posttest to make retrospective judgements about changes
in their children's nutrition awareness and eating behavior. Because these data were collected
at only one time, a repeated measure approach is not used. However, there is a second parent
measure of student change in which parents describe student behavior at multiple points in
time. For this measure also, the analysis is comparable to the student survey measures of self-reported
skills, motivation, and eating behavior.
The direction and magnitude of TN impacts as well as the associated tests of statistical
significance are estimated through regression analysis. All of the data are highly clustered, with
students grouped together in classrooms and schools. Within a cluster, students are more
similar to each other than would be true in a random sample, since they shared the same
2-11
2-V
teacher(s) and were exposed to the same school-wide programs. Given this clustering,
standard statistical procedures would underestimate the standard errors, and might lead to
some differences appearing statistically significant when tr sy were not. To correct for the
clustering effects, a mixed models approach is used.
Most of the impact analysis is cc ducted to test to the effects of Team Nutrition on students'
eating behavior, the primary interest of the TN initiative. According to Social Learning Theory,
skill-based knowledge and motivation might be considered as mediating variables through
which Team Nutrition affects behavior. However, except for a brief examination of this issue in
Appendix B, skills and motivation are considered either as dependent variables (in Chapter 5),
or are modeled using only pretest measures that do not include any TN-related impacts.
The TN impact is equivalent to the total change shown by TN participants minus the total
change shown by the comparison students (i.e. double difference). Depending on the specific
model that is used, that difference is the same as the regression coefficient representing TN
participation or the difference in the least square means when comparing the TN participants
and the comparison group. In the approach to modeling and analysis, we closely followed the
strategy recommended by Murray and Wolfinger (1994). We estimated model parameters using
the SAS procedure, Proc Mixed (SAS Institute, Inc., 1997).
Additional information about the double difference approach is provided in Section 2.4.5,
Appendix B (for the questionnaire data), and Appendix C (for the lunchroom observations). The
statistics presented in Sections 8.3.2 and 8.3.3 also are based on a double difference approach,
but require a slightly different interpretation. They represent the impact of a unit of participation
rather than the total overall impact. The interpretation of these statistics is discussed in Section
8.2.7.
2.4 OTHER KEY EVALUATION ISSUES
This section provides a summary of some of the key issues in conducting this evaluation.
Additional information about these issues and other issues is provided in the specific chapters
where the data are analyzed and in the appendices.
2.4.1 The Use of Multiple Data Sources
Information on student eating behavior was obtained from several different sources to create a
more robust and rich set of conclusions. This helps to avoid the risk that the findings might be
2-12
37
dependent on the specific measure that was used. To the extent that a finding persists across
multiple measures, the likelihood of a true change is validated.
The multiple sources of data that were used to evaluate the impact of TN include students' self-reports,
independent observations of students' food choices and consumption during school
lunches, and the opinions expressed by students' parents. Students' self-reports have the
advantage of reflecting general eating behavior, while observations during school lunches are
limited to a single meal, and parents' perspectives may ignore school lunches or other
meals/snacks eaten away from home. On the other hand, students' self-reports are subject to
error. For example, students may forget about some foods they ate, or try to give socially
desirable rather than accurate answers. Students' self-reports may reflect students' usual
patterns of behavior rather than the specific time period referred to in the questionnaire (e.g.,
foods eaten in the previous day or in the last 2 weeks). This is not a serious error because
knowledge about students' general behavior is of most interest. However, it may tend to
understate the variation in behavior from one time period to another, including the variation
between students' pretest behavior and their posttest behavior. By contrast, independent
observations of students' lunch-time behavior have the advantage of greater accuracy, but are
limited to single meals on specific days. Furthermore, because not all students participate in the
NSLP, self-selection bias may occur. Parents' opinions have the advantage of being more
comprehensive and based on a greater amount of nutrition knowledge than the fourth-grade
students would have, but it is possible that parents' tendency to give socially desirable answers
is as great or greater than that of the students. The use of all three sources of data helps to
offset the bias of any one data source.
The potential for respondents to give socially desirable answers is in part controlled by the use
of a comparison group who did not participate in Team Nutrition. However, it is possible that the
intervention would sensitize the students and parents so they are more likely to give socially
desirable answers. Even the lunch-time observations might contain some bias if students
intentionally eat a more nutritious lunch because they know their behavior is being observed.
2.4.2 Scaling Student Self-Reports
In order to estimate the impact of Team Nutrition, the study required general summary
measures of students' knowledge, motivation, and behavior relating to nutrition. Nine such
measures were developed, based largely on items that were used in previous nutrition studies
(CATCH, 1991; Borra et al., 1995; USDHHS-CDC. 1988; Domel et al., 1993; Dollahite et al..
2-13
AT
1998), but adapted to meet the particular needs of this project (i.e., in terms of topics of interest
and total length of the questionnaire). Specifically, items were chosen to reflect the three TN
messages and the general content of the classroom curriculum. These three emphases are
eating less fat; eating more fruits, vegetables and grains; and increasing the variety of foods
eaten. The wording and number of items were determined by pretesting the questionnaire with
comparable students. Factor analysis was performed to confirm that questionnaire items were
appropriately grouped together, and Cronbach's alpha was used to estimate the reliability of the
measures. A more detailed explanation of the derivation of the measures is provided in
Appendix B. The following is a brief description of each measure with sample items.
Skill-Based Knowledge
• AM/rv to identify the healthier choice [7 items; alpha«.60 to 712]
e.g., Which food is better for your health: cold or ready-to-eat cereal or eggs and
bacon? Whole milk or low-fat or skim milk?
• AbMtv to apply Food Guide Pyramid knowledge f5 items: aloha* .29 to .451
e.g., From which food group should you eat the most servings a day? If you want to
eat more fruit, which of the following would you choose?
• Ability to apply the balanced diet concept [3 items; alpha-35 to .49]
e.g., How much do you agree or disagree that it's okay to eat whatever you want,
whenever you want?
Motivation
• Generalflff/fyde[7 items; alpha=64to .70]
e.g., How much do you agree or disagree: I like to taste new foods; I like to choose
snacks lower in fat.
• Perceived consequences of more fruits, vegetables, and grains [10 items; alpha=76
to .83]
e.g., How much do you agree or disagree that if I eat fruits, vegetables, and grains
such as breads, cereals, spaghetti, and other noodles every day: my family will be
proud of me; I will become stronger.
2 The range reflects the minimum and maximum alpha values over 5 data collection periods (the pretest and posttest
for Phases I and II. and the foMowup for Phase I).
2-14
^9
• Cognitive rules for healthy choices [8 items; alpha=.66 to .76]
e.g.. If you are going to eat healthier food, then would you ever choose to eat fresh
fruit instead of a candy bar?
Self-Reported Behavior
• Usual food choices M1 items; alphas57 to .68]
e.g., Which of these two foods do you eat most often: Cookies or an apple? Buttered
popcorn or unbuttered popcorn?
• Food choices in last 2 weeks [9 items; alpha*60 to .66]
e.g., In the last 2 weeks, did you ever. Eat fruits at home for breakfast? Eat a new
grain such as brown rice, bukjur, or pita bread, that you had never tried before?
• Variety of food choices yesterday [5 items; alprNF.58 to .67]
e.g., Which of the following food groups did the foods you ate yesterday come from:
Bread, cereal, rice, and pasta group? Vegetable group? Fruit group? Milk, yogurt,
and cheese group? Meat, poultry, fish, dry beans, eggs, and nuts group?
Cronbach's alpha for each of the measures is shown in Table 2-2.
Table 2.2. Reliability of Student Survey Measures of Knowledge, Motivation, and
Behavior, Using Cronbach's Alpha
8km-b—d knowledge
Identify healthier choice
Apply FFcood Guide Pyramid knowledge
Apply beiar
Motivation
balanced diet concept
General attitude
Consequences of more fruits, vegetables.
and yams
Cognitive rule)
Usual food chi
rules for healthy choices
Choices in last 2 weeks
Variety of food choices yesterday
Phase
0.60
02T
"515"
064
"076"
0.66
0.57
0.62
061
Phase II
0.71
■042"
048
069
0.83
0.74
0.64
0.64
0.67
Poetteet
Phase I
0.60
ML
0.49
068
0.82
0.74
0.59
0.66
0.64
Phased
0.61
0.25
0.36
0.67
0.75
0.69
0.62
0.60
058
I Month
FoNowup
Phase I
0.63
0.45
0.70
0.81
0.76
0.68
0.66
0.63
Though the three scales within each category are interrelated, there are differences in
emphasis. Among the measures of nutrition knowledge, the ability to identify the healthier
choice is largely a measure of knowledge of low-fat foods. The ability to apply Food Guide
Pyramid knowledge measures students' knowledge of the 5 basic food groups, and the ability to
2-15
3*
apply the balanced diet concept focuses on the message that no food is good or bad, but that
there should be a proper balance. The three measures of motivation include a set of general
attitude items; opinions about the results of eating more fruits, vegetables, and grains; and
strategies that students would pursue to eat more healthfully. With regard to students' self-reported
behavior, the measure of usual choices largely focuses on choosing low-fat food
alternatives, while the measure of choices in the last 2 weeks captures mostly behavior
concerning fruits and vegetables. Finally, the measure of yesterday's choices focuses on
variety across food groups in the Food Guide Pyramid.
Because of these differences, one should not necessarily expect all three measures within a
category to behave in the same way. Rather, they reflect different dimensions of skill-based
knowledge, motivation, and behavior, and though they all reflect emphases of the TN initiative,
the intervention may not affect each measure equally.
A risk of using multiple measures is that the report might not be as cohesive as it would be if
fewer measures were used. However, there are important reasons to use multiple measures:
(1) Team Nutrition intends to change several different eating behaviors; (2) to the extent that the
multiple measures produce similar results, the findings are more robust and generalizable than
if a single measure were used; and (3) the extent to which the measures produce different
results, they might provide valuable clues about how the intervention affected students in some
ways but not in others. Factor analysis confirmed that the measures worked better in their
current form than when using alternative groupings of the survey items. An examination of the
correlation of each measure of skill-based knowledge and nutrition motivation with each
measure of eating behavior indicated that the measures performed sufficiently differently that
they should be treated as separate factors.
The scales also were created primarily to allow estimates of change over time (especially
change related to participation in Team Nutrition), and to facilitate comparisons among students
(e.g., some students started with more nutrition knowledge than others). Thus, they are more
useful for examining differences in scores (over time or among students) than for using actual
scores to indicate specific levels of knowledge, motivation, or behavior. For example, different
scales often have different ranges, so the same numeric score could have different
interpretations on different scales.
3i 2-16
2.4.3 Scaling Parent Perceptions of Student Change
To provide another perspective on whether students changed after participating in Team
Nutrition, two scales were created based on parents' perceptions of change. The first scale is
based on parents' observations over the last 3 months, taken at the time of the posttest, and
includes both nutrition motivation and eating behavior. The second is based on questions
asked in both the pretest and posttest questionnaires, and more specifically focuses on eating
behavior. These two scales are:
• Number of areas that parents reported seeing 'a lot more" nutritious behavior
retrospectively [5 items; alpha=71 for both phases]
e.g., Compared to 3 months ago, is your child a lot more: Aware of healthy eating?
Likely to talk about nutrition? Likely to make healthy food choices?
• Number of areas that parents reported seeing more healthy eating behavior
[6 items; alpha=63to .67*]
e.g., Does your child usually, sometimes, or almost never: Eat healthy snacks? Try
new foods? Eat too few fruits?
2.4.4 Cafeteria Observation* of Student Food Selection and Consumption
The observations of students' behavior in the cafeteria were designed to assess response to the
basic TN message. The following scales were created:
Increase the selection and consumption of grains, fruits, and vegetables.
— Number of grain servings selected and tasted.
— Amount of grains consumed.
— Number of fruit servings selected and tasted.
— Amount of fruits consumed.
— Number of vegetable servings selected and consumed.
— Amount of vegetables consumed.
Increase student preferences for lower fat milks.
— Change in distribution of types of milk (% fat) selected and consumed.
3 The range reflects the minimum and maximum alpha values over 4 data collection periods (the pretest and posttest
for Phases I and II).
2-17
3
Increase the range of foods students selected and consumed.
— Diversity of major food groups selected and consumed.
— Diversity of consumption from individual food items.
14.5 The Choice ofa Mixed Model* Repeated Measures Approach
In all of the impact analyses, we use a mixed models repeated measures approach. The one
exception is a measure of parent perceptions that is based on posttest data only, and thus does
not allow a repeated measures approach. In this case, a mixed models approach still is used.
This approach was chosen because of two key features of the study design. First, the analyses
are based on a double difference design in which pretest and posttest measures of knowledge,
motivation, and behavior are compared both for TN participants and comparison students. (For
data from the student questionnaires, data from the 6-month followup also are available.) As
noted earlier, this is an extremely powerful design because it allows one to control for the effects
of maturation, testing, instrumentation, and history from the impact of the intervention. When
data are collected from multiple points in time, however, each measurement is subject to error.
Traditional ordinary least squares regression analysis would model the error at only one point in
time. Second, as noted earlier the study design resulted in data that are highly clustered. Such
clustering results in standard errors that are too small when traditional regression analysis is
used, so that one might falsely conclude that Team Nutrition had a significant impact when the
results might have appeared by chance. A mixed models repeated measures approach
property models both types of error so one may accurately assess the statistical significance of
the findings.
Another strength of this approach is that it minimizes the chance that differences between the
implementation and comparison groups might result in mis-estimating the impact due to Team
Nutrition. Because change is measured on a student-by-student basis, even if TN participants
generally show better eating behavior because, say, they are more likely to be female, the
impact estimates will not be affected by such differences in the population. Impact estimates
would be affected only if Team Nutrition interacts with gender (or some other student
characteristic), so that females improve their scores (due to Team Nutrition) by a different
amount than do males. In acknowledgment of this possibility, the multivanate analyses in
Chapter 8 include an assessment of possible interaction effects.
Consistent with the double difference design, the estimates ^resented in Chapters 5 through 8
represent the amount of change that is specifically associated with Team Nutrition.
2-18 33
2.4.6 Aggregating Data Across Districts Vsrsus Performing Saparate Analyst for
Each District
Where possible, the data were aggregated across districts before performing the statistical
analyses. The reasons were to simplify the presentation by presenting a single summary
statistic, to increase the variation within the data so the findings would be more generalizabie,
and to allow a sufficient number of cases to perform meaningful multivariate analysis. The
survey data allowed such an approach because the same questionnaires were used for all four
districts.
In contrast, we report TN effects for the behaviors based on cafeteria observations by district for
the two phases, and aggregate across districts and phases. The analytic strategy used for
cafeteria observations was dictated by TN's design. As planned, districts largely succeeded in
harmonizing cafeteria menus during the pre- and post-implementation periods within each
phase. There was, however, no attempt to secure comparable menus across districts, or
across phases within districts.
Since we did not feel it was reasonable to assume that TN's impact is independent of food
availability, we were forced to choose between explicitly modeling the effect of menu
differences, and assessing TN's impact based on homogeneous data, that is within phase and
district. Although it was theoretically attractive, we discarded the option of modeling the effect of
menus on student selection and consumption early on, both for theoretical and for practical
reasons. The main theoretical reason was the problematic nature of using the same data twice:
to describe how menus affect food selection and consumption (which was not a focus of the this
project), and then to employ these same data, and models derived for it, to estimate TN's
impact. The main practical reason was a limitation on resources to develop and validate
models. Since menus are likely to affect food selection and food consumption in complex ways,
modeling this process was unlikely to produce any insight into the impact of Team Nutrition on
students' eating behaviors.
To compensate for the lack of homogeneity of menus across school districts and phases, we
use methods borrowed from meta-analysis to aggregate the separate effect-size estimates
across districts and phases. Meta-analysis is a two-step approach for summarizing separate
estimates. In Step 1, the Phase I and II district-level estimates for TN's effects (e.g., number
selected, number tasted, amount consumed) are divided by a corresponding Phase I and II
district-level statistical index of student variability (i.e., the residual standard deviation in the
random effect models for the effect of Team Nutrition). In Step 2, mean effect sizes are
2-19 *i
estimated as weighted averages of groups of eight Phase I and II district-level effect size
estimates. An attractive feature of this approach is that it synthesizes separate estimates
across homogeneous data sets without explicit modeling of between-district and between-phase
differences.
2.4.7 Tests of Statistical Significance
Throughout this report we have used two-sided tests of statistical significance to assess TN's
impact on outcome measures. The reason for this is that the use of a orm-sided test of
significance for an outcome measure would have required strong prior empirical evidence that,
following TN's implementation, the outcome measure in question can only change in one
direction. Since Team Nutrition was a pilot project, there was no prior evidence about its
effects, and the use of one-sided tests would not have been justified.
Statistical significance can be confusing when some results look large but are not significant.
Whether an effect estimate is or is not statistically significant depends on the size of the effect
estimate, the size of its estimated variance, and the applicable degrees of freedom. For a given
effect estimate, significance is reduced by increasing variance and increased by increasing
(numerator or denominator) degrees of freedom. In this study, all similar analyses have the
same degrees of freedom. However, estimated variances do vary among analyses, and a
larger effect with larger variance may be less significant than a smaller effect with a smaller
variance. The size of the variance depends, primarily, on the between-student variability for the
measure.
2.4.8 Exploratory Analysis of Survy Data
While the primary focus of the analysis is to assess the overall impact of participation in Team
Nutrition, the study gives some preliminary attention to identifying the factors that account for
more or less change in student eating behavior. Specifically, the analyses pursue three
approaches to providing further explanation. One of these addresses whether or not there is
any interaction between Team Nutrition and selected student/family characteristics on behavior
change. The second looks at the relationship between exposure to different dimensions of
Team Nutrition and student outcomes. The last asks whether or not the degree of improvement
in nutrition knowledge and motivation is associated with changes in eating behavior.
3^
2-20
All three of the analyses are focused on self-reported measures of eating behavior. This choice
of outcome measures reflects both the allocation of study resources and number of cases
available for examination.
2.6 SUMMARY
This chapter describes the hypotheses, design, measures and analytic approach of the TN Pilot
Project evaluation. The TN Pilot took place in four school districts among 24 schools and about
140 classrooms. The TN Pilot was conducted in widely varying settings in order to increase the
ability to generalize the results beyond the specific sites in the study-
Both the TN implementation and evaluation took place in two phases—during the Spring and
Fall semesters of 1996. The use of two phases allows a test of the possibility that impacts
might vary from one school year to another, as for example if there is a significant "learning
curve" for teachers. Students from Phase I were contacted a third time, approximately six
months after their classroom participation in Team Nutrition, in order to provide some
information on longer term effects While the process evaluation looks at the TN expenence in
different grades, the impact assessment is focused on approximately 3,300 fourth graders.
The study approach is a combinsd pre-post test comparison group design This is also known
as a double difference design which allows one to control for some of the most likely threats to
internal validity, such as maturational effects. School pairs were matched on key demographic
characteristics and randomly assigned to implementation and comparison conditions to reduce
systematic differences between the two groups "f students. This approach also helps to
maximize internal validity.
Social Learning Theory was used to guide the development of the research hypotheses.
Specific research hypotheses were developed primarily around the expected impacts of Team
Nutrition on students' food choices and eating behaviors. These include eating more fruits,
vegetables and grains; eating a greater variety of foods and making lower fat food choices.
Because the theoretical foundation of Team Nutrition suggests there will be positive changes in
skill-based nutrition knowledge and motivation, the evaluation also assessed such impacts.
Measuring impacts was a significant challenge. The general strategy was to make sure impact
scales captured dimensions that Team Nutrition was designed to affect while at the same time
using items and/or approaches that have been used successfully in the past with nine- and ten-year
olds. The limitations associated with any one data collection approach were compensated
2-21
by capturing behavioral impacts through student self-reports, observations of student food
choices and consumption in the cafeteria, and parent perceptions of changes in relevant student
behavior. Similarly, multiple scales were used with each data source (e.g., three separate
measures were developed of skill-based knowledge) to verify that the results did not depend on
the particular measure being used.
Draft surveys were informally pretested and modified to address questions about student
comprehension and time constraints. Data collected with these survey items were
systematically examined to identify the most meaningful and reliable scales. Procedures for
making cafeteria observations were similarly pretested and modified to improve efficiency and
accuracy.
Analysis involved regression techniques that control for changes over time, classroom clustering
of students, any unmeasured baseline differences between Team Nutrition and comparison
group students, and repeated measurement. Two-tailed tests were used to provide
conservative evaluations of statistical significance. The major results are presented for overall
TN Impacts where students are combined across districts-although the nature of individual data
sources requires some variation in specific technique. More complex multivariate regression
analyses are also conducted to examine whether or not particular implementation features,
number of intervention channels or any unmeasured district characteristics are related to TN
vnpects.
20
2-22
CHAMR* THE STARnMOPOWT-BASHJME SNAPSHOT
ar
This chapter provides baseline information about students, characteristics of their households
and schools before their participation in the Team Nutrition (TN) Pilot. These data establish the
context for the implementation of the TN Pilot described in Chapter 4, and for the outcomes
presented in Chapters 5 through 8. This chapter also examines how similar intervention and
comparison students were at baseline. In doing so, we also document the degree to which
there was room for improvement in students' skill-based nutrition knowledge, motivation, and
behavior.
3.1 SITE PROFILES
Each of the four districts participating in the outcome evaluation is described in detail in the
Case Studies Report. Table 3.1 provides a summary of the general elementary school make-up
in each of the four sites.
Table 3.1. Characteristics of School Districts Participating in Outcome Study
Percent of
Elsmsntary Percent of
School Elementary Required
Student School Nutrition Recent Food
Population Students Education Service Staff
Elementary Number of Composed Receiving Curricula in Training in
School Elementary of Minority Free/Reduced the District Dietary
District Population Schools Students Meals Prior to TN Guidelines
DesMoines 15,367 44 24% 46% Yes Yes
Hambien 4.205 12 8% 48% No Yes
County
Tulsa 22,047 59 43% 62% No No
VacaviKe 7,548 13 35% 31% Yes No
Des Moines, (population 195,000) n the political and economic capital of Iowa. Forty-four
elementary schools served 15,367 students in the public school system. Responsibilities for
coordinating Team Nutrition in this district were divided between three people: the Director and
Assistant Director of Food and Nutrition Management, and the Supervisor of Family and
Consumer Sciences/Health.
Situated dose to the Smoky Mountains, Hambien County (population 22,000) is approximately
40 miles east of Knoxville, Tennessee. At baseline, there were 12 elementary schools with a
total enrollment of 4,205 students, of which 8 percent were minorities. The Team Nutrition
Coordinator (TNC) was the district's Supervisor of School Nutrition Programs
3-1
31
The largest school district in the evaluation, Tulsa, Oklahoma (population 400,000), had 22,047
children enrolled in 59 elementary schools. This district also had the largest proportion of
minority students (43%), and comparatively more children who received free or reduced school
lunches (62%). The Tulsa school district appointed their Nutrition Education Project Coordinator
as the district's TNC on a full-time basis. In Tulsa, the TN Scholastic lessons were team taught
in the treatment schools; thus, some teachers taught individual lessons to multiple classes.
Therefore, each fourth-grade class did not receive all of the Scholastic lessons from the same
teacher.
VacaviHe (population 83,000) is located in California between San Francisco and the capital,
Sacramento. It is described as a "bedroom community" for the city of Sacramento. The
Vacaville Unified School District served 7,548 students in 13 elementary schools at baseline.
This was the most affluent of the districts in the study, with only 31 percent of the elementary
school students receiving free or reduced-price school lunches in 1995. The TNC in this district
was the district's Director of Child Nutrition, who worked closely with the Deputy Superintendent
of Instructional Services.
3.1.1 Characteristic* of School*
Within districts, the student population data were reviewed for each elementary school to
identify those that most closely matched the overall district with respect to racial/ethnic
distribution and percentage of children applying for free or reduced-priced school meals.
Schools were also selected to reresent different geographic areas within the district. As noted
in Chapter 1, two of the pilot communities (HamWen County and Vacaville) had two treatment
and two comparison schools, while the other two communities (Des Moines and Tulsa) had four
treatment and four comparison schools.
Table 3.2 summarizes the characteristics of the study schools. For all districts, students'
racial/ethnic distribution were comparable between implementation and comparison schools at
baseline The proportion of students eligible for subsidized lunches was also generally
comparable for three of the sites. The exception was Hamblen County, where 44 percent of
students in comparison schools vs. 31 percent in implementation schools were eligible for free
or reduced-priced school lunches.
3-2
Ut
Table 3.2. Characteristics of Outcome Study Schools by Treatment Condition at Phase I
Pretest
DesMoines
Hamblen
County Tulta Vacaville
Impl Comp Impl Comp Impl Comp Impl Comp
Total Number of
Students 1543 1523 936 642 1273 1600 1370 1652
Student Ethnicity (%)'
White/Non-Hispanic 84 85 97 100* 76 70 72 76
Black/Non-Hispanic 9 8 1 0 10 11 10 5
Hispanic 2 3 1 0 5 5 13 13
American Indian 2 2 1 0 9 9 <1 1
Asian/Pacific Islander 1 1 0 0 0 0 5 4
Percent Students with
Free/Reduced Meals
49 42 31 44 74 75 31 30
Numbers may not add up to 100 percent because of rounding.
2Schoo) records indicated no minority students in comparison schools. However, responses from parents reported in
Table 3.3 showed that a small number classified themselves as belonging to a minority group.
3.1.2 Demographic Characteristics of Students and Their Families
Tables 3.3 and 3.4 provide summary statistics on basic demographic characteristics of the
students for each phase of the pilot project. The data for these tables were provided by parent
survey respondents identified as those most knowledgeable about the students' nutrition habits
(typically the students' mothers). The data show that the intervention and comparison groups
were roughly similar in their compositions. Using a chi-square test, none of the demographic
characteristics showed a statistically significant relationship to students' participation in the TN
Pilot for the combined districts in Phase I; Phase II had differences only for the number of
children in the family under 18 years of age. Further, the intervention and comparison groups
were also generally similar within each of the four districts. In Phase I, parents' racial
background in Des Moines, and the parents' level of education in Tulsa, differed. During Phase
II, the statistically significant differences between implementation and comparison school
students i- duded student's gender and the parent's marital status in Vacaville, and total family
income in Oes Moines
3-3
H\
Table 3.3. Phase 1 Baseline: Characteristics (in percentages) of Students and Their
Families by Treatment Group and District
Characteristic Des Moines
Hamblen
County Tulsa Vacaville Totel
Impl Comp impl Comp impl Comp Impl Comp Impl Comp
Student's sex
Boy 50 45 53 48 50 46 46 50 49 47
Girl 50 55 47 52 50 54 54 50 51 53
Parent*s age
Under 30 10 11 12 8 14 9 8 12 11 10
30-40 65 71 60 71 68 70 66 71 65 71
41-50 20 14 23 19 15 18 25 16 20 16
Over SO 5 4 5 2 4 3 1 2 4 3
Parent s racerstnnicity
bochorouod
Minority Race and/or
Hispanic Origin
21* 14 2 1 30 33 28 30 22 19
While (non-Hispanic) 79 86 98 99 70 67 72 70 78 81
Parent's education
High school or less 41 49 51 39 55* 40 20 26 41 40
Some coNeoe 39 37 36 37 34 48 50 47 40 42
College graduate or more 19 14 13 24 11 12 30 27 19 19
Parent's marital status
Married 69 67 75 84 66 57 74 73 71 69
Not married, separated,
or living twill i partner
31 33 25 16 34 43 26 27 29 31
Total no. of children in
family under 18
1 12 15 23 28 16 18 14 16 15 19
2 46 38 54 46 32 40 41 37 43 40
3 30 33 16 16 33 21 26 32 28 26
4 or more 9 14 7 8 18 21 16 15 14 15
Parent's employment
status
Fulltime 58 61 61 50 47 45 45 52 53 52
Part time 15 10 19 25 21 18 26 25 20 19
Homemaker 13 16 6 11 17 13 16 12 14 13
Serf-employed 7 5 5 6 5 10 5 6 6 7
Student or unemployed 6 8 9 6 10 14 8 6 8 6
Total family income
Less than $30,000 39 35 33 34 53 56 15 16 35 36
$30,000 - $50,000 35 35 39 33 33 31 32 35 34 34
Over $50,000 21 23 24 29 7 3 45 43 24 24
Missing 6 6 5 3 6 7 6 2 7 5
*p<0.05 Measures of stati
tests of the responses for It
•>
stical ngi
iedemo(
<Vd
vficencebe
iraphicvarii
twoon ml
iWe
ervontion a
3-4
ndcompimsongroujis are besedoncrH- iquere
Table 3.4. Phase II Baseline: Characteristics (in percentages) of Students and Their
Families by Treatment Group and District
Characteristic Des Moines
Hamblen
County Tulsa Vacaville Total
Impl Comp Impl Comp Impl Comp Impl Comp Impl Comp
Students MX
Boy 50 50 59 51 54 53 45* 54 52 52
Girl 50 50 41 49 46 47 55 46 48 48
Parents ape
Under 30 18 12 14 8 13 10 13 14 14 11
30-40 61 66 64 69 67 64 63 62 64 65
41-50 18 19 20 21 17 19 23 21 19 20
Over SO 4 3 2 2 3 7 2 3 3 4
r ■! vi ii ■ rvcwmntciiy
background
Minority Race end/or
Hispanic Ongn
20 20 3 2 27 24 31 32 21 19
VvrtHe (non-Hwpenc) 80 80 97 98 73 76 69 68 79 81
Parent's education
High school or leu 44 44 37 43 46 43 27 25 38 39
Someoossgs 41 39 44 32 43 42 48 46 44 40
Cosege graduate or more 15 16 20 25 11 15 26 29 18 21
Parents marital status
Married 73 68 87 84 71 68 87* 76 79 73
NM memea separatee
or bvsiQ wan partner
27 32 13 16 29 32 13 24 21 27
Total no. of children in
family under 11
1 8 15 16 26 12 23 13 18 12* 20
2 51 44 53 45 41 37 43 43 47 42
3 30 33 20 19 29 25 27 24 27 26
4 or more 11 8 9 10 18 14 18 16 14 12
Parents employment
Ft* feme 65 62 63 54 59 56 43 52 57 56
Part ferns 12 12 14 23 18 17 25 18 17 17
Homemaker 10 14 14 11 6 10 17 15 12 12
SJM employed 7 7 6 8 8 6 8 7 7 7
Student or unemployed 6 4 3 5 9 11 7 8 6 7
Total famMy income
Less than $30,000 31' 32 28 34 48 47 12 20 29 34
$30,000 - $50,000 33 48 36 32 42 41 34 28 36 38
Over $50,000 28 19 29 29 7 8 47 46 28 24
Missing 8 2 6 6 4 5 8 6 7 4
•p<0 05 Measures of statistical significance between intervention and comparison groups are based on chi-square
tests of the responses for the demographic variable
3-5
4M
3.2 NUTRITION ENVIRONMENT
While Federal regulations stipulate the nutrient content of school lunches, nutrition education for
students and for food service staff is not required. As a result, there was considerable variability
at baseline in the exposure to nutrition information among elementary school children, their
teachers, and food service staff in the study schools. Since previous education and training can
conceivably affect the degree of impact of the TN initiative, this section briefly describes the
nutrition environment of the outcome evaluation sites at baseline. In addition, district and
school wide changes that occurred over phases are identified. More detailed information is
found in the Case Studies Report.
3.2.1 Nutrition Education Activity
Prior to the introduction of Team Nutrition, two study sites, Des Moines and Vacaville, had
district-wide nutrition education curricula in place Des Moines' curricula consisted of a 3 to 4
week unit for fourth graders as part of a Growing Healthy Program. Vacaville schools
implemented nutrition education from K to 5; however, teachers could choose from a variety of
lessons and activities that met broad criteria Before being selected as a TN Pilot
Implementation community, there were no existing district-approved nutrition education curricula
•oTutsa The district food service staff were unaware of any classroom curriculum or
instruction however thev believed mat nutrition was covered mmimaRv m some health
education classes In Hamblen County, no curriculum was required, but a number of teachers
in both implementation and comparison schools routinely incorporated nutrition lessons and
activities into the health, home living, consumer homemakmg or physical education
Teachers in this district received some training in nutribon concepts armuaty These I
ware supplemented by nutrition activities conducted by the School Nutrition Program (SNP) and
the Nutrition Advisory Council (NAC) Hamblen County was unique among the TN Psot
naoonariy sponsored by the American School Food Service Association The NAC had at least
20 members from elementary, middle, and high schools In addition to conducting nutnbon
> throughout the school year in classrooms and the community, the NAC helped in
When Phase II began, students in both implementation and comparison schools had been
exposed to community-wide TN events and the Disney PSAs instituted as part of Phase I
W
activities. Additionally, children in implementation schools who entered the fourth grade in
Phase II had been exposed to school-wide TN events during the previous semester.
3.2.2 Opinion* Toward Nutrition Education
Teacher opinions about nutrition education can potentially affect the implementation of the TN
initiative. As seen in Table 3.5, most teachers implementing Team Nutrition in all districts
indicated interest in teaching nutrition in the classroom and felt it was appropriate to do so.
More than 80 percent of teachers believed that their students would enjoy learning about
nutrition at least as much as other subjects.
Table 3.5. Teacher Opinions (in percentages) About Nutrition Education and Promotion
by District and Phase at Baseline
Motivational Kama DesMoines HaroWen Tutoa VacavHIa
Phase 1 N 1 II 1 N 1 N
N (number of teachers
completing questionnaireI)
n»28 n«22 n»17 n*20 n«30 n«23 n«21 n=28
Interest in teechmg nutrition 93% 96% 100% 100% 97% 87% 100% 91%
U1 ,...
100 100 100 100 100 91 95 95
activities into classrooms
View mat the classroom is
an appropriate place to
leach students about
nutrition
100 96 94 95 97 100 95 95
Beftef that students Mce
nutnbon subjects as well as
otter subjects
96 100 100 100 79 83 81 82
Plan to incorporats nutnbon
more often mso their
89 100 100 100 90 91 70 77
Try to influence the food
cheeses their students make
44 72 62 95 67 82 65 71
Try to nfluence the food
choices thaw students make
■ - -J - ■ afc tank OUCKW KnOOi
59 62 66 90 76 86 86 67
•Percentages based on number of T*Imptemei eringeech question
usedai
I 3-7
IX
Greater variability was seen in teachers' intent to incorporate more nutrition into the classroom,
with noticeably fewer teachers in Vacaville agreeing with this statement compared to the other
districts. Given that nutrition education was implemented in Vacaville before the TN Pilot, this
may indicate that the teachers considered their current program to be adequate. Considerably
more variability was seen when teachers were asked if they attempted to influence their
st