Skip to main content

Full text of "DTIC ADA547063: Defense Management: Actions Needed to Improve Management of Air Force's Food Transformation Initiative"

See other formats


United States Government Accountability Office 


GAO 


Report to Congressional Committees 


DEFENSE 

MANAGEMENT 

Actions Needed to 
Improve Management 
of Air Force’s Food 
Transformation 
Initiative 


i-U.S. Government Accountability Office 



YEARS 1921-2011 

ACCOUNTABILITY ★ INTEGRITY ★ RELIABILITY 


GAO-11-676 






Report Documentation Page 

Form Approved 

OMB No. 0704-0188 

Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and 
maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, 
including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington 

VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it 
does not display a currently valid OMB control number. 

1. REPORT DATE 

JUL 2011 2. REPORT TYPE 

3. DATES COVERED 

00-00-2011 to 00-00-2011 

4. TITLE AND SUBTITLE 

Defense Management: Actions Needed to Improve Management of Air 
Force’s Food Transformation Initiative 

5a. CONTRACT NUMBER 

5b. GRANT NUMBER 

5c. PROGRAM ELEMENT NUMBER 

6. AUTHOR(S) 

5d. PROJECT NUMBER 

5e. TASK NUMBER 

5f. WORK UNIT NUMBER 

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 

U.S. Government Accountability Office,441 G Street 

NW,Washington,DC,20548 

8. PERFORMING ORGANIZATION 

REPORT NUMBER 

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 

10. SPONSOR/MONITOR'S ACRONYM(S) 

11. SPONSOR/MONITOR'S REPORT 
NUMBER(S) 

12. DISTRIBUTION/AVAILABILITY STATEMENT 

Approved for public release; distribution unlimited 

13. SUPPLEMENTARY NOTES 

14. ABSTRACT 

15. SUBJECT TERMS 

16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF 

___ ABSTRACT 

18. NUMBER 19a. NAME OF 

OF PAGES RESPONSIBLE PERSON 

a. REPORT b. ABSTRACT c. THIS PAGE Same OS 

unclassified unclassified unclassified Report (SAR) 

49 


Standard Form 298 (Rev. 8-98) 

Prescribed by ANSI Std Z39-18 





July 2011 


GAO 


Accountability * Integrity * Reliability 

Highlights 


Highlights of GAO-11 -676, a report to 
congressional committees 


DEFENSE MANAGEMENT 

Actions Needed to Improve Management of Air 
Force’s Food Transformation Initiative 


Why GAO Did This Study 


What GAO Found 


To assess quality of life and identify 
areas for improvement, the Air Force 
conducts periodic surveys of its 
airmen. Recent surveys indicted that 
many of them were not satisfied with 
the quality, variety, and availability of 
food at Air Force bases. As a result, 
the Air Force implemented an initiative 
in 2010—the Food Transformation 
Initiative—to improve on-base food 
services. The Air Force piloted the 
initiative at six installations with a 
$10.3 million contract awarded in 
August 2010 and plans to eventually 
expand the initiative to other 
installations in the United States. The 
Senate and House Armed Services 
Committees directed GAO to 
undertake a comprehensive review of 
the initiative. GAO reviewed (1) the 
initiative’s objectives and performance 
measures, (2) implementation 
progress, (3) the initiative’s impact on 
food service workers at the pilot 
installations, and (4) alternative 
approaches the Air Force considered. 
GAO reviewed documentation on the 
initiative, interviewed Air Force and 
contractor officials, and visited four of 
the six pilot installations. 


What GAO Recommends 

GAO recommends that the Air Force 
take several actions to improve 
management of the Food 
Transformation Initiative, including 
developing an evaluation plan to 
assess the initiative’s results before 
moving beyond the pilot. In 
commenting on a draft of this report, 
the Department of Defense concurred 
with all the recommendations. 


The Air Force established eight objectives for its Food Transformation Initiative, 
but it has not fully developed metrics or an evaluation plan for assessing the 
initiative’s results. When GAO began its review, the Air Force had developed 
metrics for three of the eight objectives. The Air Force subsequently identified 
metrics for four additional objectives, but it did not have a robust evaluation plan, 
as called for by best practices derived from prior GAO work and various guidance 
documents, that describes how the initiative’s results will be measured. Without 
well-defined metrics, contained in an evaluation plan that is clearly linked to its 
objectives, the Air Force is not in a position to adequately assess the results of 
the pilot and make informed decisions about the initiative’s future. 

The Air Force has made some improvements to its food service operations as a 
result of the Food Transformation Initiative, but a key part of the initiative has 
been delayed, and it is too early to measure its results. The Air Force has 
obtained mostly positive feedback on the changes it has made, but the data 
collected to date are preliminary. In addition, the campus dining concept, which 
will allow airmen to use their meal cards to eat at on-base dining facilities other 
than the main dining facility, had not been implemented at the time of GAO’s 
review. Therefore, this concept, a key part of the initiative, could not be 
assessed. Furthermore, initial data show that the Air Force may have 
underestimated some of the costs of the initiative, by overestimating the number 
of hours military personnel can provide the contractor for cooking and food 
preparation. 

The Food Transformation Initiative, as implemented so far, has had varying 
effects on food service workers. All military and civilian cooks at the main dining 
facilities have maintained their jobs, but some expressed concerns about 
increases in their job responsibilities. Most contract mess attendant employees 
have retained their jobs, but the total number of these employees has been 
reduced and some contract employees lost their jobs as a result of the initiative. 
Further, most civilian employees at nonappropriated fund food and beverage 
operations, such as clubs and snack bars, accepted jobs with the new contractor, 
but their job security remains uncertain as the contractor tries to make the 
facilities profitable and the remaining parts of the initiative are implemented. 

The Air Force included three options in its analysis of alternatives in selecting its 
method for improving its food services, but did not follow its own guidance for 
identifying and assessing alternatives that would meet the program’s stated 
objectives. GAO found that two of the three alternatives were not viable options 
to be considered because they only met three of the program’s eight objectives. 

In addition, the Air Force may be able to reduce its food service costs by 
reviewing and renegotiating its existing food service contracts at bases that are 
not part of the pilot program. 


View GAO-11-676 or key components. 

For more information, contact Revae Moran at 
(202) 512-3863 or moranr@gao.gov. 


United States Government Accountability Office 









Contents 


Letter 1 

Background 3 

The Air Force Has Established Objectives for the Food 

Transformation Initiative, but Has Not Fully Developed Metrics 
or an Evaluation Plan for Assessing Results 7 

The Food Transformation Initiative Has Produced Some Food 
Service Improvements, but More Time Is Needed to Measure 
Results 11 

A Key Part of the Initiative—the Campus Dining Concept—Has Not 

Been Implemented 19 

Initial Labor Cost Information Indicates Savings May Be Less Than 

Estimated 20 

Implementation of the Food Transformation Initiative Has Had 

Varying Effects on Food Service Workers 21 

The Air Force’s Food Transformation Analysis of Alternatives Was 
Limited, and Other Options for Achieving Cost Savings May Be 
Available 27 

Conclusions 33 

Recommendations for Executive Action 34 

Agency Comments and Our Evaluation 34 


Appendix I Scope and Methodology 37 


Appendix II Comments from the Department of Defense 40 


Appendix III GAO Contact and Staff Acknowledgments 44 


Tables 

Table 1: Dining Facilities at Each Pilot Installation 5 

Table 2: Food Transformation Initiative Objectives 8 

Table 3: Increase in Hours of Operation at the Main Dining 

Facilities 13 

Table 4: Projected Cost of Dining Facility Renovations at Six Pilot 

Installations 16 

Table 5: Change in the Number and Percentage of Meals Purchased 

at the Main Dining Facilities 17 


Page i 


GAO-11-676 Defense Management 








Table t 

i: Number of Ability One Workers before and after the Food 



Transformation Initiative 

23 

Table 1 

7 : Number of Former Nonappropriated Fund Food and 
Beverage Operation Employees Who Accepted Positions 



with Aramark 

25 

Table £ 

i: Employees Eligible for and Enrolled in the 



Nonappropriated Fund Retirement System 

27 

Table i 

): Comparison of Labor Hours under Previous Contract to 
Labor Hours under the Food Transformation Initiative 



Contract 

32 


Figures 

Figure 1: Comparison of Customer Satisfaction Survey Results for 

Pilot Bases to All Air Force Bases, 2010 14 

Figure 2: Entryway at Dining Facility, MacDill Air Force Base, 

Florida, before and after Initiative 15 

Figure 3: Salad Bar at MacDill Air Force Base, Florida, before and 
after Implementation of the Food Transformation 
Initiative 16 


This is a work of the U.S. government and is not subject to copyright protection in the 
United States. The published product may be reproduced and distributed in its entirety 
without further permission from GAO. However, because this work may contain 
copyrighted images or other material, permission from the copyright holder may be 
necessary if you wish to reproduce this material separately. 


Page ii 


GAO-11-676 Defense Management 







GAO 

Accountability * Integrity* Reliability 


United States Government Accountability Office 
Washington, DC 20548 


July 26, 2011 

Congressional Committees 

Maintaining satisfaction with quality of life is a key ingredient in the Air 
Force’s ability to successfully retain needed personnel. To assess quality 
of life and identify areas where improvements are needed, the Air Force 
conducts periodic surveys of airmen. Through these surveys, the Air 
Force found many airmen were not satisfied with the quality, variety, and 
availability of food at Air Force installations. In an effort to improve 
satisfaction in this area, the Air Force developed a program—the Food 
Transformation Initiative—designed to improve the quality, variety, and 
availability of food while continuing the training of military cooks who feed 
airmen during deployments. In October 2010, the Air Force began piloting 
the initiative at six Air Force installations and plans to eventually expand it 
to other installations in the United States. 1 

On August 31,2010, the Air Force awarded the contract for the Food 
Transformation Initiative to Aramark, a large company experienced in 
food service, and on October 1, 2010, issued a delivery order for an 
estimated $10.3 million. 2 The contract covers a full range of dining 
services, including cleaning the facility and menu development, as well as 
providing input into the design of renovations for some of the existing 
dining facilities at the six pilot bases. It is anticipated that the renovations 
to the main dining facilities, 3 expected to cost an additional $9 million, will 
generally be paid for by the Air Force, subject to funding caps. While the 
initiative is focused on improving food and service at the main dining 
facilities, a second element is to have the contractor operate the 
nonappropriated fund food and beverage operations, such as clubs and 
bowling center snack bars, because many of these facilities were losing 


’The pilot installations include Elmendorf Air Force Base, Alaska; Little Rock Air Force 
Base, Arkansas; Fairchild Air Force Base, Washington; Travis Air Force Base, California; 
MacDill Air Force Base, Florida; and Patrick Air Force Base, Florida. 

2 ln a decision issued August 2, 2010, we denied a bid protest concerning the Food 
Transformation Initiative solicitation. See John P. Santry—Designated Employee Agent, 
B-402827, Aug. 2, 2010, 2010 CPDfl 177. 

3 The Air Force calls its main dining facilities “mission essential feeding facilities,” which is 
also the term used in the contract. We use the term main dining facilities to refer to these 
facilities in this report. 


Page 1 


GAO-11-676 Defense Management 




money and in danger of being closed. By including the nonappropriated 
fund food and beverage operations in the initiative, the Air Force will 
eventually be able to allow the installations to institute “campus style 
dining,” where airmen can use their meal cards to purchase food at 
different facilities at each installation. 4 

In reports accompanying proposed bills for the National Defense 
Authorization Act for fiscal year 2011, 5 the Senate and House Armed 
Services Committees directed that we undertake a comprehensive review 
of the Air Force’s Food Transformation Initiative. In response, we 
reviewed (1) the initiative’s objectives and performance measures, 

(2) progress the Air Force has made in meeting the objectives of the 
initiative, (3) the initiative’s impact on food service employees at the pilot 
bases, and (4) the Air Force’s consideration of alternative approaches for 
transforming its food service program. 

To address these objectives, we interviewed Air Force officials about the 
development of objectives and performance measures for the initiative 
and assessed the objectives and performance measures in view of 
Department of Defense (DOD) and Air Force guidance and criteria from 
our previous work. In addition, we collected and analyzed data related to 
the initiative’s objectives, such as changes in the number of meals served 
and the level of airmen’s satisfaction with food services. To assess the 
reliability of these data, we discussed the procedures for generating and 
verifying the data with knowledgeable officials and, where possible, 
examined the data for obvious anomalies. Where we found limitations 
with the data, we presented that information in the report, but otherwise 
determined that the data were sufficiently reliable for the purposes of our 
report. We also interviewed contractor officials, union representatives, 
and food service employees about their experiences under the initiative 
and reviewed information on the impact the initiative has had on civilian 
food service employees, military food service personnel, food service 
contract workers, and employees at nonappropriated fund food and 
beverage operations. Further, we assessed the process the Air Force 


4 Certain junior enlisted personnel, primarily in grades E1-E4 and who are assigned to the 
dormitory, are usually placed on essential station messing. Enlisted personnel on 
essential station messing receive meals in the dining facility by presenting proper 
identification. In this report, we use the term meal card to refer to these benefits. 

5 See S. Rep. No. 111-201, at 115 (2010) (accompanying S. 3454); H.R. Rep. No. 111-491 
at 264-265 (2010) (accompanying H.R. 5136). 


Page 2 


GAO-11-676 Defense Management 



used in selecting the Food Transformation Initiative, including alternatives 
that were considered. We compared the Air Force’s business case 
analysis, which describes the alternatives considered, to the criteria 
provided in the GAO Cost Estimating and Assessment Guide * * * * * 6 and Air 
Force guidance. We also discussed with Air Force officials options other 
than the Food Transformation Initiative they could have considered for 
improving food service at other installations. We conducted this 
performance audit from September 2010 to July 2011, in accordance with 
generally accepted government auditing standards. Those standards 
require that we plan and perform the audit to obtain sufficient, appropriate 
evidence to provide a reasonable basis for our findings and conclusions 
based on our audit objectives. We believe that the evidence obtained 
provides a reasonable basis for our findings and conclusions based on 
our audit objectives. A more detailed description of our scope and 
methodology is included in appendix I. 


Background 


In the Air Force, food service program oversight is provided by the Air 
Force Services Agency, headquartered in San Antonio, Texas. The 

agency is responsible for oversight of the Air Force’s food service 
program, including developing the menus for the main dining facilities and 
providing training for military food service workers who assist in food 
preparation and the operation of dining facilities. According to Air Force 

officials, each Air Force installation contracts for mess attendant services, 

such as cleaning and cashiering services, at their main dining facilities 
and is responsible for awarding and managing these contracts. In 
addition, although many installations use both military and civilian 
personnel as cooks, according to Air Force officials, some installations 
have expanded their mess attendant services contracts to enable them to 
use contract employees to prepare and serve food. Depending on the 
availability of and priorities given to contractors located near the 
installations, according to Air Force officials, the installations may award 

these food service contracts to small business firms, 7 contractors covered 


6 GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and 
Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: March 2009). 

According to the Small Business Administration, a food service contractor is considered a 
small business if its average annual receipts are equal to or less than $35.5 million. 
Information on calculating annual receipts is provided in 13 C.F.R. § 121.104(c). Prior to 
November 5, 2010, the threshold for food service contractors was $20.5 million. See Small 
Business Size Standards; Accommodation and Food Service Industries, 75 Fed. Reg. 
61,604 (Oct. 6, 2010) (to be codified at 13 C.F.R. pt. 121). 


Page 3 


GAO-11-676 Defense Management 







by the Randolph-Sheppard Act, 8 or contractors covered by the Ability One 
program. 9 


The Air Force Services Agency, which developed the Food 
Transformation Initiative and awarded the implementation contract for the 
six pilot installations to Aramark, is responsible for managing and 
overseeing the execution of the contract, along with a contracting officer’s 
representative from each base’s Force Support Squadron. Under the 
contract, Aramark is responsible for overseeing the day-to-day food 
service operations at the six pilot installations and planning and executing 
the menus. Aramark is also responsible for subcontracting for the 
required food service labor at the pilot installations and is required under 
its contract to subcontract with Ability One agencies for the main dining 
facilities. 10 Aramark works with the Ability One agencies to negotiate the 
labor hours needed and the cost of labor at the pilot installations. 

Air Force officials plan to implement the initiative at a second set of up to 
eight installations after assessing the results of the initiative at the six 
installations in the pilot program. If the initiative continues to be 
successful, they plan to implement it at more Air Force installations in the 
United States over the next 5 years. Potentially, the Food Transformation 
Initiative could be implemented at all of the Air Force’s 149 appropriated 
fund dining facilities nationwide. 


s The Randolph-Sheppard Act created a vending facility program in 1936 to provide blind 
individuals with more job opportunities and to encourage their self-support. See Pub. L. 
No. 74-732 (1936) (codified as amended at 20 U.S.C. §§ 107 to 107f). The program trains 
and employs blind individuals to operate vending facilities on federal property. Federal 
law gives blind vendors under the program a priority to operate cafeterias on federal 
property. See 20 U.S.C. § 107d-3(e). According to Air Force officials, the Air Force 
currently has nine Randolph-Sheppard Act contractors operating dining facilities. 

9 The Ability One program, formerly referred to as the Javits-Wagner-O’Day Act program 
and codified at 41 U.S.C. §§ 8501 to 8506, provides training and employment 
opportunities for persons who are blind or have other severe disabilities through the 
purchase of products and services from nonprofit agencies employing these individuals. If 
a product or service is on the procurement list maintained by the Committee for Purchase 
from People Who Are Blind or Severely Disabled, a government entity intending to 
procure that item or service generally must procure it from a qualified nonprofit agency. 
See § 8504. Federal Acquisition Regulation provisions implement the program. See 48 
C.F.R. subpart 8.7. 

10 The contract contains a Statement of Objectives for each pilot location, each of which 
indicates that Ability One participation is not required at nonappropriated fund food and 
beverage resale operation locations but is encouraged. 


Page 4 


GAO-11-676 Defense Management 



Initially, the Food Transformation Initiative focused on improving food 
service at the pilot installations’ main dining facilities by increasing the 
hours of service, remodeling the dining facilities (using money provided 
by the Air Force), developing new menus, and providing an executive 
chef at each location. In addition, the main dining facilities were opened to 
all installation personnel. Previously, the main dining facilities at most 
installations were not available to all installation personnel. According to 
Air Force officials, the potential increase in customers by opening the 
dining facilities to all installation personnel is a key component of the 
initiative because making greater use of a facility, which has fixed 
overhead costs, could help defray the costs of providing meals. 

According to the Air Force, the initiative also offered installation 
commanders the option of turning over their nonappropriated fund food 
and beverage operations, such as bowling center and golf course snack 
bars and officers’ and enlisted members’ clubs, to Aramark in a 
commission-based contract. 11 Table 1 lists the pilot installations and 
dining facilities included in the Food Transformation Initiative contract. 


Table 1: Dining Facilities at Each Pilot Installation 


Air Force Nonappropriated fund food and 


installation 

Main dining facilities 

beverage operations 3 

Elmendorf 

• Iditarod Dining Facility 

• Kenai Flight Kitchen 


Eagle’s Nest Cafe at the Eagle 
Glen Golf Course 

The Warehouse Grill at the 

Kashim Enlisted Club 

Paradise Cafe at the Community 
Center 

Hillberg Snack Bar b 

Fairchild 

• Warrior Inn 

• Warrior Inn Express 

• Ross Dining Facility 

• Flightline Feeding Facility 


Funspot Bowling Center Snack 

Bar 

Comingled lounge at the Bowling 
Center 0 

Final Point at the Club Annex 

Little Rock 

• Hercules Dining Facility 

• Flight Line Kitchen 


Deer Run Golf Course 

Strike Zone Cafe at the Bowling 
Center 

Hangar 1080 Club 


"The contract and orders under the contract use the terms concessionaire or commission 
rate in describing the nature of the contract. 


Page 5 


GAO-11-676 Defense Management 






Air Force 
installation 


Nonappropriated fund food and 
beverage operations 3 


MacDill 


Patrick 


T ravis 


Main dining facilities 

• Diner’s Reef 

• Flight Kitchen (colocated 
with Diners’ Reef) 


• Riverside Dining Facility 

• Riverside Flight Kitchen 


• Sierra Inn 

• Sierra Inn Grab and Go 

• Golden Bear Flight 
Kitchen 


Bay Palms Golf Course Snack 
Bar 

MacDill Lanes Snack Bar at the 
Bowling Center 

Bayshore Officers’ Club 

Boomer’s Bar and Grill - Enlisted 
Club 

Seascapes Beach Club - All 
Ranks Club 

Manatee Cove Golf Course Cafe 
and Lounge 

Rocket Lanes Ten Pin Cafe 
The Tides Club and The 
Blockhouse - Enlisted Club 

Gatsby’s Grill at the Golf Course 

Strike Zone Grill at the Bowling 
Center 

Delta Breeze Colocated Club 
Rickenbacker’s Coffee Shop 


Source: GAO analysis of Air Force information. 

a As represented by the Air Force. 

b The Statement of Objectives includes seasonal operation of the Hillberg T-Bar Grill among the 
potential future dining opportunities, available for incorporation into the delivery order at the 
installation’s discretion. A modification to the initial nonappropriated fund order for Elmendorf adds a 
line item to allow the contractor to operate the Hillberg facility snack bar from February 7 to March 31, 
2011, based on specified terms and conditions. The modification states that “[n]o commissions will be 
paid due to the partial season and timeline” and that a “reassessment of the operation will occur after 
completion of the season before additional seasons are addressed.” 

c This facility does not appear on the nonappropriated fund order with the contractor, but the 
Statement of Objectives for Fairchild Air Force Base indicates that the installation desires to develop 
a comingled bar/lounge adjacent to the snack bar. 


Traditionally, nonappropriated fund food and beverage operations have 
been part of the installations’ morale, welfare, and recreation program run 
by nonappropriated fund personnel employed by nonappropriated fund 
instrumentalities at each Air Force installation. 12 According to Air Force 
officials, these food and beverage operations have been losing money, 
and including them in the contract with Aramark was an attempt to make 


12 Nonappropriated fund instrumentalities are organizations that typically provide for the 
morale, welfare, and recreation of government officers and employees. Employees of 
armed forces nonappropriated fund instrumentalities are generally not deemed to be 
employees of the United States except as specifically provided by statute. 


Page 6 


GAO-11-676 Defense Management 





them more profitable. For example, according to Air Force data, the 
nonappropriated fund food and beverage operations at the six pilot 
installations lost almost $950,000 during the 5-year period from fiscal year 
2005 to fiscal year 2009. In fiscal year 2009, the clubs, golf course snack 
bars, and bowling center snack bars at the Travis and Elmendorf Air 
Force bases lost about $122,000 and $99,000 respectively. In the new 
arrangement, Aramark operates the facilities, including hiring personnel to 
run the operations and developing menus, and plans to invest its own 
money to remodel some of these nonappropriated fund food and 
beverage operations. In return, the nonappropriated fund instrumentality 
will receive a commission on each sale. At each of the pilot locations, the 
installation commanders decided to turn over operation of many of their 
nonappropriated fund food and beverage operations to Aramark. 


The Air Force Has 
Established 
Objectives for the 
Food Transformation 
Initiative, but Has Not 
Fully Developed 
Metrics or an 
Evaluation Plan for 
Assessing Results 


The Air Force has established eight objectives for its Food 
Transformation Initiative, but it has not fully developed the metrics or an 
evaluation plan for assessing the initiative’s results. Best practices, such 
as those prompted by principles derived from our previous work, 13 and 
from DOD and Air Force guidance, 14 suggest that programs need specific 
metrics against which their performance can be measured. These metrics 
should be well defined and include measurable targets. Our previous 
work further suggests that, ideally, these objectives and metrics should be 
described in a single document, such as an evaluation plan that defines 
how results can be measured. This is especially important for a pilot 
program so that performance information can be used to make changes 
and adjustments to make the program more effective before expanding its 
implementation to other installations. Without an evaluation plan with well- 


13 See GAO, Agency Performance Plans: Examples of Practices That Can Improve 
Usefulness to Decisionmakers, GAO/GGD/AIMD-99-69 (Washington, D.C.: Feb. 26, 
1999). Although this report focuses on agency plans under the Government Performance 
and Results Act, it presents useful principles for measuring program performance. 

14 For example, although DOD Instruction 5000.02, which includes guidance on the 
acquisition of services, may not directly apply to the Food Transformation Initiative, it 
provides useful information concerning considering performance evaluation in the context 
of acquisition planning. See DODI 5000.02, Operation of the Defense Acquisition System, 
enc. 9, § 4(f) (Dec. 8, 2008). Similarly, although officials indicated that Air Force guidance 
regarding performance-based acquisition of services would not apply in the context of the 
Food Transformation Initiative, the guidance provides principles that we believe would be 
useful in the context of the initiative’s objectives. See, for example, Air Force Instruction 
63-124, Performance-Based Services Acquisition (Aug. 1,2005). 


Page 7 


GAO-11-676 Defense Management 







defined metrics that are clearly linked to the objectives, the Air Force may 
not be in a position to adequately assess the results of the pilot program 
and make informed decisions about the initiative’s future. 


The Air Force Developed 
Eight Objectives for Its 
Food Transformation 
Initiative 


The Air Force developed eight objectives for its Food Transformation 
Initiative, as shown in table 2. 15 Some of the objectives focus on activities 
that expand food options for airmen and other personnel with access to 
the installation, while others focus on training, efficiencies, and costs. For 
example, part of one of the objectives is to open the main dining facilities 
to everyone on the installation and to allow airmen to use their meal cards 
at locations other than the main dining facility. 


Table 2: Food Transformation Initiative Objectives 


Objective 

Description 

1 . 

Enhance food quality, 
variety, and availability 

Improve the visual appearance and taste of menu items; 
offer healthy food; increase menu items; increase hours. 

2. 

Maintain home base 
and war fighting 
feeding capabilities 

Ensure that airmen receive experience and training in food 
service and are ready to deploy. 

3. 

Focus on airmen’s 
changing needs, 
lifestyles and 
preferences 

Better align food service with the needs of the customers. 

4. 

Enhance sense of 
community 

Make the main dining facility a community food 
establishment, making airmen feel more a part of their 
community by allowing them to use their meal card 
privileges at food outlets other than the main dining facility. 

5. 

Improve programs and 
facilities 

Renovate dining facilities to the atmosphere that is similar 
to what is found on college campuses; bring additional food 
service availability to the installation by enhancing remote 
feeding capabilities. 

6. 

Provide enhanced 
nutritious meals 

Continue to provide nutritious food that meets DOD 
nutritional standards. 

7. 

Improve efficiency 

Improve food service delivery approaches to improve 
efficiency. 

8. 

Decrease costs 

Maintain or decrease the cost of operating the main dining 


facilities. 


Source: GAO analysis of Air Force information. 


15 These objectives are not explicitly provided in the contract, but rather are contained in 
Air Force briefing materials and other documents. 


Page 8 


GAO-11-676 Defense Management 



The Air Force Has Not 
Fully Developed Metrics 
and an Evaluation Plan to 
Use in Assessing the 
Initiative’s Results 


Our initial review of the Air Force’s documentation on the Food 
Transformation Initiative showed it had limited information on how it 
planned to measure implementation of the objectives and, ultimately, the 
results of the pilot program. We initially found that the Air Force had 
defined metrics for three of the initiative’s eight objectives: 16 

• To measure progress on its first objective—enhancing food quality, 
variety, and availability—the Air Force plans to use its annual 
customer satisfaction survey and set a goal score of 70 out of 100 for 
the survey questions pertaining to hours of operation, food variety, 
food pricing, and the overall dining experience. 17 However, the Air 
Force’s annual customer satisfaction survey could be biased and 
therefore may not provide the data it needs to measure results for this 
objective because the response rate for recent surveys has been low. 
For example, although 58,805 active duty airmen responded to the 
survey, the response rate was about 17 percent. Further, the Air 
Force did not perform a nonresponse analysis to determine whether it 
is likely that those who did not respond to the survey would have 
provided substantively different answers from those who responded. 

• To measure progress in meeting its seventh objective—improving 
efficiency—the Air Force plans to track and monitor Aramark’s 
quarterly financial statements to identify whether the contractor was 
adhering to its original budget for labor costs. The Air Force defined 
this objective as improving food service delivery to improve efficiency 
in maintaining food, labor, and nonfood costs. However, based on the 
documentation we reviewed, it was not clear that a specific goal had 
been established for the various cost elements. 

• For its eighth objective—decreasing costs—the Air Force plans to use 
information from the same sources to measure results as the 
information for its seventh objective, improving efficiency. Specifically, 
the Air Force plans to monitor Aramark’s quarterly financial 
statements to track the operating expenses of the dining facilities. The 
Air Force currently defines the metric for this objective as maintaining 
or decreasing costs as compared to previous food service contracts. 


16 As with the Food Transformation Initiative objectives, these metrics were not clearly 
contained in the initial contract, but rather provided to us in written responses to our 
questions to Air Force officials. 

17 Survey respondents were asked to rank items on a scale of 1 to 10, with 1 considered 
poor and 10 considered excellent. These rankings were then converted to scores of 0 to 
100 . 


Page 9 


GAO-11-676 Defense Management 



However, initially the Air Force estimated that the Food 
Transformation Initiative would achieve a 30 percent cost savings 
when compared to existing food service contracts. Since that time, the 
Air Force has reduced that estimate to 27 percent. However, neither 
figure was being used as an actual metric for measuring results for 
this objective. 


During the course of our review, we brought to the Air Force’s attention 
the lack of defined metrics for all objectives and an evaluation plan for 
measuring progress in achieving the initiative’s objectives. As a result, the 
Air Force developed four additional metrics for its objectives. 

• To measure progress in meeting its second objective—maintaining 
home base and warfighting feeding capabilities—the Air Force plans 
to monitor the readiness of military food service personnel to deploy 
using DOD’s readiness system. The goal is to achieve a readiness 
score of C-1, which means that these personnel have the required 
resources and are trained to undertake the full mission for which they 
are trained without any compensation for deficiencies. 

• To measure progress in meeting its third objective—to focus on 
airmen’s changing needs, lifestyles, and preferences—the Air Force 
plans to use its annual customer satisfaction survey. Specifically, the 
Air Force has defined goal scores for two specific survey sections: 
food and satisfaction. Under the food section of the survey, the Air 
Force has defined a goal score of 76 out of 100 for the part titled “type 
of food you want.” Under the satisfaction section of the survey, the Air 
Force will use the score on how well the dining facility rates 
“compared to expectations” and has defined a goal score of 76. In 
addition, the Air Force will measure the number of available meals 
that meal card holders eat on the installation, with a goal of having 
meal card holders eat 70 percent of available meals. 

• For its fifth objective—to improve programs and facilities—the Air 
Force will again use its annual customer satisfaction survey to 
measure results and has a goal score of 85 out of 100 on the facilities 
section of the survey. 

• To measure progress in meeting its sixth objective—providing 
enhanced nutritious meals—the Air Force has defined a target score 
of 80 out of 100 on the “availability of healthy food choices” question 
in the customer satisfaction survey. 


At the time we completed our review, the Air Force had not yet defined a 
metric for its fourth objective—to enhance a sense of community. The Air 
Force stated that it will use its annual customer satisfaction survey to 


Page 10 


GAO-11-676 Defense Management 



measure the effect of increased use of the dining facility on having a 
sense of community, but it had not defined goal scores in this area. 

Most of the evidence we obtained on the Air Force’s planned 
assessments of the initiative was in the form of written responses to our 
inquiries and a modification to a delivery order under the contract that laid 
out some specific goals the Air Force had developed for the contractor. 
The modification contains an incentive fee plan with key performance 
indicators that will be used to determine the availability of an incentive 
fee. Several of the key performance indicators may link to performance 
metrics for the objectives, 18 but neither the contract itself nor the delivery 
order explicitly outlines the eight Food Transformation Initiative objectives 
or provides all performance metrics to measure those objectives. At the 
time of our review, Air Force officials stated that they did not have 
planning documents that lay out the metrics, data sources, or planned 
assessment schedules for the pilot program. They also told us that they 
recognized the need to further refine their metrics and develop an 
evaluation plan. 


The Food 
Transformation 
Initiative Has 
Produced Some Food 
Service 

Improvements, but 
More Time Is Needed 
to Measure Results 


Although the Air Force has made some improvements to its food service 
operations as a result of the Food Transformation Initiative, it is too early 
to measure the results of the initiative. The Air Force has obtained mostly 
positive feedback on the changes it has implemented, but the data 
collected to date are preliminary, and parts of the initiative have not been 
implemented. For example, implementation of a key part of the initiative, 
the campus dining concept, which will allow airmen to use their meal 
cards to eat at food service locations other than the main dining facility, 
has been delayed from January 2011 and had not been implemented at 
the time of our review. Furthermore, the data we reviewed on labor costs 
for the first 6 months of the Aramark contract indicated that the actual 
savings from the initiative may be less than the Air Force’s initial 30 
percent cost savings estimate. In its initial estimates of the costs of the 
initiative, it appears that the Air Force overestimated the number of hours 


1s For example, one of the key performance indicators would require a review of customer 
satisfaction scores in the categories of hours of operation, variety, pricing, and overall 
dining experience. The overall rating must exceed a customer satisfaction index of 75 and 
any one survey factor must not fall below a score of 70. Another key performance 
indicator would measure performance against the annual budget in reducing contractor 
nonfood costs per meal. 


Page 11 


GAO-11-676 Defense Management 



military personnel would be able to work in the dining facilities and 
underestimated the wages of contract employees. As a result, it is unclear 
whether the Air Force will successfully meet its objective of reducing 
costs as a result of the initiative. 


The Air Force Has Made 
Improvements to Its Food 
Service Operations and 
Progress in Meeting Some 
Objectives of the Food 
Transformation Initiative 


The Air Force has made improvements to its food service operations and 
cited progress in meeting five of its eight objectives for the Food 
Transformation Initiative at the six pilot installations. Specifically, officials 
stated that they have made significant improvements to the menus and 
increased the hours of operation, which has led to increased customer 
satisfaction. According to Air Force officials, these activities show 
progress in meeting three of the program’s objectives: enhancing food 
quality, variety, and availability (objective 1); providing enhanced 
nutritious meals (objective 6); and focusing on airmen’s changing needs, 
lifestyles, and preferences (objective 2). 


To achieve its goals, Aramark added a pizza station, improved the salad 
bar, and enhanced the deli menu in the main dining halls. In addition, 
Aramark revitalized the menus, adding new recipes, such as chicken 
marsala and shrimp scampi, while keeping some traditional Air Force 
recipes that were popular among the airmen. According to the Air Force 
officials, more healthy cooking options are now available because the 
menu selections include more fresh vegetables. At each of the four bases 
we visited, Air Force officials stated that there has been increased 
customer satisfaction with the quality of the food. 


Air Force officials also stated that increasing the hours of operation to 
make food service more available was an integral part of the objective to 
enhance food quality, variety, and availability. Under the Food 
Transformation Initiative, the six pilot installations have increased the 
hours of operation at their main dining facilities. The hours of operation 
increased to 112 hours per week at the main dining facilities at each base 
except Patrick Air Force Base, which does not have a requirement for a 
midnight meal period. 19 Officials at several of the installations we visited 
noted that the customers were pleased with the expanded hours. 


19 The midnight meal is designed to provide food to airmen who work overnight shifts and 
cannot eat at normal designated meal times. 


Page 12 


GAO-11-676 Defense Management 




Table 3: Increase in Hours of Operation at the Main Dining Facilities 


Air Force base 

Number of weekly hours 
before the Food 
Transformation Initiative 

Number of weekly hours 
after the Food 
Transformation Initiative 

Increase in number 
of weekly hours 

Percentage increase 
in weekly hours 

Elmendorf 

66.0 

112.0 

46.0 

69.7% 

Fairchild 

58.5 

112.0 

53.5 

91.5% 

Little Rock 

45.5 

112.0 

66.5 

146.2% 

MacDill 

56.3 

112.0 

55.8 

99.1% 

Patrick 

42.0 

98.0 

56.0 

133.3% 

T ravis 

63.0 

112.0 

49.0 

77.8% 


Source: GAO analysis of Air Force data. 


The data the Air Force plans to use to measure its progress in meeting 
these three objectives are from its annual customer satisfaction survey. 
According to the most recent customer satisfaction survey, conducted 
from December 2010 to January 2011, survey respondents at the six pilot 
installations indicated higher levels of satisfaction than respondents at Air 
Force bases that had not implemented the Food Transformation Initiative 
in the areas of having the type of food they wanted, hours of operation, 
and availability of healthy food choices (see fig. 1). The respondents 
indicated higher levels of satisfaction at the six pilot locations, but the 
survey goals of 76 out of 100 for “type of food you want” and 80 out of 
100 for “availability of healthy food choices” were not met. The goal of 70 
out of 100 for “hours of operation” was achieved. Air Force officials note 
they used this survey, as well as additional information gathered from the 
private sector, to help define goal scores and will reevaluate their goals 
each year of the program. However, as noted previously, due to the low 
response rate, these survey data may not be reliable enough for 
comparison purposes. 


Page 13 


GAO-11-676 Defense Management 





Figure 1: Comparison of Customer Satisfaction Survey Results for Pilot Bases to 
All Air Force Bases, 2010 


Score 

100 

90 

80 78 78 



Type of food Hours of Availability of 

you want operation healthy food choices 

Survey questions 


Food Transformation Initiative locations 
Non-Food Transformation Initiative locations 
Source: GAO analysis of Air Force data. 


Note: Survey respondents were asked to rank items on a scale of 1 to 10, with 1 considered poor and 
10 considered excellent. These rankings were then converted to scores of 0 to 100. Due to the low 
response rate, these survey data may not be reliable enough for comparison purposes. 


Air Force officials stated that, while there is still work to be done, they 
have started to transform the dining facilities to create an atmosphere 
similar to what is found on college and university campuses, which is an 
important part of objective 5—improve programs and facilities. Although 
major renovations of the main dining facilities have not begun, Aramark 
has made some changes, such as enhancing food displays and 
improving the appearance of dining facility entryways. The photographs in 
figures 2 and 3 illustrate improvements made to the entryway and the 
salad bar at MacDill Air Force Base. 


Page 14 


GAO-11-676 Defense Management 
























Figure 2: Entryway at Dining Facility, MacDill Air Force Base, Florida, before and after Initiative 



Source: Air Force. 


Page 15 


GAO-11-676 Defense Management 


















Figure 3: Salad Bar at MacDill Air Force Base, Florida, before and after Implementation of the Food Transformation Initiative 



Source: Air Force. 

The Air Force plans to spend almost $9 million in renovations to the main 
dining facilities at the six pilot installations. Air Force officials stated that 
they expect the renovations to begin in June or July 2011. Plans for 
renovating the dining facilities include expanding the cashier area to allow 
for more cashiers during peak periods, removing walls to open up the 
food service area and create a more efficient customer flow, adding more 
seating capacity, and making general cosmetic upgrades. Table 4 lists 
the projected costs of renovations at each of the six pilot installations. 


Table 4: Projected Cost of Dining Facility Renovations at Six Pilot Installations 

Air Force base 

Cost (dollars) 

Elmendorf 

$1,238,439 

Fairchild 

2,176,729 

Little Rock 

865,029 

MacDill 

1,610,745 

Patrick 

1,402,358 

T ravis 

1,696,545 

Total 

$8,989,845 


Source: Air Force. 


Page 16 


GAO-11-676 Defense Management 


















Another objective in which Air Force officials said they have made some 
progress is objective 4—enhancing a sense of community. They told us 
an essential part of this objective is to open the dining facilities to 
everyone on the base. Prior to the initiative, five of the six pilot 
installations allowed only active duty enlisted personnel to dine at the 
main dining facilities. Installation officials noted that opening the main 
dining facilities up to everyone who has access to the installation has had 
a very positive effect, improving the sense of community. 

Table 5 shows a comparison of the number of meals purchased at the 
pilot installations for the first 6 months after implementing the initiative 
(October 2010 to March 2011) to the same 6-month period the previous 
year. Over 250,000 more meals were purchased in the main dining 
facilities after the initiative was implemented. Over 187,000 of those 
meals reflected an increase in the number of cash paying customers. 


Table 5: Change in the Number and Percentage of Meals Purchased at the Main Dining Facilities 


Increase (decrease) in total meals purchased 


Total meals purchased, Total meals purchased, 

October 2009 through October 2010 through Meals purchased by Meals purchased by 

March 2010 March 2011 meal card customers cash customers 


Base 

Meals 
purchased by 
meal card 
customers 

Meals 
purchased 
by cash 
customers 

Meals 
purchased by 
meal card 
customers 

Meals 
purchased 
by cash 
customers 

Number 

Percentage 

Number 

Percentage 

Elmendorf 

94,979 

38,511 

98,941 

64,715 

3,962 

4 

26,204 

68 

Fairchild 

85,716 

54,998 

86,549 

76,602 

833 

1 

21,604 

39 

Little 

Rock 

68,914 

38,546 

111,991 

46,853 

43,077 

63 

8,307 

22 

MacDill 

38,073 

58,494 

42,814 

111,576 

4,741 

12 

53,082 

91 

Patrick 

23,012 

33,776 

17,614 

95,612 

(5,398) 

(23) 

61,836 

183 

T ravis 

132,659 

46,455 

150,106 

62,981 

17,447 

13 

16,526 

36 

Total 

443,353 

270,780 

508,015 

458,339 

64,662 

15 

187,559 

69 


Source: GAO analysis of Air Force data. 


Meal card holders have always been able to use the main dining facilities, 
but Air Force officials said they believe the increase in usage is a further 
indication of the effect of the increased hours of availability and improved 
food choices. The number of airmen choosing to use their meal card 
benefits at the main dining facility after implementation of the initiative 
increased at five of the installations, including Little Rock Air Force Base, 
where customers using meal cards increased by more than 60 percent. 


Page 17 


GAO-11-676 Defense Management 





Air Force officials attributed part of the increase to an increase in the 
number of airmen at the installation during this period, but also to the 
expanded menus and increased hours that resulted from the initiative. 
Only Patrick Air Force Base had a decrease in the number of meal card 
customers served. Air Force officials attributed this change to a decrease 
in the number of airmen at the installation during this period. The Air 
Force had not fully developed a metric for measuring progress in meeting 
this objective and had not implemented all changes related to this 
objective, but Air Force officials stated that, based on the increased 
usage of the dining facilities, they believe some progress has been made. 


Additional Time Is Needed 
to Measure the Results of 
the Food Transformation 
Initiative 


Although the Air Force reported improvements to its food service 
program, it is too early to measure the results of the Food Transformation 
Initiative, even at the pilot installations. According to service officials, data 
on the impact of changes made to date are preliminary and may not 
predict future results. For example, Air Force and base officials stated 
that some of the increases in the number of customers may be due to a 
“honeymoon phase” in which customers visit the main dining facility 
because it is new or different. Thus, there may be some decrease in the 
number of customers as the newness of the changes in the dining 
facilities wear off. Installation officials also told us that increased 
satisfaction with the food selection and hours of operation may also 
decrease over time. Customers have expressed some dissatisfaction 
already, primarily with the increase in prices for cash-paying customers. 
As part of the Food Transformation Initiative, the Air Force raised the 
prices paid by cash customers at the main dining facilities by increasing 
the surcharge paid on the cost of food to 90 percent. 20 Previously, cash 
customers paid the cost of food plus a 60 percent surcharge. According to 
Air Force officials, this price increase helps offset the cost of the contract. 
Officials at the four installations we visited noted that cash-paying 
customers expressed some dissatisfaction with the price increases. 
According to the Air Force’s recent customer satisfaction survey, the six 
pilot installations scored lower on satisfaction with the price of food at the 
main dining facilities compared to other Air Force installations. Air Force 
officials said that they have adjusted some prices in response to the 


20 The contract statement of work lists two categories of patrons for the main dining 
facilities: essential station messing customers, to be fully reimbursed by the government 
as part of the delivery order and all other eligible patrons, considered “cash customers.” 


Page 18 


GAO-11-676 Defense Management 




A Key Part of the 
Initiative—the 
Campus Dining 
Concept—Has Not 
Been Implemented 


negative comments, especially for the salad bar, and will continue to 
make necessary adjustments as they obtain more customer feedback. 


At the time of our review, the campus dining concept, in which airmen will 
be able to use their meal cards to eat at dining facilities on the base other 
than the main dining facility, had not been implemented at any of the six 
pilot bases. The Air Force originally expected this part of the initiative to 
be implemented in January 2011. This was later changed to April 2011 
and then to June and July 2011. Air Force officials told us they expected 
to implement campus dining at Patrick Air Force Base in June 2011 and 
at the remaining five pilot bases in July 2011. 

Implementation of the campus dining concept was delayed for several 
reasons, according to Air Force officials, including the need to develop a 
policy on how the campus dining concept will work—such as what 
purchases will be allowed for those using meal cards at facilities other 
than the main dining facilities—and delays in developing the software 
needed to track all meal card purchases. Air Force officials told us they 
only recently acquired the software needed to track meal card purchases 
and were beginning to test it. In addition, they upgraded the hardware, if 
necessary, for the main dining facilities and bought additional hardware to 
support campus dining at some of the nonappropriated fund food and 
beverage operations. When operational, this software will allow the Air 
Force to determine how often and where airmen use their meal cards. In 
addition, Air Force officials told us that they recently completed 
negotiations with Aramark on the menu options that will be available for 
airmen to purchase with their meal cards at the nonappropriated fund 
food and beverage operations. 

Officials at Elmendorf Air Force Base raised additional concerns about 
the campus dining concept because the installation is a joint base with 
the Army’s Fort Richardson. At the time of our review, soldiers stationed 
at Fort Richardson were allowed to use their meal card privileges at the 
main dining facility at Elmendorf Air Force Base. Installation officials said 
they would like that privilege to be extended to the nonappropriated fund 
food and beverage operations under the campus dining concept and did 
not want to implement campus dining until this issue had been resolved. 
Air Force officials told us that, initially, campus dining was going to be 
restricted to Air Force personnel stationed at Elmendorf Air Force Base. 

The campus dining concept is an important part of the Food 
Transformation Initiative and of meeting the Air Force’s fourth objective— 
to enhance the sense of community—by allowing airmen to use more of 


Page 19 


GAO-11-676 Defense Management 



the base community when eating their daily meals. The Air Force recently 
addressed the issues that led to the delay of the campus dining concept, 
but it had not fully implemented this part of the initiative at the time of our 
review. Therefore, the Air Force cannot assess the concept or make 
necessary adjustments to this key part of the initiative. 


Initial Labor Cost 
Information Indicates 
Savings May Be Less 
Than Estimated 


It is too early to determine cost savings resulting from the Food 
Transformation Initiative, but initial information on labor costs indicates 
that actual savings may be less than the 30 percent cost savings 
originally estimated by the Air Force. The Air Force underestimated 
contract labor costs because it overestimated the number of military labor 
hours it could provide to the contractor for cooks and food preparers. 
Specifically, the Air Force estimated that military personnel would be able 
to provide approximately 32,000 hours per month of labor in the dining 
facilities at the six pilot installations. However, the military personnel for 
whom these estimates were provided have not been able to provide the 
projected number of hours to the contractor because of other duties. As a 
result, Aramark has hired additional contract staff to make up some of the 
difference, which could increase contract labor costs. 


We found that, for the first 6 months of the contract, the number of military 
labor hours was about 14,000 hours less than originally estimated for the 
six pilot installations. At Elmendorf Air Force Base, labor costs were 
projected to increase by $105,000 for the remainder of the fiscal year as a 
result of fewer military personnel being available. Air Force officials told 
us they plan to hold Aramark to its original budget, but they indicated that, 
in certain situations, after Aramark provides adequate documentation, the 
Air Force will pay the higher labor costs. 

The Air Force also underestimated the subcontractor labor costs to be 
incurred by Aramark. For example, we found that, for the first quarter of 
fiscal year 2011, Aramark submitted requests for reimbursement for 
subcontractor labor costs that were about $400,000 more than those 
originally estimated by the Air Force. For the second quarter of fiscal year 
2011, subcontractor labor costs submitted for reimbursement were about 
$600,000 more than originally estimated. According to the Air Force, 
these costs may be the result of a pay raise resulting from an annual 
wage determination that was unaccounted for in the original budget and 
not under the control of Aramark. Air Force officials told us that they have 
requested supporting documentation for the higher labor costs but have 
not paid this amount to Aramark. However, the officials indicated that, 


Page 20 


GAO-11-676 Defense Management 



with adequate support from the subcontractor, the Air Force would pay 
the higher labor costs. 


Implementation of the 
Food Transformation 
Initiative Has Had 
Varying Effects on 
Food Service Workers 


Preliminary results indicate that the Food Transformation Initiative has 
had varying effects on the three categories of food service workers at the 
six pilot bases: (1) appropriated fund civilian and military cooks in the 
main dining halls, (2) contract mess attendant workers in the main dining 
halls, and (3) nonappropriated fund civilian employees who work in the 
nonappropriated fund food and beverage operations. All civilian 
employees and military cooks who worked in the main dining halls 
retained their jobs, 21 but some employees expressed concern to us and 
their union representatives that their job responsibilities under the 
initiative have increased. Although most contract mess attendants 
retained their jobs, the total number of contract mess attendant 
employees was reduced and some employees lost their jobs as a result 
of the Food Transformation Initiative. Finally, most civilian employees at 
the nonappropriated fund food and beverage operations accepted 
employment with Aramark, but their job security remains uncertain 
because if these operations do not become more profitable, they may 
close. 


Civilian and Military Cooks 
Retained Their Jobs, but 
Their Responsibilities 
Increased 


All appropriated fund civilian employees and military food service workers 
at the main dining halls retained their jobs, but some employees 
expressed concern that they had taken on increased responsibilities as a 
result of the initiative. At the six pilot bases, 202 military and 20 civilian 
employees worked in the main dining facilities prior to the implementation 
of the Food Transformation Initiative. These workers were not affected by 
the Food Transformation Initiative contract; they continue to be employed 
directly by the Air Force. According to Air Force officials, the workers’ 
primary responsibilities are cooking and serving food. 


During our visits to four of the pilot installations, we spoke with small 
groups of civilian and military food service workers and individual workers 


21 During the course of the bid protest on the Food Transformation Initiative, the Air Force 
amended the solicitation to provide that appropriated fund wage grade employees would 
not be “displaced, reassigned, subjected to reduction in force, or otherwise adversely 
affected by the implementation of FTI phase I.” See John P. Santry—Designated 
Employee Agent, B-402827, Aug. 2, 2010, 2010 CPD IT 177 at 5 (citing Letter to Offerors, 
June 30, 2010, HI.a). 


Page 21 


GAO-11-676 Defense Management 




who were available at the time of our visit and agreed to speak with us. In 
addition, we interviewed civilian union representatives about 
implementation of the initiative at the four installations we visited. They 
raised some concerns about the increased workloads of these workers 
and noted that, prior to the Food Transformation Initiative, contract 
workers performed some of the food preparation work, such as chopping 
vegetables and replenishing the salad bar. Since implementation of the 
initiative, this work has now become part of the civilian and military cooks’ 
job responsibilities. In addition, because the new menus rely on an 
increased amount of fresh ingredients, there has been an increase in the 
amount of food preparation they must perform. While these tasks were 
new to the civilian and military cooks, according to Air Force officials, they 
should have been performing these tasks prior to implementation of the 
initiative. Contract workers were less affected by these changes because, 
according to Air Force officials, over time, they had taken on additional 
duties that might not have been in their original job descriptions. Air Force 
officials also told us the initiative helped clarify the required 
responsibilities of the civilian and military cooks and those of the contract 
employees. 

In addition, some military food service workers at the installations we 
visited told us that, because of the extended hours and increase in 
number of meals served, there is no down time and they are no longer 
getting to take their lunch time or other breaks. They also said they are 
now expected to do physical training and other required training on their 
own time rather than during work hours as was the case prior to the 
initiative. Some of the military cooks to whom we spoke said they are 
putting in 12- or 13-hour days to keep up with the increased workload. 
They also told us they are feeding many more people than prior to 
implementation of the initiative, but they have the same number of people 
cooking. Aramark officials stated that they are continuing to determine the 
appropriate size of dining facility operations, including the number of 
required labor hours, and will make adjustments as more information is 
gathered. 


Page 22 


GAO-11-676 Defense Management 



Most Contract Mess 
Attendant Workers Were 
Retained under the 
Initiative, but Some 
Workers Lost Their Jobs 


Most contract mess attendant workers in the main dining facilities at the 
six pilot installations retained their jobs after implementation of the Food 
Transformation Initiative. However, some contract mess attendant 
workers—Ability One workers 22 and small business employees—lost their 
jobs as a result of the initiative, and others’ work hours were reduced. 
Prior to the initiative, five of the six pilot installations used Ability One 
providers for their mess attendant contracts and the other installation 
contracted with a small business to provide mess attendants for its dining 
halls. According to Air Force officials, after implementation of the initiative, 
Aramark hired some Ability One workers to provide mess attendant 
services at the sixth installation in place of the small business employees. 


Following implementation of the initiative, the five installations continued 
to use Ability One workers as mess attendants through subcontracts 
between Aramark and Ability One providers. However, although many of 
the Ability One workers retained their jobs, the total number of workers 
was reduced at four of the five installations. According to Air Force 
officials, the five pilot installations that initially employed Ability One 
workers had 273 of these workers prior to the initiative. As of March 2011, 
the number of Ability One workers at these five installations decreased to 
233 (see table 6). According to Air Force officials, 72 Ability One workers 
initially lost their jobs as a result of the Food Transformation Initiative. 
However, they noted that, after the initial layoffs, 32 new Ability One 
workers were hired at the five bases as Aramark adjusted staffing levels 
at the main dining facilities. The officials could not tell us, however, 
whether any of the 32 new Ability One workers hired were among the 72 
workers who initially lost their positions after implementation of the 
initiative. 


Table 6: Number of Ability One Workers before and after the Food Transformation Initiative 

Air Force base 

Number of Ability 
One workers as of 
October 1,2010 

Number of Ability One workers 
terminated as a result of the 
Food Transformation Initiative 

Number of new Ability One 
workers hired from October 
2010 to March 31, 2011 

Number of Ability 
One workers as of 
March 31, 2011 

Elmendorf 

47 

21 

0 

26 

Little Rock 

63 

26 

1 

38 


22 We use the term Ability One workers to refer to employees participating in the Ability 
One program through employment by a nonprofit agency for the blind or a nonprofit 
agency for employing persons with severe disabilities. 


Page 23 


GAO-11-676 Defense Management 




Air Force base 

Number of Ability 
One workers as of 
October 1,2010 

Number of Ability One workers 
terminated as a result of the 
Food Transformation Initiative 

Number of new Ability One 
workers hired from October 
2010 to March 31,2011 

Number of Ability 
One workers as of 
March 31, 2011 

MacDill 

49 

2 

10 

57 

Patrick 

47 

6 

16 

57 

T ravis 

67 

17 

5 

55 

Total 

273 

72 

32 

233 


Source: GAO analysis of Air Force data. 


In addition, a representative from NISH, one of the central nonprofit 
agencies that facilitates the Ability One program, reported that the work 
hours of many of those who retained their jobs were reduced. Aramark 
officials noted that the hours of many contract employees were reduced 
as the company worked to achieve the staffing levels it deemed 
appropriate as it gained experience under the new contract. 

Prior to the initiative, one of the pilot installations, Fairchild Air Force 
Base, contracted with a small business to provide services at its main 
dining facility. According to officials, because the Air Force requires that 
installations under the Food Transformation Initiative obtain mess 
attendant services for their main dining facilities through contracts with 
Ability One, as of March 1, 2010, Ability One took over the mess 
attendant services contract at Fairchild Air Force Base. According to Air 
Force officials and Aramark officials, prior to implementation of the Food 
Transformation Initiative, 41 workers employed by the small business 
contractor worked in the installation’s main dining facility. As a result of 
the initiative, 17 of these workers lost their jobs. According to Aramark 
officials, the remaining 24 workers will be retained until September 30, 

2011, to provide services such as cashiering and cooking and they will 
maintain the contract with the small business contractor until that time. 
Subsequently, Aramark may attempt to directly hire some of these 
workers as cashiers since, according to Air Force officials, the Air Force 
does not require Aramark to contract with Ability One for cashier services. 


Page 24 


GAO-11-676 Defense Management 




Most Former 
Nonappropriated Fund 
Food and Beverage 
Operation Workers 
Accepted Jobs with 
Aramark 


Most of the civilian employees at the former nonappropriated fund food 
and beverage operations at the six pilot installations accepted jobs with 
Aramark. However, the future of these facilities is uncertain, and therefore 
the job security of these workers is uncertain. Prior to implementation of 
the initiative, a combination of regular and flexible civilian food operation 
workers were employed at the nonappropriated fund food and beverage 
operations at each of the six pilot installations: 117 regular and 183 
flexible workers. 23 As shown in table 7, most of these workers accepted 
jobs with Aramark. According to Air Force officials, most of these workers 
now receive higher pay and are eligible for increased benefits through 
their employment with Aramark. For example, flexible workers who did 
not receive any benefits while employed by the nonappropriated fund 
instrumentality are now eligible for benefits. However, according to Air 
Force officials, both regular and flexible workers employed by Aramark 
may opt to receive an additional $3.50 more per hour in lieu of benefits, 
and 187 workers accepted the higher pay rather than receiving benefits. 


Table 7: Number of Former Nonappropriated Fund Food and Beverage Operation 
Employees Who Accepted Positions with Aramark 


Air Force base 

Type of employee 

Number of former 
employees 

Number who 
accepted position 
with Aramark 

Elmendorf 

Regular 

13 

9 


Flexible 

14 

6 

Fairchild 

Regular 

2 

1 


Flexible 

17 

11 

Little Rock 

Regular 

11 

8 


Flexible 

27 

18 

MacDill 

Regular 

26 

23 


Flexible 

47 

31 


23 According to Air Force Manual 34-310, Nonappropriated Fund Personnel Management 
and Administration Procedures (Jan. 19, 2011), regular employees are guaranteed a 
minimum of 20 hours to a maximum of 40 hours of work per week, and they receive 
benefits. Flexible employees have work schedules that depend on the needs of the 
activity. These employees may work a minimum of zero hours to a maximum of 40 hours 
per week, but do not receive benefits. The version of the Manual in existence prior to 
January 2011 contained the same information regarding the two employment categories. 
See Air Force Manual 34-310, Nonappropriated Fund Personnel Program Management 
and Administration Procedures § 1.7 (Dec. 1, 1995) (obsolete). 


Page 25 


GAO-11-676 Defense Management 


Air Force base 

Type of employee 

Number of former 
employees 

Number who 
accepted position 
with Aramark 

Patrick 

Regular 

34 

32 


Flexible 

42 

31 

T ravis 

Regular 

31 

27 


Flexible 

36 

27 

Total 

Regular 

117 

100 


Flexible 

183 

124 


Source: Air Force. 


Regular workers who had worked at the nonappropriated fund food and 
beverage operations for at least 1 year of continuous service were 
generally eligible for severance pay; flexible workers were not. According 
to Air Force officials, a regular worker could accept employment with 
Aramark and also receive severance pay. According to documents 
provided by the Air Force, of the 300 former nonappropriated fund food 
operation employees, severance payments were authorized and paid to 
102 workers at the six pilot installations for a total of over $750,000. 

In some cases, regular workers with the nonappropriated fund food and 
beverage operations were earning credit towards federal nonappropriated 
fund retirement benefits. Under employment with Aramark, these workers 
will not have access to the same retirement system. However, not all 
former nonappropriated fund food and beverage operation workers were 
enrolled in the nonappropriated fund retirement system. According to Air 
Force officials, enrollment in the retirement system is not automatic and 
many workers opt not to enroll because of the additional deduction from 
their paychecks. However, officials further stated that workers who had 
enrolled in the system and then took jobs with Aramark could either defer 
their retirement benefits or have their contributions returned to them, 
depending on their age and length of service. Table 8 shows the number 
of workers at the six pilot installations’ nonappropriated fund food and 
beverage operations who were identified as eligible for and were enrolled 
in the nonappropriated fund retirement system prior to implementation of 
the initiative. 


Page 26 


GAO-11-676 Defense Management 





Table 8: Employees Eligible for and Enrolled in the Nonappropriated Fund 

Retirement System 

Air Force base 

Number of eligible 
employees 

Number enrolled in 
retirement system 

Elmendorf 

13 

5 

Fairchild 

2 

0 

Little Rock 

11 

1 

MacDill 

26 

23 

Patrick 

35 

24 

T ravis 

31 

25 

Total 

118 

78 


Source: GAO analysis of Air Force data. 


Note: Eligibility was identified by the Air Force. 


Air Force officials also stated that, because many of the nonappropriated 
fund food and beverage operations were losing money, the Air Force may 
not have been able to keep some of them open and running. For 
example, in 2007, 14 employees from nonappropriated fund food and 
beverage operations at Elmendorf Air Force Base were laid off because 
the operations were losing money. According to Air Force officials, if the 
Food Transformation Initiative had not been implemented at the six pilot 
installations, more workers might have lost their jobs. Aramark officials 
stated that they were making changes to these operations and hope to 
increase efficiencies; however, they expressed concern about keeping all 
of the facilities open in the future. Employees with whom we spoke at the 
installations also expressed concern about the possibility of losing their 
jobs in the future. 


The Air Force’s Food 
Transformation 
Analysis of 
Alternatives Was 
Limited, and Other 
Options for Achieving 
Cost Savings May Be 
Available 


The analysis the Air Force conducted for selecting its Food 
Transformation Initiative model was very limited in the options it 
considered. The Air Force included three options in its business case 
analysis, but did not follow its own guidance for identifying and assessing 
alternatives that would meet the program’s stated objectives. In preparing 
its business case analysis, the Air Force considered alternatives that 
were not feasible ways of satisfying its objectives and did not consider 
other alternatives that may have been feasible options. The Air Force 
officials who prepared the analysis stated that they were unaware of the 
guidance and that, due to the limited time they had to conduct the 
analysis, they did not explore additional alternatives when it became 
apparent that two of the options under consideration were not feasible 


Page 27 


GAO-11-676 Defense Management 





ways of satisfying the objectives. Moreover, the Air Force may have 
opportunities for reducing costs at all of its installations using methods it 
did not consider as part of its business case analysis for the pilot 
program. 


Some Alternatives the Air 
Force Considered Were 
Not Viable, and It Did Not 
Consider All Alternatives 


Air Force guidance on business case analysis procedures calls for the 
inclusion of feasible alternatives to satisfy the objectives and suggests 
that all feasible alternatives should be considered. Where an alternative 
was considered but dismissed as infeasible, the guidance directs a 
discussion of the reasons for considering it infeasible. 24 Further, best 
practices for preparing business case analyses, such as those described 
in the GAO Cost Estimating and Assessment Guide, 25 state that all 
meaningful alternatives should be presented and analyzed in a consistent 
manner. However, the Air Force considered, and included in its business 
case analysis, alternatives that were not feasible methods of satisfying 
the objectives and did not consider other alternatives that were feasible. 


The Air Force Services Agency prepared and presented a business case 
analysis in October 2008 with three alternatives to the Air Force Smart 
Operations for the 21st Century 26 office, which provided start-up funding 
for the Food Transformation Initiative. However, two of the alternatives 
considered were not viable options because they did not satisfy all eight 
objectives of the program. 


The first alternative presented was to close four dining facilities at 
selected Air Force installations and put all eligible airmen stationed at 
these installations on basic allowance for subsistence. 27 According to the 
business case analysis, this option would potentially save money by 
reducing facility and contract costs, even with the additional expense of 
providing basic allowance for subsistence funds to airmen. Our review of 
this option found that it was not viable because it would meet only three of 


24 Air Force Manual 65-510, Business Case Analysis Procedures (Sept. 22, 2008). 
25 GAO-09-3SP. 

26 The Air Force Smart Operations for the 21 st Century program began in 2006. Its goal 
was to integrate improvement and efficiencies into Air Force programs. 

27 Basic allowance for subsistence is a payment designed to defray the costs of an eligible 
servicemember’s meals. 


Page 28 


GAO-11-676 Defense Management 






the Air Force’s eight objectives for improving food service: maintaining 
home station and warfighting feeding capability, improving program 
efficiency, and reducing costs. This option would not have satisfied the 
remaining five objectives: enhancing food quality, focusing on airmen’s 
changing needs, enhancing sense of community, improving programs 
and facilities, and providing enhanced nutritious meals. 

The second alternative presented in the business case analysis was to 
close the four dining facilities presented in the first option, centralize the 
dining facility contracts at the other installations, and replace the food 
service contract workers with nonappropriated fund employees. Also, the 
Air Force would develop “hybrid” food venues where airmen could use 
their meal cards to eat at locations other than the main dining facilities. 
This option would save the same facility and contract costs as the first 
option and further reduce contract costs by eliminating the food service 
contracts at the bases and having the Air Force operate the dining 
facilities. The Air Force estimated that this option would have a 43 
percent return on investment. The Air Force has operated some of its 
main dining facilities using nonappropriated fund employees at other 
bases, including Seymour Johnson Air Force Base in North Carolina. 
However, similar to the first alternative, we found that this option also was 
not viable in that it would meet only three of the eight objectives: 
maintaining home station and warfighting feeding capability, improving 
program efficiency, and reducing costs. Further, this option could be 
inconsistent with a DOD policy providing that nonappropriated fund 
instrumentalities shall not enter into contracts or agreements with DOD 
elements for the provision of goods and services that will result in the loss 
of jobs created under Ability One, Randolph-Sheppard, or small business 


Page 29 


GAO-11-676 Defense Management 



programs. 28 However, Air Force officials stated that they may have been 
able to obtain a waiver of this policy if this option had been selected. 29 

The third alternative considered was to close the four dining facilities and 
develop hybrid dining facilities to be managed by a third-party food 
service contractor. This option was basically the same as option two, but 
with a food service contractor taking over the facilities rather than having 
them being run directly by personnel from the nonappropriated fund 
instrumentality. The Air Force found that this option would save the most 
money with a 108 percent return on investment and would also meet all of 
its eight objectives. Therefore, this is the option that was chosen. 

In developing the business case analysis, Air Force officials told us they 
had discussed at least one other potential alternative, but they dismissed 
the idea as too difficult and did not include it in the business case 
analysis. This option was to form a large nonappropriated fund 
instrumentality, similar to the Army Air Force Exchange Service, to run 
food service facilities at Air Force bases. 

Air Force officials also told us that they would have included more 
alternatives in the business case analysis, but they did not have enough 
time. Due to the time constraints imposed by the Air Force Smart 
Operations for the 21st Century office on the Air Force Services agency, 
they had a very limited time—approximately 2 to 3 weeks—to complete 
the analysis. Because one of the purposes of a business case analysis is 
to provide financial justification for an investment decision, it is important 
to follow guidance and best practices in its preparation. However, even 


28 See Memorandum from the Under Secretary of Defense for Personnel and Readiness, 
Limitations on the Use of Contracts and Other Agreements with DOD Nonappropriated 
Fund Instrumentalities (NAFIs) Pursuant to 10 U.S.C. 2492 (Dec. 29, 2004). Section 2492 
of Title 10 of the United States Code provides authority for DOD nonappropriated fund 
instrumentalities to enter into contracts or other agreements with other DOD elements in 
certain circumstances. 

29 The Air Force ultimately requested and received approval for a waiver to this policy to 
implement the Food Transformation Initiative by using a nonappropriated fund 
instrumentality memorandum of agreement to provide the installation appropriated fund 
food service mission. See Memorandum from the Under Secretary of Defense for 
Personnel and Readiness, Request for Waiver to OSD Policy (Aug. 4, 2010). The waiver 
approval recognized efforts of the Air Force to partner with Ability One, use an acquisition 
strategy requiring right of first refusal for qualified small business and nonappropriated 
fund employees as applicable, and omit from the initiative installations with existing 
Randolph-Sheppard Act dining facility contracts. See id. 


Page 30 


GAO-11-676 Defense Management 



though the Air Force has specific guidance on how to prepare a business 
case analysis, Air Force officials told us that they were unaware of this 
guidance as it was not included in the specific instructions from the Air 
Force Smart Operations for the 21st Century program. 


The Air Force May Have 
Other Options for Meeting 
Its Objective to Reduce 
Costs throughout Its Food 
Service System 


Using lessons learned from the implementation of the Food 
Transformation Initiative, the Air Force may have additional opportunities 
to meet its objective to reduce costs throughout its food service system. 

To determine the impact of the initiative on labor requirements and costs, 
we compared the estimated amount of food service labor for which the Air 
Force contracted at the six pilot installations prior to the initiative to the 
amount for which Aramark planned to contract under the initiative. We 
found that, even with expanded hours of operation and anticipated 
increases in the number of meals served, Aramark significantly reduced 
the number of hours for contract mess attendant labor it needed at the 
pilot installations. 


According to Air Force and Aramark officials, most of the cost savings 
under the Food Transformation Initiative resulted from reducing labor 
hours at the main dining facilities. 30 Prior to the initiative, the food service 
contracts at all six pilot installations were fixed-price contracts, according 
to Air Force officials, including for labor costs. 31 Officials added that, prior 
to implementation of the initiative, the Air Force did not actually know or 
closely monitor the number of labor hours required to provide food 
services. However, we obtained documentation from the installations, 
including memoranda showing how the prices were negotiated and 
contractor price proposals that estimated the number of labor hours for 
these contracts. Although these documents do not contain the precise 
number of labor hours for the main dining facilities, they provided the best 
estimates of labor costs available. We compared the estimated labor 
hours for these prior food service contracts to those presented in the 
contractor’s projected work schedules for the Food Transformation 
Initiative contract and found that the total number of labor hours at five of 


30 Air Force officials told us that additional savings under the Food Transformation Initiative 
come from optimizing the use of military personnel in the main dining facilities. 

31 Generally, under a firm fixed-price contract, the installation pays the fixed price even if 
the actual total cost of the product or service—that is, the contractor’s cost experience in 
performing the contract—falls short of or exceeds the contract price. See 48 C.F.R. § 
16.202-1. 


Page 31 


GAO-11-676 Defense Management 




the six pilot installations had been reduced by 53 percent. For example, at 
Travis Air Force Base, the number of labor hours for the mess attendant 
contract had been reduced by more than half—from approximately 2,042 
hours per week to 920 hours per week. At Elmendorf Air Force Base, 
labor hours had been reduced from approximately 1,350 hours per week 
to 588 hours per week. 32 Table 9 shows the change in the number of 
labor hours at all six pilot installations. 


Table 9: Comparison of Labor Hours under Previous Contract to Labor Hours under 
the Food Transformation Initiative Contract 

Air Force 
base 

Estimated weekly labor hours 
under the previous contract 

Estimated weekly labor hours 
under the new contract 

Elmendorf 

1,350 

588 

Fairchild 

979 

476 

Little Rock 

1,548 

303 

MacDill 

1,201 

1,063 

Patrick 

1,218 

1,349 

T ravis 

2,042 

920 

Total 

8,338 

4,699 


Source: GAO analysis of Air Force data. 


Patrick Air Force Base was the only pilot base where the labor hours 
were not reduced and the only one of the pilot installations where the 
previous food service contract had recently been audited. The results of 
that audit, conducted by the Air Force Audit Agency in 2009, showed that 
the food service personnel did not align the contract workload estimates 
with actual meals served. Specifically, meal counts were overstated, 
resulting in the installation paying more for contracted food services than 
necessary. As a result of this audit, in October 2009, Patrick Air Force 
Base renegotiated its workload estimates and pay rates, resulting in 
savings of approximately $77,000 annually. 

The Air Force may have opportunities to reduce its food service costs at 
Air Force installations that are not part of the Food Transformation 
Initiative pilot by reviewing and, where and when feasible, renegotiating 
its current food service contracts to adjust the labor costs. Air Force 


32 The labor hours presented here do not include any additional labor hours that might be 
needed because of a lack of available military personnel. 


Page 32 


GAO-11-676 Defense Management 





officials told us that they did not previously consider this option because 
they did not realize how poorly their food service contracts were 
structured. Although renegotiating the existing food service contracts 
would not meet all of the objectives of the Food Transformation Initiative, 
given the large number of food service contracts within the Air Force, and 
the length of time it may take to implement the Food Transformation 
Initiative at all remaining Air Force installations in the United States, the 
Air Force may be able to achieve significant savings prior to expanding 
the initiative. Throughout our site visits, at which Air Force officials 
accompanied us, we discussed this potential opportunity for savings. As a 
result, the Air Force recently issued a memorandum to the Major 
Commands directing a review of existing food service contracts to 
determine if the contracts meet current mission needs. For example, the 
memorandum indicates that special attention must be given to whether 
the food service contract workload estimates were properly aligned with 
the actual number of meals served. We believe that this is a good first 
step towards addressing this issue, but it is important for the Air Force to 
continue to monitor this process and continue to take actions in this area, 
where appropriate. 


Conclusions 


The Air Force is in the process of developing an approach for assessing 
the results of the Food Transformation Initiative, but has not yet 
developed well-defined metrics that are clearly linked to the objectives of 
the initiative and adequately described in an evaluation plan to assess the 
results of the initiative at the pilot installations and make informed 
decisions about the initiative’s future. Because the Air Force had not yet 
implemented the campus dining concept at the six installations in the pilot 
at the time of our review, it did not have the information available to 
assess the results of this key part of the initiative to make any 
adjustments that may be needed before expanding the initiative to other 
Air Force installations. In addition, until the Air Force reevaluates the 
actual number of hours its military personnel can provide the contractor, 
the cost savings generated by the initiative will not be clear. Finally, by 
reviewing the actual labor costs of its current contracts at Air Force 
installations that are not part of the pilot, the Air Force may have 
opportunities to reduce its food service costs prior to expanding the Food 
Transformation Initiative. 


Page 33 


GAO-11-676 Defense Management 



Recommendations for before implementing the Food Transformation Initiative at other Air Force 

installations, we recommend that the Secretary of the Air Force direct the 
Executive Action Air Force Services Agency to take the following three actions: 

• Develop an evaluation plan to accurately and reliably assess the 
results of the pilot phase of the Food Transformation Initiative. This 
evaluation plan should include, at a minimum, well-defined, clear, and 
measurable metrics to use in determining the results of the initiative 
and time frames for collecting data and reporting results. 

• Evaluate the results of the implementation of the campus dining 
concept at the six pilot locations to determine if adjustments are 
needed before expanding the initiative to additional installations. 

• Reevaluate the number of military personnel hours the Air Force can 
provide the contractor for food service, make adjustments where 
possible in projected labor hours, and take into account the increased 
labor costs in revising the cost savings estimates for the future. 


We further recommend that the Secretary of the Air Force monitor the 
actions taken by the Air Force Major Commands in response to the recent 
direction to review food service contracts, and take actions, as 
appropriate, to ensure that cost savings measures are implemented. 


Agency Comments 
and Our Evaluation 


In written comments on a draft of this report, DOD stated that it concurred 
with all of our recommendations and that the Air Force is already taking 
steps to implement them. For example, DOD has taken action to 
implement our recommendation that the Secretary of the Air Force 
monitor the actions taken by the Air Force Major Commands to review 
food service contracts, and take actions, as appropriate, to ensure that 
cost savings measures are implemented. In its response, DOD stated that 
each Air Force Major Command has tasked its bases to conduct a 100 
percent review of existing food service contracts to determine if their 
current contracts require modification to meet current mission 
needs. DOD further noted that it intends to share the results of the Air 
Force’s review of its food service labor costs to achieve cost savings with 
the other services. DOD’s comments are printed in their entirety in 
appendix II. 

DOD also commented on our discussion of the response rate for the Air 
Force’s recent customer satisfaction survey. In our report, we stated that 
the low response rate (17 percent of active duty airmen) for the Air 
Force’s annual customer satisfaction survey could bias the data it uses to 
measure the results of the Food Transformation Initiative. DOD stated 


Page 34 


GAO-11-676 Defense Management 


that, according to the contractor who conducted the survey, the response 
rate achieved in the Air Force’s customer satisfaction survey met and, in 
most cases, exceeded the response rate that is usually achieved for 
these types of surveys. However, we continue to believe that this 
response rate decreases the likelihood that the survey results are 
representative of the views and characteristics of the population. 
According to the Office of Management and Budget’s standards for 
statistical surveys, a nonresponse analysis is recommended for surveys 
with response rates lower than 80 percent to determine whether the 
responses are representative of the surveyed population. Conducting a 
nonresponse analysis would allow the Air Force to more confidently rely 
on the survey results in assessing the results of the initiative. 


We are sending copies of this report to appropriate congressional 
committees, the Secretary of Defense, and the Secretary of the Air Force. 
The report is also available at no charge on the GAO website at 
http://www.gao.gov. 

If you or your staff have any questions regarding this report, please 
contact me at (202) 512-3863 or moranr@gao.gov. Contact points for our 
Offices of Congressional Relations and Public Affairs may be found on 
the last page of this report. GAO staff who made key contributions to this 
report are listed in appendix III. 

£ 

Revae Moran 

Director, Defense Capabilities and Management 



Page 35 


GAO-11-676 Defense Management 




List of Committees 


The Honorable Carl Levin 
Chairman 

The Honorable John McCain 
Ranking Member 
Committee on Armed Services 
United States Senate 

The Honorable Daniel Inouye 
Chairman 

The Honorable Thad Cochran 
Ranking Member 
Subcommittee on Defense 
Committee on Appropriations 
United States Senate 

The Honorable Howard P. McKeon 
Chairman 

The Honorable Adam Smith 
Ranking Member 
Committee on Armed Services 
House of Representatives 

The Honorable C.W. Bill Young 
Chairman 

The Honorable Norman D. Dicks 
Ranking Member 
Subcommittee on Defense 
Committee on Appropriations 
House of Representatives 


Page 36 


GAO-11-676 Defense Management 



Appendix I: Scope and Methodology 


To assess the Food Transformation Initiative’s objectives and 
performance measures, we reviewed pertinent documentation and 
interviewed Air Force headquarters and Air Force Services Agency 
officials about the initiative’s background, goals, and how the results of 
the initiative would be determined. We also reviewed Department of 
Defense (DOD) and Air Force guidance, as well as criteria from our 
previous work, for developing metrics and plans for assessing the results 
of programs such as the initiative. We compared principles we derived 
from this guidance and criteria with the Air Force’s metrics and 
assessment plans for the initiative and where there were differences we 
discussed the reasons why with Air Force officials. 

To review the Air Force’s progress in meeting its objectives for the 
initiative, we collected available quantitative data related to each of the 
initiative’s objectives and compared the data before and after 
implementation of the initiative. For example, we obtained and reviewed 
changes in hours of operation of the main dining facilities, the menus 
being used, the number of meals served, and the cost of food service 
contracts. Also, to discuss with installation officials changes made under 
the initiative and progress towards the initiative’s objectives, we visited 
four of the six installations where the initiative was being piloted— 
Elmendorf Air Force Base, Alaska; Fairchild Air Force Base, Washington; 
MacDill Air Force Base, Florida; and Travis Air Force Base, California. 

We selected the bases to visit to include various geographic locations and 
other factors. In addition, we reviewed the Air Force’s customer 
satisfaction survey results on changes in the reported level of airmen 
satisfaction with various aspects of their food service experience. In areas 
where initiative progress to date differed from Air Force plans, such as in 
the timing of the implementation of the campus dining concept and 
changes in labor costs, we discussed with Air Force officials the reasons 
why and the associated impact on measuring the progress of the initiative 
at the pilot installations. In examining the meal count data, we looked for 
anomalies by comparing data month to month. We discussed anomalies 
with knowledgeable officials. As a result, the Air Force went through a 
revalidation process to verify the numbers and provided us with corrected 
data. Therefore, we believe that the meal count data we assessed are 
sufficiently reliable for the purposes of this report. To assess the reliability 
of the customer survey data, we discussed the data with knowledgeable 
officials and examined the survey response rate. The survey response 
rate is low and the Air Force did not perform a nonresponse analysis to 
clarify whether those who did not respond to the survey may provide 
substantively different answers than those who did respond. The survey 
results could be interpreted more confidently if a nonresponse analysis 


Page 37 


GAO-11-676 Defense Management 




Appendix I: Scope and Methodology 


was done to establish whether or not it is likely that there are any 
systematic biases due to some people being more or less likely to 
respond to the survey. However, we are presenting these data as 
descriptive information that the Air Force is using to measure its progress 
in meeting these objectives and have noted its limitations. 

To document the initiative’s impact on the food service employees at the 
pilot bases, we considered the three types of food service employees: 
appropriated fund civilian and military food service employees at the main 
dining halls, contract workers at the main dining halls, and 
nonappropriated fund employees at nonappropriated fund food and 
beverage operations. Specifically, for the civilian and military food service 
employees, we interviewed union representatives and groups of food 
service workers, as well as individual workers, who were available to 
meet with us during our visit about their experiences under the initiative, 
and also spoke to Air Force and Aramark officials about how the initiative 
affected these employees, including any changes in employment 
numbers. For the contract employees at the main dining facilities, we 
obtained and compared the number of employees prior to and after the 
implementation of the initiative and also spoke with Air Force, Aramark, 
and Ability One officials about changes under the initiative. Aramark 
officials stated that the numbers represent the number of employees the 
day Aramark took over the contract and accurately reflect the 
terminations and hires since October 1, 2010. However, the contract 
employee numbers cannot be validated since the names of the 
individuals are not available. These are small numbers of employees, and 
Aramark was able to count the number of Ability One workers at the 
dining halls on the first day of the contract. Because these are small 
numbers of employees, we believe there is less possibility for error. 
Therefore, we believe the data are sufficiently reliable for the purpose of 
this report. For the employees at the nonappropriated fund food and 
beverage operations, we obtained and compared the number of 
employees prior to and after the implementation of the initiative and 
documented the employment options taken by these employees after the 
implementation of the initiative. We also interviewed some former 
nonappropriated fund food and beverage operation employees who were 
available to speak with us during our visit to discuss their experiences 
under the initiative and discussed with Air Force headquarters and 
installation officials employee severance payments at the bases we 
visited. 

To assess the Air Force’s consideration of alternative approaches for 
transforming its food service program, we reviewed the Air Force 


Page 38 


GAO-11-676 Defense Management 




Appendix I: Scope and Methodology 


business case analysis that was used to evaluate alternatives for meeting 
the Air Force’s food service improvement objectives. Specifically, we 
discussed the business case analysis process with Air Force officials and 
compared the methodology used in the analysis with Air Force guidance 
for conducting such analyses and with guidance contained in the GAO 
Cost Estimating and Assessment Guided In areas where the business 
case analysis did not follow guidance, we discussed the reasons why with 
Air Force officials and also asked about options for meeting the Air 
Force’s food service improvement objectives that were not included in the 
analysis. Further, we talked with Air Force officials about opportunities for 
reducing food service costs outside of the Food Transformation Initiative. 

We provided a draft of this report to DOD officials for their review and 
incorporated their comments where appropriate. We also provided the 
pertinent factual information contained in this draft to the contractor, 
Aramark, for review and incorporated Aramark’s comments as 
appropriate. 

We conducted this performance audit from September 2010 to July 2011, 
in accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 


^AO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and 
Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: March 2009). 


Page 39 


GAO-11-676 Defense Management 





Appendix II: Comments from the Department 
of Defense 


OFFICE OF THE UNDER SECRETARY OF DEFENSE 

4000 DEFENSE PENTAGON 
WASHINGTON, D C. 20301-4000 


JUL 8 2011 

Ms. Revae Moran 

Acting Director, Delense Capabilities and Management 
Unites States Government Accountability Office 
Washington DC 20548 

Dear Ms. Moran: 

Thank you for the opportunity to comment on GAO Draft Report, GAO-11-676, 
"DEFENSE MANAGEMENT: Actions Needed to Improve Management of Air Force's Food 
1 ranslormation Initiative,' dated June 9, 2011 (GAO Code 351536). The Department concurs 
with the four recommendations and appreciates the comprehensive review the GAO conducted 
on this important initiative. The enclosure to this letter outlines the actions Air Force is taking in 
response to the recommendations. 

Ol particular note is GAO's recommendation to use lessons learned from the Food 
Transformation Initiative to review and renegotiate current food service contracts to adjust labor 
costs as necessary at all Air Force dining facilities, potentially achieving immediate savings prior 
to expanding the initiative. Wc intend to share this lesson learned with the other Services. 

With regard to the annual customer satisfaction survey that Air Force will use to measure 
progress for several Food Transformation objectives, it is important to clarify the survey 
methodology and reliability. The CFI Group, a management consulting firm conducts the survey 
and specializes in the application of the American Customer Satisfaction Index (ACSI) 
methodology to individual Air Force program objectives. The ACSI, established in 1994. is a 
uniform, cross-industry measure of satisfaction with goods and services in a number of national 
industry sectors from transportation and health care to hotels and food sendee. 

CFf group uses the ACSI methodology to identify the causes ol" satisfaction, and relates 
satisfaction to business performance measures. Using CFI Group results. Air Force can identify 
and improve those factors that will improve satisfaction and other measures of business 
performance. ACSI ratings are presented on a 0-100 scale. Within the context of the survey 
results, scores in the 60s are characterized as “fine but could use work," the 70s as “good job but 
keep working on if" and the 80s as “excellent - keep it up." At the program area level, CS1 
scores in the mid-70s are expected; it is unlikely that any program area will achieve a score 
greater than 85. An increase of tw'o points or more is considered significant for increased 
customer satisfaction (and business performance). Therefore, the comparison on page 15 
between customer satisfaction with several criteria for the pilot bases and non pilot bases show’s 
significant improvement in customer satisfaction where Food Transformation has been 
implemented and Air Force's stated goals for future CFI scores are commendable. 



Page 40 


GAO-11-676 Defense Management 








Appendix II: Comments from the Department 
of Defense 


The GAO expresses concern about a low (17 percent) response rate among active duty 
personnel for the recent surveys. In consultation with CFI group personnel, we have 
benchmarked the achieved response rate to several surveys conducted by CFI of a similar 
methodology (i.e., “outbound” email invitation with some pre-notification, reminder/follow up 
emails sent, no incentives offered for completion). For example, a survey of institutional 
investors of financial services had a 24 percent response rate; account holders at credit unions a 
1 -8 percent response rate; businesses and organizations within government agencies a 5-20 
percent response rate; and subscribers to satellite television a 3-5 percent response rate. 

The response rates achieved in the Air Force 2010 Caring for People survey meet and in 
most cases far exceed the rates CFI Group is accustomed to seeing in its other client work. For 
the sample sizes used, the confidence intervals around the scores are within 2-3 points on the 0 to 
100 point reporting scale at the 90 percent level of confidence (some programs areas will be 
higher), which is sufficient to identify substantive differences in performance over time and/or 
across various segmentations of the data. 

Thank you again for the excellent review. The professional and experienced conduct of 
you and your staff was exceptional and contributed to a very productive study. 



^—-"'"’"Robert L. Gordon III 

Deputy Assistant Secretary of Defense 
(Military Community and Family Policy) 


Enclosure: 
As stated 


Page 41 


GAO-11-676 Defense Management 






Appendix II: Comments from the Department 
of Defense 


GAO DRAFT REPORT DATED JUNE 9, 2011 
GAO-11-676 (GAO CODE 351536) 

“DEFENSE MANAGEMENT: ACTIONS NEEDED TO IMPROVE 
MANAGEMENT OF AIR FORCE'S FOOD TRANSFORMATION 
INITIATIVE” 

DoD COMMENTS 

TO THE GAO RECOMMENDATIONS 


RECOMMENDATION 1: Before implementing the Food Transformation 
Initiative at other Air Force bases, the GAO recommends that the Secretary of the 
Air Force direct the Air Force Services Agency to develop an evaluation plan to 
accurately and reliably assess the results of the pilot phase of the Food 
Transformation Initiative. This evaluation plan should include, at a minimum, 
well-defined, clear, and measurable metrics to use in determining the results of the 
initiative and time frames for collecting data and reporting results. 

DoD RESPONSE: Concur. Air Force Services will complete the evaluation plan, 
with well defined measureable metrics, in August 2011. There are currently 
measureable metrics for seven of the eight Air Force objectives, and work 
continues on the development of measureable data for the eighth objective, 
“providing a sense of community.” Final efforts are being made on the Balanced 
Scorecard and evaluation plan, and the completed plan will be in effect well before 
implementation of Food Transformation at the next portfolio of bases. 

RECOMMENDATION 2: Before implementing the Food Transformation 
Initiative at other Air Force bases, the GAO recommends that the Secretary of the 
Air Force direct the Air Force Services Agency to evaluate the results of the 
implementation of the campus dining concept at the six pilot locations to 
determine if adjustments are needed before expanding the initiative to additional 
bases. 

DoD RESPONSE: Concur. The implementation of Campus Dining began June 
6, 2011 at Patrick AFB. Implementation of Campus Dining at all six pilot bases is 
scheduled for completion by the end of July 2011. As each implementation 
occurs, Campus Dining will be subject to initial and ongoing analysis, with 
adjustments made as issues are identified. The Air Force delayed Campus Dining 
until an effective method for capturing Essential Station Messing (ESM) usage at 
nonappropriated fund (NAF) food service operations could be implemented. The 
requirement to discontinue use of the member’s social security account number as 


Page 42 


GAO-11-676 Defense Management 






Appendix II: Comments from the Department 
of Defense 


2 

a means to identify enlisted members on ESM required the identification and 
development of an alternative method of identifying ESM members and the meals 
they consume. Use of the alternate identifier on a military member’s identification 
card was pursued, and the appropriate software product developed. Air Force is 
confident they can now provide a Campus Dining benefit to enlisted members on 
ESM with the level of accountability required. 

RECOMMENDATION 3: Before implementing the Food Transformation 
Initiative at other Air Force bases, the GAO recommends that the Secretary of the 
Air Force direct the Air Force Services Agency to reevaluate the number of 
military personnel hours the Air Force can provide the contractor for food service, 
make adjustments where possible in projected labor hours, and take into account 
the increased labor costs in revising the cost savings estimates for the future. 


AF RESPONSE: Concur. The Air Force Services Agency (AFSVA) has tracked 
the availability of military food service manpower since the initial implementation 
of Food Transfonnation, and this process continues. The pilot bases are currently 
conducting a 100% revalidation of available military manpower for use in a final 
4 th quarter FY11 cost projection, and for determining the cost estimate for their 
food service contract for FY12. Applying the lessons learned from the initial 
implementation, additional planning factors for use in determining the amount of 
available military food service manpower were sent to each base in the next 
portfolio and their parent Major Command. Once the final manpower estimates 
are provided to AFSVA, the Food Transformation team will contact each base to 
insure all planning factors have been properly considered prior to releasing any 
due diligence data for contract bidder use on the next portfolio. 

RECOMMENDATION 4: The GAO recommends that the Secretary of the Air 
Force monitor the actions taken by the Air Force Major Commands in response to 
the recent direction to review food service contracts, and take actions, as 
appropriate, to ensure that cost savings measures are implemented. 

DoD RESPONSE: Concur. On March 28, 2011, the Commander, AFSVA 
requested each Major Command task their bases to conduct a 100% review of 
existing food service, to determine if their current contract workload estimates 
meet current mission needs or if the contract requires modification. Results of this 
review are currently being analyzed. 


Page 43 


GAO-11-676 Defense Management 





Appendix III: GAO Contact and Staff 
Acknowledgments 


GAO Contact 


Revae Moran, (202) 512-3863 or moranr@gao.gov 


Staff 

Acknowledgments 


In addition to the contact named above, key contributors to this report 
were Laura Durland, Assistant Director; Leslie Bharadwaja; Dawn 
Godfrey; Sharon Reid; Joanne Landesman; Carol Petersen; Michael 
Shaughnessy; and Amie Steele. In addition, Michele Fejfar, Kirsten 
Lauber, Brian Lepore, Pat Owens, Cheryl Williams, and William Woods 
provided their expertise and guidance. 


( 351536 ) 


Page 44 


GAO-11-676 Defense Management 




GAO’s Mission 

The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting its 
constitutional responsibilities and to help improve the performance and 
accountability of the federal government for the American people. GAO 
examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding decisions. 
GAO’s commitment to good government is reflected in its core values of 
accountability, integrity, and reliability. 

Obtaining Copies of 
GAO Reports and 
Testimony 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO’s Web site (www.gao.gov). Each weekday afternoon, 
GAO posts on its Web site newly released reports, testimony, and 
correspondence. To have GAO e-mail you a list of newly posted products, 
go to www.gao.gov and select “E-mail Updates.” 

Order by Phone 

The price of each GAO publication reflects GAO’s actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black and 
white. Pricing and ordering information is posted on GAO’s Web site, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 
MasterCard, Visa, check, or money order. Call for additional information. 

To Report Fraud, 
Waste, and Abuse in 
Federal Programs 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 

E-mail: fraudnet@gao.gov 

Automated answering system: (800) 424-5454 or (202) 512-7470 

Congressional 

Relations 

Ralph Dawn, Managing Director, dawnr@gao.gov, (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 
Washington, DC 20548 

Public Affairs 

Chuck Young, Managing Director, youngd@gao.gov, (202) 512-4800 

U.S. Government Accountability Office, 441 G Street NW, Room 7149 
Washington, DC 20548 



Please Print on Recycled Paper