Skip to main content

Full text of "DTIC ADA583901: Defense Acquisition Research Journal. Volume 20, Number 2, Issue 66, July 2013"

See other formats


DEFENSE 


JOURNAL 


Defense Acquisition Research JournaF 

’•■A Publication of the^Defense Acquisition University 


lm i n k mg ur se Ives 


Best Practices to I 
Work Smarter f 


Defense Acquisition University 


ISSUE 








Report Documentation Page 

Form Approved 

OMB No. 0704-0188 

Public reporting burden for the col lection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and 
maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, 
including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington 

VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it 
does not display a currently valid OMB control number. 

1. REPORT DATE 

JUL 2013 2. REPORT TYPE 

3. DATES COVERED 

4. TITLE AND SUBTITLE 

Thinking for Ourselves:Understanding Best Practices to Work Smarter 

5a. CONTRACT NUMBER 

5b. GRANT NUMBER 

5c. PROGRAM ELEM ENT NUM BER 

6. AUTHOR(S) 

5d. PROJECT NUMBER 

5e. TASK NUMBER 

5f. WORK UNIT NUMBER 

7. PERFORMING ORGANIZATION NAM E(S) AND ADDRESS(ES) 

Defense Acquisition Research Journal,Attn: DAU Press,9820 Belvoir Rd. 
Ste 3,Fort Belvoir,VA,22060 

8. PERFORMING ORGANIZATION 

REPORT NUMBER 

9. SPONSORING/MONITORING AGENCY NAM E(S) AND ADDRESS(ES) 

10. SPONSOR/MONITOR'S ACRONYM (S) 

11. SPONSOR/MONITOR'S REPORT 
NUMBER(S) 

12. DISTRIBUTION/AVAILABILITY STATEMENT 

Approved for public release; distribution unlimited. 

13. SUPPLEMENTARY NOTES 

The original document contains color images. 

14. ABSTRACT 

15. SU BJ ECT TERMS 

16. SECURITY CLASSIFICATION OF: 17. LIM ITATION OF 

ARCTRAfT 

18. NUMBER 19a. NAME OF 

nr parfqphmqiri f pfrqhm 

a. REPORT b. ABSTRACT c.THISPAGE 

unclassified unclassified unclassified 

259 


Standard Form 298 (Rev. 8-98) 

Prescribed by AN SI Std Z39-18 






Defense Acquisition Research Journal 

A Publication of the Defense Acquisition University 



We’re on the Web at: 

http://www.dau.mil/pubscats/Pages/ARJ.aspx 




oau 

Defense Acquisition University 


Mr. Frank Kendall 


Under Secretary of Defense for Acquisition, Technology and Logistics 


Mrs. Katharina G. McFarland 


Assistant Secretary of Defense for Acquisition 


Dr. James McMichael 


President (Acting), Defense Acquisition University 


Research Advisory Board 


Mr. Patrick Fitzgerald 

Defense Contract Audit Agency 

Mr. Richard T. Ginman 

Office of Defense Procurement and 
Acquisition Policy 

Mr. Andre J. Gudger 

Office ofDoD Small Business Programs 


Dr. Nayantara Hensel 

National Defense University 

Mr. Brett B. Lambert 

Office of DoD Manufacturing and 
Industrial Base Policy 

Ms. Heidi Shyu 

Office of the Assistant Secretary of the 
Army for Acquisition, Logistics and 
Technology 


Mr. James E. Thomsen 

Office of the Assistant Secretary of the 
Navy for Research, Development and 
Acquisition 

Mr. David M. Van Buren 

Office of the Assistant Secretary of the Air 
Force for Acquisition 


Editorial Board 

Dr. Larrie D. Ferreiro Dr. Mary C. Redshaw 


Chairman and Executive Editor Deputy Executive Editor 


Mr. Richard Altieri 

Industrial College of the Armed Forces 

Dr. Michelle Bailey 

Defense Acquisition University 

Dr. Don Birchler 

Center for Naval Analyses Corporation 

Mr. Kevin Buck 

The MITRE Corporation 

Dr. Richard Donnelly 

The George Washington University 

Dr. J. Ronald Fox 

Harvard Business School 

Dr. Jacques Gansler 

University of Maryland 

RADM James Greene, USN 
(Ret.) 

Naval Postgraduate School 


Dr. Donald E. Hutto 

Defense Acquisition University 

Dr. Ned Kock 

Texas A&MInternational University 

Dr. Mike Kotzian 

Defense Acquisition University 

Dr. Craig Lush 

Defense Acquisition University 

Dr. Mark Montroll 

Industrial College of the Armed Forces 

Dr. Andre Murphy 

Defense Acquisition University 

Dr. Christopher G. Pernin 

RAND Corporation 

Mr. Tim Shannon 

Defense Acquisition University 


Dr. Richard Shipe 

Industrial College of the Armed Forces 

Dr. Keith Snider 

Naval Postgraduate School 

Dr. John Snoderly 

Defense Acquisition University 

Dr. David M. Tate 

Institute for Defense Analyses 

Dr. Trevor Taylor 

Cranfield University (UK) 

Prof. Jerry Vandewiele 

Defense Acquisition University 

Dr. J. Robert Wirthlin 

Air Force Institute of Technology 

Dr. Roy Wood 

Defense Acquisition University 


ISSN 2156-8391 (print) ISSN 2156-8405 (online) 


The Defense Acquisition Research Journal, formerly the Defense Acquisition Review Journal, is published quarterly by the Defense 
Acquisition University (DAU) Press. Postage is paid at the U.S. Postal facility, Fort Belvoir, VA, and at additional U.S. Postal facilities. 
Postmaster, send address changes to: Editor, Defense Acquisition Research Journal, DAU Press, 9820 Belvoir Road, Suite 3, Fort 
Belvoir, VA 22060-5565. Some photos appearing in this publication may be digitally enhanced. 


Articles represent the views of the authors and do not necessarily reflect the opinion of DAU or the Department of Defense. 















DEFENSE 




JOURNAL 


A Publication of the Defense Acquisition University 






A Publication of the Defense Acquisition University 


July 2013 Vol. 20 No. 2 ISSUE 66 


CONTENTS I Featured Research 


Digging Out the Root Cause: Nunn-McCurdy Breaches in Major 
Defense Acquisition Programs 


P-128 


Irv Blickstein, Charles Nemfakos, and Jerry M. Sollinger 

Continuing concern over defense acquisition has led Congress to direct 
the establishment of an office in the Department of Defense to oversee 
the conduct of root cause analyses on programs that have incurred Nunn- 
McCurdy breaches. Analyses of six programs that have incurred such 
breaches reveal that many of the causes of the breaches are common to 
several programs. However, each program is different, and those differences 
suggest that policymakers should be wary of applying policies that assume 
all program cost increases stem from common causes. 


Reusing DoD Legacy Systems: Making the Right Choice 

Meredith Eibandy Timothy J. Eveieigh, Thomas H. Hoizer, 
p ^34 anc ^ Shahryar Sarkani _ 

Department of Defense (DoD) programs often experience cost overruns 
and technical difficulties due to reuse of legacy systems. With today's fiscal 
climate of resource-constrained DoD budgets, reuse of legacy systems is 
frequently touted as the solution to cost, efficiency, and time-to-delivery 
problems; however, cost overruns and technical difficulties can significantly 
diminish any perceived benefits. Through evaluation of eight diverse DoD 
programs, this research shows that the state of a legacy system's documen¬ 
tation, availability of subject matter expertise, and complexity/feasibility of 
integration are key factors that must be analyzed. Based on these three key 
factors, the authors propose a framework to aid both the DoD and defense 
contractors in the evaluation of legacy systems for potential efficient and 
effective reuse. 


Valuing the Cost of an Economic Price Adjustment Clause to 
the Government 

p Scot Arno/dy Bruce Harmon, Susan /?ose, and John Whitley _ 

An Economic Price Adjustment (EPA) clause in a contract allows for adjust¬ 
ment of contract price if certain conditions are met. The Department of 
Defense (DoD) often uses an EPA clause in contracts where there is an 
increased risk that the costs of inputs used by the contractor will diverge 
from the forecasts used in the original pricing of the contract. EPA clauses 
transfer risk from the contractor to the government; thus, they are of 
economic value to the contractor. This article reviews EPA clauses, analyzes 
the value of risk transfer, and discusses how DoD could account for this 
value in negotiating fees for contracts that contain EPA clauses. Other 
government costs and risks associated with EPA clauses are also discussed. 






Featured Research 



Current Barriers to Successful Implementation of FIST Principles 

p TQ4 Ca P t Brandon Keller; USAF, and Lt Col J. Robert Wirth/in, USAF _ 

The Fast, Inexpensive, Simple, and Tiny (FIST) framework proposes a 
broad set of organizational values, but provides limited guidance on prac¬ 
tical implementation. Implementing FIST principles requires clarifying 
the definitions of “fast,” “inexpensive,” and “simple,” recognizing where 
FIST does and does not apply. Additionally, a subset of the FIST heuristics 
was expanded upon to increase their usefulness for practitioners. The 
primary research findings are that FIST principles are less conducive for 
highly complex or novel systems, immature technologies, future needs, 
acquisitions in early development phases, or when performance is the 
foremost value. FIST principles were also found to be constrained by the 
acquisition process, the requirements process, and oversight. 

Defense Logistics Agency Disposition Services as a Supply 
Source: A DoD-Wide Opportunity 

Capt Nate Leon, USMC, Capt Todd Paulson, USMC, and Geraldo Ferrer 

The Defense Logistics Agency Disposition Services (DDS) provide 
centralized disposal management of excess and surplus military prop¬ 
erty. An important component of its mission is the reutilization of excess 
equipment within the military services to prevent wasteful purchases 
within the Department of Defense. This research analyzes the extent 
to which the U.S. Marine Corps (USMC) is implementing reutilization 
through DDS as a source of supply. The results and recommendations of 
this study will enable decision makers within the USMC and the Defense 
Logistics Agency to address institutional and systemic obstacles to maxi¬ 
mize reutilization. Some of the lessons learned herein may be useful to all 
the military services, resulting in more value from their operations and 
maintenance budgets through reutilization. 





A Publication of the Defense Acquisition University July 2013 Vol. 20 No. 2 ISSUE 66 


CONTENTS I Featured Research 


n 242 _ 

Professional Reading List 

Democracy's Arsenal—Creating a Twenty-First-Century Defense Industry 

by Jacques S. Gans/er 

P-245 _ 

Call for Authors 

We are currently soliciting articles and subject matter experts for the 
2013-2014 Defense Acquisition Research Journal (ARJ) print years. 

p. 246 _ 

Guidelines for Contributors 

The Defense Acquisition Research Journal (ARJ) is a scholarly peer- 
reviewed journal published by the Defense Acquisition University (DAU). 
All submissions receive a blind review to ensure impartial evaluation. 

P-254 _ 

Defense Acquisition University Web site 

Your online access to acquisition research, consulting, information, and 
course offerings. 

p.255 _ 

Defense ARJ Survey 

We want to know what you think about the content published in the 
Defense ARJ. 











From the Deputy 
Executive Editor 


The articles included in this month's issue suggest a 
time-honored maxim that is consistent with the philosophy 
and guidelines espoused by Under Secretary of Defense for 
Acquisition, Technology and Logistics Frank Kendall (2013) 
in a recent memorandum. The maxim is Caveat Emptor —Let 
the buyer beware! One reasonably might ask: How does that maxim relate to Mr. 
Kendall's memorandum, or to the articles in this issue? 


In the commercial world, caveat emptor means that the buyer bears the 
risk for the quality of goods purchased unless they are covered by the seller's 
warranty. A standard commercial principle is that product use other than for 
intended purposes may void any manufacturer's warranties. Extending that 
principle, acquisition managers inadvertently may negate anticipated benefits 
of best practices if they fail to thoughtfully and deliberately analyze their appro¬ 
priateness to the context and conditions under which they are to be applied. 

Mr. Kendall emphasized that key enduring acquisition principles and 
evolving best practices work when they are applied effectively with a thorough 
understanding of the program context and an understanding of the risks. He 
stressed the need to “apply our education, training, and experience through 
analysis and creative, informed thought" to program decisions (Kendall, 
2013, p. l).His principal guideline was succinct: Think. 

In our first article, RAND coauthors (Blickstein, Nemfakos, and Sollinger) 
shared lessons learned from their analyses of nine major defense acquisition 
programs that experienced Nunn-McCurdy breaches. Their analyses provided 
insight into the breaches and some lessons for how other programs can avoid 
them. The authors also note that every program is different, and they caution 
that managers should be wary of applying policies founded on the premise that 
all program cost increases stem from common causes. 

Coauthors from Lockheed Martin (Eiband) and The George Washington 
University (Eveleigh, Holzer, and Sarkani) examined reuse of legacy systems as 
an “oft-touted" approach to achieve affordability and reduce acquisition time- 
frames. Failure to analyze the implications of reuse may result in adverse cost, 
schedule, and system performance outcomes. The authors developed a Reuse 
Evaluation Framework to aid program planners identifying opportunities for 
reuse that offer the greatest chance of success. 





Coauthors from the Institute for Defense Analyses (Arnold, Harmon, Rose, 
and Whitley) discussed the need for adjusting target fees when using Economic 
Price Adjustment (EPA) clauses in contracts. They noted that EPA clauses can 
entail unintended risks and drive unwanted behavior; assessing those risks and 
behavior motivations requires in-depth assessment of the specific contract and 
the contractor. 

Similarly, coauthors from the Air Force Institute of Technology (Keller and 
Wirthlin) examined the Fast, Inexpensive, Simple, and Tiny (FIST) principles 
touted in various circles as a means to reforming acquisition. The authors 
offered their own planning considerations to augment the FIST heuristics and 
identified barriers and limitations to successful implementation of the concept. 

Finally, authors from the Marine Corps Logistics Base Albany, GA, Defense 
Language Institute, and Naval Postgraduate School (Leon, Paulson, and Ferrer, 
respectively) explored the extent to which equipment reutilization can achieve 
cost savings from wasteful, duplicative purchases—savings that can be used to 
cover other shortfalls in a dwindling defense budget. Ironically, one obstacle that 
potentially prevents managers from aggressively seeking those savings is the 
DoD culture, implying that managers must constantly examine and challenge 
patterns of shared, basic assumptions about their business. 

These articles represent research results from a variety of organizations 
and perspectives. I hope that you will enjoy them and can use the information 
they contain to inform your selection of best practices. Enjoy! 


( 


Dr. Mary C. Redshaw 
Deputy Executive Editor 
Defense ARJ 


Reference 


Kendall, F. (2013, April 24). Implementation directive for Better Buying Power 2.0— 

Achieving greater efficiency and productivity in defense spending [Memorandum]. 
Washington, DC: Office of the Under Secretary of Defense (Acquisition, Technology 
and Logistics). 



Defense ARJ and AT&L have become 
online-only publications for 

individual subscribers 


If you would like to start or 
continue a membership with 

Defense ARJ or AT&L, you must 
register a valid e-mail 
address in our LISTSERV 

ALL READERS: Please 
subscribe or resubscribe 
so you will not miss out on 
receiving future publications. 

• Send an e-mail to 

darjonline@dau.mil and/ 
or datlonline@dau.mil, 

giving the e-mail address 
at which you want to be 
notified when a new 
issue is posted. 

• Please type “Add to 
LISTSERV” in the 
subject line. 

• Please also use this 
address to notify us 
if you change your 
e-mail address. 

1 ■ 



http://www.dau.mil 

















































A Publication of the Defense Acquisition University 


http: //www.dau .mil 


DAU Center for 
Defense Acquisition 
Research 


Research Agenda 2013 


The Defense Acquisition Research Agenda is intended to make 
researchers aware of the topics that are, or should be, of partic¬ 
ular concern to the broader defense acquisition community 
throughout the government, academic, and industrial sectors. 
The purpose of conducting research in these areas is to provide 
solid, empirically based findings to create abroad body of knowl¬ 
edge that can inform the development of policies, procedures, and 
processes in defense acquisition, and to help shape the thought 
leadership for the acquisition community. 

Each issue of the Defense ARJ will include a different selection 
of research topics from the overall agenda, which is at: http:// 
www.dau.mil/research/Pages/researchareas.aspx 

Affordability and cost growth 


• Define or bound “affordability” in the defense portfolio. What is it? 
How will we know if something is affordable or unaffordable? 


• What means are there (or can be developed) to measure, manage, and 
control “affordability” at the program office level? At the industry 
level? How do we determine their effectiveness? 



What means are there (or can be developed) to measure, manage, 
and control “Should Cost” estimates at the Service, Component, 
program executive, program office, and industry levels? How do we 
determine their effectiveness? 


What means are there (or can be developed) to evaluate and compare 
incentives for achieving “Should Cost” at the Service, Component, 
program executive, program office, and industry levels? 


July 2013 


• Recent acquisition studies have noted the vast number of programs 
and projects that do not make it successfully through the acquisition 
system and are subsequently cancelled. What would systematic 
root cause analyses reveal about the underlying reasons, whether 
and how these cancellations are detrimental, and what acquisition 
leaders might do to rectify problems? 

• Do Joint programs—at the inter-Service and international levels— 
result in cost growth or cost savings compared with single-Service 
(or single-nation) acquisition? What are the specific mechanisms 
for cost savings or growth at each stage of acquisition? Do the data 
support “jointness” across the board, or only at specific stages of a 
program, e.g., only at research and development or only with specific 
aspects, e.g., critical systems or logistics? 



• Can we compare systems with significantly increased capability 
developed in the commercial market to DoD-developed systems of 
similar characteristics? 


Is there a misalignment between industry and the government 
priorities that causes the cost of such systems to grow significantly 
faster than inflation? 


If so, can we identify why this misalignment arises? What relation¬ 
ship (if any) does it have to industry's required focus on shareholder 
value and/or profit, versus the government's charter to deliver 
specific capabilities for the least total ownership costs? 


A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


DEFENSE ACQUISITION RESEARCH JOURNAL 



126 




July 2013 


ISSUE 66 JULY 2013 VOL. 20 NO. 2 


A Publication of the Defense Acquisition University 


http://www.dau.mil 


Keywords: Nurm-McCurdy Breaches, Defense 
Acquisition Cost Growth, Selected Acquisition 
Report (SAR), Wideband Global Satellite (WGS), 
Performance Assessments and Root Cause Analyses 
(PARCA) 


Digging Out the Root Cause 

Nunn-McCurdy Breaches in 
Major Defense Acquisition 
Programs 


f Irv Blickstein, Charles Nemfakos, 
and Jerry M. Sollinger 


Continuing concern over defense acquisition has led 
Congress to direct the establishment of an office in 
the Department of Defense to oversee the conduct of 
root cause analyses on programs that have incurred 
Nunn-McCurdy breaches. Analyses of six programs 
that have incurred such breaches reveal that many 
of the causes of the breaches are common to several 
programs. However, each program is different, and 
those differences suggest that policymakers should be 
wary of applying policies that assume all program cost 
increases stem from common causes. 


128 


DefenseARJ, July 2013, Vol. 20No. 2:128-153 


Image designed by Diane Fleischer » 


• • 












A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


Congress has long been concerned about cost overruns in Major 
Defense Acquisition Programs (MDAPs). Beginning in the 1970s when 
it expropriated the Selected Acquisition Report (SAR) as a gauge of pro¬ 
gram performance, Congress has continued to create mechanisms to 
gain insights into program execution. 1 However, SARs did not become a 
legal reporting requirement until 1975, with Public Law (Pub. L.) 94-105 
(Leach, 2003). In 1981, Senator Samuel Nunn and Congressman David 
McCurdy introduced the Nunn-McCurdy Amendment to the Department 
of Defense Authorization Act, 1982 (Pub. L. 97-86,1981). The purpose 
of the amendment was to establish congressional oversight of defense 
weapon systems acquisition programs whose costs rise above certain 
limits. The Nunn-McCurdy Amendment defines two types of unit cost. 
The first is total program acquisition unit cost (PAUC), which is the 
sum of development cost, procurement cost, and system-specific mili¬ 
tary construction for the acquisition program, divided by the number of 
fully configured end-items to be produced for the acquisition program. 
The second is average procurement unit cost (APUC), which is the 
procurement funding divided by the number of units procured. Cost 
growth of a weapon system was measured by how much the unit costs 
in 1982 exceeded the same respective unit costs reported in the weapon 
system's SAR dated March 31,1981. Hence, the amendment applied only 
to those major weapon systems reported in SARs dated March 31,1981. 
The original amendment required the Secretary of Defense to notify 
Congress when a major weapon system's unit cost growth exceeded 
15 percent. If unit cost growth exceeded 25 percent, the program was 
assumed terminated unless the Secretary of Defense submitted written 
certifications to Congress within 60 days of determining that a breach 
had occurred. The provisions were made permanent in the Department 
of Defense Authorization Act, 1983, and these breaches are commonly 
referred to as Nunn-McCurdy breaches. 

Over time, the Department of Defense (DoD) leadership promulgated 
many external as well as internal initiatives to reform the acquisition 
system. Figure 1 captures the DoD Issuances as well as a few of the major 
initiatives pushed by Congress and by the DoD leadership, where the 
acquisition system has been the prime focus. Clearly, over time these 
efforts for reform have increased. 

The National Defense Authorization Act for Fiscal Year 2006 
changed the Nunn-McCurdy reporting requirements to include the 
original baseline as a benchmark against which to measure cost growth. 


130 


Defense ARJ, July 2013, Vol. 20 No. 2:128-153 



Digging Out the Root Cause: Nunn-McCurdy Breaches in Major Defense Acquisition Programs 


FIGURE 1. DoD ISSUANCES AND REFORM OVER TIME 



IDIDIDIDIDIDIDIOIDIDIDIDIOIDIDIDIDIDIDID 
OOOOOOOOOOOOOOOOOOOOlDlDlDlDCDCDtOCDCDCD 
O ~' 1 MWJiWOl'J00(OO J 'N>C/-J4^<-ncn^lOOtD 


U1 CT) ^1 00 ID 


The Weapon Systems Acquisition Reform Act (WSARA) of 2009 is the 
latest effort, and it incorporates definitions for two categories of weapon 
system breaches: significant and critical (Pub. L. 111-23,2009). A breach 
is determined by comparing original and current PAUC and APUCs, 
and a breach can occur if the unit costs exceed either the current or the 
original baseline by a specific percent. Thresholds appear in Table 1. 


The National Defense Authorization Act for Fiscal 
Year 2006 changed the Nunn-McCurdy reporting 
requirements to include the original baseline as a 
benchmark against which to measure cost growth. 


Congressional interest in, and efforts to contain spending on, 
defense acquisition have continued (Government Accountability Office, 
2011). The Weapon Systems Acquisition Reform Act (WSARA) of 2009 
established a number of requirements that affected the operation of 
the Defense Acquisition System and the duties of the key officials who 


DefenseARJ, July 2013, Vol. 20 No. 2:128-153 


131 









A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


TABLE 1. BREACH THRESHOLDS 


Level 

Unit Cost 

Baseline 

Threshold 

Significant 

PAUC 

Current 

>=15% 


APUC 

Current 

>=15% 


PAUC 

Original 

>=30% 


APUC 

Original 

>=30% 

Critical 

PAUC 

Current 

>=25% 


APUC 

Current 

>=25% 


PAUC 

Original 

>=50% 


APUC 

Original 

>=50% 


support it, including the requirement to establish a new organization in 
the Office of the Secretary of Defense (OSD) with the mandate to con¬ 
duct and oversee Performance Assessments and Root Cause Analyses 
(PARCA) for MDAP (Pub. L. 111-23, 2009). 

Pub. L. 111-23 assigned the resultant PARCA organization five pri¬ 
mary responsibilities: 

1. Carrying out performance assessments of MDAPs; 

2. Performing root cause analysis (RCA) of MDAPs whose 
cost growth exceeds the threshold as detailed in the Nunn- 
McCurdy provision; 

3. Issuing policies, procedures, and guidance governing the 
conduct of performance assessments and RCAs; 

4. Evaluating the utility of performance metrics used to mea¬ 
sure the cost, schedule, and performance of MDAPs; and 

5. Advising acquisition officials on performance issues that 
may arise regarding an MDAP. 

The PARCA office has a relatively limited staff, and reporting dead¬ 
lines for breaches are short—less than 2 months. Therefore, the director 
has asked outside organizations, primarily federally funded research and 
development centers, to assist in conducting the RCAs directed by the 


132 


Defense ARJ, July 2013, Vol. 20 No. 2:128-153 













Digging Out the Root Cause: Nunn-McCurdy Breaches in Major Defense Acquisition Programs 


law. RAND has supported the PARCA office by analyzing six programs: 
the Zumwalt-Class Destroyer (DDG-1000), the Joint Strike Fighter F-35, 
Longbow Apache Helicopter, Wideband Global Satellite, Excalibur artil¬ 
lery round, and the Navy Enterprise Resource Program. Further, RAND 
has recently completed the analysis of the Joint Tactical Radio System 
Ground Mounted radio, the P-8A Poseidon aircraft, and modifications 
to the Global Hawk Unmanned Aerial Vehicle. 2 



Purpose 

This article has four purposes. First, it briefly describes the meth¬ 
odology RAND developed to carry out RCAs. The approach to RCAs has 
matured over time and may prove useful to other organizations that 
either must do an RCA or wish to understand what the process involves. 
Second, it presents an example of such analyses—the Wideband Global 
Satellite, a program with both significant and critical breaches. Third, 
the article provides insight into the causes of breaches across several 
programs. Fourth, it offers lessons learned about breaches and how to 
avoid them. 


DefenseARJ, July 2013, Vol. 20 No. 2:128-153 


133 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


Methodology for Root Cause Analysis 

Congressional deadlines for an RCA are tough to meet for two rea¬ 
sons. First, the time available to do them is short. Depending on the 
circumstances, the RCA must be done in either 45 or 60 days. 3 Second, 
each RCA is unique because each program is unique. Thus, no “cookbook” 
spells out all the components and identifies key documents and their loca¬ 
tions. RAND has developed a generic methodology, depicted in Figure 2. 

The generic process is designed to use the short time available as 
efficiently as possible. The process is general enough that it can apply to 
the RCA of any system yet still accommodate the unique attributes of 
each system. It begins with a hypothesis about what caused the program 
to breach the threshold. That hypothesis guides many of the subsequent 
activities, including setting up interviews with key players both in indus¬ 
try and government, which can take some time to arrange. Work has to 
proceed in parallel so that the required products can be delivered to the 
PARCA office in a timely manner. In the RCAs performed to date, the 
PARCA office has requested the following deliverables: 

• a completed root cause matrix in the format supplied by the 
PARCA office; 

• a summary narrative; 

• a set of briefing charts based on the narrative; and 

• a full RCA report. 

All deliverables except the full RCA report should be supplied by 
PARCA office deadlines to ensure that these materials can be used to 
support the recertification decision. 


134 


Defense ARJ, July 2013, Vol. 20 No. 2:128-153 



Digging Out the Root Cause: Nunn-McCurdy Breaches in Major Defense Acquisition Programs 


FIGURE 2. GENERIC RCA METHODOLOGY 




Construct a timeline of cost 
growth relevant events from 
the program history 



l 
1 

* s 

V \ i? 

* Electronic record of \ 
i data sources and J 
data sources archive 

... 

* \ 

I 

I 


Create the program cost 
profiles and pinpoint 
occurrences of cost growth 



Match timeline with cost 
profiles and derive root 
causes of cost growth 


Root Cause Analysis of Wideband 
Global Satellite <WGS) Program 

The WGS program was funded in 2001 to acquire an unprotected 
wideband satellite communications (SATCOM) capability by using a 
commercial off-the-shelf satellite bus and Ka-band technology, thereby 
meeting DoD's demand for military SATCOM. WGS provides both 
X-band communications compatible with the older Defense Satellite 
Communications System (DSCS) platforms and Ka-band broadcast 
capability like the Global Broadcast System (GBS). Throughput for each 
satellite is estimated at over two gigabits per second (U.S. Air Force 
[USAF], 2007). 

The program consists of two phases or “blocks,” as shown in the 
first row of Table 2. Block I of WGS comprises three satellites, the last 
of which went into orbit in December 2009. WGS Block II consists of 
three additional satellites—two contracted for the United States to 
replace aging DSCS and GBS satellites, and a third wholly purchased by 
Australia in exchange for a percentage of global WGS bandwidth. Block 
II satellites are essentially the same as Block I, with a high-bandwidth 
bypass feature for aerial intelligence, surveillance, and reconnaissance 
platforms (Block I, 2010, p. 16.) With the delays and eventual cancella¬ 
tion of the Transformational Satellite Communications System, DoD 


DefenseARJ, July 2013, Vol. 20 No. 2:128-153 


135 








A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


TABLE 2. WGS AVERAGE PROCUREMENT UNIT COST (EXCLUSIVE 
OF LAUNCH COSTS) 



Original 

APB 

Current 

APB/ 

Original APB 

Estimate/ 

Current 

APB 

Estimate/ 

Original 

APB 

Block 

1 

1 & II 

1, II, Ilf 

1, II, Ilf 

Satellites 

1-3 

1-5 

1-8 a 

1-8 a 

Contract type 

FFP 

FPIF 

FPIF 

FPIF 

APUC 

$268m 

$294m 

$374m 

$374 m 

Unit cost b 

$239m 

$377m c 

$574m 

$574m 

% A APUC 

- 

110% 

127% 

140% 

% A Unit Cost 

- 

158% 

152% 

240% 


Note. APB = Acquisition Program Baseline; FFP = Firm Fixed Price; FPIF = Fixed Price 
Incentive Firm (Target Price). 

a WGS 6 was purchased for Australia and does not show up in U.S budget accounts. 
b That is, cost to the government. 

c Cost claims currently made by Boeing would suggest that the true cost of the first 
three satellites was roughly $377m. 

decided to procure the seventh and eighth WGS satellites—Block Ilf— 
with a planned total buy of 12 WGS satellites to meet future broadband 
communication requirements (Edwards, 2010). 

The Nunn-McCurdy Breach 

The unit cost to the government of WGS Block II was roughly 50 
percent more expensive than Block I ($377 million compared with $239 
million), and Block Ilf is again roughly 50 percent more expensive than 
Block II ($574 million compared with $377 million), as shown in the 
bottom row (Table 2). 

Table 2 illustrates the breach. The 27 percent increase between the 
current estimate and the current Acquisition Program Baseline (APB) 
(third column) exceeds the 25 percent threshold for a “critical” breach. 
(The 40 percent increase [fourth column] between the current estimate 
and the original APB represents a “significant” but not “critical” Nunn- 
McCurdy breach.) 

The averages, in turn, permit calculation of a unit cost for Blocks 
I, II, and Block Ilf, but not in a straightforward manner. 4 In real (Base 
Year [BY] 2001 $) terms, the PAUC of the WGS satellite rose 58 percent 
between Block I and II (from $239 million to $377 million). Unit costs 
between Block II and Block Ilf are projected to rise 52 percent (from $377 


136 


Defense ARJ, July 2013, Vol. 20 No. 2:128-153 












Digging Out the Root Cause: Nunn-McCurdy Breaches in Major Defense Acquisition Programs 


TABLE 3. WGS ORDER AND LAUNCH YEARS 



Satellite 

Budget 

Year 

Launch 

Year 

Difference 
in Years 

Block 1 

1 

2002 

2007 

5 


2 

2002 

2009 

7 


3 

2003 

2009 

6 

Block II 

4 

2007 

2011* 

5* 


5 

2008 

2012* 

4* 


6 (Aus.) 

2009 

2013 

4* 

Block Ilf 

7 

2011 

2016 

5 


8 

2012 

2017 

5 


Note. Aus. = Australia 

* These are the launch dates taken from the President’s 2012 budget. 


FIGURE 3. WGS PRODUCTION/LAUNCH PERIODS 



Year 


Note. Launches do not include MexSAT-1 and MexSAT-2. Launch dates will follow 
MexSAT-3. 


million to $574 million). Table 3 indicates when each WGS satellite was 
ordered, when delivered, and the difference in years; Figure 3 indicates 
the interval during which the USAF-purchased WGS satellites were built 
and launched. Table 3 indicates a large gap between WGS Block I and 
WGS Block II, and a smaller gap between WGS Block II and WGS Block 
Ilf. However, the time between program approval and launch for WGS 
Block I was 5 to 7 years, and the expected cycle time for WGS Block II is 


DefenseARJ, July 2013, Vol. 20 No. 2:128-153 


137 


















A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


shorter—4 to 5 years. If current launch dates for Block Ilf prove accurate, 
then the gap between Block I and Block II will be somewhat smaller than 
the gap between Block II and Block Ilf. 

Sources of the Nunn-McCurdy Breach 

The WGS cost breach has two components: the increase in unit 
costs between Block I and Block II satellites, and the increase in unit 
costs between Block II and Block Ilf satellites. The first difference was 
ascribed to “what proved to be an artificially low cost for the original 
three vehicles under a firm fixed-price contract” (Secretary of the Air 
Force M. B. Donley, personal communication, March 8, 2010). We focus 
on the latter cost increase, largely because it is the current one and, thus 
far, more relevant to decisions to be made on the WGS program. 

Table 4 shows both blocks in terms of target and ceiling costs. The 
latter includes margin sufficient to account for the possibility of cost 
overruns on the FPIF work (combining advanced procurement, base 
procurement, and launch support costs). 

How do $555 million and $410 million in current dollars (Table 4) 
compare with the $574 million and $377 million (in BY 2001 $)? Table 5 
illustrates the difference. 


TABLE 4. PROGRAM OFFICE UNIT COST BREAKDOWN 
(CURRENT $) 



BY 

Target 

Ceiling 

Block II 

2007 

$355m 

$410m 

Block Ilf 

2011 

X 

$555m 

Note. $ shown are program estimates. 


Several features merit note. First, storage and factory restart costs 
were very small in going from Block I to Block II, but substantial in going 
from Block II to Block Ilf even though the gap before restarting produc¬ 
tion was 4 years for Block II and only 2^2 years for Block Ilf. We could 
not explain this difference. Second, in both cases, Other Government 
Costs (estimated based on data from the program office and Secretary of 
the Air Force) are fairly large, but roughly the same in both cases. These 
costs include contracting office and engineering costs; it was estimated 
by subtracting known cost components from total cost components and 
checked for overall reasonableness and consistency. 


138 


Defense ARJ, July 2013, Vol. 20 No. 2:128-153 







Digging Out the Root Cause: Nunn-McCurdy Breaches in Major Defense Acquisition Programs 


TABLE 5. RELATING BASE YEAR AND CURRENT YEAR COSTS 
($ IN MILLIONS) 



Block II 

Block Ilf 

Unit cost (BY01 $) 

377 

574 

Inflation factor to current costs 

1.14 (BY07) 

1.207 (BY11) 

Unit cost current year dollars 

430 

693 

Less storage and factory restart 

4 

73 

Subtotal 

426 

620 

Less other government costs 

71 

65 

Subtotal (from Table 4) 

355 

555 


Third, and most importantly, Boeing's price figure for the Block II 
satellite, as a basis for comparison, is $355 million each rather than 
the $410 million ceiling price. Why? The $355 million represented the 
contracted, hence targeted, price of the satellites; if Boeing costs were 
higher than $355 million, then, under the terms of the contract, the fed¬ 
eral government would reimburse Boeing only for 80 percent of those 
additional costs. The $410 million was the ceiling price; Boeing would 
have to absorb all costs in excess of that amount. Building the Current 
APB APUC (for Blocks I and II) out of Boeing's price, but building the 
Expected APB APUC (for Blocks I, II, and Ilf) out of the ceiling price 
essentially compares apples and oranges. In effect, the WGS program 
office built a 15 percent factor—essentially an accounting artifact—into 
the price. We cannot explain the programmers' motivation for doing 
so, particularly because it led to a critical Nunn-McCurdy breach that 
otherwise could have been avoided. Whether this difference represents 
their lack of confidence in the estimate can only be a matter of specula¬ 
tion. Were this 15 percent removed, then the unit cost of Block Ilf would 
have been $516 million (in current $) rather than $574 million, yielding 
an APUC of $357 million or an increase of 22 rather than 27 percent, 
representing a “significant” rather than “critical” breach. Nonetheless, 
$555 million is still a substantial increase over $355 million—and needs 
to be explained. Table 6 lists the various factors. 

We start with Boeing's price of $355 million. Next we add the current 
cost overrun of 3 percent ($11 million). (Although the final cost overrun 
may be higher or lower, we presume that cost overruns experienced to 
date establish a new baseline for what it really costs to build a WGS, 
hence $366 million.) The next adjustment, line 4, factors in 4 years' 


DefenseARJ, July 2013, Vol. 20 No. 2:128-153 


139 












A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


TABLE 6. COST INCREASE BETWEEN BLOCK II AND IIF 
(CURRENT YEAR $) 


Increase Component 

Block II 

1 . 

Boeing price (BY 2007 $) 

$355m 

2. 

3% cost overrun 

$11m 

3. 

Actual unit costs (BY 2007 $) 

$366m 

4. 

Four years 5 inflation at 3.5% per year 

1.147* 

5. 

Expected unit cost circa 2011 

$420m 

6. 

Extra tests 

$2m 

7. 

Higher component prices for 3 items 

$35m 

8. 

Higher component prices overall 

$25m 

9. 

Subtotal 

$482m 

10. 

Risk premium of 15% 

$555m 


* (1.035 x 1.035 x 1.305 x 1.035 = 1.147 x $366m = $420m) 


worth of inflation at 3.5 percent per year (as calculated by the program 
office based on historic experience in satellite component and manufac¬ 
turing costs), 5 hence the $420 million in line 5. Next comes $2 million 
for additional tests not required for Block II, $35 million (as calculated 
by Boeing) to pay for three critical components that might otherwise 
go out of production, 6 and $25 million (also as calculated by Boeing) for 
cost increases in other components at risk in the supply chain, hence the 
subtotal of $482 million in line 9. The last adjustment arises from the 
accounting artifact noted previously—the difference between contract 
costs used to calculate Block II prices and the ceiling cost used to calcu¬ 
late Block Ilf prices. This brings us to the $555 million that the program 
office uses to calculate unit costs for Block Ilf. 

Explaining the Cost Differences 

The $60 million in component cost inflation (over and above the 
normal 3.5 percent a year) shown in rows 7 and 8 of Table 6 requires 
further explanation. Reflecting a general shift in market requirements, 
Boeing shifted its commercial satellite offerings from its HS702HP 
(high-power) bus to its HS702MP (medium-power) bus. This shift has 
left WGS supporting the production of parts that no longer have much 
commercial demand, thereby raising the cost of these components. That 
noted, Boeing also reports that the cost ratio between bus and payload 
is expected to remain constant, and the cost ratio between component 
costs and Boeing's costs is also expected to remain constant. Both imply 


140 


Defense ARJ, July 2013, Vol. 20 No. 2:128-153 















Digging Out the Root Cause: Nunn-McCurdy Breaches in Major Defense Acquisition Programs 


that its internal costs have also risen more or less proportionately with 
component costs. This may be reflected in the charges associated with 
the cold factory restart noted earlier. Figure 4 indicates a sharp decline 
in commercial satellite production at about the same time that WGS 
production started. In the 8 years before 2008, Boeing launched 11 com¬ 
mercial satellites; from 2008 to 2016, it plans to launch six. Although the 
pace of satellite construction has recovered, it has not returned to earlier 
levels that characterized the first few years of this century. 


FIGURE 4. LAUNCH DATES FOR BOEING-PRODUCED SATELLITES 



Fiscal year 


Component cost inflation also reflects a broader phenomenon— 
the growing divergence between WGS and its civilian counterpart. 
Commercial products change constantly; military products change 
infrequently (but in relatively large chunks) and, in the case of Military 
Specification products, may not change at all precisely because product 
qualification is complex. In effect, the WGS, born as a modification to 
a commercial business line, has evolved to a program that is primarily 
military. As noted, the WGS satellite bus has diverged from its civilian 
counterpart. The payload of the WGS satellite consists of Ka-band tran¬ 
sponders, and X-band transponders and channelizers to switch between 
the two. X-band is primarily military to begin with. The commercial 


DefenseARJ, July 2013, Vol. 20 No. 2:128-153 


141 





A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


market had flirted with Ka-band 10 years ago, but the trend toward 
terrestrial (fiber optics and cell phones) rather than satellite-based 
communications has dampened industry's interest in exploring differ¬ 
ent spectra whose primary virtue is that they are largely unclaimed. 
Furthermore, the global business of U.S. satellite manufacturers has 
been hampered by increasingly stringent application of International 
Traffic in Arms Regulations starting 10 years ago. Components that once 
could be supported from both WGS and commercial sales increasingly 
rely on the WGS market, and suppliers must be paid a premium to remain 
in the market. Similarly, former WGS workers who could count on trans¬ 
ferring their skills into very similar commercial work when gaps appear 
in WGS, face a harder transition. As one observer (Mecham, 2009) notes: 

In its 10-year history, the Boeing division's main platform, the 
702, has commonly served big commercial requirements, such 
as the three current orders for DirecTV and two for Sky Terra. 
But the platform also has been used for many of the company's 
major government programs, most prominently the Wideband 
Global Satcom (WGS) network of six spacecraft that replaces the 
Defense Satellite Communications System.... WGS and two other 
major government programs—the Global Positioning System 
IIF and GOES N-P series—have provided 90 percent of Boeing's 
recent work. To redress that imbalance, the company began 
looking for new commercial market entries four years ago and 
concluded it could take advantage of the 702's flight software, 
avionics and power management systems to develop a smaller 
bus. (p. 66) 

The days when commercial sales could buoy the resources put into 
the WGS program between one buy and the next are gone. The econom¬ 
ics of WGS increasingly depend on the pace and scheduling of WGS 
buys alone. 

Root Cause Analysis 

The 52 percent increase between Block II and Block Ilf unit pricing 
is primarily due to the first three factors listed in Table 7. Such results 
are necessarily limited by the 60-day window allowed for investigation 
under the Nunn-McCurdy legislation that curtailed RAND's ability to 
question subcontractors and analyze many of the cost claims that had 
to be accepted as valid over the course of the analysis. 


142 


Defense ARJ, July 2013, Vol. 20 No. 2:128-153 



Digging Out the Root Cause: Nunn-McCurdy Breaches in Major Defense Acquisition Programs 


TABLE 7. PRIMARY FACTORS FOR BLOCK II TO BLOCK IIF UNIT 
COST INCREASE(BY 2001 $) 


Factor 

$ Amount 

Percent 

Risk premium accounting artifact 

$60m 

30% 

Storage and restart costs 

$57m 

29% 

Increased component costs 

$51m 

26% 

Other (e.g., SATCOM industry 
inflation, cost overruns) 

$29m 

15% 


The largest factor—almost one-third of the increase—is an account¬ 
ing artifact where the Block Ilf prices, as calculated by the program 
office, include a 15 percent risk premium, whereas Block II unit costs do 
not (because they largely reflect expended rather than projected costs). 
These results represent an apples-and-oranges comparison. Inasmuch 
as the Block Ilf is practically identical to the Block II units that Boeing 
is already building, Boeing can be realistically expected to produce the 
satellites at near the target cost, which is 15 percent below the ceiling 
cost—although Block II is running 3 percent over target. But the ceil¬ 
ing price is what was reported. Next, Boeing is charging for storage 
and restart costs for the 2V2-year hiatus between Blocks II and Ilf. On 
the surface, the cause appears to be the interruption in production, but 
the 4-year hiatus (measured, as noted, in terms of when satellites were 
ordered, not when they launched) between Block I and Block II had a cost 
of only $3.5 million, or less than 7 percent of the current estimate. One 
explanation is that significant aspects of WGS production are no longer 
supported by the commercial market and, therefore, require storage and 
restart expenses during production breaks. Finally, key components of 
WGS that are no longer supplied to the commercial market will have 
greatly increased procurement costs, accounting for another 26 percent 
of the cost increase. The second and third factors support the argument 
that the root causes of the breach are changes in the commercial market 
without corresponding changes in the WGS design and procurement, 
and obsolescence. 

Despite these large cost increases, the WGS program is essentially 
healthy and relatively well managed. The satellites work; three of them 
are already on-orbit serving customers. These customers are gener¬ 
ally happy, which is part of the reason that the currently planned WGS 
constellation is larger than the one originally planned (more often, total 


DefenseARJ, July 2013, Vol. 20 No. 2:128-153 


143 









A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


buys decline over the life of a contract). There is no reason to expect that 
the cost of subsequent satellites after WGS 8 will increase—quite the 
contrary. Boeing's bid proposals for WGS 9 through 12 suggest that these 
variants will run $100 million less than WGS 7 did (once due account is 
taken of the baseline inflation in the satellite industry). Thus, although 
the cost increases in what should be a stable program may appear star¬ 
tling (and remain somewhat startling even after explanation), this is no 
indicator of a program facing technological or production problems that 
cannot be reasonably solved. 

The broader lesson learned for this program is that when DoD pro¬ 
curement piggybacks on a commercial base—notably the commercial 
base of a particular company—it takes a risk. The base may shrink, leav¬ 
ing it with less capacity to cover total overhead costs. Even if the base 
does not shrink, it will evolve. If DoD requirements do not evolve in par¬ 
allel—and there is no inherent reason why they should—the divergence 
between DoD's requirements and the market's requirements means 
that either the requirements are compromised (admittedly, this may be 
acceptable in some circumstances) or, eventually, such programs have 
to stand or fall on their own merit. They can no longer be free riders, so 
to speak. This suggests that a certain procurement discipline is called 
for, or DoD will pay the difference. Start-stop programs cost more than 
steady-state programs (i.e., when buys are consistent from one year to 
the next), which, in turn, are somewhat more costly than total-buy pro¬ 
grams. Although DoD cannot necessarily commit to even procurements 
for a variety of reasons (e.g., changing requirements, risk management, 
congressional politics), everyone concerned should understand that 
maximizing acquisition flexibility entails costs. 

WGS Conclusions 

Three primary factors contribute to the Nunn-McCurdy breach: an 
accounting artifact, increase in the cost of component parts, and stor¬ 
age and restart costs. Each contributes to about one-third of the cost 
increase between Block II and Ilf. An underlying factor of the increase, 
particularly with respect to the storage and restart costs, is the change 
that occurs in the commercial product base that affected the WGS costs. 
The government incurred additional costs because the commercial base 
of Boeing no longer supported the WGS. 


144 


Defense ARJ, July 2013, Vol. 20 No. 2:128-153 



Digging Out the Root Cause: Nunn-McCurdy Breaches in Major Defense Acquisition Programs 


Common Root Causes and Lessons Learned 

Table 8 displays the root causes of breaches in the six programs 
examined. It places the causes of the various program breaches in three 
categories: planning, changes in the economy, and program manage¬ 
ment. The check marks indicate either a root cause or a root cause with 
relatively greater effect in causing the program to breach. 

As can be noted, while these six programs reveal certain cost growth 
characteristics, they also reflect important differences in how and why 
cost growth occurred. This point is an important one for policymakers 
to keep in mind because they sometimes attempt to universalize policies 
as if all program cost increases stem from common causes. 


Understanding the principle that quantity change 
is rarely a governing root cause for cost growth is 
fundamental to investigating cases where quantity 
changes accompany unit cost threshold breaches. 


Table 8 indicates that quantity increases or decreases figured into 
all six of the programs listed. However, RAND's experience suggests 
that while quantity change can affect a program in important ways, such 
change is rarely the root cause of a Nunn-McCurdy breach. For example, 
the DDG-1000 program went from 10 ships to 3, which naturally raised 
the unit cost and signaled a breach. But the reason for the quantity change 
stemmed from a recognition of changes in the operational environment. 
Similarly, the increase in the Apache quantities was driven by a decision 
to procure additional helicopters for operational reasons. Understanding 
the principle that quantity change is rarely a governing root cause for cost 
growth is fundamental to investigating cases where quantity changes 
accompany unit cost threshold breaches. The RAND experience to date 
shows that although programs had associated quantity changes when 
they incurred Nunn-McCurdy breaches that triggered RCA examinations, 
in each case the quantity change was grounded in other program-specific 
factors that resulted in unit cost growth. Uncovering the grounds upon 
which quantity changes are founded is an important part of the thorough 
and insightful RCAs demanded by the WSARA. 


DefenseARJ, July 2013, Vol. 20 No. 2:128-153 


145 





A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


£ 

o 

a 

o 

i— 

</) 

o 

o 

z 

< 

on 

o 

O 

on 

a. 

LL 

o 

c/> 

LU 

(/> 

D 

< 

U 

H 

o 

o 

on 

LL 

o 

>< 

on 

< 

z 

z 

o 

t/) 

on 

2 

Z 

o 

u 

06 

ui 

00 

< 


m 

UI 

>1 

> 

re 


re 

8 

ui 


ui 


ho 


O 


01 

</> 

3 

re 

u 

o 

£ 


UI 

a 

5 


re 

2 

CQ 

>i 


§> 

01 

re 

u 












ro 

_q 

M— 

O 

CD 

A-> 

fO 

E 


CD 

"O 

c 

=) 


cn 

c 

'c 

c 

ro 








cn 

c 

3 

"O 

CD 

_c 

u 

co 

CO 

D 

O 


CD 


3 E 

E +J 
E 00 

< CD 






CO — 

c ro 
O 

u u 
> ro 
^ 3 

o C 
O ° 
CL U 


\ 


S 






s 


o o 


ro o 
CD CD 




\ 


U) 

c 


.E u 
. ro 

> s- 
_ro 

O O 
Cl o 










"O 

c 

ro 


ro 

CD ^ 
co r- 
CD 

a ' ‘ 


C 

+-» CD _ 

.1 | § 

t) E- -t-* 

.E O ro 

^ > ro 

£ £ > 

£ Q LU 


\ 


\ 






* 


c 

CD 

c 

O 

a 

E 

o 

o 


co 

CD 

U) 

C 

ro 

-C 


\ 








U CD 


146 


DefenseARJ, July 2013, Vol. 20 No. 2:128-153 


Increase in labor costs 
Discontinued/ 
decreased production of 
components 

















Digging Out the Root Cause: Nunn-McCurdy Breaches in Major Defense Acquisition Programs 


III 

> 

re 


re 

S 

ui 


i/l 


II 

v 

v 

re 


i/I 

O 

£ 


^ V 

re 

o m 

Q) £ 
ui ™ 
3 >i 

re ^ 

u 


o j£ 

tt S 


8d 

01 

re 

u 


s 


<T3 cn 

e -i 


■o o 

CD CD 
co a-> 
H3 l_ 
0) _f0 

CD q 

CD E 
Q 


u 

CD 

co 


-i— 1 CD 

ro u 


co Q- O 












CD 

CD 

C 

CD 

-C 

U 






\ 








c 

O 


- ~o 76 

^ c Lr 

^ fD D) 

~ ~ CD 


"O 

CD 

+-> 

CD 

Q. 

U 

4 -> 

c 

CD 

c 

=) 


CD 

C 


> 

CD 

O 

o «, 
£ ® 
CD co 






O ™ 

^ S2 

U CD 


a 5 
o § 

CD 


> CD 

o a 


c 

CD 


CD CD 
CD 


CD 

O 




\ 


\ 






\ 


_CD 

_q 


c 

D 

5 ^_ 

o 

CD 

4 -> 

ro 

D 

CT 

CD 

"O 

ro 

c 


_ a 




a E 


DefenseARJ, July 2013, Vol. 20 No. 2:128-153 


147 


Note. DDG = Guided Missile Destroyer; ERP = Enterprise Resource Planning; JSF = Joint Strike Fighter. 















A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


Based upon our research into the root causes of breaches of the 
programs analyzed thus far, and an examination of similarities and 
differences as reflected in Table 8, RAND offers three overarching 
recommendations: 

1. In the development of early program planning, understand 
thoroughly the implication of the testing regimes and the 
numbers of test articles required to execute those regimes. 
Planning for the testing regime and use of simulation cannot 
be overstated. As noted in previous RAND research, the F-35 
exemplified that problem (Blickstein et al., 2011, pp. 1,15-16). 

2. Clearly stipulate costing methodologies that rely on commer¬ 
cial production or even commercial production practices. 

The danger is both that necessary cost controls will not 
be implemented and that important cost analysis alterna¬ 
tives will not be recognized and used. Based on research 
conducted by RAND with the PARCA at the WGS program 
office, there does not appear to be a good understanding 
that fabricating a vehicle to be used by the military can cost 
significantly more than a commercial vehicle with an inter¬ 
national “list price.” 

3. Where a program depends upon planned product 
improvements over time, ensure a clear understanding 
of relationships among several factors, primarily time in 
inventory, ongoing research and development, and periodic 
platform upgrades or blocks through the entire out-year 
period. Failure to understand this can cause program man¬ 
agers to lose sight of program cost growth, as was the case 
with the Apache Longbow. 


148 


Defense ARJ, July 2013, Vol. 20 No. 2:128-153 



Digging Out the Root Cause: Nunn-McCurdy Breaches in Major Defense Acquisition Programs 


A ckn o wle dgm ent 

We are indebted to our RAND colleague, Martin Libicki, for his 
analysis and description of the root causes of the Nunn-McCurdy breach 
on the Wideband Global Satellite. 



DefenseARJ, July 2013, Vol. 20 No. 2:128-153 


149 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


Author Biographies 

Mr. Irv Blickstein is a senior engineer at 
the RAND Corporation. Before joining 
RAND, he served in both the Departments 
of the Navy and Defense. In 1994, he became 
the director of Acquisition Program 
Integration in the Office of the Under 
Secretary of Defense for Acquisition and 
Technology. In 1996, he returned to the 
Navy as the Assistant Deputy Chief of Naval 
Operations for Resources, Warfare 
Requirements and Assessments. Mr. 
Blickstein holds a BS in Industrial 
Engineering from Ohio State University 
and an ME A in Engineering Management 
from The George Washington University. 

(E-mail address: irving@rand.org) 

Mr. Charles Nemfakos is a senior fellow 
at the RAND Corporation, providing 
research, analyses, support, and advice to 
clients, as he did for private domestic and 
international entities in Nemfakos Partners 
LLC and Lockheed Martin Corporation. 
His service in government culminated as 
Deputy Assistant Secretary for Installations 
and Logistics, and as Deputy Under 
Secretary and Senior Civilian Official for 
Financial Management and Comptroller. 
He received four Presidential Rank Awards, 
American University's Roger W. Jones 
Award for Executive Leadership, was 
elected Fellow of the National Academy of 
Public Administration, and honored as one 
of nine Career Civilian Exemplars in the 
history of the Armed Forces. Mr. Nemfakos 
holds a BA in History from the University 
of Texas (Pan American College) and an 
MA in Government from Georgetown 
University. 

(E-mail address: nemfakos@rand.org) 




150 


Defense ARJ, July 2013, Vol. 20 No. 2:128-153 



Digging Out the Root Cause: Nunn-McCurdy Breaches in Major Defense Acquisition Programs 



Dr. Jerry M. Sollinger is currently 
a communications analyst at RAND. 
Prior to joining RAND in 1990, Dr. 
Sollinger was an Army officer, retiring 
at the rank of colonel. He served tours 
in Vietnam, Korea, Germany, and the 
United States. He is a graduate of the 
Armed Forces Staff College and the 
National War College. Dr. Sollinger 
holds a PhD in English from the 
University of Pittsburgh. 

(E-mail address: jerrys@rand.org) 


DefenseARJ, July 2013, Vol. 20 No. 2:128-153 


151 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


References 

Blickstein, I., Boito, M., Drezner, J. A., Dryden, J., Horn, K., Kallimani, J. G., 

... Wong, C. (2011). Root cause analyses of Nunn-McCurdy breaches: 
Zumwalt Class Destroyer ; Joint Strike Fighter, Longbow Apache, and 
Wideband Global Satellite (Vol. 1). Santa Monica, CA: RAND National 
Defense Research Institute. 

Block I of WGS Constellation completed. (2010, March 8). Aviation Week & 
Space Technology, 772(10), 16. 

Department of Defense Authorization Act, 1982, Pub. L. No. 97-86 (1981). 

Department of Defense Authorization Act, 1983, Pub. L. No. 97-252 (1982). 

Edwards, J. S. (2010, January 25). Military space becomes increasingly 
important as capabilities mature. Aviation Week & Space Technology, 
774(4), 122. 

Leach, B. (2003, May). Acquisition reporting overview & unit cost 

reporting/Nunn-McCurdy breaches. Office of the Under Secretary 
of Defense (Acquisition, Technology and Logistics) briefing to AT&L 
personnel (pp. 9-11). Retrieved from http://www.google.com/search 
?q=Leach%2C+B.+%282002%2C+June%29.+Acquisition+reporting+ 
overview&rls=com.microsoft:en-us:IE-Address&ie=UTF-8&oe=UTF- 
8&sourceid=ie7&rlz=1l7GGHP_enUS476 

Mecham, M. (2009, December 7). Hosted payloads growth eyed by U.S. 
satellite makers. Aviation Week & Space Technology, 777(21), 66. 

U.S. Air Force. (2007). The Air Force handbook. Retrieved from http:// 
www.fas.org/irp/agency/usaf/handbook.pdf 

U.S. General Accounting Office. (1980). Comptroller General’s report to the 
Congress: ‘SARs’ - Defense Department reports that should provide 
more information to the Congress (Report No. PSAD-80-37). Retrieved 
from http://archive.gao.gov/f0102/112285.pdf 

U.S. Government Accountability Office. (2011). High-risk series: An update 
(Report No. GAO-11-278). Washington, DC: Author. 

Weapon Systems Acquisition Reform Act of 2009,10 U.S.C., Pub. L. 111-23 
(2009). 


152 


DefenseARJ, July 2013, Vol. 20 No. 2:128-153 



Digging Out the Root Cause: Nunn-McCurdy Breaches in Major Defense Acquisition Programs 


Endnotes 

1. The SAR’s initial purpose was to act as a vehicle to keep its 
sponsor, the Assistant Secretary of Defense (Comptroller), 
apprised of the progress of selected acquisitions and to 
compare this progress with planned technical, schedule, and 
cost performance. In February 1969, the Chairman of the Senate 
Armed Services Committee asked the Secretary of Defense to 
provide status reports on major weapons systems. The parties 
agreed in April 1969 that the SAR would be the vehicle to satisfy 
the committee’s needs (U.S. General Accounting Office, 1980). 

2. RAND is a nonprofit institution whose mission is to help improve 
policy and decision making through research and analysis. The 
name is an acronym for “research and development.” 

3. The 45-day period between program manager report of a breach 
and military department secretary notification of a critical unit 
cost breach to Congress starts the day after the initial report 

of the breach to the Service Acquisition Executive. The 60-day 
period within which the Secretary of Defense must submit a 
program recertification decision to Congress starts on the day 
after the due date of the first SAR that reports the breach. 

4. Note that the original APB was $268 million (fifth row, Table 
2) per satellite, but the unit cost is now estimated to be $239 
million (fourth row, Table 2). The difference between the two is 
accounted for by the fact that other government costs ended up 
$29 million per satellite lower than estimated. 

5. Note that this 3.5 percent exceeds the 1.8 percent used as an 
overall price deflator by the Office of the Secretary of Defense to 
convert constant into current dollars. 

6. The three critical components that might otherwise go out of 
production were the Xenon Ion Propulsion System (XIPS), certain 
transponders, and a crypto box. 


DefenseARJ, July 2013, Vol. 20 No. 2:128-153 


153 



A Publication of the Defense Acquisition University 


http://www.dau.mil 


Keywords: Reuse, Legacy, Reuse Framework, 
Evaluating DoD Legacy Systems 


Reusing DoD 
Legacy Systems: 

Making the Right Choice 


i Meredith Eiband, Timothy J. Eveieigh, 
Thomas H. Holzer, and Shahryar Sarkani 


Department of Defense (DoD) programs often experi¬ 
ence cost overruns and technical difficulties due to 
reuse of legacy systems. With today’s fiscal climate of 
resource-constrained DoD budgets, reuse of legacy 
systems is frequently touted as the solution to cost, 
efficiency, and time-to-delivery problems; however, 
cost overruns and technical difficulties can significantly 
diminish any perceived benefits. Through evaluation of 
eight diverse DoD programs, this research shows that 
the state of a legacy system’s documentation, availability 
of subject matter expertise, and complexity/feasibility 
of integration are key factors that must be analyzed. 
Based on these three key factors, the authors propose 
a framework to aid both the DoD and defense contrac¬ 
tors in the evaluation of legacy systems for potential 
efficient and effective reuse. 


154 


DefenseARJ, July 2013, Vol. 20No. 2:154-173 


Image designed by Diane Fleischer » 





JVi i*' 

-- ■« «s 


















A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


Within the DoD, there is an increasing need to deliver products that 
are both technologically cutting-edge and affordable. Currently, the DoD 
budget is facing sequestration and planned reductions, cost overruns 
of DoD acquisition programs over a 7-year period were approximately 
$919 billion (Defense Business Board, 2010). At the same time, a survey 
of all DoD programs shows that the DoD acquisition life cycle, which 
begins at the identification of needs, goals, and objectives and completes 
at the disposal of the system was on average 11 years (Tomczykowski, 
2001). One potential solution to the issues of cost overruns and prolonged 
acquisition timeframes is to reuse DoD legacy systems. While this may 
seem like an ideal solution due to the legacy system being complete, 
tested and even operational, reusing legacy systems can lead to unfore¬ 
seen technical complications and financially prohibitive difficulties 
when integrating with newer technologies. A prime example of this is 
the potential to have a technological gap between the legacy system and 
the newly created system. In this instance, additional cost is frequently 
incurred when developing the solution for the systems to work in unison. 

Interestingly, even the terms “reuse” and “legacy” have multiple defini¬ 
tions depending on their source. In the software engineering domain, the 
term reuse may imply that a software product was designed as a reusable 
building block. For this study, it was imperative to derive a definition from 
established sources that did not limit the study or exclude other forms of 
reuse that are common within the DoD. Similarly, definitions of legacy 
systems abound, and often the term is used to simply describe a system 
that is considered old. However, within the DoD, the term legacy system 
has a much more specific meaning. In the DoD context, a legacy system's 
age does not distinguish it as legacy, but merely denotes that the system 
is one in which DoD has a substantial investment of both time and money 
(Defense Acquisition University [DAU], 2009). To investigate this topic, 
the authors used the following taxonomy of terms: 

• Reuse - the integration of an already developed part (e.g., 
engine), product (e.g., inventory database), or legacy system 
(e.g., telemetry processing system) into another context or 
component. 

• Legacy System - “a system or application in which an orga¬ 
nization has already invested considerable time and money” 
(DAU, 2009). 


156 


DefenseARJ, July 2013, Vol. 20 No. 2:154-173 



Reusing DoD Legacy Systems: Making the Right Choice 


Despite the challenges involved in the reuse of legacy systems, defense 
contractors, whether required by contract or by design, are regularly 
agreeing to do so as a cost and schedule mitigation strategy without either 
the U.S. Government or the defense contractor fully analyzing what the 
effect of reuse may actually be on the cost, schedule, risk, and perfor¬ 
mance of the product life cycle (General Accounting Office [GAO], 1993). 
In some cases, reuse of a legacy system provides an affordable and efficient 
alternative to a newly developed system, as in the KC-135 Stratotanker. 
In this case, the original fleet has been updated, retrofitted, and modified 
over 12 times in the last 50 years, each time saving the DoD the estimated 
$40 billion cost of developing a new aircraft for a similar purpose (GAO, 
2004; Clark, 2010). On the other hand, the Navy Marine Corps Intranet 
(NMCI) contract was grossly underestimated by both the prime contrac¬ 
tor and the U.S. Government, and as of 2007, the prime contractor had 
lost $3 billion (Jordan, 2007). One of the many continuing struggles on 
the NMCI program is the incorporation of tens of thousands of different 
legacy software versions and applications into a common operating envi¬ 
ronment (Jordan, 2007). Given the valuable lessons observed (and maybe 
learned) from these and many other programs, what factors are critical 
to consider before deciding to reuse a legacy system? 


Reusing legacy systems can lead to unforeseen 
technical complications and financially prohibitive 
difficulties when integrating with newer technologies. 


Research into the application of software reuse is plentiful, and gen¬ 
erally falls into three common themes: theoretical work, cost impacts, 
and software tools used to aid in the reuse process. In the area of theoret¬ 
ical work, researchers have developed software legacy and reuse-based 
acquisition life-cycle frameworks (Ahrens & Prywes, 1995), described 
the causes of technological uncertainty (Fleming, 2001), discussed 
implementing design reuse (Gil & Beckman, 2007), formalized reuse 
processes (Redwine & Riddle, 1989), defined strategies for reuse (Frakes 
& Terry, 1996), and created a better reuse design based on knowledge 
management techniques (Hicks, Culley, Allen, & Mullineux, 2002). The 
literature surrounding the cost and economic impacts of reuse include 
works tying cost to software development (Wang, Valerdi, & Fortune, 
2010), updating software cost models for current issues (Boehm et al., 


DefenseARJ, July 2013, Vol. 20No. 2:154-173 


157 





A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


2000), and evaluating the impacts of the cost of software reuse (Boehm, 
1981; Gaffney & Durek, 1989). Perhaps the most expansive studies in 
reuse are derived from the creation of software tools and applications. 
Examinations in this area include work in evaluating reuse through a 
total system approach (Kim & Stohr, 1998; Mili, Mili, & Mili, 1995; Isoda, 
1995) and exploring reuse abstraction (Freeman, 1983). 

Regardless of all of the theoretical work, tools, and cost models 
available, one key area remains inadequately researched: how program 
managers should determine whether or not they will efficiently and 
effectively reuse hardware and software legacy systems based on cost, 
schedule, risk, operations and maintenance (O&M), and performance. To 
investigate this, an interpretive case study approach was used to evaluate 
a group of DoD programs to accomplish three objectives: 

• Identify the key factors decision makers need to consider 
when determining whether or not to reuse legacy systems. 

• Determine how often the key factors have an impact on 
studied programs and what preventative measures could be 
applied to diminish unsuccessful reuse of legacy systems. 

• Create a framework of imperative questions and quantifi¬ 
able answers that can improve the decision maker's ability 
to pinpoint which legacy system opportunities for reuse 
stand the greatest chance of success. 



158 


DefenseARJ, July 2013, Vol. 20 No. 2:154-173 




Reusing DoD Legacy Systems: Making the Right Choice 


Identifying Key Factors in 
Reusing Legacy Systems 

Eight existing DoD programs spanning the areas of aircraft, infor¬ 
mation technology, systems of systems, communications, satellites, and 
facilities were used as a sample of DoD program domains where reuse 
of legacy systems exists. Programs were delineated by their capacity to 
successfully reuse DoD legacy systems. For this study, successful reuse 
was based on each program's capacity to reuse a legacy system within the 
projected cost, schedule, risk, and performance baselines. Data—includ¬ 
ing GAO reports; program-specific lessons observed; and independent 
third-party analyses that explored cost, schedule, risk, performance, and 
O&M impacts—were used to determine the success or failure of legacy 
system reuse. Additionally, data were analyzed to ascertain the funda¬ 
mental reasons that cost, schedule, or risk increased on the program. 

To substantiate the findings of this study, the collected data were 
then validated by experts in the field of systems engineering who were 
familiar with the programs selected. Data were also controlled for factors 
that were outside the control of either the DoD or the defense contractors. 
For example, six of the programs studied have acquisition life cycles of 
10 years or more, and thus were more susceptible to volatility in their 
budgets. Since budget fluctuations are often out of the control of both 
the DoD as a whole and defense contractors in particular, any results 
that were directly influenced by these types of external causes were not 
included in the final analysis. 

Upon initial review of the eight programs, three recurring factors 
were found when programs were unsuccessful in reusing legacy systems: 

• Substandard, inadequate, or nonexistent systems engineer¬ 
ing documentation including: requirements, architectures, 
statements of work, work breakdown structures, concept 
of operations, test documentation, and standard operating 
procedures. 

• Insufficient subject matter expertise including: inadequate 
identification of current users of the legacy system, little or 
no accounting for training existing employees on the system, 


DefenseARJ, July 2013, Vol. 20No. 2:154-173 


159 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


and assuming experts within a specific field should be able 
to operate the system simply because of the interrelation¬ 
ship between the newer system and the older legacy system. 

• Inadequate analysis of the cost, schedule, and risk of 
integrating a legacy system including: incompatible tech¬ 
nologies, inadequate security postures of the legacy systems 
against the current security landscape, substandard pro¬ 
cessing of data after integration, and creation of additional 
systems or functions to connect the new pieces of the sys¬ 
tem to the legacy system. 

Conversely, for the programs that successfully reused legacy sys¬ 
tems, these factors were either addressed early in the program life cycle 
or accounted for in the reengineering work associated with the program. 
To fully validate the dominance of these key factors, the eight programs 
and their software and hardware projects were evaluated against the 
following three hypotheses: 

• Hypothesis 1: Decision makers overestimate the quantity 
and quality of legacy system documentation available. 

• Hypothesis 2: Decision makers underestimate the criticality 
of legacy system subject matter expertise. 

• Hypothesis 3: Decision makers underestimate the time, 
cost, and feasibility of the integration phase. 

Frequency of Factors 

Of the eight programs analyzed, six of the programs overestimated 
the quantity and quality of legacy system documentation (Figure 1). 
When unsuccessful programs did have documentation, the quality of the 
documentation did not meet the industry or military standard, and thus 
required additional effort to meet these standards. Similarly, a different 
set of six of the programs also overestimated the cost and time to deliv¬ 
ery of integrating new technology with the applicable legacy systems, 
while another set of five program decision makers underestimated the 
criticality of legacy system subject matter expertise. Unsuccessful pro¬ 
gram teams that did not understand the importance of subject matter 
expertise often employed personnel who were experts in a specific field 


160 


DefenseARJ, July 2013, Vol. 20 No. 2:154-173 



Reusing DoD Legacy Systems: Making the Right Choice 


related to the legacy system, but who had never worked on that system 
specifically. In this situation, all five of the program managers quickly 
exhausted the budget and schedule resources allocated for training their 
staffs, and even then fell short in the area of system knowledge. 


FIGURE 1. FREQUENCY OF KEY FACTORS 


Number of Programs that 
Overestimate the Quality and 
Quantity of Documentation 



Number of Programs that 
Underestimate Subject 
Matter Expertise 



Number of Programs that 
Underestimate the Cost 
of Integration 



■ 6 of 8 Programs ■ 6 of 8 Programs ■ 5 of 8 Programs 

■ 2 of 8 Programs ■ 2 of 8 Programs ■ 3 of 8 Programs 


Quality and Quantity of Documentation 

Data analysis shows that decision makers overestimate the quantity 
and quality of on-hand legacy hardware and software system documen¬ 
tation 72.7 percent of the time. In fact, three of the programs studied 
had little to no requirements, architecture, statement of work, or work 
breakdown structure artifacts for the legacy systems involved. Due to 
ever-evolving military and industry standards for documentation within 
the systems and software engineering fields, gaps often exist in quality 
and quantity of requirements, operational concepts, and legacy archi¬ 
tecture documentation, which must be understood prior to beginning 
the program life cycle. These artifacts frequently must be redocumented 
to current standards during the initial phase of the program to satisfy 
contractual deliverables. If the level of redocumentation is not bid into 
the contract, the unscoped efforts directly impact program planning, 
resources, and performance. This leads to higher risk, additional costs 
for either the DoD or defense contractor depending on the contract vehi¬ 
cle type, and the possibility of schedule impacts, thus having a negative 
impact on successful legacy system reuse. 

This conclusion held true for 100 percent of programs that reused 
legacy software systems, but it only held true for 50 percent of programs 
that reused hardware components (Figure 2). Since the relative age of the 
software engineering field is less than that of the hardware engineering 
field, these results are not entirely surprising. The software engineering 


DefenseARJ, July 2013, Vol. 20No. 2:154-173 


161 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


field is still maturing in the frameworks used to apply it, which includes 
how systems are documented and to what degree. The initial results indi¬ 
cate that while reviewing and obtaining documentation is a challenge 
for the vast majority of programs regardless of the hardware or software 
system being built, added diligence is warranted when reusing DoD 
legacy systems. Based on these findings, the hypothesis that decision 
makers overestimate the quality and quantity of legacy documentation 
is supportable. 

FIGURE 2. COMPARISON OF HARDWARE AND SOFTWARE 

PROJECTS 

100 % 

90% 

80% 

70% 

60% 

50% 

40% 

30% 

20 % 

10 % 

0 % 

Hypothesis 1 Hypothesis 2 Hypothesis 3 

Documentation Subject Matter Integration 
Expertise 

Subject Matter Expertise 

The analysis illustrates that decision makers do in fact underesti¬ 
mate the importance of having the correct subject matter expert at the 
right time for the program 62.5 percent of the time on both software and 
hardware projects. When proper subject matter experts are not engaged, 
knowledge recovery becomes a critical endeavor in understanding the 
legacy system. In particular, knowledge recovery activities included 
legacy system training, additional documentation of system operating 
procedures, and specialized use cases. During the planning phase, four of 
the programs studied had unquantified knowledge recovery efforts. Data 
show this added time to schedules and raised the cost of the program 
while inexperienced employees were brought up to speed. Programs 
on which underestimation of subject matter expertise occurred shared 
the common problem of hiring experts in a field of study that includes 
the legacy system, while assuming that the experts could immediately 
begin working on that system. The field subject matter experts were often 



162 


DefenseARJ, July 2013, Vol. 20 No. 2:154-173 














































Reusing DoD Legacy Systems: Making the Right Choice 


very knowledgeable, but did not have the specific knowledge that comes 
from working directly on or with the legacy system, which increased the 
learning curve and, subsequently, program cost and schedule. Similarly, 
hiring additional manpower after these realities were established typi¬ 
cally occurred too late to effectively mitigate their impacts. 

When obtaining experts for hardware and software components, 
a common problem is the complex and unique nature of the DoD appli¬ 
cation of a given capability. The DoD regularly pushes the bounds of 
common hardware and software tools by using commercial equipment 
that is often designed for smaller and less intricate applications. In 
these cases, expertise is imperative, as even the experts in the field are 
challenged by the application of a legacy system. To lessen the effects of 
inexperienced staff, the DoD and defense contractors should determine 
the complexity of the legacy system and what, if any, legacy experts 
should be employed on the program to ensure successful delivery. 

Of the programs that reused legacy software systems, 80 percent 
underestimated the importance of this factor, while this was only true 
of 50 percent of hardware programs (Figure 2). This result emphasizes 
the importance of subject matter expertise to both hardware and par¬ 
ticularly software programs. Since legacy DoD software can be unique, 
special attention should be paid to hiring staff with particular expertise 
for the given legacy system. These results show that underestimation in 
this area can significantly degrade the success of reuse. 

Feasibility of Integration 

Data show that decision makers on the programs in the analysis 
underestimated the time and cost of integrating with legacy systems 
72.7 percent of the time. For example, on one program in the study, the 
original bid included an assumption that the outdated software code 
could be converted and ported to new hardware to reduce the cost of 
purchasing or developing completely new software. Despite careful 
analysis and hiring subcontractors specializing in performing this 
task, the integration failed. Further, the schedule was impacted, and the 
subcontractor was still performing work to create a usable system at the 
time the data were collected. With work still being performed on this 
system, the benefit of reusing the legacy software cannot be calculated, 
but it is likely that this rework activity will add to the schedule, cost, 
and risk of the program. In the previous example and others like it, the 
program risk profile will be increased, and the probability of impacts to 


DefenseARJ, July 2013, Vol. 20No. 2:154-173 


163 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


both cost and schedule is greater without active risk mitigation strate¬ 
gies in place (Bennett, 1995). As identified by Orrego and Mundy (2007), 
there is little research into the level of risk impact when reusing systems. 
Because of this, an opportunity exists for future research in this area to 
further refine the decision-making process for reuse and develop risk 
mitigation techniques that program managers can leverage to better 
manage reuse-related technical risks. 

Security implications must also be considered in any analysis of a 
legacy system's integration potential. The rapid pace of change in today's 
security environment will likely necessitate significant penetration 
testing, security scanning, and hardening to identify vulnerabilities 
and retrofit any legacy system to meet current DoD and industry stan¬ 
dards. Additionally, the cost and risk of reusing hardware or software 
in a classified environment can increase the complexity of integration. 
Intensive systems engineering and security architecture analysis will 
likely be required to ensure that classified data security is not put at risk 
due to latent vulnerabilities that may be exposed when integrating with 
a legacy system. As observed on one of the programs studied, underes¬ 
timation of these efforts at the beginning of a project drives significant 
unplanned investments later in legacy system reuse projects—even if 
only to navigate the complex government processes required to pursue 
waivers or deviations for any vulnerabilities that cannot be overcome 
without prohibitively high additional costs (Jordan, 2007). 

Legacy System Reuse Framework 

Documentation, subject matter expertise, and feasibility of integra¬ 
tion were all found to impact legacy reuse success individually, but they 
were consistently found to overlap with compounding effects (Figure 
3). On programs with little documentation, 87.5 percent of the programs 
underestimated the criticality of obtaining the correct experts at the 
proper time, and this directly impacted the time and cost required for 
integration. An additional finding shows that there may be a relationship 
between the age of a legacy system and the feasibility of its reuse due to a 
confluence of the factors discussed here. Data show that programs that 
reuse increasingly older legacy systems had not only larger documenta¬ 
tion gaps, but also difficulty bridging the technological divide between 
the new and old parts of the system. All five of the software programs 
had documentation gaps, but of those systems, the two attempting to 
use software systems older than 10 years had virtually no requirements, 
architecture, or operational concept documentation to leverage. Given 


164 


DefenseARJ, July 2013, Vol. 20 No. 2:154-173 



Reusing DoD Legacy Systems: Making the Right Choice 


the clear impact that this has on subject matter expertise retention and 
integration facilitation, this demonstrates that there may, in fact, be 
a tipping point at which a legacy system's age directly determines the 
feasibility, or lack thereof, of reusing that system. 

FIGURE 3. KEY FACTORS IN REUSE 

Key Documentation Questions 

• What types of documentation exist? 

• How much documentation exists for each type? 

• If documentation does not exist or is deemed 
insufficient for current needs, what reengineering 
efforts must be done to understand and document the 
system moving forward? 


Key Subject Matter Expertise 
Questions 

What experts are necessary? 
Do experts have experience 
working on or with the legacy 
system? 

If no experts exist, what and 
how much training is required 
for current team members? 



Key Feasibility of Integration 
Questions 

Are there technological gaps 
that exist? 

How broad are the 
technological gaps? 

If there are technological 
gaps, is there a path forward 
that can enable integration? 


In these cases, by the time problems are identified, efforts must be 
made to provide proof of why the legacy system is not suitable, thereby 
adding yet more cost to the effort. At the point that a subsequent judg¬ 
ment on suitability is rendered, pursuit of a more optimal solution may 
no longer be an option. Due to the investment and development already 
done on a reuse-based system, creation of an optimal, new solution may 
be outside of the acceptable cost for the final product. Interrelation of 
these factors necessitates their consideration individually and collec¬ 
tively to properly assess areas of compounding risk. 

Within each of the key reuse factors shown in Figure 3, imperative 
questions should be answered prior to the decision point of determining 
whether to reuse a legacy system (Table). These questions were devel¬ 
oped by isolating the problem areas identified from the research that 
contributed to cost and schedule impacts on each program. Similarly, 
programs that were successful were analyzed for mitigation strate¬ 
gies applied. Based on this analysis, a question-based framework was 
developed, and standard quantification methods were applied to each 
area. Program decision makers employing this methodology will need 


DefenseARJ, July 2013, Vol. 20No. 2:154-173 


165 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


TABLE. REUSE EVALUATION FRAMEWORK 


Reuse Factors Key Questions for Analysis Quantification Method 


Documentation • What types of documentation exist? 
° Operational Concept 

- Use cases 

° Requirements 
° Architecture 

- Functional Flow Diagrams 

- Activity Diagrams 

- Block Definition Diagrams 
° Work Breakdown Structure 
° Design 

- Software Design Documents 

- Hardware Design Documents 

- Interface Control Documents 
° Test 

- System Acceptance Test 
Plans/Results 

- System Integration Test 
Plans/Results 

- Security Test and Evaluation 
Plans/Results 

° Security Analysis 

- System Security Plan 
° System Operations and 

Maintenance Procedures 
° Industry or Military Standards 

• How much documentation exists for 
each type? 

• If documentation does not exist or 
is deemed insufficient for current 
needs, what reengineering efforts 
must be done to understand and 
document the system moving 
forward? 

° What are the contract line item 
deliverables? 

° Are there documents that are 
necessary, but not listed in the 
contract line item deliverables? 


Cost - based on prior 
documentation or 
redocumentation efforts 
Schedule - based 
on prior basis of 
estimates for length 
of documentation 
activities 
Risk - based on 
risk assessment 
of documentation 
availability 


166 


DefenseARJ, July 2013, Vol. 20 No. 2:154-173 






Reusing DoD Legacy Systems: Making the Right Choice 


Reuse Factors 

Key Questions for Analysis 

Quantification Method 

Subject Matter 

• What experts are necessary to 

• Cost - based on training 

Expertise 

understand the legacy system? 

and personnel hours 


° Are there experts within the DoD 

• Schedule - based on 


or within industry? 

training efforts and 


° Will the contractor need 

transition period 


assistance in locating experts if 

• Risk - based on risk 


they reside within the DoD? 

assessment of subject 


• Do they have experience working on 

matter expertise 


or with the legacy system? 

• If no experts exist, what training and 
how much training is required for 

current team members? 

° Is there a similar system where 
experts may have overlapping 

skills? 

availability 

Feasibility of 

• Are there technological gaps that 

• Cost - if the legacy 

Integration 

exist? 

technology can be 


° Compatibility of legacy software 

integrated 


and/or hardware with the new 

• Schedule - if the legacy 


system 

technology can be 


° Data transfers and protocol 

integrated 


° Performance requirements in the 

• Risk - based on 


new environment 

risk assessment of 


° Platform differences 

technological gaps 


° Security standards and 

and cost and schedule 


accreditation 

• How broad are the technological 
gaps? 

° Would a technical solution be 

more difficult to implement than 
selecting nonlegacy hardware or 

software? 

• If there are technological gaps, 
is there a path forward that can 
enable integration? 

° Is there a common technical 

solution, how often is it used, and 

with what results? 

flexibility 


DefenseARJ, July 2013, Vol. 20No. 2:154-173 


167 







A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


to collect and apply their own program-specific data to feed the frame¬ 
work. In turn, a determination on a legacy system's candidacy for reuse 
success may be more easily obtained. The framework can be used to 
augment these and other traditional analysis methods, thereby allowing 
decision makers to bring the frequently overlooked or underestimated 
legacy system factors into the decision-making process. 

Based on the answers to the questions outlined in the framework, 
the decision maker can associate cost, schedule, and risk with any 
redocumentation effort. These quantification methods should be based 
on historical data collected and applied for analogous proposal activi¬ 
ties. Similarly, cost, schedule, and risk can be associated with subject 
matter expertise. Feasibility of integration can be linked with risk, cost, 
and schedule; and if there are technological gaps that can be solved, the 
program can associate cost and schedule impacts. If a technological 
gap cannot be reasonably overcome, the program manager should not 
reuse the legacy system and instead begin work to identify alternative 
solutions. By utilizing these measurements, program managers can 
make an informed and grounded estimation of the costs, schedule, risk, 
performance, and O&M needed to successfully reuse the legacy system. 

Conclusions 

Reuse of DoD legacy systems is a tempting enterprise for both the 
DoD and defense contractors, but the perceived value of reusing a legacy 
system is often outweighed by the very real technical difficulties and 
costs associated with doing so. 

With improved upfront analysis, a smarter application of reuse can 
play an important role in diminishing time to market and affordability 
initiatives. However, early analysis is rarely done. Despite the fact that 
two of the programs within this study were able to successfully reuse 
legacy systems, the overall findings suggest that the decision to do so 
is not being assessed properly on these programs, particularly since 
no reuse analysis was performed prior to the decision to go forward. 
In fact, all five of the program managers who reused software in this 
study overestimated the quality and quantity of documentation needed 
as well as the feasibility of integration; and 80 percent of the program 
managers who reused software underestimated the criticality of legacy 
system subject matter expertise. While legacy hardware reuse was more 
successful, 50 percent of these programs also succumbed to improper 


168 


DefenseARJ, July 2013, Vol. 20 No. 2:154-173 



Reusing DoD Legacy Systems: Making the Right Choice 


estimation of the key factors outlined here. With so many unaccounted 
activities, program managers—not surprisingly—will see overruns in 
cost and schedule on programs where legacy system reuse is attempted. 


Reuse of DoD legacy systems is a tempting 
enterprise for both the DoD and defense contractors, 
but the perceived value of reusing a legacy system 
is often outweighed by the very real technical 
difficulties and costs associated with doing so. 


These findings underscore the necessity of utilizing a framework 
to quantitatively evaluate legacy systems prior to the decision to reuse 
them. Both the DoD and defense contractors can benefit from applica¬ 
tion of this framework. Contractors can use it to justify the inclusion 
of reuse in a proposed solution, or alternatively to justify higher initial 
costs to perform ground-up development and avoid reuse altogether. The 
DoD can additionally leverage this framework to perform an indepen¬ 
dent analysis of contractor bids and ensure that reuse feasibility was 
adequately evaluated by each contractor. All too frequently, proposals 
including reuse in the solution space are enticing because of their lower 
cost estimates and other perceived benefits, but when these benefits fail 
to materialize, the damage is already done. Since no two programs are 
alike, applying this framework in conjunction with developing a compre¬ 
hensive risk profile and performing a cost-benefit analysis will provide 
a more complete examination of reuse potential. A combination of these 
techniques to perform such analyses could also be a valuable subject for 
future research. 

Of importance to note is that even if the cost of reusing a legacy 
system is more than what was budgeted, reusing the legacy system may 
still be a more efficient and effective alternative in terms of cost, sched¬ 
ule, performance, and risk than building an entirely new system. In this 
instance, the framework should be used to aid in better cost estimation 
during the discovery and contracts phase of the acquisition life cycle. 
The results of such rigor would benefit both the DoD and defense contrac¬ 
tors alike. Unless an analysis is performed, the implications of reusing a 
legacy system are entirely unknown. 


DefenseARJ, July 2013, Vol. 20No. 2:154-173 


169 





A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


Author Biographies 



Dr. Meredith Eiband is a systems engineer 
at Lockheed Martin. Her career includes 
extensive experience working with the United 
States Army, Navy, Air Force, Missile Defense 
Agency, and DoD on a variety of technical 
projects. She holds two U.S. patents; a BA in 
Business Administration from Trinity 
University in San Antonio, Texas; and an MS 
and a PhD in Systems Engineering from The 
George Washington University (GWU). 

(E-mail address: meredith.eiband@imco.com) 


Dr. Timothy J. Eveleigh is an adjunct pro¬ 
fessor at GWU and an International Council 
on Systems Engineering (INCOSE) Certified 
Systems Engineering Professional. Dr. 
Eveleigh has over 30 years' industry experi¬ 
ence working in such diverse areas as DoD 
and intelligence community information 
technology (IT) acquisition challenges, 
research and development, enterprise archi¬ 
tecting, and IT governance. Dr. Eveleigh holds 
a DSc in Systems Engineering from GWU 
and an MS in Remote Sensing/Physical 
Geography from the University of Delaware. 

(E-mail address: eveleigh@gwu.edu) 


170 


DefenseARJ, July 2013, Vol. 20 No. 2:154-173 



Reusing DoD Legacy Systems: Making the Right Choice 




Dr. Thomas H. Holzer is an adjunct profes- 
sor at GWU. He is the former director, 
Engineering Management Office, Enterprise 
Operations Directorate, National Geospatial- 
Intelligence Agency. He has over 35 years' 
experience in life-cycle systems engineering, 
leading large-scale IT programs and process 
improvement initiatives. Dr. Holzer holds a 
DSc and an MS in Engineering Management 
from GWU, and a BS in Mechanical 
Engineering from the University of Cincinnati. 

(E-mail address: holzert@gwu.edu) 

Dr. Shahryar Sarkani is an adjunct profes- 
sor in the Department of Engineering 
Management and Systems Engineering at 
GWU. He has over 20 years of experience in 
the field of software engineering, focusing on 
architecture and design. Dr. Sarkani holds a 
DSc in Systems Engineering from GWU, an 
MS in Mathematics from University of New 
Orleans, and a BS in Electrical Engineering 
from Louisiana State University. 

(E-mail address: emseor2003@yahoo.com) 



DefenseARJ, July 2013, Vol. 20No. 2:154-173 


171 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


References 

Ahrens, J. D., & Prywes, N. S. (1995). Transition to a legacy- and reuse-based 
software life cycle. Computer, 28(10), 27-36. doi: 10.1109/2.467576 

Bennett, K. (1995). Legacy systems: Coping with success. IEEE Software, 
72(10), 19-23. 

Boehm, B. W. (1981). Software engineering economics. Upper Saddle River, 

NJ: Prentice-Hall. 

Boehm, B., Abts, C., Brown, A. W., Chulani, S., Clark, B., Horowitz, E., ... Steece, 
B. (2000). Software cost estimation with COCOMO //. Upper Saddle 
River, NJ: Prentice-Hall. 

Clark, C. (2010, July 9). Boeing touts KC-X cost, jobs. DoD Buzz [Online 

journal]. Retrieved from http://www.dodbuzz.com/2010/07/09/boeing- 
touts-kc-x-cost-jobs/ 

Defense Acquisition University. (2009, April 21). Legacy systems [Online 

forum comment]. Defense Acquisition University “Ask A Professor” Web 
site. Retrieved from https://dap.dau.mil/aap/pages/qdetails.aspx7cgiSub 
jectArealD=1&cgiQuestionlD=28188 

Defense Business Board. (2010). Best business practices for fixed-price 
contracting (Report No. FY 10-03). Washington, DC: Author. 

Fleming, L. (2001). Recombinant uncertainty in technological search. 
Management Science, 47(1), 117-132. 

Frakes, W., & Terry, C. (1996). Software reuse: Metrics and models. Computing 
Surveys (CSUR), 28(2), 415-435. 

Freeman, P. (1983). Reusable software engineering: Concepts and research 
directions. ITT Proceedings of the Workshop on Reusability in 
Programming (pp. 129-137). New York, NY: ITT Programming. 

Gaffney, J. E., & Durek, T. A. (1989). Software reuse—Key to enhanced 
productivity: Some quantitative models. Information and Software 
Technology, J7(5), 258-267. 

General Accounting Office. (1993). Software reuse: Major issues need 
to be resolved before benefits can be achieved (Report No. GAO/ 
IMTEC-93-16). Washington, DC: Author. 

General Accounting Office. (2004). Military aircraft: DoD needs to determine 
its aerial refueling aircraft requirements (Report No. GAO-04-349). 
Washington, DC: Author. 

Gil, N., & Beckman, S. (2007). Design reuse and buffers in high-tech 

infrastructure development: A stakeholder perspective. IEEE Transaction 
Engineering Management, 54(3), 484-497. 

Hicks, B. J., Culley, S. J., Allen, R. D., & Mullineux, G. (2002). A framework 
for the requirements of capturing, storing and reusing information and 
knowledge in engineering design. International Journal of Information 
Management, 22(4), 263-280. 

Isoda, S. (1995). Experiences of a software reuse project. Journal of Systems 
and Software, 30(3), 171-186. 

Jordan, K. (2007). The NMCi experience and lessons learned: The 

consolidation of networks by outsourcing. Washington, DC: Center for 
Technology and National Security Policy. 


172 


DefenseARJ, July 2013, Vol. 20 No. 2:154-173 



Reusing DoD Legacy Systems: Making the Right Choice 


Kim, Y., & Stohr, E. A. (1998). Software reuse: Survey and research directions. 

Journal of Management Information Systems, 14(4), 612-623. 

Mili, A., Mili, F., & Mili, H. (1995). Reusing software: Issues and research 

directions. IEEE Transactions of Software Engineering, 27(6), 528-562. 
Orrego, A. S., & Mundy, G. E. (2007). A study of software reuse in NASA 

legacy systems. Innovations in Systems and Software Engineering, J(3), 
167-180. doi: 10.1007/s11334-007-0027-y 
Redwine, S. S., & Riddle, W. E. (1989). Software reuse processes. Proceedings 
from ISPW ‘88: 4th International Software Process Workshop on 
Representing and Enacting the Software Process (pp. 133-135). doi: 
10.1145/75111.75135 

Tomczykowski, W. (2001). DMSMS acquisition guidelines: Implementing 
parts obsolescence management contractual requirements (Rev. 3.0). 
Sacramento, CA: Defense Microelectronics Activity. 

Wang, G., Valerdi, R., & Fortune, J. (2010). Reuse in systems engineering. IEEE 
Systems Journal, 4(3), 376-384. doi: 10.1109/JSYST.2010.2051748 



DefenseARJ, July 2013, Vol. 20No. 2:154-173 


173 















A Publication of the Defense Acquisition University 


http://www.dau.mil 


Keywords: Contracting, Economic Price Adjustment 
(EPA) C/auses, Risk, Target Fee 


Valuing the Cost of an 
Economic Price Adjustment 
Clause to the Government 

( Scot Arnold, Bruce Harmon, Susan Rose, 
and John Whitley 


An Economic Price Adjustment (EPA) clause in a 
contract allows for adjustment of contract price if certain 
conditions are met. The Department of Defense (DoD) 
often uses an EPA clause in contracts where there is 
an increased risk that the costs of inputs used by the 
contractor will diverge from the forecasts used in the 
original pricing of the contract. EPA clauses transfer 
risk from the contractor to the government; thus, they 
are of economic value to the contractor. This article 
reviews EPA clauses, analyzes the value of risk transfer, 
and discusses how DoD could account for this value in 
negotiating fees for contracts that contain EPA clauses. 
Other government costs and risks associated with EPA 
clauses are also discussed. 


174 


DefenseARJ, July 2013, Vol. 20 No. 2:174-193 


Image designed by Diane Fleischer » 









A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


An Economic Price Adjustment (EPA) clause in a contract allows for 
adjustment of contract price if certain conditions are met. The Federal 
Acquisition Regulation (FAR) (2005) permits use of an EPA clause when 
“there is serious doubt concerning the stability of market or labor condi¬ 
tions that will exist during an extended period of contract performance” 1 
The DoD uses EPA clauses in areas like multiyear procurement (MYP) 
contracts; for example, recent C-17, F/A-18 E/F, and AH-64D Apache 
Longbow MYP contracts all contained EPA clauses covering certain 
labor costs and contracts for highly volatile commodities, e.g., fuel. 

EPA clauses transfer risk from the contractor to the government; 
thus, they are of economic value to the contractor. For example, a con¬ 
tractor may be able to get better financing terms for a project, given the 
contractor's lower risk exposure. In other areas of government contract¬ 
ing, hedging a contractor's risk is grounds for adjusting the target fee 
used in establishing contract price. For example, the Defense Federal 
Acquisition Regulation Supplement (DFARS) recommends using a 
higher target fee in a Firm Fixed Price (FFP) contract than in a Fixed 
Price Incentive Firm (FPIF) contract, where more risk is shared with 
the government. But, the weighted guidelines method contained in the 
DFARS does not clearly address how to adjust target fee 2 when an EPA 
clause is used. 3 After a brief review on the background of EPA clauses, 
this article analyzes the potential value of EPA clauses and discusses 
how this value could be taken into account in negotiating a contract. 



176 


DefenseARJ, July 2013, Vol. 20 No. 2:174-193 



Valuing the Cost of an Economic Price Adjustment Clause to the Government 


Background 

A fixed price contract commits the contractor to absorb the cost risk 
associated with providing the agreed-upon product or service. Cost risk 
can result from unexpected changes in input prices, unfavorable changes 
in a manufacturing process, labor strikes that shut down production, or 
other unforeseen events. This works both ways for contractors. If they 
are not able to control costs, they are exposed to losses; if they are able 
to control and reduce costs, they retain the higher profit. 

Different types of contracts distribute risk between contracting 
parties in various ways. An FPIF requires the contractor to share cost 
changes from a negotiated target while an FFP contract puts all of the 
cost risk on the contractor. EPA clauses place the inflation risk for cer¬ 
tain elements of cost—e.g., steel, titanium, labor, or a combination of cost 
elements—with the government. 

The typical EPA clause specifies “[adjustments based on cost indexes 
of labor or material” (FAR, 2005). The indexes chosen should be corre¬ 
lated with the cost elements at risk, but should also be broad enough to 
be outside of the contractor's control (DFARS Procedures, Guidance, 
and Information, 2012). Most EPAs are written with symmetry between 
upward and downward price adjustments. However, contractors who 
have the greatest exposure to upward pressure on input costs will more 
likely prefer an EPA clause. An EPA clause would be disadvantageous to 
those expecting a decrease in input prices (which would normally lead to 
higher profits); to the degree that contractors can influence whether an 
EPA clause is included, this would result in a higher incidence of upward 
price adjustments. 4 

In addition to the intended transfer of risk for particular labor or 
material inputs, EPA clauses can entail unintended risks from such 
things as poorly chosen indexes and strategic behavior driven by the 
existence of the EPA clause. The choice of the price index is important. 
Researchers of past studies have found difficulties in the application of 
EPA clauses. In some cases, the EPA clause was linked to price changes 
that were not sufficiently coupled to the actual underlying inputs to the 
contract that established the need for the clause. We refer to this as 
“basis risk.” For example, Keating, Murphy, Schank, and Birkler (2008) 
found that the Steel Vessel Index, constructed in the 1950s to track the 
prices of common materials used in ships, was not representative of 


DefenseARJ, July 2013, Vol. 20No. 2:174-193 


177 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


modern naval vessels and had been substantially more volatile than the 
prices of common input materials for modern naval ships. To overcome 
this inaccuracy, several ship program managers have created their own 
materials cost indexes. 

The DFARS directs that the costs subject to the EPA be fixed at 
contract start, including the proportions of labor and material, and 
their allocation across time. This is intended to limit the contractor's 
ability to shift resources and “game” the EPA clause once a contract has 
started. Choice of index is also important in limiting gaming of an EPA 
clause, as some indexes that have been used could allow the contractors' 
actions to affect index values. For example, in the first F/A-18E/F MYP 
contract, an index based on the contractor's actual labor rates was used. 5 
The Department of Defense Inspector General (DoDIG, 2008) found that 
Boeing's prefunding of pension liabilities directly affected the Bureau of 
Labor Statistics' aircraft industry labor compensation index, which was 
used in calculating EPAs for three Boeing contracts. These unintended 
risks may result in payments to the contractors that otherwise would 
not have occurred. Updates to the DFARS and improvements in govern¬ 
ment/industry practice have better regulated these issues; the F-22 MYP 
contract includes a good example of a well-written EPA clause. In this 
case, the portions of contract cost affected were narrowly defined, and 
the labor indexes specified used a broad formulation for fringe benefits. 
However, given imperfect information and the limitations of available 
indexes, the possibility of using an inappropriate index remains. 

EPA-like clauses are also used to mitigate risks in commercial, 
long-term supply agreements in capital-intensive industries (Goldberg 
& Erickson, 1987). A common objective of these agreements in com¬ 
mercial transactions is to stabilize supply availability; the purpose of 
the EPA-like component is to transfer pricing risk to the party most 
able to manage it. Like an EPA clause in a government contract, the 
private contracts use a price index to adjust the transaction price in 
the long-term agreement. For example, a supplier of wrought titanium 
might index processed mill product prices to the cost of titanium sponge. 
Public firms must estimate the value of these EPA-like clauses for their 
quarterly and annual financial reporting if the language of the clause 
implies an embedded risk option. In some cases, the firm can use market 
prices for similar options for a valuation; in other cases, it must use a 


178 


DefenseARJ, July 2013, Vol. 20 No. 2:174-193 



Valuing the Cost of an Economic Price Adjustment Clause to the Government 


model. For estimating the value of an EPA clause in a defense weapons 
system contract, the same process can be applied. The valuation method 
is dependent on the type of commodity covered under the clause. 

Value of Risk Transfer in Government Contracts 

EPAs in government contracting transfer risk from the contractor to 
the government. In commercial transactions of this sort, the party that 
“sells” risk is expected to pay a premium to the party that “buys” risk. The 
financial and insurance industries have developed sophisticated tools 
for estimating the value of risk, thus the “premiums” that should be paid 
for various types of risk transfers. In government contracting, the pre¬ 
mium would be paid by a downward adjustment to the target fee earned 
by the contractor, set during determination of the contract price. The 
DFARS does not clearly address fee adjustments to account for the risk 
transfer when including an EPA clause. 6 Presently, contracting officers 
use their own judgment in determining whether to reduce the contract 
fee to reflect the lower cost-risk exposure, and no guidance is provided to 
contracting officers as to what might be an appropriate adjustment level. 

The DFARS does take into account other forms of risk transfer and 
provides recommendations on target fee adjustments to account for 
their value, e.g., moving from an FPIF to an FFP contract. These recom¬ 
mendations can provide rules of thumb for valuing other types of risk 
transfer. The Table lists the range of fees paid for contract risk based on 
contract type. 


TABLE. DFARS CONTRACT RISK FEE POLICY 


Contract Type 

Normal 

Value 

Designated 

Range 

Firm-fixed-price (FFP), no financing 

5.0% 

4 to 6% 

FFP, with performance-based payments 

4.0% 

2.5 to 5.5% 

FFP, with progress payments 

3.0% 

2 to 4% 

Fixed-price incentive (FPI), no financing 

3.0% 

2 to 4% 

FPI, with performance-based payments 

2.0% 

0.5 to 3.5% 

FPI, with progress payments 

1.0% 

0 to 2% 

Cost-plus-incentive-fee 

1.0% 

0 to 2% 

Cost-plus-fixed-fee 

.5% 

0 to 1% 


DefenseARJ, July 2013, Vol. 20No. 2:174-193 


179 













A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


For example, if an FFP contract with progress payments has a 15 
percent fee, 3 percentage points are for contract risk. The value of the 
risk transfer (as indicated by the DFARS “profit” rules) associated with 
an FPIF contract vice an FFP contract is 1 percent. 7 Note that the rules 
make no distinction between an FPIF with a high share ratio (e.g., 80 per¬ 
cent of overruns/underruns absorbed by the government) and one with 
a low share ratio. However, the designated range allows the contracting 
officer some leeway in accounting for the different levels of risk transfer 
possible in an FPIF contract. 

Valuing EPA Risk Transfers 
Using Financial Models 

The value of an EPA clause is what the finance literature calls the 
“risk premium”—the minimum price (or fee reduction) that the govern¬ 
ment might charge for taking the specific risk from a contractor. 8 The 
financial tools used to determine the market price of risk implied in 
hedging debt and commodities form the basis for valuing an EPA clause. 

Keynes (1930, pp. 142-44) and Hicks (1946, pp. 146-47) were the 
first to develop theories on the returns associated with commodities 
futures markets. Their normal backwardation theory postulated that the 
risk premium would accrue, on average, to buyers of futures (analogous 
to the government for an EPA). This was due to producers (the contrac¬ 
tor) selling futures—thereby hedging their profits—to speculators (the 
government), who required in return a price below the expected spot 
price at maturity (potential decrease in negotiated fee). This is similar 
to a hedger buying insurance from an insurance firm that serves as the 
speculator. The insurer expects that the premium includes compensation 
for administration and other management expenses associated with the 
insurance policy. These are all analogous to an EPA. 

To apply this to valuing the cost of risk to the government associated 
with EPA clauses, consider two extremes: a contract that is exposed to 
general inflationary risks in all elements of cost, e.g., labor and materi¬ 
als, and a contract exposed exclusively to risk in its commodity costs. 
In the first case, the entire contract is exposed to general inflation and 
the risk that this inflation deviates from the forecasted inflation used 
in developing the contract. The risk that, overall, all inflation deviates 
from forecasted inflation is called the Inflation Risk Premium (IRP). 
In this case, if the government were to charge a risk premium to hedge 


180 


DefenseARJ, July 2013, Vol. 20 No. 2:174-193 



Valuing the Cost of an Economic Price Adjustment Clause to the Government 


the contractor from the entire amount of this risk, it could be estimated 
directly as the IRP. This premium would be reflected as a reduction of 
the fee paid to the contractor. 

To understand the IRP, begin by examining risk premium generi- 
cally. A risk premium is the discount required on an investment whose 
cash flows are subject to fluctuations in value due to its exposure to a 
particular risk. The price discount is computed as the price relative to the 
same asset that is free of the risk exposure. For example, the equity risk 
premium, as used in the capital asset pricing model, is the discount inves¬ 
tors require on an investment in the market portfolio of equity securities 
relative to the risk-free rate. For U.S. stocks, this can be estimated by 
calculating the rate of return implied by the Standard & Poor's 500 index 
over 10 years and subtracting the yield on the 10-year Treasury Note. 


The value of an EPA clause is what the finance 
literature calls the “risk premium”—the minimum 
price (or fee reduction) that the government might 
charge for taking the specific risk from a contractor. 


This equity risk premium example is merely an illustration of the 
purpose of a risk premium. An EPA clause is designed to target very 
specific risks, in most cases inflation. Fortunately, this type of risk can 
be decoupled from certain types of publicly traded debt instruments. 
For a fixed rate note, the value of the interest payments is eroded should 
the rate of inflation exceed the rate that was assumed when the note was 
originally issued. 9 The IRP compensates investors for bearing the risk 
of inflation. 

The other extreme case is when the risk is due to exposure to price 
volatility of specific commodities, only a fraction of the overall cost of 
the contract value is at risk. In this case, if the government hedges the 
contractor from commodity price inflation, the value of the risk premium 
is more like that embedded in the related commodities futures market—if 
one exists—in which commodities producers hedge price risk by selling 
futures contracts. Commodities make up only a small fraction of the cost 
of major defense acquisition programs. Even with the largest historical 
price swings for such commodities as titanium or nickel, the overall cost 


DefenseARJ, July 2013, Vol. 20No. 2:174-193 


181 





A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


of the contract is unlikely to change by more than 1 percentage point 
(Arnold, Patel, & Harmon, 2011; Tran-Le & Thompson, 2005). While a 
risk premium based on specific commodities can be estimated, it may be 
the case that such an EPA would not justify the cost of its implementation 
and management effort. 10 


The U.S. Government raises debt through two main 
offerings: U.S. Treasury securities, which pay 
nominal interest rates, and Treasury inflation- 
protected securities (TIPS), which pay “real rates.” 


An EPA clause is most likely to be used in situations between these 
two extremes where most of the input price volatility is correlated with 
volatility in the overall inflation rate. 11 In these cases, one approach is 
to begin with the IRP and adjust it for the fraction of total contract cost 
represented by the inputs covered by the EPA clause. 

The U.S. IRP can be estimated by analyzing U.S. Treasury securi¬ 
ties along with a consistent inflation forecast. 12 The U.S. Government 
raises debt through two main offerings: U.S. Treasury securities, which 
pay nominal interest rates, and Treasury inflation-protected securities 
(TIPS), which pay “real rates.” The term structure of interest reflects 
the set of yields on fixed interest rate notes maturing in the future. 
Comparing the effective yield of U. S. Treasury notes against their matu¬ 
rity date shows the term structure that reflects the market's expectation 
of future interest rates. When the economy is expected to grow, the curve 
is usually upward-sloping. The market's expected inflation—inferred 
from price forecasts, commodity futures, and other economic data—is 
also embedded in this term structure. The Figure shows the yield curve 
for nominal Treasury securities and TIPS from September 26, 2011 
(Board of Governors, 2011). 


182 


DefenseARJ, July 2013, Vol. 20 No. 2:174-193 





Valuing the Cost of an Economic Price Adjustment Clause to the Government 


The difference between the yields on similarly maturing nominal 
Treasury securities and TIPS is the “break even” inflation (BEI) rate. 
The BEI rate can be deconstructed into the expected inflation rate and 
the IRP, as follows: 

Break Even Rate = IRP + Expected Inflation 

The inflation-protected and nominal Treasury securities parallel the 
pricing of contracts with and without an EPA (linked to general infla¬ 
tion), respectively. The government saves the IRP by providing inflation 
risk protection; alternatively, the IRP is the cost of paying nominal rates. 
The government implicitly charges TIPS investors this premium relative 
to the buyers of nominal Treasuries. In similar fashion, an FFP contract 
with an EPA (linked to general inflation) is like providing inflation pro¬ 
tection and the value of this IRP. 

FIGURE. NOMINAL TREASURY AND TIPS YIELD CURVE WITH 

BREAK EVEN INFLATION RATE 



Note. Adapted from "Selected Interest Rates - H. 15: Daily Updates" by Board of 
Governors of the Federal Reserve System (U.S.) (September 26, 2011). Retrieved from 
http://www.federalreserve.gov/releases/hl5/update/ 


DefenseARJ, July 2013, Vol. 20No. 2:174-193 


183 






A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


Application 

The process of applying this to a procurement contract starts with 
a clear identification of what input is being covered and whether the 
contracting officer can identify a good market index or price series. Next, 
the overall effect of the commodity's price volatility on the contract cost 
must be estimated. Large price fluctuations for inputs such as titanium 
in the F-35 have a relatively insignificant effect on the overall cost of 
the contract, because they represent a small fraction of the cost. On the 
other hand, even mild fluctuations to general inflation can affect all of 
the contract's inputs, leading to relatively large cost changes. 

Although the IRP concept is relatively simple, computing an estimate 
from interest rate data can be a relatively complicated task. 13 It has been 
done using time series analyses of interest rate data and both historical 
and forecast inflation rates (Grishchenko & Huang, 2008). The IRP can 
also be estimated from prices for fixed income securities other than 
Treasuries. 


The risk exposure from materials and other 
specialty commodities 9 price volatility may be too 
small to merit an EPA clause . 


Examples of IRP estimates show that the premium varies overtime. 
Inflation volatility is not stationary, and the IRP varies with economic 
uncertainty and expectations of high or low inflation. 14 Recent estimates 
of the IRP show it as low during periods of low inflation expectations and 
high during periods of high uncertainty. Shiller and Campbell (1996) 
estimated the IRP to be between 50 and 100 basis points by analyzing 
nominal 5-year Treasury yields over the period 1953 to 1994. 15 More 
recently, Durham (2006) of the Federal Reserve reported the IRP rang¬ 
ing from 15 to 120 basis points over the period from late 2000 to 2005. 
Grishchenko and Huang (2008) reported a smaller IRP—2 to 63 basis 
points—from their vector autoregression analysis of TIPS prices. A 
more recent staff report by Adrian and Wu (2009) of the Federal Reserve 
points to a higher IRP ranging from around 40 to over 250 basis points. 


184 


DefenseARJ, July 2013, Vol. 20 No. 2:174-193 





Valuing the Cost of an Economic Price Adjustment Clause to the Government 


As stated earlier, the risk exposure from materials and other spe¬ 
cialty commodities' price volatility may be too small to merit an EPA 
clause. One exception to this could be a contract for a product or service 
for which commodities such as food services or fuel were a high fraction 
of the cost. The commodities risk premium is typically higher than the 
IRP. Estimates from commodities futures data find the premium is simi¬ 
lar to that for equities—about 4 to 5 percent. This premium was estimated 
by Fama and French (1987), among others (Gorton & Rouwenhorst, 2005; 
Basu & Miffre, 2011), using an equally weighted portfolio of commodities. 

One way to deal with this would be for an FFP contract with an 
EPA clause to have a fee decrement of 50 basis points relative to an FFP 
contract without the clause, reflecting the IRP estimates for the present 
period of low inflation risk. The fee decrement could be adjusted by the 
cost share ratio if the contract type was an FPIF. This fee adjustment 
reflects the cost of bearing the risk that input prices could differ from 
expectations. The contract should already reflect the expected inflation 
rate so that the bearer of the risk exposure can reasonably expect to get 
the same degree of good news versus bad news. 



DefenseARJ, July 2013, Vol. 20No. 2:174-193 


185 




























A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


Other Considerations 

Other issues of risk and cost arise from the inclusion of an EPA 
clause in a contract. For example, effort required to manage the clause 
once the contract is executed carries an additional administrative cost. 
Also, the government faces a number of risks, discussed previously in 
the background section, by accepting the clause. These risks are closely 
related to the concept of an EPA clause as insurance. The risks include 
the basis risk associated with the indexes used; other risks can be related 
to risks inherent in any insurance: adverse selection (contractors with 
higher inflation risk opting into an EPA) and, to some extent, moral 
hazard (the contractor having an incentive to change its behavior to 
manipulate the EPA). 

In addition to adjusting the contract fee by the IRP, the govern¬ 
ment should also consider managing basis, adverse selection, and moral 
hazard risks the way insurance companies deal with these risks. The 
government should evaluate its level of understanding of the contractor's 
costs and its incentives given an EPA clause. If the government deems 
itself at a significant informational disadvantage, it may need to apply 
insurance-like provisions to its contracts to share risks. One common 
insurance practice is coinsurance—only insuring a fraction of the loss 
exposure, perhaps 75 percent. A variation on coinsurance is a trigger 
band that is directed in the DFARS and is common in EPAs: The contrac¬ 
tor is exposed to a narrow band of volatility—say ±3 percent—outside of 
which the government is fully exposed to the loss or gains. 

Ultimately, assessing these other risks is idiosyncratic and requires 
an in-depth assessment of the specific contract and contractor. This 
is in contrast to the methodology described in this article to use 
market-derived risk premiums to price specific, but not supplier, 
idiosyncratic risks. 


186 


DefenseARJ, July 2013, Vol. 20 No. 2:174-193 



Valuing the Cost of an Economic Price Adjustment Clause to the Government 


Discussion 

An EPA clause transfers risk from the contractor to the government; 
in essence, it constitutes an insurance contract. EPA clauses, there¬ 
fore, provide value to the contractor and cost to the government, and 
the government could take this into account in determining contract 
price. Setting the target fee used to establish contract price provides an 
opportunity to account for the value of an EPA clause, and the DFARS 
Weighted Guidelines now provide contracting officers with some flexibil¬ 
ity to do so. If the government wanted to account for the value of the risk 
transfer systematically, it could develop adjustment factors for inclusion 
in the weighted guidelines. 

The IRP, which is based on the Consumer Price Index (CPI), reflects 
a more diversified portfolio of goods than a typical EPA clause linked to 
a single commodity such as steel. Further study could also be performed 
to gauge the risk exposure of the various contract elements for which the 
government is willing to allow EPA clauses. Simplicity in constructing 
these clauses is important, and it may be that a single risk premium 
is sufficient to equitably price the EPA clause. To develop systematic 
guidelines, the government would have to consider adjustments to the 
EPA fee decrement to reflect changes in the IRP during periods of very 
high inflation expectations. While the literature does provide estimates 
of the IRP, a consistently applied method, possibly based on TIPS and 
nominal notes, might provide an effective pricing tool that captures 
changing inflation trends. 

It is important to remember that the IRP will not keep the govern¬ 
ment from paying inflation adjustments associated with an EPA clause; 
rather, compensation to the government for bearing the volatility risk may 
drive the adjustments. Furthermore, the fee adjustment concept outlined 
herein does not consider the premium for bearing other risks associated 
with EPAs, namely basis risk, adverse selection, or moral hazard. 


DefenseARJ, July 2013, Vol. 20No. 2:174-193 


187 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


Author Biographies 

Dr. Scot Arnold has worked at the Institute 
for Defense Analyses (IDA) for 10 years, 
focusing on economic analyses and defense 
industrial policy issues. Prior to joining IDA, 
he worked in finance at the Visteon 
Corporation and the Ford Motor Company. 
Dr. Arnold holds a PhD in polymer science 
from the Massachusetts Institute of 
Technology, an MBA from the University of 
Michigan, and a BA from Vassar College. 

(E-mail address: sarnold@ida.org) 

Mr. Bruce Harmon works for the IDA, where 
he has been a professional research staff 
member for over 25 years. Mr. Harmon has 
extensive experience modeling the costs and 
schedules of various aerospace systems, as 
well as analyses of other acquisition issues. 
He is a PhD candidate in Economics at 
American University, Washington, DC. 

(E-mail address: bharmon@ida.org) 




188 


DefenseARJ, July 2013, Vol. 20 No. 2:174-193 








Valuing the Cost of an Economic Price Adjustment Clause to the Government 



Dr. Susan Rose works as a research staff 
member at the IDA. During her 5 years at IDA, 
Dr. Rose has worked on a variety of research 
analyses for the Office of the Secretary of 
Defense, including forecasting of healthcare 
costs. She holds a PhD in Economics from 
Ohio State University. 

(E-mail address: srose@ida.org) 




Mr. John Whitley is a senior fellow at the 
IDA. Prior to joining IDA, he was director of 
Program Analysis and Evaluation (PA&E) at 
the Department of Homeland Security (DHS); 
and prior to joining DHS, Mr. Whitley worked 
in the Department of Defense PA&E, the U.S. 
Senate, academia, and served in the U.S. Army. 
He holds a PhD in Economics from the 
University of Chicago and undergraduate 
degrees from Virginia Polytechnic Institute 
and State University. 

(E-mail address: jwhitley@ida.org) 


DefenseARJ, July 2013, Vol. 20No. 2:174-193 


189 











A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


References 

Adrian, T., & Wu, H. (2010). The term structure of inflation expectations (Staff 
Report No. 362). New York: Federal Reserve Bank of New York Staff 
Reports. 

Arnold, S. A., Patel, P., & Harmon, B. (2011). The material breakdown of the 
F-35 Joint Strike Fighter and the Mine Resistant Ambush Protected 
Vehicle (IDA Paper P4648). Alexandria, VA: Institute for Defense 
Analyses. 

Basu, D., & Miff re, J. (2009). Capturing the risk premium of commodity 
futures: The role of hedging pressure. Retrieved from http://ssrn.com/ 
abstract=1340873 

Board of Governors of the Federal Reserve System (U.S.). (2011, September 
26). Selected interest rates - H.15: Daily updates. Retrieved from http:// 
www.federalreserve.gov/releases/h15/update/ 

Department of Defense Inspector General. (2008). Effect of payment into 
Boeing pension funds on economic price adjustment clauses in DoD 
contracts (Report No. D2008099). Arlington, VA: Author. 

Department of Defense Inspector General. (2009). Cost increases related 
to the Producer Price Index for titanium mill shapes on DOD multiyear 
contracts (Report No. D-2010-004). Arlington, VA: Author. 

DFARS Procedures, Guidance, and Information, § 216.202-4 (2012). 

Durham, J. B. (2006). An estimate of the inflation risk premium using a three- 
factor affine term structure model (Finance and Economics Discussion 
Series). Washington, DC: Divisions of Research & Statistics and Monetary 
Affairs, Federal Reserve Board. 

Fama, E., & French, K. (1987, January). Commodity futures prices: Some 
evidence on forecast power, premiums, and the theory of storage. 
Journal of Business, 60(1), 55-73. 

Federal Acquisition Regulation, Vol. 1 § 16-203-2(i) (2005). 

Goldberg, V. P., & Erickson, J. R. (1987). Quantity and price adjustment in 
long-term contracts: A case study of petroleum coke. Journal of Law 
and Economics, 30(2), 369-398. Retrieved from http://www.jstor.org/ 
stable/725501 

Gorton, G., & Rouwenhorst, K. (2005). Facts and fantasies about commodity 
futures. National Bureau of Economic Research Working Paper 04-20. 
Cambridge, MA: Yale School of Management. 

Grishchenko, O. V., & Huang, J. (2008, April). Inflation risk premium: Evidence 
from the TIPS market. Paper presented at the meeting of the 18th Annual 
Derivatives Securities and Risk Management Conference, Arlington, VA. 

Hicks, J. R. (1946). Value and capital (2nd ed.). Oxford: The Clarendon Press. 

Keating, E. G., Murphy, R., Schank, J. F., & Birkler, J. (2008). Using the steel- 
vessel material-cost index to mitigate shipbuilder risk (Tech. Rep.). Santa 
Monica, CA: RAND National Defense Research Institute. 

Keynes, J. M. (1930). A treatise on money (Vol. II). New York: Harcourt, Brace, 
& Company. 


190 


DefenseARJ, July 2013, Vol. 20 No. 2:174-193 



Valuing the Cost of an Economic Price Adjustment Clause to the Government 


Shiller, R. J., & Campbell, J. Y. (1996). A scorecard for indexed government 
debt. In B. S. Bernanke and J. Rotemberg (Eds.), National Bureau of 
Economic Research Macroeconomics Annual 1996. Cambridge, MA: MIT 
Press. 

Tran-Le, J., & Thompson, S. (2005). China’s impact on metals prices in 
defense aerospace. 


DefenseARJ, July 2013, Vol. 20No. 2:174-193 


191 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


Endnotes 

1. In the FAR and DFARS, fixed price contracts (encompassing contacts that 
would otherwise be FFP or FPIF) with an EPA clause are considered a 
unique contract type. Thus, guidance regarding the EPA clause is included 
in sections describing contract types, specifically, FAR part 16 and DFARS 
subpart 216. 

2. This article uses the term fee to refer to the difference between the contract 
price and the underlying cost of the contract to the contractor. This is to 
reflect the distinction with contractor economic or accounting profit, which 
is unlikely to be identical to the negotiated fee. The concept of fee here is 
referred to as profit in DFARS subpart 215.4. 

3. Section 215.404-71-3 of the DFARS titled “Contract type risk and working 
capital adjustment” provides guidance on pricing contract type risk. If an 
EPA clause is included under the contract type category “Fixed-price with 
redetermination provision,” the guidance is to set the fee as if it were a fixed- 
price incentive contract with below normal conditions. If, without the EPA 
clause, the contract would be priced as an FFP contract with a “normal fee” 
for contract risk of 3 percent, then this means that adding an EPA would 
reduce the fee to less than 1 percent. This fee adjustment may be reasonable 
for fixed-price contracts with prospective price redetermination (FP-PPR) 
where the price of the entire item being purchased could be adjusted upward 
in the future. However, for an FFP contract with an EPA, this could be a severe 
fee reduction if the clause references direct labor or materials that could be 
small fractions of the overall contract value. 

4. In the finance and economics literature, this is referred to as “adverse 
selection” and is addressed later in our article. 

5. Although there was no evidence of manipulation in this case, using an index 
driven by a contractor’s own labor rates opens the possibility of increasing 
enterprise-wide profits though cost-shifting. 

6. For government contracts, the tools for pricing FAR part 15 contracts are 
limited to percentage-of-contract-cost fee guidelines that outline the amount 
that should be paid as a function of the level of cost risk and management 
effort to which the contractor is exposed. 

7. The fee difference between FFP and FPIF contracts could also partially 
reflect the greater level of government management effort required for an 
FPIF contract. 

8. There may be other costs associated with the EPA that the government 
may seek to recover from the contractor. For example, there could be 
recompensable costs associated with administering the EPA. In this article, 
we are restricted to examining the cost of quantifiable risk. 

9. Inflation is not the only economic factor that can erode the value of a bond; 
other factors are credit risk and the risk that market interest rates may rise. 

10. Note that titanium may be vulnerable to potential gaming as the majority of 
domestic titanium metal is used in aerospace applications (DoDIG, 2009). 

11. The expected inflation rates for contract inputs should already be reflected in 
the contract price excluding the EPA. 


192 


DefenseARJ, July 2013, Vol. 20 No. 2:174-193 



Valuing the Cost of an Economic Price Adjustment Clause to the Government 


12. Alternatively, if the general forecast is believed to be not significantly 
different from the recent past, the IRP could be estimated from historical U.S. 
Treasuries and Consumer Price Index data. 

13. For example, Adrian and Wu (2009) use a Kalman filter to estimate the 
parameters of a generalized autoregressive conditional heteroskedasticity 
model (GARCH) of the inflation rate risk premium. 

14. Adrian and Wu (2009) found the IRP was strongly correlated with the equity 
Chicago Board Options Exchange Volatility Index or VIX. 

15. Shiller and Campbell (1996) also estimated that the option value of inflation 
protection was about 140 basis points. 


DefenseARJ, July 2013, Vol. 20No. 2:174-193 


193 



A Publication of the Defense Acquisition University 


http://www.dau.mil 


Keywords: Fast , Inexpensive, Simple, Tiny (FIST); 
Program Management; Heuristics; Innovation; 
Oversight 


Current Barriers to 
Successful Implementation 
of FIST Principles 

f Capt Brandon Keller, USAF, 
and Lt Col J. Robert Wirthlin, USAF 


The Fast, Inexpensive, Simple, and Tiny (FIST) frame¬ 
work proposes a broad set of organizational values, 
but provides limited guidance on practical implemen¬ 
tation. Implementing FIST principles requires clarifying 
the definitions of “fast,” “inexpensive,” and “simple,” 
recognizing where FIST does and does not apply. Addi¬ 
tionally, a subset of the FIST heuristics was expanded 
upon to increase their usefulness for practitioners. The 
primary research findings are that FIST principles are 
less conducive for highly complex or novel systems, 
immature technologies, future needs, acquisitions in 
early development phases, or when performance is the 
foremost value. FIST principles were also found to be 
constrained by the acquisition process, the requirements 
process, and oversight. 


194 


DefenseARJ, July 2013, Vol. 20 No. 2:194-217 


image designed by Diane Fleischer » 





SIMPLE 

















A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


The Fast, Inexpensive, Simple, and Tiny (FIST) articles first 
appeared in the Defense Acquisition University (DAU)'s Program 
Manager, a periodical later renamed Defense AT&L in January 2004 
(Ward, Quaid, & Mounce, 2008). The articles were evaluated, iterated, 
and compiled into a cohesive thesis by Air Force Lt Col Dan Ward (2009) 
in “The Effect of Values on System Development Project Outcomes .” 
To this day, Ward's theories and adept writing style have stimulated 
significant debate in the Department of Defense (DoD) acquisition 
community and academia. The FIST framework proposes a broad set 
of organizational values, but provides limited guidance on practical 
implementation. Implementing FIST principles requires clarifying the 
definitions of “fast,” “inexpensive,” and “simple,” recognizing where FIST 
does and does not apply, and offering additional FIST heuristics based 
on the recommendations provided herein, to increase their usefulness 
for practitioners. 

The purpose of this article is neither to discredit nor to aggrandize 
FIST. The intent is to impartially evaluate FIST concepts to increase 
knowledge and understanding. 



196 


DefenseARJ, July 2013, Vol. 20 No. 2:194-217 




Current Barriers to Successful Implementation of FIST Principles 


FIST Principles 

Ward mentions in his various writings, the “tiny” aspect is an “ines¬ 
capable outcome” of accomplishing the first three (Ward & Quaid, 2006a, 
p. 31); therefore, the focus will be on the fast, inexpensive, and simple 
tenets of the FIST framework. These tenets should also be thought of 
as a single idea rather than a value set having separate parts. As a single 
entity, “an attempt to remove some portion of this value set is likely 
to impact the program manager's ability to implement any of it at all” 
(Ward, 2009, p. 8). Therefore, all the FIST principles must be present 
for a program to succeed. For example, the Bazooka is a success story 
because the program (and product) was simple and inexpensive and fast 
(therefore tiny as well). It adhered to all of the FIST principles. 

Scoping “Fast” 

One principle to delivering systems quickly is to get early and 
iterative feedback from users (Hebert, 2011). The assertion that early 
feedback from users leads to rapid development and shorter timeframes 
is accurate (Ward, 2004), but the limitations should also be discussed. 
Is it possible to get early user feedback on a Naval carrier? What about 
early operator feedback on a satellite program? This is nearly impos¬ 
sible unless a satellite or prototype is launched solely for this reason, 
which is often cost-prohibitive. Historically, around 80 percent of a 
space system's life-cycle cost is consumed prior to operations (Hebert, 
2011). Therefore, operator feedback is often delayed until the system is 
fielded because launching a satellite solely for testing and user feedback 
is cost-prohibitive. To be fair, operator prototypes and simulators obtain 
a degree of operator feedback. This reduces the risk, but rarely is actual 
operator feedback with operational assets obtained in the space domain. 

The “fast” aspect of the FIST framework also has a fair amount 
of overlap with rapid acquisitions. Rapid acquisition requires stable 
requirements (Ford, Colburn, & Morris, 2012). As requirements are 
usually not fully stable prior to Milestone B for major programs, FIST 
must be scoped to a certain phase in the acquisition system. The earli¬ 
est phase for FIST implementation would likely be the Engineering 
and Manufacturing Development phase (post-Milestone B approval), 
because a Capability Development Document will be complete with all 
technologies at a Technology Readiness Level of 6 or greater. 


DefenseARJ, July 2013, Vol. 20No. 2:194-217 


197 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


For these reasons, FIST is less conducive in the early phases (pre- 
Milestone B) of the acquisition process, and therefore is less beneficial 
for delivering future needs. FIST is also less conducive for complex, large 
programs in which early operator feedback is not feasible. 

Scoping “Inexpensive” 

Ward suggests several times that large budgets hinder communica¬ 
tion with the user community (Ward, 2004). Real feedback from users 
is extremely important, as Ward would agree, but no evidence is offered 
as to why this cannot be done with high-dollar programs. One theory 
is that high-dollar programs are generally for the complex, highly inte¬ 
grated, and interrelated systems. These systems tend to have a variety 
of users and stakeholders whose exact roles can be vague or undefined. 
For example, who is the user of an F-22? If the sole answer is the pilot, 
we are limiting our decisions to one of many users. A “user” with real 
combat feedback beneficial to acquirers includes air liaison officers, 
aircraft maintainers, air traffic controllers, instructor pilots, and the 
training schools, to name a few. Whenever these users have conflicting 
feedback and desires for the system, the program office must make engi¬ 
neering trade-off decisions. If users have conflicting desires, a subset of 
users will inevitably be unsatisfied and may view the program office as 
unresponsive if their desires were not met. 

Therefore, large budgets are not the root cause of communication 
issues with users. Large budgets usually accompany complex, major 
weapons systems, which have various users and stakeholders with dif¬ 
fering values. Consequently, FIST is less conducive for systems in which 
many users and stakeholders exist. 

Scoping “Simple” 

Figure 1 is the graphical representation of Ward's Simplicity Cycle. 
The graph depicts a certain turning point (shown by a “2” on the graph) 
in which adding complexity decreases “goodness” (ability of a system to 
do what it's supposed to do). 

Understandably, at a certain turning point, adding complexity to 
a system actually decreases its “goodness.” However, how do we know 
where this turning point is? Program managers and engineers do not 
add unnecessary complexity to systems without reason. “An inadequate 
appreciation for simplicity can result in an overvalued perspective of 


198 


DefenseARJ, July 2013, Vol. 20No. 2:194-217 



Current Barriers to Successful Implementation of FIST Principles 


FIGURE 1. THE SIMPLICITY CYCLE (WARD, 2005) 



complexity, which can cause programmatic disaster” (Ward, 2005, p. 
20). The opposite is also true, which causes another set of conflicting 
values. Holding to simplicity because the genius behind the complexity 
is not understood can also cause programmatic disaster. This concept 
is better understood with an example. 


Holding to simplicity because the genius behind 
the complexity is not understood can also cause 
programmatic disaster. 


Consider a team meeting to decide if solar retroreflectors are 
required on the exterior of a space plane. The viewpoint of the chief 
engineer is that they add complexity, cost too much, and will extend the 
program schedule. The materials expert contends that the retroreflec¬ 
tors are required because the sun's rays will burn the exterior before 
the payload will reach the proper orbit. Which to choose? One cannot 
blindly say that the retroreflectors go against all four FIST principles 
and, therefore, should not be pursued. FIST principles must be tempered 
with mission assurance. 


DefenseARJ, July 2013, Vol. 20No. 2:194-217 


199 








A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


As a result, the FIST principle of “simple” is less conducive for pro¬ 
grams and technologies that are complicated and not well understood. 
One way to utilize FIST principles in immature or uncertain program¬ 
matic environments is to budget and plan for a simple, fast prototype. By 
doing this, much of the uncertainty and technical risk is reduced, remov¬ 
ing these barriers to successful FIST implementation. The majority of 
uncertainty occurs in the early acquisition phases, so once again FIST is 
less applicable in the early phases of an acquisition. As Mathiassen and 
Munk-Madsen (1986, p. 20) state,"... in reality the [product development] 
situation is rarely well defined from the start.” 


One way to utilize FIST principles in immature 
or uncertain programmatic environments is to 
budget and plan for a simple, fast prototype. 


Will a FIST Program Meet DoD Technical Guidance? 

Current DoD technical regulations and guidance do not support 
FIST principles. This can be easily seen for programs that must comply 
with current DoD technical direction, such as the DoD Net-Centric 
Services Strategy or the Net Ready Key Performance Parameter (NR 
KPP) from Chairman of the Joint Chiefs of Staff Instruction (CJCSI) 
6212. The Net-Centric Services Strategy from 2007 promotes integrated 
systems employing net-centric principles, service-oriented architec¬ 
tures, and global information grid-compliant systems. This strategy 
ensures warfighters receive the right information at the right level of 
detail, from trusted and accurate sources, when and where it is needed 
(DoD, 2007). The NR KPP makes net-centric operations a KPP for all 
applicable systems (Chairman of the Joint Chiefs of Staff, 2012). These 
collaborative requirements are program-dependent, meaning they rely 
on other programs to comply with interface specifications before they 
can be compliant. Anything taking control away from the program 
manager goes against FIST because if PMs are dependent on external 
stakeholders, they will be less able to ensure speed and cost. Additionally, 
a graduate systems engineering certificate capstone project by Wong and 
Thompson (2006) cites the numerous cost and complexity issues related 
to technical interface management. Therefore, requirements mandated 
as part of the NR KPP are a current barrier to FIST implementation. 


200 


DefenseARJ, July 2013, Vol. 20No. 2:194-217 





Current Barriers to Successful Implementation of FIST Principles 


Of course the goal is to be compliant as fast and simply as possible, 
but complying with the NR KPP is neither a fast nor simple process. 
Once interoperability and net-centricity become better understood and 
operationalized, fast and simple concepts should be pursued to optimize 
performance in these areas. Therefore, if the system must comply with 
complex, undefined requirements (not all systems do), it will be more dif¬ 
ficult to implement the FIST methodology. The point here is that Ward 
is absolutely correct that simplicity has many tangible benefits, but the 
thick waters of complexity must be waded through first, which many 
programs and technologies are still in the process of doing (most often 
in the complex, long-standing programs). 

In summary, implementation of FIST principles is limited by DoD 
technical guidance. When guidance mandates compliance with techni¬ 
cally complex requirements, achieving FIST principles is very difficult. 

FIST is for Evolutionary (Not Revolutionary) Innovations 

Ward states that “small teams + thin budgets + short timelines = sig¬ 
nificant innovation and combat effectiveness” (Ward, 2004, p. 34). This 
statement is true for today's fight; however, is it less applicable if the focus 
is on winning tomorrow's war? If the military simply has small teams 
with thin budgets delivering products and services quickly, we will lose 
the innovative edge with respect to our novel, complex systems. Some 
complexity is required, as the Simplicity Cycle states, before simplicity 
can be achieved. 

Books on innovation and Lean principles describe the different 
strategies of “Invest in Evolution'' versus “Invest in Revolution.'' Figure 
2 maps common verbiage for similar concepts. The incremental improve¬ 
ment strategy is very similar to the FIST strategy. Both require a steady 
industrial base, mature technology, and the existence of a capability or 
performance gap in the current system. The risk to this strategy is that 
key new opportunities (radical innovations) go unexplored for incremen¬ 
tal or evolutionary upgrades (Murman, 2002). Although incremental 
innovations sustain current capability, they don't produce the radical 
innovation necessary to address an asymmetric threat. This strategy 
does have value by delivering newer versions of existing systems faster. 
The DoD must be careful not to perpetuate existing monuments (in Lean 
speak), or not to let core capabilities become core rigidities. 1 


DefenseARJ, July 2013, Vol. 20No. 2:194-217 


201 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


FIGURE 2. COMMON LEXICON FOR EVOLUTIONARY AND 
REVOLUTIONARY STRATEGIES 


Strategy 

Reference 

Author 

Year 

Invest in Evolution 

Lean Enterprise 

Value 

Murman et al. 

2002 

= Directional ideas 

The Medici Effect 

Frans Johannson 

2006 

= Incremental 

innovation 

Making Innovation 

Work 

Davila et al. 

2005 

= Sustaining 

innovation 

The Innovators DNA 

Dyer et al. 

2011 

Invest in Revolution 

Lean Enterprise 

Value 

Murman et al. 

2002 

= Intersectional ideas 

The Medici Effect 

Frans Johannson 

2006 

= Breakthrough 

innovation 

Making Innovation 

Work 

Davila et al. 

2005 

= Disruptive 

innovation 

The Innovators DNA 

Dyer et al. 

2011 

= Radical innovation 

Making Innovation 

Work 

Davila et al. 

2005 


The opposite of an Invest in Evolution strategy is an Invest in 
Revolution strategy. The Invest in Revolution strategy involves game¬ 
changing innovations that result in current systems and technologies 
becoming obsolete. When a revolutionary innovation emerges, no fur¬ 
ther evolutionary upgrades are value-added. For example, the advent 
of electricity made upgrading candles (for practical lighting) obsolete. 
The advent of low-profile, stealth-like characteristics made many sur¬ 
face-to-air defenses obsolete. The downside of an Invest in Revolution 
strategy includes costliness, no guarantee a new capability will be 
fielded, and the risk of a gap in current capabilities (Murman, 2002). 
However, this is the primary strategy to take advantage of breakthrough 
technologies to remain a step ahead of the competition (Dyer, Gregersen, 
& Christensen, 2011). This is not trivial when the nation's defense is at 
stake. Herein lies the heart of a major barrier to successful implementa¬ 
tion of FIST principles. 

First, incremental improvements are normally completed faster, 
with less complexity (more simplicity) and at lower costs (Dyer, 
Gregersen, & Christensen, 2011; Johannson, 2006; Davila, Epstein, & 


202 


DefenseARJ, July 2013, Vol. 20No. 2:194-217 














Current Barriers to Successful Implementation of FIST Principles 


Shelton, 2006). Radical innovations are characterized by their novelty, 
technical immaturity, and mission uncertainty—all contrary to the 
FIST framework. Therefore, the FIST methodology closely aligns to 
incremental, vice disruptive, innovation. FIST success stories may not 
seem incremental based on the extent of the improvements. However, 
based on the fact that existing, mature technologies were used and the 
original platforms still have value, the improvements are, by definition, 
incremental. Although FIST principles have before and can continue to 
field radical innovations, these results are the exception. As Maier and 
Rechtin (2009, p. 405) state, “proven” and “state-of-the-art” are mutu¬ 
ally exclusive properties. 

Additionally, FIST enhances project stability (Ward, 2009). 
A corresponding limitation to project stability is the reduction of 
radical innovations. Radical innovation does not come from stable, 
secure, assured delivery environments. Rather, these game-chang¬ 
ing innovations are born from organizations that embrace failure, 
are not risk-averse, and have a degree of instability as novel ideas are 
investigated. 


When guidance mandates compliance with 
technically complex requirements, achieving FIST 
principles is very difficult . 


Lastly, Ward agrees that a key to FIST implementation is the use of 
mature technologies (Ward, 2009), which is often the antithesis of inno¬ 
vation. A FIST program, as with a rapid acquisition program, does not 
have time to struggle with immature technologies. Unfortunately, many 
new weapon systems, especially space systems, are relying on immature 
and complex technologies (Government Accountability Office, 2006). 
This creates a barrier that must be overcome when trying to implement 
the fast and simple aspects of FIST. 

For these reasons, FIST principles reduce radical innovations. 
Additionally, FIST principles are not conducive for immature technolo¬ 
gies (as Ward would agree, citing mature technology as a key to FIST 
implementation). 


DefenseARJ, July 2013, Vol. 20No. 2:194-217 


203 





A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


Adding Realism to FIST 

FIST is a set of guidelines, or heuristics, to help steer program man¬ 
agers to better decisions. However, many of the core aspects FIST urges 
program managers to embrace are simply out of the program man¬ 
ager's control. In these cases, research highlighting the lack of control 
and authority program managers have, especially in a Major Defense 
Acquisition Program (MDAP), in the current acquisition system is cited. 
A realistic set of guidelines for FIST must help program managers decide 
between available alternatives, not areas that are outside their control. 
One opportunity in which program managers can make engineering 
and programmatic trade-offs favoring FIST principles is early in a pro¬ 
gram, before the requirements, technologies, acquisition category level, 
and other decisions have been made more permanent. However, when 
program managers inherit programs later in development, many times 
implementation of FIST principles is out of their control. 

“Simple” Realism 

In terms of simplicity, a program manager is given a set of require¬ 
ments validated by the Air Force Requirements Oversight Council and 
Joint Requirements Oversight Council, as required. Although a degree 
of requirements tailoring can be achieved through discussions between 
the acquirers and users, by and large the requirements have been vetted 
when the acquiring organization receives them. The requirements for 
complex, novel systems will consequently force the program office into 
complexity rather than simplicity. 


Whenever and wherever possible, simplicity 
is an extremely valid heuristic to help manage 
a program. 


In addition, the approval process and program oversight have been 
shown to be overly complex, very costly, and—to a large degree—out¬ 
side the control of the program manager (Assessment Panel, 2006; 
Neal, 2004; Knue, 1991). Therefore, in the reality of complex, novel 
systems, not only does the required performance force complexity, but 
the acquisition process forces complexity as well. This is a barrier to 
implementing the FIST methodology, but should not be confused with 


204 


DefenseARJ, July 2013, Vol. 20 No. 2:194-217 





Current Barriers to Successful Implementation of FIST Principles 


the fact that whenever and wherever possible, simplicity is an extremely 
valid heuristic to help manage a program. Current research investigates 
the applicability of rapid acquisition methods for traditional develop¬ 
ment programs with promising initial results. Ford et al. (2012) identify 
expedited systems engineering and rapid acquisition concepts that can 
potentially improve processes for traditional programs. 

In summary, the requirements process reduces a program manager's 
ability to implement FIST principles. The acquisition process and over¬ 
sight also constrain FIST implementation. 

“Fast” and “Inexpensive” Realism 

A program manager has a bit more control with respect to cost and 
schedule variables. Still, the acquisition process can have major effects 
on these as well, regardless of the program manager's intent. Ward high¬ 
lights in “Putting the Pieces Together" that the common saying “better, 
faster, cheaper: pick two" is short-sighted and unjustifiable (Ward & 
Quaid, 2006a, p. 32). All program managers should desire better, faster, 
and cheaper each and every time. The problem lies in the DoD acquisition 
system, as the military reformers 2 found out while fighting tooth-and- 
nail to overcome it. A good example is the F-16 program as described in 
The Pentagon Wars (Burton, 1993). The development of the F-16 involved 
a bitter fight between the military reformers and existing senior lead¬ 
ership. The reformers wanted a cheap, focused air superiority fighter 
utilizing an existing airframe to reduce costs. At the time, military 
leadership lobbied for an all-purpose, air superiority aircraft with all 
the “bells and whistles." In the end, the F-16 emerged as a very capable, 
inexpensive, and quickly fielded aircraft (qualities the reformers valued). 
However, the program continually faced stringent resistance from the 
acquisition system and leadership. The normal acquisition processes 
had to be circumvented by nothing short of heroic efforts (Burton, 1993). 
Therefore, rather than trying to train heroes and ignore the root cause of 
the problem, the system should set the average program manager up for 
success. “Pick two" is forced upon program managers, and the following 
example will highlight how cost and schedule can quickly be taken out 
of the program manager's hands. 

Program X is an MDAP approaching Milestone B with a cost 
estimate of $100 million. The Office of the Secretary of Defense Cost 
Assessment and Program Evaluation (CAPE) staff may disagree with 
the program office cost estimate when conducting their 80 percent 


DefenseARJ, July 2013, Vol. 20No. 2:194-217 


205 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


estimate. Therefore, to ensure a successful Milestone, the cost estimate 
is reconciled and increased to the CAPE's estimate. Subsequently, the 
budget approved at Milestone B will reflect this higher cost estimate. In 
this simplistic yet realistic example, the program office is forced into an 
increased budget. The same example holds true for schedule as well. A 
decision authority often regards a condensed schedule as unrealistic, and 
either increases the cost estimate to accomplish the condensed work, or 
forces the schedule (using independent schedule analyses, which tend 
to be more conservative) to expand. The key to passing a Milestone is to 
have a low-risk, high-confidence program in an executable cost within 
the budget. In other words, offering a strategy that's faster and cheaper 
than comparable programs is often viewed by oversight personnel as 
the program office staff not fully understanding the scope of the effort or 
overestimating a learning curve. In this case, the historical acquisition 
deficiencies work against the program offices' efforts to streamline and 
plan in efficiencies. Because of this, a “better, faster, cheaper'' program 
may not receive Milestone approvals as the program is unlikely to be a 
highly confident, executable program. 


The acquisition system limits the strategy to the 
Iron Triangle concept of cost, schedule, and scope 
(performance): pick two . 


Ward states in his thesis that simultaneously improving cost, sched¬ 
ule, and performance without reducing complexity leads to failure. 
“Excessive complexity in the organization and the system virtually 
requires project leaders to improve only two sides of the “Program 
Manager's Iron Triangle," while simple organizations can produce simple 
technologies that are simultaneously faster, better and cheaper" (Ward, 
2009, p. 87). We must temper this statement with the realization of what 
is acquired from simple organizations producing simple technologies: 
simple systems. As mentioned earlier, complex, traditional MDAPs do 
not meet this criteria. 

Therefore, the “better, faster, cheaper" strategy is not practical. 
The acquisition system limits the strategy to the Iron Triangle concept 
of cost, schedule, and scope (performance): pick two. Additionally, few 
simple organizations producing simple technologies exist in the complex 


206 


DefenseARJ, July 2013, Vol. 20 No. 2:194-217 





Current Barriers to Successful Implementation of FIST Principles 


business of defense acquisition. Program managers must actively man¬ 
age the trade-offs between cost, schedule, and scope, and be cognizant 
of how altering one will inevitably alter at least one of the other pillars. 

FIST Principles and Performance 

Interestingly, the FIST framework does not include performance 
or quality, at least not in the acronym. Ward states that users must be 
satisfied with system performance to have value; however, the FIST 
framework does not foster high performance. In general, a product 
delivered quickly, cheaply, and simply will not perform as well as one 
with more time, money, and arguably more complexity. In developing a 
new iPhone, would a manager rather have 3 months and $100 thousand, 
or 6 months and $400 thousand? Logically, the performance of the more 
costly program should be greater. The exception is when acquiring 
known capabilities, in which acquiring them cheaper and quicker leads 
to the ability to acquire more, therefore increasing overall performance 
(think bombs and bullets). However, when discussing performance, 
requirements must be revisited. If the requirement is such that it can 
be met using FIST principles, by all means FIST should be adhered 
to. Defense acquisitions have, at times, lost sight of a requirement's 
underlying purpose and delivered gold-plated solutions (solutions with 
unnecessary functionality and capability). This is very important. If 
FIST principles allow a program manager to effectively meet a require¬ 
ment, by all means the FIST methodology should be used. 

For known capabilities, FIST principles are valid and should be 
valued more than gold-plating. For less known capabilities, minimal 
cost and minimal schedule should not be valued above performance, but 
effectively controlled and managed. As opposed to acquiring a known 
capability, “unknown-unknown” risks will surface during development 
that could not have been predicted. Managing a thin budget with no 
schedule slack for these unknown-unknowns is not smart management. 
FIST most certainly reduces unknown-unknown risks if the principles 
were followed during initial concept development and program initia¬ 
tion. However, applying FIST principles after program initiation would 
reduce the program's ability to handle uncertainty. 


DefenseARJ, July 2013, Vol. 20No. 2:194-217 


207 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


The FIST principles are not conducive for higher performance 
systems. Additionally, FIST principles, applied retroactively, limit a 
program's ability to mitigate unknown-unknown risks surfacing during 
development. 

In review, Figure 3 compiles the FIST limitations discussed thus 
far. Now, a logical question would be: Can FIST be applied retroactively 
to programs already drowning in complexity? Additional research must 
be done to more thoroughly answer this question; however, it is gener¬ 
ally believed that rapid and traditional programs are distinct in their 
requirements, goals, priorities, speed, and complexity. To this end, a 
recent Defense Science Board concluded that the Secretary of Defense 
should formalize a dual acquisition path separating rapid and deliberate 
acquisitions (Defense Science Board, 2009). In this case, FIST would be 
much more implementable in the realm of rapid acquisitions due to the 
limitations listed in Figure 3. Whenever the limitations listed in Figure 3 
are not present in an acquisition, or if they can be influenced early during 
program conception, the FIST principles seem to be highly valuable and 
effective in meeting warfighter and taxpayer needs. 

FIGURE 3. FIST LIMITATIONS 

FIST is less conducive for:_ 

• the early phases (pre-Milestone B) of the acquisition system 

• complex, novel programs 

• immature technologies 

• radical innovations 

• delivering future needs 

• mitigating unknown-unknown risks 

FIST Is less conducive when:_ 

• early operator feedback is not feasible 

• multiple users and stakeholders exist 

• performance is foremost value 

• DoD technical guidance mandates complexity 

Implementing FIST principles is constrained by:_ 

• the acquisition process 

• the requirements process 

• oversight 


208 


DefenseARJ, July 2013, Vol. 20 No. 2:194-217 










Current Barriers to Successful Implementation of FIST Principles 


FIST as a Set of Heuristics 

A heuristic is an aid to learning, discovery, or problem solving by 
experimental and trial-and-error methods (Heuristic, n.d.). Maier and 
Rechtin (2009, p. 29) provide a more useful definition in terms of product 
development, describing heuristics as a problem-solving approach “using 
guidelines, abstractions, and pragmatics generated by lessons learned 
from experience” Heuristics can be considered the “art” side of the “art 
and science” of project management and/or systems engineering. The 
human test of a good heuristic is whether an experienced listener knows 
within seconds that it fits the domain it refers to and cannot be proven 
false (Maier & Rechtin, 2009). The value of a good set of heuristics, and 
the practitioner's ability to know when they are applicable in different 
situations, should not be undervalued. The acronym for FIST in itself 
can be considered a set of heuristics: 

• Deliver weapon systems as quickly as practical [Fast]. 

• Deliver at minimal expense [Inexpensive]. 

• Minimize design and system complexity [Simple]. 

• Minimize the size of products and processes [Tiny]. 

Ward concludes his 2009 thesis with a list of FIST heuristics, clearly 
stating the importance of heuristics. Heuristics are particularly useful 
in program management because program management is not a hard 
science, but rather a social discipline (Dyer, Gregersen, & Christensen, 
2011). The existing FIST heuristics are generally too broad or contradic¬ 
tory to be useful or actionable. Meaningful heuristics must be actionable 
to the maximum extent possible (Maier & Rechtin, 2009). For example, 
heuristic No. 3 from Ward's thesis is: “The tortoise was faster than the 
hare.” Heuristic No. 6, however, is the opposite: “The best way to run a 
program is quickly” (Ward, 2009, p. 102). Opposite heuristics degrade 
usability. To render a heuristic more usable, the heuristic usually must 
be de-scoped and more directed to a particular topic. In other words, a 
heuristic that says, “optimally expending funds is vital to success” is 
much less useful than the more focused “rarely expend more than 90 
percent of current fiscal year funds in the first half of the fiscal year.” 


DefenseARJ, July 2013, Vol. 20No. 2:194-217 


209 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


The FIST principles lend themselves well as a set of heuristics 
because each FIST term is relative. A tiny unmanned aerial vehicle 
and a tiny tank are not the same size. A complex fighter aircraft and a 
complex rocket launcher do not have the same complexity. These FIST 
concepts are, by their nature, relative terms that cannot be bounded for 
all situations. No checklist exists proving a system to be sufficiently 
simple, inexpensive, or fast. Therefore, describing FIST through a set of 
heuristics fits nicely because heuristics are generally agreed upon and 
cannot be proven false. 



210 


DefenseARJ, July 2013, Vol. 20 No. 2:194-217 





Current Barriers to Successful Implementation of FIST Principles 


In reviewing the multitude of materials related to FIST and in light 
of the heuristics discussion, the authors offer here a review of the points 
made thus far as a set of heuristics, with the intent of increasing the 
set's usefulness for practitioners. As with all heuristics, we leave it to 
the community of scholars and practitioners to validate the efficacy of 
our recommended additions for themselves. Each grouping of heuristics 
relates to the FIST limitations highlighted in Figure 3 and previously 
discussed. 

The following heuristics relate to the early phases of the acquisition 
system: 

1. For the most relevant end product, start early. 

2. To account for uncertainty, start early (Defense Acquisitions, 

2010 ). 

3. Without flexible requirements, unconstrained schedule analysis 
should be completed before accepting a constrained schedule. 

The following heuristics relate to complex or novel programs: 

1. Complexity must first be understood, then minimized (Ward, 
2005). 

2. At a certain program turning point, increased complexity 
reduces system “goodness” (Ward, 2005). 

3. Define reliability requirements, then minimize complexity to 
achieve these requirements (Ward, 2005). 

4. Minimize complexity until the point when the cost or time 
required becomes more burdensome than the complexity itself. 


DefenseARJ, July 2013, Vol. 20No. 2:194-217 


211 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


The following heuristics relate to innovation and delivering future 
needs with immature technology: 

1. Rapid development approaches involve the user much earlier 
(Ward, 2004). 

2. Rigorous independent analyses hold much more weight than 
internal, program office analyses (for cost, schedule, and techni¬ 
cal maturity in particular). 

3. Overfunding leads to tinkering and restrains innovation (Ward, 
2004). 

The following heuristics relate to tailoring DoD technical guidance 
and processes to each particular system: 

1. Tailor processes to specific systems (Blanchard & Fabrycky, 
1998). 

2. Ensure processes are tempered with rationalism (Naur, 1982). 

3. Don't let a process force a bad decision (Mathiassen & Munk- 
Madsen, 1986). 

4. Don't let a process hold up a good decision (Mathiassen & Munk- 
Madsen, 1986). 

5. Utilizing simple or standard interfaces can help reduce complex¬ 
ity, in turn reducing development costs (Ford et al., 2012). 

6. Utilize “It Depends'' management - maximizing knowledge of the 
environment and situation at hand optimizes decision-making. 


212 


DefenseARJ, July 2013, Vol. 20No. 2:194-217 



Current Barriers to Successful Implementation of FIST Principles 


Lastly, the following heuristics relate to the overall DoD acquisition 
process, including the requirements process and oversight: 

1. Employ simplicity in both acquisition processes and engineering 
development. 

2. Contractors should be allowed to bid their expected schedules 
without fear of being labeled “nonresponsive” (Ward & Quaid, 
2006b). 

3. Pick three from the beginning , or else be prepared to pick three 
and get two (see “Adding Realism to FIST” section of this article, 
discussion on “Fast” and “Inexpensive”). 

4. The project leader's influence over the development is inversely 
proportional to the budget and schedule. 

Conclusions 

Acquisition professionals should carefully consider the current bar¬ 
riers to successful FIST implementation. Realism was added to several 
FIST concepts to impartially assess how the framework relates to cur¬ 
rent practice. Finally, Ward's heuristics were expanded upon to increase 
the usability for practitioners. Interestingly, the Air Force announced 
that its Next Generation Bomber will be managed under the auspices of 
the Rapid Capabilities Office (Reed, 2012). The outcome of this program 
will undoubtedly offer a variety of lessons learned. The degree of innova¬ 
tion, either evolutionary or revolutionary, will be of particular interest 
for the FIST debate. 

Once again, the FIST framework is an excellent revival of what the 
military reformers started: thoughtful inquisition, unyielding drive for 
excellence, wariness of the trade-offs between complexity and simplicity, 
and the needs of warfighters over the needs of politicians and programs. 
However, barriers and limitations exist to successful implementation of 
FIST in all types of acquisition scenarios. 


DefenseARJ, July 2013, Vol. 20No. 2:194-217 


213 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


Author Biographies 

Capt Brandon Keller, USAF, is currently 
the Program Manager, Space Command and 
Control at the Air Force Research Laboratory 
Information Directorate. A graduate of the 
University of Pittsburgh, he also holds an MS 
from the Air Force Institute of Technology. 
Capt Keller has served as a program manager 
in the Global Positioning System Next 
Generation Operational Control System (GPS 
OCX) program, a $1 billion software-centric 
ground control system. Capt Keller's next 
assignment was a staff job leading contractor 
performance assessment processes and vari¬ 
ous staff briefings for the GPS program 
director. His research interests include 
defense acquisition reform and program 
management oversight. 

(E-mail address: brandon.keller@rl.af.mil) 

Lt Col J. Robert Wirthlin, USAF, is an 

assistant professor of Engineering Systems 
at AFIT. A graduate of the U.S. Air Force 
Academy, he also holds an MS and PhD from 
the Massachusetts Institute of Technology. 
He is a member of the International Council 
on Systems Engineering, the American 
Institute of Aeronautics and Astronautics, 
and the Design Society. His research interests 
include: acquisition, engineering manage¬ 
ment, risk, and lean. Previously, he served as 
a systems engineer and a program manager 
at Hill AFB, UT; Los Angeles AFB, CA; and 
Buckley AFB, CO. 

(E-mail address: joseph.wirthlin@afit.edu) 




214 


DefenseARJ, July 2013, Vol. 20No. 2:194-217 



Current Barriers to Successful Implementation of FIST Principles 


References 

Assessment Panel of the Defense Acquisition Performance Assessment 
Project. (2006). Defense acquisition performance assessment report. 
Washington, DC: Author. 

Blanchard, B. S., & Fabrycky, W. J. (1998). Systems engineering and analysis 
(3rd ed.). Upper Saddle River, NJ: Prentice-Hall. 

Burton, J. G. (1993). The Pentagon wars: Reformers challenge the Old Guard. 
Annapolis, MD: Naval Institute Press. 

Chairman of the Joint Chiefs of Staff. (2012). Net-ready key performance 
parameter (NR KPP) (CJCSI 6212). Washington DC: Author. 

Davila, T., Epstein, M. J., & Shelton, R. (2006). Making innovation work. Upper 
Saddle River, NJ: Wharton School Publishing. 

Defense acquisitions: Managing risk to achieve better outcomes. Testimony 
Before the Subcommittee on Defense, Committee on Appropriations, 
House of Representatives. (2010). Testimony of Paul Francis, Michael 
Golden, & William Woods (Report No. GAO-10-374T). Washington, DC: 
Government Accountability Office. 

Defense Science Board. (2009). Fulfillment of urgent operational needs. 
Report of the Defense Science Board Task Force. Washington, DC: 

Office of the Under Secretary of Defense (Acquisition, Technology and 
Logistics). 

Department of Defense. (2007). DoD net-centric services strategy. 
Washington DC: Chief Information Officer. 

Dyer, J., Gregersen, H., & Christensen, C. M. (2011). The innovator’s DNA. 
Boston: Harvard Business School Press. 

Ford, J. S., Colburn, R. M., & Morris, Y. A. (2012). Principles of rapid acquisition 
and systems engineering [Graduate research project]. Wright-Patterson 
AFB, OH: Air Force Institute of Technology. 

Hebert, B. (2011, August). Appropriations, Congress & budget execution: Color 
of money 101. Defense Acquisition University presentation to DMSMS and 
Standardization Conference, Hollywood, FL. 

Heuristic, (n.d.). In Merriam-Webster’s online dictionary. Retrieved from 
http://www.merriam-webster.com/dictionary/heuristic 

Johannson, F. (2006). The Medici effect. Boston: Harvard Business School 
Press. 

Knue, T. G. (1991). Oversight of and within the Department of Defense: Is 
it becoming counterproductive? Unpublished manuscript, Air Force 
Institute of Technology, Wright-Patterson AFB, OH. 

Maier, M., & Rechtin, E. (2009). The art of systems architecting (3rd ed.). Boca 
Raton, FL: CRC Press. 

Mathiassen, L., & Munk-Madsen, A. (1986). Formalizations in systems 
development. Behaviour and Information Technology, 5(2), 145-155. 

Murman, E. (2002). Lean enterprise value: Insights from MIT’s lean 
advancement initiative. New York: Palgrave Macmillan. 


DefenseARJ, July 2013, Vol. 20No. 2:194-217 


215 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


Naur, P. (1982). Formalizations in program development. Behaviour and 
Information Technology, 22, 437-453. 

Neal, M. J. (2004). Establishing a foundation to capture the cost of oversight 
for a major defense program within the information technology 
acquisition community. Unpublished manuscript, Air Force Institute of 
Technology, Wright-Patterson AFB, OH. 

Reed, J. (2012, February 24). AFA: New bomber program ‘underway’. 

Retrieved from DoD Buzz Online Defense and Acquisition Journal at 

http://www.dodbuzz.corm/2012/02/24/afa-new-bomber-program- 

underway/ 

Ward, D. (2004). Doing less with more. Defense AT&L, JJ(6), 30-34. 

Ward, D. (2005). The simplicity cycle. Defense AT&L, 34(6), 18-21. 

Ward, D. B. (2009). The effect of values on system development project 

outcomes (Thesis No. AFIT/GSE/ENV/09-M08). Wright-Patterson AFB, 
OH: Air Force Institute of Technology. 

Ward, D., & Quaid, C. (2006a). FIST Part 5: Putting the pieces together. 
Defense AT&L, 35(3), 30-33. 

Ward, D., & Quaid, C. (2006b). It’s about time. Defense AT&L, J5(1), 14-17. 

Ward, D., Quaid, C., & Mounce, G. (2008). The FIST handbook. Beavercreek, 
OH: Rogue Press. 

Wong, J., & Thompson, R. (2006). Assessment of technical interface 
management for complex Department of Defense satellite systems. 
Graduate Systems Engineering Certificate Capstone Project. Wright- 
Patterson AFB, OH: Air Force Institute of Technology. 


216 


DefenseARJ, July 2013, Vol. 20 No. 2:194-217 



Current Barriers to Successful Implementation of FIST Principles 


Endnotes 

1. “Core Capabilities and Core Rigidities” is the subject of a seminal paper 
by Leonard-Barton in her 1992 Core Capabilities and Core Rigidities: A 
Paradox in Managing New Product Development. 

2. A group of military and civilian analysts emerging in the 1980s opposed 
lengthy, high-technology, complex weapon systems. 


DefenseARJ, July 2013, Vol. 20No. 2:194-217 


217 



A Publication of the Defense Acquisition University 


http://www.dau.mil 


Keywords: Equipment Reutilization, Supply, Surplus 
Property, Operations and Maintenance (O&M), Defense 
Logistics Agency Disposition Services (DDS), Materiel 
Management 


Defense Logistics Agency 
Disposition Services as a 
Supply Source: 

A DoD-Wide Opportunity 


( Capt Nate Leon, USMC, Capt Todd Paulson, USMC, 
and Geraldo Ferrer 


The Defense Logistics Agency Disposition Services (DDS) 
provide centralized disposal management of excess and 
surplus military property. An important component of 
its mission is the reutilization of excess equipment within 
the military services to prevent wasteful purchases within 
the Department of Defense. This research analyzes 
the extent to which the U.S. Marine Corps (USMC) is 
implementing reutilization through DDS as a source of 
supply. The results and recommendations of this study 
will enable decision makers within the USMC and the 
Defense Logistics Agency to address institutional and 
systemic obstacles to maximize reutilization. Some of the 
lessons learned herein may be useful to all the military 
services, resulting in more value from their operations 
and maintenance budgets through reutilization. 


218 


DefenseARJ, July 2013, Vol. 20 No. 2:218-241 


image designed by Diane Fleischer » 






A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


Adopting economical business practices in the Department of 
Defense (DoD) is a national priority. Media coverage of the federal debt 
debate in Congress and in the White House focuses much attention on 
the largest contributors to U.S. federal spending—and the DoD is one of 
them. A low-hanging fruit readily available for immediate DoD savings 
is equipment reutilization—the reuse or initial use of excess or surplus 
property to meet known or anticipated requirements. Reutilization 
already saves the DoD millions of dollars each year by enabling both 
internal and intra-Service transfer of excess supplies and equipment, 
thereby preventing unnecessary purchase of property for which a suit¬ 
able substitute already exists. However, according to reports from the 
Government Accountability Office (GAO, 2005a; GAO, 2005b; GAO, 
2006) and a Department of Defense Inspector General (DODIG, 2011) 
audit, the DoD can and should do much more to capitalize upon the eco¬ 
nomic benefits of reutilization. 


Reutilization already saves the DoD millions 
of dollars each year by enabling both 
internal and intra-Service transfer of excess 
supplies and equipment, thereby preventing 
unnecessary purchase of property for which 
a suitable substitute already exists. 


Within the U.S. Marine Corps (USMC), reutilization occurs at the 
headquarters level via intra-unit transfers of principal end items, and at 
the unit level through the use of Defense Logistics Agency Disposition 
Services (DDS) field sites. DDS is typically used by the USMC for req¬ 
uisitions of consumable supply items, repair parts, garrison furniture, 
clothing, and many other items. 

In this study we show that, to prevent excess purchase of supplies 
and equipment, and to realize significant organizational savings that 
might release funds to be used in other critical areas, the USMC should 
expand use of DDS inventory to meet its current and anticipated require¬ 
ments. The USMC must develop doctrine, standard operating procedures 
(SOPs), supply techniques, and automated systems with this objective. 
The USMC supply and logistics community must foster and reinforce 
the utilization of DDS as truly a first source of supply. 


220 


Defense ARJ, July 2013, Vol. 20 No. 2:218-241 





Defense Logistics Agency Disposition Services as a Supply Source: A DoD-Wide Opportunity 


In the following section, we present the concerns raised inside and 
outside DoD regarding the practices and culture of DoD supply systems. 
We start by discussing reutilization in the USMC, the information tech¬ 
nology (IT) interfaces within DDS and the USMC, and how they enable 
or hinder the use of DDS as a source of supply. Then, we discuss current 
attitudes, assumptions, and initiatives by USMC company-grade supply 
officers (0-2 and 0-3) regarding equipment reutilization, as provided in 
anonymous feedback during the early stage of this study. This is followed 
by a brief discussion and recommendations for improvement. Although 
our discussion is focused on data collected from USMC sources, and the 
lessons herein are specific to the USMC, we believe that similar benefits 
could also be achieved by other military services. 

Prior Concerns 

The War on Terror that has driven DoD operations since September 
11, 2001, has enjoyed much support from the U.S. Congress in the form 
of seemingly limitless financial outlays. A report by Vanguard Advisors 
(Del Mar, 2010) identified a “blank check of sorts” that occurred between 
2001 and 2010 for DoD acquisition, bonuses, pay increases, medical 
care, and morale programs. Because of the warfighting focus during this 
period, significant efforts to transform DoD business practices and econ¬ 
omize within the agency took a back seat to operational requirements. 

However, the ongoing U.S. financial crisis that began in 2007 has 
reminded us of the importance to be good stewards of the taxpayer's 
money. In that vein, making the right choices in purchasing and recog¬ 
nizing the savings opportunities that do not compromise our operational 
requirements remain defense budget imperatives. We believe that reuti¬ 
lization of surplus property is one of these opportunities, as expressed 
by the GAO and other observers. 

GAO Findings 

The reutilization of DoD supplies and equipment continues to be 
a focus of the U.S. Congress (Hast & Warren, 2000; GAO, 2005a). The 
2005 report identified $2.2 billion dollars in “substantial waste and 
inefficiency” (p. 4) because “new, unused, and excellent condition items 
were being transferred or donated outside of DoD, sold on the Internet 
for pennies on the dollar, or destroyed rather than being reutilized” (p. 4). 
The report also found that the DoD purchased at least $400 million of 


Defense ARJ, July 2013, Vol. 20 No. 2:218-241 


221 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


identical commodities in Fiscal Years (FY) 2002 and 2003 instead of 
reutilizing excess items in like-new conditions available in DDS. GAO 
identified numerous examples of DoD equipment sales and donation of 
items that were later requisitioned by the DoD at full acquisition cost. A 
portion of the report (GAO, 2005a) reads: 

We [GAO] requisitioned at no charge a medical instrument chest, 
two power supplies, and two circuit cards. Although these items 
had an original DoD acquisition cost of $55,817, we paid only 
about $5 shipping cost to obtain them. (p. 4) 

We also purchased at minimal cost, over the Internet at 
govliquidation.com, tents, boots, gasoline burners (stove/heating 
units), a medical suction apparatus, and bandages and other 
medical supply items. Although the total reported acquisition 
cost for these items was $12,310, we paid a total of $1,466 to 
obtain them—about 12 cents on the dollar, including buyer's 
premium, tax, and shipping cost. (pp. 4-5) 

Moreover, the report offered 13 recommendations to DoD for improv¬ 
ing reutilization, many of which the Defense Logistics Agency (DLA) 
already had underway or subsequently implemented. These recom¬ 
mendations fell under three major headings: Data Reliability, Physical 
Control of Property, and Commodity Inventory Systems. All of them are 
directed to DLA and/or DDS, with the exception of the fifth and sixth 
recommendations, which are directed at the military services. Table 1 
displays the 13 recommendations (GAO, 2005a). 

Recommendations 2 and 3 are concerned with data reliability, a 
problem that we uncovered in our conversations with USMC supply 
officers. Recommendations 5 and 6 highlighted the importance of the 
military services “to do their part” to help DDS succeed in their mission. 
The last three recommendations, focused on excess property in prime 
condition, would help uncover reutilization opportunities more easily 
than what is done today, possibly generating even more savings than we 
discuss in this article. Most important: We were able to observe in 2011 
many of the same issues that were raised in the GAO (2005a) report. 


222 


Defense ARJ, July 2013, Vol. 20 No. 2:218-241 



Defense Logistics Agency Disposition Services as a Supply Source: A DoD-Wide Opportunity 


TABLE 1. GOVERNMENT ACCOUNTABILITY OFFICE 
RECOMMENDATIONS EXTRACTED FROM REPORT 
NO. GAO-05-277 (2005) 


1 . 

to waive the requirement to verify quantities on turn-ins under 
exempted conditions 

2. 

to assure that excess property receipts are verified and 
processed in an accurate and timely manner 

3. 

to develop a mechanism for linking prime vendor purchase 
transactions to National Stock Numbers (NSN) or other unique 
product identification 

4. 

to develop written guidance and formal training to assist 
personnel and military service turn-in generators in the proper 
assignment of condition codes 

5. 

to provide accurate excess property turn-in documentation, 
including proper assignment of condition codes and NSNs 
based on available guidance [directed to the Services] 

6. 

to establish appropriate accountability mechanisms, including 
supervision and monitoring, for assuring the reliability of turn-in 
documents [directed to the Services] 

7. 

to review excess property loss reports to identify systemic 

weaknesses 

8. 

to resolve identified uncorrected security weaknesses 

9. 

to identify the appropriate number and liquidation sales 
locations needed to handle the sales of excess DLA depot 
property 

10. 

to inspect liquidation contractor facilities and take immediate 
action to correct structural impairments and other deficiencies 

11. 

to consider available options and implement an interim process 
for identifying turn-ins of excess new, unused, and excellent 

condition items 

12. 

to coordinate on the identification of key data elements for 
identifying excess property that should be reutilized 

13. 

to include edit controls in Business Systems Modernization 
(BSM) system design that would reject a purchase transaction 
or generate an exception report when A-condition excess items 
are available, but are not selected for reutilization 


Defense ARJ, July 2013, Vol. 20 No. 2:218-241 


223 

















A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


A Cultural Analysis 

Reusing valuable assets is a practice not as common as desired. To 
institute it as standard practice, the supply organization must change. 
With similar concerns, Doane and Spencer (1997) conducted a cultural 
analysis of the acquisition system within the DoD. Their analysis shed 
light on how individuals in the DoD resisted process changes, in par¬ 
ticular in its acquisition activities. They studied two major Navy and 
Air Force acquisition programs through the lens of mission and strategy, 
goals, means, measurement, and correction. 

Doane & Spencer (1997) defined culture as: 

... a pattern of shared basic assumptions that the group learned 
as it solved its problems of external adaptations and internal 
integration, that has worked well enough to be considered valid, 
therefore, to be taught to new members as a correct way to per¬ 
ceive, think, and feel in relation to those problems, (p. 25) 

They found two prominent cultural obstacles to DoD acquisition 
reform: 

• Little incentive for the workforce to change. Most govern¬ 
ment employees believe there is little competition or threat 
to their organization's existence. Since the DoD operated 
without a profit and loss sheet, the workforce did not feel the 
pressure to meet the bottom line, or the need to take risks. 

• The acquisition system is risk-averse. The acquisition 
system has been quick to penalize employees who make 
mistakes or take risks (also discussed in Ferrer & Dew, 2010). 
The workforce is conservative, strict about following rules, 
and self-preservationists. They are accustomed to routine 
and ordinary work, and are skeptical of initiatives and major 
change. 

According to the authors, aligning the culture of the organization 
with the philosophies of acquisition reform is critical to achieve true ben¬ 
efits. However, this is difficult within the DoD because “most incentives 
and motivations are not apparent for either government or industry” 
(Doane & Spencer, 1997, p. 84). Industry incentives and motivation seem 


224 


Defense ARJ, July 2013, Vol. 20 No. 2:218-241 



Defense Logistics Agency Disposition Services as a Supply Source: A DoD-Wide Opportunity 


to be based on the same profit and loss theories that were present before 
acquisition reform. Concurrently, the only incentives for government 
employees are personal pride in their jobs and respect of their peers. 
However, due to the constant rotation of supervisors within the DoD, 
change is often difficult to achieve (Doane & Spencer, 1997). So what can 
be done to transform culture in pursuing better DoD business practices? 
The researchers recommended leadership that questions old assump¬ 
tions and can overcome organizational inertia and apprehension. 

Additionally, leaders in a changing organization must foster open 
lines of communication and cooperation among other leaders and the 
organization's members. They must be accountable for their actions 
and empower the members of the organization, allowing them to fail 
and question authority without fear of reprisal. This empowerment will 
allow for new perspectives that may encourage innovation and generate 
better business practices (Ferrer & Dew, 2010). Reutilization is a prime 
example that fits this situation: Given that it is an uncommon practice, 
the habit of acquiring used assets from DDS—rather than buying them 
brand new—would require a cultural change. This, in turn, would require 
the support of civilian and military leaders for a continued period of time 
until it coalesced into the SOR However, if the new process is cumber¬ 
some, and if no organizational incentives are in place to change a process 
that is generally accepted, the typical supply professional is unlikely to 
seize and apply the reutilization opportunity to its full potential. 



Defense ARJ, July 2013, Vol. 20 No. 2:218-241 


225 









A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


Analysis of USMC Reutilization Efforts 

In this section, we analyze how current USMC reutilization activi¬ 
ties compare to the acquisition practices in other military services. By 
analyzing actual requisitions processed by USMC supply activities in F Y 
2010 and 2011, we show the potential cost savings that could be achieved 
by an increased use of DDS as a source of supply. 

Potential USMC Cost Savings Through Reutilization 

To get a clearer picture of the potential cost savings that the USMC 
could realize through increased reutilization, we analyzed all Marine 
Corps NSN requisitions inducted during four, 2-week periods through¬ 
out FY 2010 and 2011. These requisitions alone do not show the full 
breadth of total USMC requisitions, and therefore cannot fully capture 
potential cost savings. We had to remove from our analysis many items 
that do not have an NSN or are assigned a 'local NSN” generated by the 
supply management unit. Nonetheless, NSN requisitions exemplify the 
volume of requisition traffic conducted through the USMC's standard 
supply system. We chose specific dates to represent each of the seasons 
of the year to capture a wide breadth of demand patterns. The specific 
dates of analysis were: 

• November 8-22, 2010 

• February 14-28, 2011 

• April 4-18, 2011 

• August 8-22, 2011 

We compared the demand for all NSNs over the specified sample 
periods with on-hand DDS inventory for the same 2-week periods using 
data retrieved from the DDS Management Information Distribution 
and Access System. We extrapolated the information to project the 
cost savings that the USMC could have achieved if it had used on-hand 
and available DDS inventory, to the maximum extent possible, to fill its 
requisitions. We conducted this analysis for two sample groups. Group 1 
included a cost savings roll-up for NSNs listed only in Supply Condition 
Code (SCC) A (like-new condition), while Group 2 consisted of a cost 
savings roll-up for NSNs listed in SCCs B and C (serviceable condition) 


226 


Defense ARJ, July 2013, Vol. 20 No. 2:218-241 



Defense Logistics Agency Disposition Services as a Supply Source: A DoD-Wide Opportunity 


TABLE 2. POTENTIAL SAVINGS FOR DDS AS SOURCE OF SUPPLY 
(SCC A) 



Nov8-22 r 

2010 

Feb 14-28, 
2011 

Apr 4-18, 
2011 

Aug 8-22, 
2011 

Number of 

unique NSNs 

196 

193 

284 

857 

Total 

Acquisition 

Value 

$464,329 

$1,465,013 

$315,153 

$1,104,550 


Sources: DLA, USMC Logistics Command, the authors 
Total potential savings for 8-week period: $3,349,045 

Total potential annual savings: $21,768,793 


Table 2 shows the results of our analysis of Group 1, and Table 3 
shows the results of our analysis of Group 2, indicating the potential cost 
savings for each of the four, 2-week periods. We computed the total acqui¬ 
sition value in the tables by multiplying the quantity available at DDS for 
each unique NSN in the specified period, by the full acquisition price for 
the NSN. Extrapolating the 8-week data, we estimate a potential USMC 
annual cost savings of $21.8 million using SCC A, in addition to $6.7 mil¬ 
lion using SCC B items. Extrapolating further, we estimate that the full 
adoption of condition SCC A and B items from DDS could have provided 
savings of approximately $28.5 million for the USMC in FY 2011. 


TABLE 3. POTENTIAL SAVINGS FOR DDS AS SOURCE OF SUPPLY 
(SCC B) 



Nov 8-22, 
2010 

Feb 14-28, 
2011 

Apr 4-18, 
2011 

Aug 8-22, 
2011 

Number of 

unique NSNs 

66 

104 

2 

143 

Total 

Acquisition 

Value 

$123,853 

$194,053 

$3,080 

$708,296 


Sources: DLA, USMC Logistics Command, the authors 
Total potential savings for 8-week period: $1,029,282 

Total potential annual savings: $6,690,333 


Defense ARJ, July 2013, Vol. 20 No. 2:218-241 


227 











A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


Example of Potential DLA Disposition Services Use 

In the previous section, we used the four sampled periods to describe 
how USMC requisitions represent a wide variety of supplies and equip¬ 
ment—from inexpensive repair parts to major pieces of equipment—all 
of which were available within the DDS inventory at no cost. To further 
isolate potential cost savings that the USMC could have achieved by 
utilizing DDS, Table 4 provides a snapshot of the two most expensive 
items (SCC A only) from each of the four sampled periods. 

Using the USMC NSN requisition data from the four sampled peri¬ 
ods, we compared the total quantities of requisitioned items with the 
total on-hand quantities available at DDS for the same NSNs, during 
the same ordering periods. This analysis allowed us to see what types of 
supplies had the greatest probability of being filled from DDS inventory. 
Table 5 shows some of the most requested supplies for each of the four 
sampled periods that were simultaneously available for issue within 
the DDS inventory. In this small sample, the total value of the requested 
items was $216.8 thousand, of which $188.4 thousand could be supplied 
by DDS—87 percent of the total. The large number of requisitions and 
the high level of availability are enough to warrant reutilization. This 
is especially true when most of these items are SCC A, like-new items. 

Sales Through Government Liquidation 

At the time of this writing, Government Liquidation (GL) was the 
privately contracted company used by DLA to sell excess DDS property 
to the public after the property has undergone the full Reutilization, 
Transfer and Donation (RTD) screening cycle. Sales are conducted using 
a Web-based auction format. According to the DLA, sales through GL 
generated a total of $31.4 million in FY 2009, and $29.6 million in FY 
2010. We analyzed every sale of NSN items conducted by GL for FY 2010 
using a list provided by DLA. We compared this list with the requisi¬ 
tions conducted by the USMC during the four sample periods previously 
described. Our goal was to identify cases in which the DLA sold supplies 
and equipment through GL for which the Marine Corps had a valid need 
in the same fiscal year. We found that 9,909 unique NSNs in SCC A were 
sold in FY 2010 through GL that were also requisitioned by the USMC 
during the four sampled periods. Although we could not determine 
whether these items were available at precisely the same time they were 
requisitioned, our findings nonetheless show that DLA is selling supplies 
to the public for which the USMC possesses a valid need and continues 
to order at full acquisition cost. 


228 


Defense ARJ, July 2013, Vol. 20 No. 2:218-241 



TABLE 4. LIST OF MOST EXPENSIVE REQUISITIONS FOR SAMPLE PERIOD (SCC A) 

Acquisition Quantity Potential 
NSN Nomenclature Value Available Savings 

1010012589638 Slip Ring, Twelve CH $8,023 1 $8,023 


Defense Logistics Agency Disposition Services as a Supply Source: A DoD-Wide Opportunity 


CO 

CO 

LO 

o 

D 

G> 

O 

CN 

G) 

LO 

LO 

CO 

CO 

O) 

CO 

o 

K) 

2 

CD 

CN 

G) 




CO 

NT 

LO 

CN 

cd 

CO 

K) 

GO 

09 - 

09- 

09- 

09- 

09- 

09- 

09- 


- 

- 

00 

CN 

- 

CN 

- 


CO 

o 

CO 

LO 

o 

K) 

2 

D 

LO 

LO 

G) 

CN 

CN 

NT 

CO 

G) 



2 

CO 

R 

CN 

CN 

K) 

CN 

CO 


oo 

09- 

09- 

09- 

09- 

09- 






4— j 





D 


> 

c 

CD 


> 

u 

c 




_Q 

E 


LU 


— 


E 


CO 



_ 


CD 


CO 

4-> 


fD 


CD 

(/) 

a 


< 

c 


CD 


CO 

a 


CD 

'fD 


C 


< 

D 



Z 


CD 

<s> 

4 -J 

o 

o 

-1— * 

F 

CO 

4-J 

CD 

C 

,fD 

F 

"D 

C 

4— » 

-C 

D) 


CD 

o 

"D 

CO 

1— 

(D 

'CD 


CO 

4-> 

z 

C 

CD 

o 

sD 

CD 

"CD 

£ 

4 — 1 


_C 

ro 


E 


CD 

-C 


D) 


C 

CD 

'fD 

_C 

D) 


_l 

Z 

CL 

< 

F 


_l 


CD 

CD 

CD 

N 1 

LO 

CN 

G) 

CN 

G> 

CD 

LO 

s. I 

K) 

CN 

O 

1^ 

CO 

ro 

LO 

D 

CN 

D 

LO 

D 

K) 

|\ 

CO 

o 

G) 

N" 

CO 

N" 

K) 

CO 

K) 

D 

LO 



nJ 







O 


O 

O 

o 

O 

o 

—I 

o 

CD 

LO 

CO 

K> 

LO 

o 

o 

o 

o 

< 

K) 

LO 


K) 

K) 


r 

CN 

CO 

LO 

K) 

LO 

K) 

o 

D 

LO 

CN 

CN 

CN 

CO 

H 


CO 

i- 

O 

_c 

4-> 

=3 

ro 

CD 

_c 

4-> 

~o 

c 

co 

E 

E 

o 

u 

co 

U 

4 — 1 
(/) 

'o> 

o 

—I 

u 

Z 

co 

D 


O 

Co 


Defense ARJ, July 2013, Vol. 20 No. 2:218-241 


229 














A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


TABLE 5. DLA DISPOSITION SERVICES SUPPLY AVAILABILITY FOR 
USMC REQUISITIONS (SCC A) 


NSN 

DDS 

Inventory 

USMC 

Requisitions 

Nomenclature 

Unit 

Price 

8465011150026 

235 

2032 

Canteen, Water 

$5.08 

5660002701510 

1139 

1800 

Post, Fence, 

Metal 

$6.75 

7105009350422 

1063 

1238 

Cot, Folding 

$70.06 

8440005437779 

2300 

1193 

Socks 

$1.45 

6515015217976 

1133 

852 

Tourniquet, 

Non-pneumatic 

$43.50 

8465008600256 

7503 

500 

Cover, Water 

Canteen 

$5.85 

8465011178699 

460 

498 

Bag, Duffel 

$22.90 

5310012349416 

3598 

470 

Washer, Flat 

$0.01 

2590015762424 

329 

274 

Cutter, Cable, 

Vehicle Mounted 

$14.28 

1095015216087 

477 

263 

Bayonet, Knife 

$116.18 

8465014783009 

5010 

240 

Strap, Webbing 

$6.62 

8415012968878 

424 

186 

Vest, Tactical 

Load Carrying 

$48.68 

8460006068366 

320 

158 

Kit Bag, Flyer's 

$28.98 

7240000893827 

74 

154 

Can, Military 

$18.77 

1005005506573 

91 

121 

Case, Small 

Arms Cleaning 

Rod 

$6.82 

6220015164926 

468 

113 

Light, Marker, 

Clearance 

$9.63 

6240000802012 

112 

96 

Lamp, 

Incandescent 

$0.25 


Sources: DLA, USMC Logistics Command, the authors 


230 


Defense ARJ, July 2013, Vol. 20 No. 2:218-241 






















Defense Logistics Agency Disposition Services as a Supply Source: A DoD-Wide Opportunity 


Comparison of USMC Reutilization With Other DoD Services 

Savings from DoD equipment reutilization impacts each DoD 
Service's Operations and Maintenance (O&M) budget. O&M appropria¬ 
tions traditionally finance those items for which the utility is derived 
for a short period of time. They usually comprise expenses rather than 
investments. Examples of costs financed by O&M funds are travel, fuel, 
expenses of operational military forces, training and education, recruit¬ 
ing, depot maintenance, spare parts, and base operations support. O&M 
appropriations are normally available for obligation for one fiscal year 
and are budgeted using the annual funding policy. Equipment reutiliza¬ 
tion has the potential to save O&M funds that could then be reallocated 
to other uses. 

We define reutilization rate as the acquisition value of reutilized 
equipment as a percentage of O&M funding. Table 6 compares the reuti¬ 
lization rates of each military service from FY 2008 to 2010. The table 
shows that the USMC had the lowest reutilization rates over the three 
fiscal years. Although O&M budgets in the other three Services increased 
between 7.8 percent and 9.6 percent, their reutilization expenses shrank 
-11.5 percent (Air Force), -27.2 percent (Army), and -33.0 percent (Navy), 


TABLE 6. DoD REUTILIZATION RATES (2008-2010) 


Service 

Fiscal 

Year 

Operations & 
Maintenance 
Budget 

Amount 

Reutilized 

Reutilization 

Rate 

Marines 

2008 

$9,256,100,000 

$7,715,701 

0.083% 


2009 

$9,757,100,000 

$9,022,663 

0.092% 


2010 

$10,327,300,000 

$8,608,010 

0.083% 

Air Force 

2008 

$43,490,600,000 

$61,250,572 

0.141% 


2009 

$45,388,500,000 

$77,291,963 

0.170% 


2010 

$46,869,800,000 

$54,194,481 

0.116% 

Army 

2008 

$82,838,400,000 

$136,513,483 

0.165% 


2009 

$82,877,200,000 

$104,777,760 

0.126% 


2010 

$90,793,300,000 

$99,352,677 

0.109% 

Navy 

2008 

$39,923,200,000 

$74,296,155 

0.186% 


2009 

$39,847,100,000 

$73,495,085 

0.184% 


2010 

$43,129,600,000 

$49,757,887 

0.115% 


Sources: DLA, Office of the Secretary of Defense, the authors 


Defense ARJ, July 2013, Vol. 20 No. 2:218-241 


231 

















A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


as shown in Figure 1. The Marine Corps was the only Service for which 
the reutilization expense kept pace with the O&M budget increase. 
However, the USMC reutilization rate was steady, but it was low com¬ 
pared to other Services. Considering the reutilization opportunity of 
$28.5 million, previously estimated, the USMC seizes approximately 30 
percent of all excess property that matches its current needs each year. 
In the following sections, we discuss some reasons why the reutilization 
rates at the Marine Corps remain low, and offer some recommendations 
for improvement. 


FIGURE 1. EVOLUTION OF REUTILIZATION IN DoD (2008-2010) 



♦ Marines 
■ Air Force 
a Army 
x Navy 


Summary of Requisition Analysis 

We estimate that the potential annual savings for USMC adoption 
of DDS as a source of supply is $28.5 million in FY 2011. This amount 
represents the full acquisition value of supplies that were on-hand and 
ready-for-issue within the DDS inventory in the same year that USMC 
personnel requisitioned them. Rather than these orders being filled from 
DDS on-hand inventory, they were instead filled with brand new supplies 
and equipment from standard inventory control points and from prime 
vendors at full acquisition cost. 


In some cases, filling non-DDS orders might be necessary because of 
the need for expedited delivery. The lead-time for DDS order fulfillment 
is longer than that of standard order fulfillment. Hence, supplies ordered 
with Force Activity Designator priority 3 or higher are better serviced 
from standard inventory control points. In the case of lower priority sup¬ 
plies with no requirement for expedited delivery, the DDS is invariably 


232 


Defense ARJ, July 2013, Vol. 20 No. 2:218-241 













Defense Logistics Agency Disposition Services as a Supply Source: A DoD-Wide Opportunity 


the most cost-effective choice for requisition fulfillment. The following 
sections discuss some of the reasons why reutilization is not being used 
to its full potential. 

Nonintegrated Systems Impair Reutilization 

The GAO reported that supply management IT systems at DL A that 
enable integration are outdated and are not integrated with the supply 
systems used by the Services (GAO, 2005a). The DLA has IT initiatives 
underway to correct this deficiency; however, the integration of these 
efforts with the systems of all military services is critical. The following 
sections provide more detail on the legacy and emerging IT systems that 
will be key components in improved reutilization efforts. 

USMC Standard Automated Materiel Management System 

The legacy automated materiel management system for the Marine 
Corps is the Supported Activities Supply System (SASSY), which inter¬ 
faces neither with DLAs Enterprise Business System (EBS) nor with 
the automated Digital Accessible Information System (DAISY) used by 
DDS. Instead, USMC supply transactions are first filled, if possible, at 
the supply management unit. If the unit does not have the item in stock, 
it passes the requisition to the Marine Corps Logistics Base Automated 
Information Systems Transaction Router (MAISTR). Such transactions 
are grouped at each unit and transmitted at the end of the working day. 
MAISTR then interfaces with the Defense Automated Address System, 
finally resulting in a requisition at the DLA that is screened by EBS, 
possibly ordered by the Distribution Standard System (DSS), and finally 
shipped to the customer. The entire process from requisition to delivery 
has a wait time of several days, depending upon customer priority and 
conditioned on item availability. 

When the Marine Corps developed SASSY, it was not intended for 
supply chain integration with other DoD supply systems. Soon, SASSY 
and other legacy systems will be replaced by Global Combat Support 
System-Marine Corps (GCSS-MC), currently under implementation— 
a major DoD acquisition program that is aligned with other Services' 
Global Combat Support Systems. It seeks to seamlessly integrate with 
the DLAs inventory and asset visibility systems, providing real-time vis¬ 
ibility to USMC customers. However, GCSS-MC does not improve upon 
SASSY's ability to screen DDS stock since it will not directly interface 


DefenseARJ, July 2013, Vol. 20 No. 2:218-241 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


with DAISY, the DDS inventory management system. Therefore, the 
GCSS-MC alone will not be the solution for reutilization efforts. The 
solution resides in DLAs ability to seamlessly integrate DAISY with its 
national system (EBS). 

DLA Systems 

Prior to the GACfs findings on outdated DLA systems in 2005, the 
DLA had begun an IT transformation effort known as Business Systems 
Modernization. As part of that effort, EBS was introduced in 2006. 
DAISY—fielded in 1990—is still in use and unable to communicate 
directly with the EBS. However, a current DLA initiative known as 
Reutilization Business Integration will integrate all DSS business pro¬ 
cesses within the DLA suite of business applications, by moving all data 
and functions from DAISY into the DLAs DSS and the EBS. Once this 
occurs, it may be possible to directly source supplies from DDS inventory 
in fulfillment of requisitions from all military services. The implications 
for improved reutilization will be profound, provided that DDS can be 
used as a source of supply with the same credibility and accuracy as cur¬ 
rent DoD suppliers. Figure 2 shows the current DLA systems construct. 

Organizational Climate for 
Reutilization at USMC 

Before we started this study, we sought advice from approximately 
300 Marine Corps company-grade supply officers on the types of data and 
reutilization topics we should explore. We e-mailed them using contact 
information available at the Marine Online personnel management Web 
site. Our goal was to use their various insights to help steer our research 
into the most relevant and timely areas. Although this outreach did not 
constitute a scientific survey or formal interview—responses were volun¬ 
tary, anonymous, and used solely for background information to review 
the community's perception on reutilization—we nonetheless received 
some comments that were useful in our study. 

Supply Officer Feedback 

A total of 62 supply officers in the grades 0-2 and 0-3 provided gen¬ 
eral comments about their experiences using DDS as a source of supply, 
as well as their usage level and predominant method of DDS requisition. 
All respondents consented to anonymous citation in this report, sum¬ 
marized as follows. 


234 


Defense ARJ, July 2013, Vol. 20 No. 2:218-241 



Defense Logistics Agency Disposition Services as a Supply Source: A DoD-Wide Opportunity 


FIGURE 2. REUTILIZATION USING DEFENSE LOGISTICS AGENCY 
IT SYSTEMS 






Provide item 

w 

Replenish item 


vendor 


DLA supply depot 





L 





\ 


Request excess items 

w 


Deliver excess 

w 

Receive item 

DLA item mgr 



items 

W 

unit 



■ 

DDS 




Sources: DLA, the authors 


Of the 62 respondents, 41 indicated that they had used DDS on at 
least one occasion to order supplies, while the remainder never used 
DDS for requisitions. The most common method for requisition was the 
RTD Web site, followed by a physical walk-in at an installation DDS field 
site. Most respondents who had used DDS as a source of supply were at 
least somewhat satisfied with the results, though some mentioned errors 
in the accuracy of SCC and item identification—a concern previously 
raised in Report No. GAO 05-277 (GAO, 2005a). 


Defense ARJ, July 2013, Vol. 20 No. 2:218-241 


235 




































A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


Of the respondents who did not use DDS as a source of supply, the 
most common reason was the lack of financial incentive to do so. They 
mentioned that their budgets were sufficiently large to procure new sup¬ 
plies at full cost. 

The procedures to turn in excess, damaged, or obsolete property 
place heavy reliance on the person turning in equipment (known to 
DDS as the “generator”) to ensure paperwork accuracy. However, the 
generators have little incentive to ensure 100 percent accuracy because 
the equipment will no longer be in their custody. DDS employees are often 
unable to provide a redundant check because of manpower constraints 
or lack of expertise concerning the property. The end result is that items 
often enter DAISY with an improper NSN, nomenclature, SCC, or demili¬ 
tarization code (DLA, 2008). 

These inaccurate inventory data were also mentioned in Report 
No. GAO 05-277 (GAO, 2005a), which cites “unreliable excess property 
inventory data” as a root cause for “billions of dollars in waste and inef¬ 
ficiency.” According to Kutz' testimony (DoD Excess Property Systems, 
2005), the DLA implemented several changes, including the consolida¬ 
tion of numerous field sites for better property control, changes to their 



236 


Defense ARJ, July 2013, Vol. 20 No. 2:218-241 












Defense Logistics Agency Disposition Services as a Supply Source: A DoD-Wide Opportunity 


process, and the installment of a Senior Executive Service director to 
oversee the organization. However, our research shows that many USMC 
supply officers remain skeptical of inventory accuracy, based on their 
experience turning in equipment at a DDS site. 

Finally, many supply officers did not use DDS because they were 
simply unaware of the RTD Web site to screen inventory nationally and 
internationally. Students at Marine Corps supply schools are trained to 
use the RTD Web site, but this training should be analyzed for unifor¬ 
mity, rigor, and skills retention. Moreover, occasional refreshers may 
be necessary. This resonates with GAO Recommendation No. 4 (Table 
1). Although DDS does have some resources, they are not sufficient to 
ensure a broad reutilization of excess property in their possession by the 
Services that need them. 


Discussion 

We have shown that some of the reasons for not using DDS are that 
supply officers are frequently unaware of the benefits of using DDS, that 
they lack confidence in the DDS supply chain, or that the IT system is 
inconvenient, hindering frequent utilization. Based on the analysis of 
equipment reutilization with the USMC, we make some observations 
regarding the most notable characteristics of current reutilization efforts. 

Many USMC supply community leaders are not aware of the breadth 
and utility of the DDS inventory as a source of supply, assuming most 
items to be “junk” and therefore ignoring the system. Although supply 
officers receive training on DDS screening at the Ground Supply Officers 
Course, this one-time class is not reinforced in the fleet, and may be eas¬ 
ily forgotten. For this reason, many supply officers opt to use the standard 
supply system, the General Services Administration, and open purchases 
for all orders despite the availability of DDS inventory. 

At the same time, USMC supply professionals may consider DDS, but 
typically do not trust it as a source of supply due to previous experiences 
with the cumbersome turn-in and reutilization processes or misun¬ 
derstanding of the DDS fulfillment process; therefore, they ignore the 
system. Further, DLA and USMC distribution, requisition, and inventory 
management systems are not integrated, and therefore prohibit seamless 
requisitions of DDS supplies using the standard USMC supply system. 


Defense ARJ, July 2013, Vol. 20 No. 2:218-241 


237 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


Consequently, DLA sells supplies and equipment to the public for 
which the USMC holds an ongoing requirement. The Marine Corps is 
not maximizing procurement of available DDS on-hand inventory, lag¬ 
ging behind the other Services, and annually forfeiting approximately 
$28.4 million in potential cost savings. We present a course of action 
that, if undertaken, will enable all military services to achieve substan¬ 
tial cost savings. 


Recommendations 

Until such time that GCSS-MC and DDS are seamlessly linked, the 
Marine Corps Deputy Commandant, Installations and Logistics, should 
establish Service policy in orders, directives, and SOP that requires 
screening available DDS inventory prior to inducting standard supply 
system requisitions, particularly Class II supplies. It should support 
these directives with incentives to the supply community to use DDS 
more frequently. This can be accomplished through performance 
appraisal and through an awards program for reutilization in conjunc¬ 
tion with DLA. 

Moreover, the USMC Logistics Command should maximize the 
untapped potential of the DoD EMALL, which provides on-hand vis¬ 
ibility of DDS supplies in SCC A, by enabling USMC requisitions through 
Military Standard Requisitioning and Issue Procedures. For account¬ 
ability purposes, such requisitions must be visible to the USMC Standard 
Accounting Budget and Reporting System. 

DLA has a national marketing program to raise awareness for the 
potential of DDS reutilization for all military services. A key part of this 
program should be to conduct frequent training visits to supply activities 
of all Services to educate supply officers on the benefits of working with 
DLA Disposition Services. 

DLA Disposition Services should establish a seamless interface 
between its automated information system (DAISY) and DLA's enter¬ 
prise resource planning system (EBS), so that all requisitions placed via 
GCSS from any Service can be filled, to the maximum extent possible, 
by DDS on-hand inventory. 


238 


Defense ARJ, July 2013, Vol. 20 No. 2:218-241 



Defense Logistics Agency Disposition Services as a Supply Source: A DoD-Wide Opportunity 


Finally, DDS must ensure 100 percent accuracy in inventory man¬ 
agement data—most importantly the SCC, NSN, and quantity—so that 
all deliveries meet customer expectations. This reputation-establish¬ 
ment effort must begin at the property receipt process and continue 
through the entire reutilization cycle. If necessary, capacity at property 
receipt points should be overhauled so that DDS employees can effec¬ 
tively manage workload and ensure 100 percent accuracy in inventory 
management data. 

We should not let inertia take control of business decisions. With 
sound leadership, we can start to pay attention to this opportunity 
to improve our expenditures and to do more while spending less. 
Nonetheless, seamless IT integration will be necessary before reuti¬ 
lization becomes common practice. 


Author Biographies 



Capt Nate Leon, USMC, is currently aground 
supply officer in the Logistics Capabilities 
Center at Marine Corps Logistics Base in 
Albany, GA. He has 17 years of active service, 
10 of which he served as an enlisted Marine. 
His career experience encompasses a wide 
range of operational and staff assignments, 
including a tenure as warfighting instructor 
at the U.S. Marine Corps Basic School in 
Quantico, VA. Capt Leon earned a BA in 
Political Science from University of 
Mississippi and an MBA from the Naval 
Postgraduate School. 


(E-mail address: nathanael.leon@usmc.mil) 


Defense ARJ, July 2013, Vol. 20 No. 2:218-241 


239 




A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 




Capt Todd Paulson, USMC, is currently a 
company commander at the Defense Language 
Institute. After attaining a commission as a 
second lieutenant in the Marine Corps in 
August 2001, he served with the 3rd Battalion, 
6th Marines; the Marine Recruit Depot at 
San Diego; a combat logistics unit; and the 
Tactical Training and Exercise Control Group. 
Capt Paulson received his BSc from Arizona 
State University and an MBA from the Naval 
Postgraduate School. 

(E-mail address: todd.paulson@usmc.mil) 

Dr. Geraldo Ferrer is an associate professor 
at the Naval Postgraduate School. His areas 
of expertise include global operations, supply 
chain tracking technologies, sustainable 
technologies, and reverse logistics. He 
received his PhD from INSE AD, a multicam¬ 
pus international graduate business school 
and research institution. Dr. Ferrer also holds 
an MBA from Dartmouth College, a BSc in 
Mechanical Engineering from the Military 
Institute of Engineering in Rio de Janeiro, 
and a BA in Business Administration from 
Federal University of Rio de Janeiro. 

(E-mail address: gferrer@nps.edu) 


240 


Defense ARJ, July 2013, Vol. 20 No. 2:218-241 





Defense Logistics Agency Disposition Services as a Supply Source: A DoD-Wide Opportunity 


References 

Defense Logistics Agency. (2008). The Defense Reutilization and Marketing 
Service (DRMS) program. Retrieved from Defense Acquisition University 
Web site at https://acc.dau.mil/CommunityBrowser.aspx?id=210297 

Del Mar, D. S. (2010). Responding to Secretary Gates' challenge to eliminate 
waste: The time for a true government/industry "team effort” is now! The 
Armed Forces Comptroller ; 55(3), 49-51. 

Department of Defense Inspector General. (2011). Changes are needed 
to the Army contract with Sikorsky to use existing DoD inventory 
and control costs at the Corpus Christi Army Depot (Report No. 

DO DIG-2012-004). Retrieved from http://www.dodig.mil/Audit/reports/ 
fy12/DO DIG-2012-004_REDACTED.pdf 

Doane, D. R., & Spencer, S. D. (1997). Cultural analysis case study: 

Implementation of acquisition reform within the Department of Defense. 
Retrieved from http://lean.mit.edu/search.html?f=3&q=Doane 

DoD excess property systems: Throwing away millions. Hearing before 
the Subcommittee on National Security Emerging Threats, and 
International Relations, The Committee on Government Reform, House 
of Representatives, 109th Cong. 51 (2005) (testimony of Gregory Kutz). 
Retrieved from http://www.gpo.gov/fdsys/pkg/CHRG-109hhrg22905/ 
pdf/CH RG-109hhrg22905.pdf 

Ferrer, G., & Dew, N. (2010). The stigma of failure in organizations. Journal of 
Operations and Supply Chain Management, J(1), 15-33. 

Government Accountability Office. (2005a). DoD excess property: 
Management control breakdowns result in substantial waste and 
inefficiency (Report No. GAO-05-277). Washington, DC: Author. 

Government Accountability Office. (2005b). DoD excess property: 
Management control breakdowns result in substantial waste and 
inefficiency (Report No. GAO-05-729T). Washington, DC: Author. 

Government Accountability Office. (2006). DoD excess property: Control 
breakdowns present significant security risk and continuing waste and 
inefficiency (Report No. GAO-06-943). Washington, DC: Author. 

Hast, R. H., & Warren, D. R. (2000). Better controls needed to prevent misuse 
of excess DoD property (Report No. OSI/NSIAD-00-147). Retrieved from 
http://www.gao.gov/products/OSI/NSIAD-00-147 


Defense ARJ, July 2013, Vol. 20 No. 2:218-241 


241 



A Publication of the Defense Acquisition University 


http://www.dau.mil 



The Defense Acquisition Professional 
Reading List is intended to enrich 
the knowledge and understanding of 
the civilian, military, contractor, and 
industrial workforce who participate in 
the entire defense acquisition enterprise. 
These book reviews/recommendations 
are designed to complement the education 
and training that are vital to developing 
the essential competencies and skills 
required of the Defense Acquisition 
Workforce. Each issue of the Defense 
Acquisition Research Journal (ARJ) will 
contain one or more reviews of suggested 
books, with more available on the Defense 
ARJWeb site. 


We encourage Defense ARJ readers 
to submit reviews of books they believe 
should be required reading for the 
defense acquisition professional. The 
reviews should be 400 words or fewer, 
describe the book and its major ideas, 
and explain its relevance to defense 
acquisition. Please send your reviews 
to the Managing Editor, Defense 
Acquisition Research Journal: 
Norene.Fagan-Blanch@dau.mil. 



iSSSfesK*' 


Featured Book 

Democracy's Arsenal: Creating a 
Twenty-First-Century Defense Industry 

Author(s): 

Jacques S. Gansler 

Publisher: 

Massachusetts Institute of 
Technology Press 

Copyright Date: 

2011 

ISBN: 

978-0262072991 

Hard/Softcover: 

Hardcover, 432 pages 
Reviewed by: 

Professor Steven A. Jones, 

Defense Systems Management College, 
Defense Acquisition University 




Professional Reading List 


Book Review 

Winston Churchill once said “You can always count on Americans to 
do the right thing—after they've tried everything else.” When it comes 
to our nation's defense acquisition system, you would think Winston 
Churchill's quote is spot on. Jacques Gansler describes in his book one 
more transformation needed in government and industry to achieve a 
new, more effective system of national defense. The author contends 
that this transformation is required if this country intends to meet all of 
the challenges to national security the new century brings. As suggested 
by the title, Creating a Twenty-First-Century Defense Industry , 

Gansler clearly outlines the changes he sees as essential in industry 
and the Department of Defense (DoD) business practices. He makes 
a compelling case for globalization of defense business and greatly 
reduced competition. 

This is a well-researched and engaging book. Drawing upon his 
decade of experience in industry, government, and academia, Gansler 
argues that the old model of ever-increasing defense expenditures on 
largely outmoded weapons systems must be replaced by a strategy 
that combines a healthy economy, effective international relations, 
and a strong (but affordable) national security posture. The author 
describes many significant policy decisions made by the DoD over the 
last 30 years. He provides a significant bibliography to support policy 
decisions in the past and provides rationale for why these policies 
should change in the future. His extensive analysis provides the reader 
with detailed pros and cons of each hypothesis made and clearly 
articulates his conclusions. 

One of the many “Globalization Thrusts'' Gansler suggests is to 
increase purchases from foreign sources and to codevelop more 
weapons systems. He refutes those that claim this approach increases 
U.S. Forces' vulnerability and contends globalization brings greater 
efficiency and innovation. He provides specific recommended changes 
in legislation to recognize this global defense market; he also claims 
this approach would establish a more appropriate, risk-based set of 
considerations associated with the vulnerability of U.S. Forces. 

This book is of particular importance in today's defense acquisition 
community as we approach a significant transition. It provides 
the reader with a vast knowledge of studies performed by DoD and 


Defense ARJ, July 2013, Vol. 20 No. 2:242-244 


243 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


Congress that led to policy decisions and legislative actions in the 
past. Having this clear lens of hindsight is important to any decision 
maker in the DoD. This is especially important to Executive Coaches, 
providing a wealth of knowledge in the cache of tools available to 
coaches and their clients. 

This book may spark a new twist to acquisition reform. Is it possible 
Winston Churchill is right? Have we tried everything else? 


244 


Defense ARJ, July 2013, Vol. 20 No. 2:242-244 



Call for Authors 


We are currently soliciting articles and subject matter experts for the 
2013-2014 Defense Acquisition Research Journal (ARJ) print years. 
Please see our guidelines for contributors for submission deadlines. 


Even if your agency does not require you to publish, consider these 

career-enhancing possibilities: 

■ Share your acquisition research results with the acquisition, 
technology, and logistics (AT&L) community. 

■ Change the way Department of Defense (DoD) does business. 

■ Help others avoid pitfalls with lessons learned or best practices 
from your project or program. 

■ Teach others with a step-by-step tutorial on a process or approach. 

■ Share new information that your program has uncovered or 
discovered through the implementation of new initiatives. 

■ Condense your graduate project into something beneficial to 
acquisition professionals. 

Enjoy These Benefits: 

■ Earn 25 continuous learning points for publishing in a refereed 
journal. 

■ Get promoted or rewarded. 

■ Become part of a focus group sharing similar interests. 

■ Become a nationally recognized expert in your field or speciality. 

■ Be asked to speak at a conference or symposium. 


We welcome submissions from anyone involved with or interested 
in the defense acquisition process—the conceptualization, initia¬ 
tion, design, testing, contracting, production, deployment, logistics 
support, modification, and disposal of weapons and other systems, 
supplies, or services (including construction) needed by the DoD, 
or intended for use to support military missions. 


If you are interested , contact the Defense ARJ managing editor (DefenseARJ@ 
dau.mil) and provide contact information and a brief description of your 
article. Please visit the Defense ARJ Guidelines for Contributors at http:// 
www.dau.mil/pubscats/Pages/ARJ.aspx. 











Defense ARJ 

Guidelines for Contributors 


The Defense Acquisition Research Journal (ARJ) is a scholarly 
peer-reviewed journal published by the Defense Acquisition 
University (DAU). All submissions receive a double-blind review 
to ensure impartial evaluation. 


We welcome submissions from anyone involved in the defense acqui¬ 
sition process. Defense acquisition is defined as the conceptualization, 
initiation, design, development, testing, contracting, production, deploy¬ 
ment, logistics support, modification, and disposal of weapons and other 
systems, supplies, or services needed for a nation's defense and security, 
or intended for use to support military missions. 

Research involves the creation of new knowledge. This generally 
requires using material from primary sources, including program docu¬ 
ments, policy papers, memoranda, surveys, interviews, etc. Articles are 
characterized by a systematic inquiry into a subject to discover/revise 
facts or theories with the possibility of influencing the development of 
acquisition policy and/or process. 

We encourage prospective writers to coauthor, adding depth to 
manuscripts. It is recommended that a mentor be selected who has 
been previously published or has expertise in the manuscript's subject. 
Authors should be familiar with the style and format of previous Defense 
ARJs and adhere to the use of endnotes versus footnotes, formatting of 
reference lists, and the use of designated style guides. It is also the respon¬ 
sibility of the corresponding author to furnish a government agency/ 
employer clearance with each submission. 

MANUSCRIPTS 


Manuscripts should reflect research of empirically supported experi¬ 
ence in one or more of the areas of acquisition discussed above. 


246 





Research articles may be published both in print and online, or as 
a Web-only version. Articles that are 4,500 words or less (excluding, 
abstracts, biographies, and endnotes) will be considered for both print as 
well as Web publication. Articles between 4,500 and 10,000 words will 
be considered for Web-only publication, with an abstract included in the 
print version of the Defense ARJ. In no case should article submissions 
exceed 10,000 words. 

Audience and Writing Style 

The readers of the Defense ARJ are primarily practitioners within 
the defense acquisition community. Authors should therefore strive to 
demonstrate, clearly and concisely, how their work affects this commu¬ 
nity. At the same time, do not take an overly scholarly approach in either 
content or language. 

Format 

Please submit your manuscript with references in APA format 
(author-date-page number form of citation) as outlined in the Publication 
Manual of the American Psychological Association (6th Edition). For all 
other style questions, please refer to the Chicago Manual of Style (15th 
Edition). 

Contributors are encouraged to seek the advice of a reference librar¬ 
ian in completing citation of government documents because standard 
formulas of citations may provide incomplete information in reference 
to government works. Helpful guidance is also available in Garner, D. 


247 





A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


L., and Smith, D. H., 1993, The Complete Guide to Citing Government 
Documents: A Manual for Writers and Librarians (Rev. Ed.), Bethesda, 
MD: Congressional Information Service. 

Pages should be double-spaced and organized in the following order: 
title page, abstract (120 words or less), two-line summary, list of key¬ 
words (five words or less), body of the paper, reference list (works cited), 
author's note (if any), and any figures or tables. 

Figures or tables should not be inserted (or embedded, etc.) into the 
text, but segregated (one to a page) following the text. When material is 
submitted electronically, each figure or table should be saved to a sepa¬ 
rate , exportable file (i.e., a readable EPS file). For additional information 
on the preparation of figures or tables, see CBE Scientific Illustration 
Committee, 1988, Illustrating Science: Standards for Publication, 
Bethesda, MD: Council of Biology Editors. Restructure briefing charts 
and slides to look similar to those in previous issues of the Defense ARJ. 

The author (or corresponding author in cases of multiple authors) 
should attach to the manuscript a signed cover letter that provides all of 
the authors' names, mailing and e-mail addresses, as well as telephone 
and fax numbers. The letter should verify that the submission is an 
original product of the author; that it has not been previously published 
in another journal (monographs and conference proceedings, however, 
are okay); and that it is not under consideration by another journal for 
publication. Details about the manuscript should also be included in 
this letter: for example, title, word length, a description of the computer 
application programs, and file names used on enclosed CDs, e-mail 
attachments, or other electronic media. 

COPYRIGHT 

The Defense ARJ is a publication of the United States Government 
and as such is not copyrighted. Because the Defense ARJ is posted as 
a complete document on the DAU homepage, we will not accept copy¬ 
righted manuscripts that require special posting requirements or 
restrictions. If we do publish your copyrighted article, we will print only 
the usual caveats. The work of federal employees undertaken as part of 
their official duties is not subject to copyright except in rare cases. 


248 



July 2013 


Web-only publications will be held to the same high standards and 
scrutiny as articles that appear in the printed version of the journal and 
will be posted to the DAU Web site at www.dau.mil. 

In citing the work of others, please be precise when following the 
author-date-page number format. It is the contributor's responsibility to 
obtain permission from a copyright holder if the proposed use exceeds 
the fair use provisions of the law (see U.S. Government Printing Office, 
1994, Circular 92: Copyright Law of the United States of America, p. 15, 
Washington, D.C.). Contributors will be required to submit a copy of the 
writer's permission to the Managing Editor before publication. 

Policy 

We reserve the right to decline any article that fails to meet the 
following copyright requirements: 

• The author cannot obtain permission to use previously 
copyrighted material (e.g., graphs or illustrations) in the 
article. 

• The author will not allow DAU to post the article in our 
Defense ARJ issue on our Internet homepage. 

• The author requires that usual copyright notices be posted 
with the article. 

• To publish the article requires copyright payment by the 
DAU Press. 


SUBMISSION 

All manuscript submissions should include the following: 

• Cover letter 

• Author checklist 

• Biographical sketch for each author (70 words or less) 


249 



A Publication of the Defense Acquisition University 


http: //w w w. dau. mil 


• Headshot for each author should be saved to a CDR disk 
as a 300 dpi (dots per inch) or high-print quality JPEG or 
Tiff file saved as no less than 5x7. Please note: images from 
Web, PowerPoint, or e-mail will not be accepted due to low 
image quality. 

• One copy of the typed manuscript, including: 

° Title (12 words or less) 

° Abstract of article (120 words or less) 

° Two-line summary 
° Keywords (5 words or less) 

° Document excluding abstract and references (4,500 
words or less for the printed edition and 10,000 words 
or less for online-only content) 

• These items should be sent electronically, as appropri¬ 
ately labeled files, to Defense ARJ Managing Editor, Norene 
Fagan-Blanch at: Norene.Fagan-Blanch@dau.mil. 


250 








Defense ARJ 

Print Schedule 


The Defense ARJ is published in quarterly theme editions. Please 
consult the DAU homepage for current themes being solicited. See print 
schedule below. 


2013 

-2014 

Author Due Date 

Publication Date 

July 

January 

November 

April 

January 

July 

April 

October 


In most cases, the author will be notified that the submission has 
been received within 48 hours of its arrival. Following an initial review, 
submissions will be referred to referees and for subsequent consideration 
by the Executive Editor, Defense ARJ. 


252 








Contributors may direct their questions to the Managing Editor, 
Defense ARJ, at the address shown below, or by calling 703-805-3801 
(fax: 703-805-2917), or via the Internet at norene.fagan-blanch@dau.mil. 



The DAU Homepage can be accessed at: 

http://www.dau.mil. 


DEPARTMENT OF DEFENSE 
DEFENSE ACQUISITION UNIVERSITY 
ATTN: DAU PRESS (Defense ARJ) 

9820 BELVOIR RD STE 3 
FORT BELVOIR, VA 22060-5565 


253 






Defense Acquisition University 

WEB SITE 

http://www.dau.mil 


Your Online Access to Acquisition Research, Consulting, 
Information, and Course Offerings 



Training 

• General Information 

• Login—Distance Learning Courses 

• Student Course Material 

• Course Schedules 

• Course Registration/Cancellation 

Continuous Learning 

• Register for Continuous Learning Modules 

• Browse Continuous Learning Modules 

• Login—Continuous Learning Modules 

• Conferences and Symposiums 

Mission Assistance 

• Acquisition Workshops 

• Consulting Services 

• Team Training 

• Individual Focus 

• Senior Service College Fellowship 


Knowledge Sharing 

• Defense Acquisition Portal (DAP) 

• Defense Acquisition Policy Center 

• Acquisition Community Connection (ACC) 

• Service Acquisition Mall (SAM) 

• ACQuipedia 

• Defense Acquisition Guidebook (DAG) 

• Life Cycle Chart 

• Program Managers e-Tool Kit 

• Ask A Professor (AAP) 

• DAU Virtual Library 

• Media Library 

Research 

Now you can search 
the DAU Web site and 
our online publications! 













Please rate this publication based on the following scores: 

5 - Exceptional 4 - Great 3 - Good 2 - Fair 1 - Poor 


1) How would you rate the overall publication?_ 

2) How would you rate the design of the publication? 

3) Please list all that apply: _ 


True False 

a) This publication is easy to read 



b) This publication is useful to my career 



c) This publication contributes to my job effectiveness 



d) 1 read most of this publication 



e) 1 recommend this publication to others in the acquisition field 




4) What themes or topics would you liketo see covered in future/4R/s? 


5) What topics would you like to see get less coverage in future ARJsi 


6) How can we improve this publication? Provide any constructive criticism to help us to 
improve this publication: _ 


7) 


Please specify your organization: 































NO POSTAGE 
NECESSARY 
IF MAILED 
IN THE 

UNITED STATES 


BUSINESS REPLY MAIL 

FIRST-CLASS MAIL PERMIT NO. 12 FORT BELVOIR,VA 


POSTAGE WILL BE PAID BY ADDRESSEE 


DEFENSE ACQUISITION UNIVERSITY 

ATTN: DAU PRESS 
9820 BELVOIR RD STE 3 
FORT BELVOIR VA 22060-9910 


ImI.ImI.IIImmIImII.mI.ImI.I.I 





















FREE 

ONLINE 

SUBSCRIPTION 


I I Defense ARJ Defense AT&L O 


Thank you for your interest in Defense AT&L magazine and Defense Acquisition Research 
Journal. To receive your complimentary online subscription, please answer all questions 
below—incomplete forms cannot be processed. 

*When registering, please do not include your rank, grade, service, or other personal identifiers. 


□ New Online 
Subscription 


□ Cancellation 

□ Change of E-mail Address 


Date 

Last Name: _ 

First Name: _ 

Day/Work Phone: 


E-mail Address: 


Signature: (Required), 


PLEASE FAX TO: 703-805-2917 

The Privacy Act and Freedom of Information Act 

In accordance with the Privacy Act and Freedom of Information Act, we will only 
contact you regarding your Defense ARJ and Defense AT&L subscription. If you provide us 
with your business e-mail address, you may become part of a mailing list we are required 
to provide to other agencies who request the lists as public information. If you prefer not 
to be part of these lists, please use your personal e-mail address. 









SUBSCRIPTION 









ver 11/28/11 
































BUSINESS REPLY MAIL 

FIRST-CLASS MAIL PERMIT NO. 12 FORT BELVOIR,VA 
POSTAGE WILL BE PAID BY ADDRESSEE