How the Army Ought to Write Requirements
Lt. Col. Thomas “Bull” Holland, PhD, U.S. Army
Download the PDF
The U.S. Army’s recent history is replete with spectacular acquisition program failures. An incalculable number of meetings, symposia, working groups, and studies have been dedicated to “righting the wrongs” in Army acquisition. As the failure of Defense Innovation Unit Experimental 1.0 proved, mimicking the behavior of innovators is not the same as adopting a culture of innovation.1 The “fail fast” mentality of successful innovators is predicated on collecting and analyzing evidence about customer needs (i.e., requirements). The Army’s lack of an evidence-based requirements system is a consistent cause of failure in Army acquisition programs. The Army should adopt a consistently proven industry method for writing the best requirements.
In his 2015 testimony to the Senate, Secretary of the Army John McHugh stated, “The Army’s track record on acquisition programs is too often a tale of failure.”2 There is rarely a single, identifiable root cause for the failure of any acquisition program. Many of the problems with an acquisition program can be overcome after the program is initiated and the error is detected. Decision makers may end a program because the cost of correcting management errors or funding errors is too expensive, but the regulatory tools to make those changes are available. The one error no acquisition program can survive is the one the Army makes all too often—the wrong requirement.
The Army’s most significant acquisition program failure is the Future Combat System (FCS). With a planned cost of almost $200 billion, the FCS is still the most financially ambitious program ever attempted by the Army. The FCS failed for many reasons but shortcomings with requirements were cited as central to the program’s failure. One of the many requirements failure-related lessons learned from the analysis was that “insufficient analysis and mismanagement of expectations can lead to unrealistically ambitious requirements.”3
FCS may have been the largest acquisition failure in the Army’s history, but it certainly was not the only significant one in recent history. The Crusader self-propelled artillery and Comanche helicopter programs were both expensive failures, costing the Army $9 billion.4 As was the case with FCS, these two programs were based on unrealistic requirements, which no amount of time or money could overcome.
The Army’s challenges with acquisition programs has not been limited to developing new-to-the-battlefield technologies. The Army has also recently struggled to procure some of warfare’s most mature technologies. Chief of Staff of the Army Gen. Mark Milley asked the following rhetorical questions about the requirements document for the Army’s replacement of the 9 mm pistol: “This thing has been out there for nine years, ten years? Requirements? A 367-page requirement document? Why?”5 Each of the cases listed have a common thread: opinions were substituted for evidence in the requirements development process.
The Way the Current Process Works
A surface-level understanding of the Army acquisition processes reveals why getting requirements right is critical for any acquisition program. An Army acquisition program must use three systems to produce a result: a funding system, a management system, and a requirements system. The funding system—the Planning, Programming, Budgeting and Execution (PPBE) system—is directed to the Office of the Secretary of Defense (OSD) and by the Office of Management and Budget.6 The management system—the Defense Acquisition Management System—is directed by multiple public laws. However, the requirements system—the Joint Capabilities Integration and Development System (JCIDS)—was created by the Department of Defense (DOD) and is still administered by the DOD.7 All of the acquisition management decisions and budgeted costs for an Army acquisition program are based on the JCIDS requirements documents; if the requirement is wrong, nothing else can be right.
The OSD and each of the services have been requesting changes to acquisition laws for as long as there have been acquisition laws.8 Requests for changes to the Goldwater-Nichols Act have recently given the service chiefs more power in the acquisition system.9 Also, the Weapons System Acquisition Reform Act is a recent law that changed the standards for acquisition management decisions.10 Additionally, the Defense Acquisition Workforce Improvement Act changed certification standards across the DOD.11 The Clinger-Cohen Act established the role of a chief information officer in each service, and the recent Federal Information Technology Acquisition Reform Act further empowered these chief information officers to execute acquisition programs.12 The number of additional requested changes that became neither policy nor law are too numerous to list. Through all of these legal and policy changes, the one system that has remained mostly unchanged is also the one system over which the DOD has almost total change authority—JCIDS. Understanding the requirements development process reveals why the process does not change.
JCIDS sets standards for requirements formatting, staffing, and approval, but this process also relies on the content of the requirements to come from the Army and its sister services. The Army’s process is very robust and is designed to ensure that every aspect of a JCIDS requirement is derived from a defined capability gap, concept, and Army Warfighting Challenge.13 This process also has robust oversight with approvals required from the vice chief of staff of the Army, the Army G-8, the G-8 director of capabilities integration, the director of Army Capabilities Integration Center (ARCIC), the ARCIC Concept and Learning Directorate, and the commanding general of the appropriate Army center of excellence.
The ARCIC director sought to improve this process for writing the requirement for the Ground Combat Vehicle program by adding experts and senior leaders from the Program Executive Office Ground Combat Systems, the Army Research Laboratory, the Tank and Automotive Research Development and Engineering Center, the Army Materiel Command, and the Army G-3.14 Where this process succeeds in collaboration it fails completely in evidence-based content. Every person involved in the writing of these documents offers their opinion but no one is required to offer any evidence to support that opinion.
There are no footnotes, endnotes, or references necessary to define a requirement in a JCIDS document. Neither experimentation nor research is necessary to propose and approve a threshold attribute, key performance parameter, or key supportability attribute in a JCIDS document. This does not mean that decisions in the process are irrational. This also does not mean that there are no interoperability necessities that will dictate space and weight requirements. It does mean that no one is required to justify and then test a hypothesis about any aspect of the requirement. Therefore, the absence of an evidence-based decision-making system leaves a process where only opinions can be provided and discussed.
To mitigate this problem, there are lessons the Army should learn from successful companies to move to an evidence-based requirements process.
Successful Industry Practices
Business models for developing technologies for a profit are end-to-end solutions that cannot be adopted by the Army acquisition enterprise for several reasons, not the least of which is a lack of competition. Every successful business model relies on market forces and competition to drive innovation, efficiency, and productivity. Consumer choices in a competitive marketplace provide companies with evidence for business decisions. The Army cannot adopt models based on a competitive marketplace because the Army is prohibited by law from competing with industry.15 Consequently, the Army requires a model for technology development that is not designed with the end goal of making a profit and does not rely on competition to make evidence-based decisions about requirements.
In his most recent book, Competing against Luck, Dr. Clayton Christensen describes how successful innovators gather evidence about customer requirements to test hypotheses about developing technologies.16 Christensen provides examples across multiple industries that show how successful innovators are those who use an evidence-based approach to correctly define requirements, which he terms “jobs to be done.”17 Christensen is not the only successful author and entrepreneur who saw the power of favoring evidence over opinion and provided concepts the Army could adopt.
For over a decade, Steve Blank has been teaching students across the globe how to be successful entrepreneurs. His course at Stanford University, “Lean Launchpad”, was adopted by the National Science Foundation to teach scientists how to apply this evidence-based approach in order to find the right customer requirements for their discoveries. The National Science Foundation’s adaptation of Lean Launchpad is called Innovation Corps. Blank applied the principles he teaches in Lean Launchpad to see that the Army had a need that no current business model, including his own, could address. His partnership with Alexander Osterwalder, best-selling author of Business Model Generation, produced a business model targeted at the Army’s requirements development shortcomings: the “Mission Model Canvas.”18
Osterwalder’s initial concept, The Business Model Canvas, provides a proven methodology that for-profit businesses may use to connect key segments of their businesses. Osterwalder and Blank adapted the Business Model Canvas after considering the viability of the model in organizations that have no profit motive. The result of this collaboration is the Mission Model Canvas.
Blank continued to evolve the concept of focusing on nonprofit businesses by adapting his Lean Launchpad course to a new course focusing specifically on innovation in defense technology development. Where Lean Launchpad used the Business Model Canvas, the Mission Model Canvas is the basis for Stanford’s new course, “Hacking for Defense.” Hacking for Defense provides a pedagogy that the Army can adopt to move from an opinion-based requirements generation process to an evidence-based requirements generation process.
Hacking for Defense
Hacking for Defense implements an evidence-based requirements system by introducing the concept of a “minimum viable product.”19 The minimum viable product is the most rudimentary prototype that will allow a requirements developer to test a hypothesis. For example, when testing a hypothesis about whether thermal scans of farming fields are useful for farmers or not, a minimum viable product would be a mock spreadsheet of the data produced, not a mock-up of the thermal sensor. The value in this approach is not only the depth of thinking required to form a good hypothesis, but the speed with which requirements developers can confirm or deny each hypothesis. This is what it means to “fail fast.”
This hypothesis testing process is iterative and constantly adds targeted, valuable information to the requirements development process. This is the same way the scientific method works and, more importantly, it is why the scientific method works. Expert opinions matter in developing hypotheses, but only the facts produced from experimentation matter in supporting hypotheses.
The Army University Press announces the publication of Through the Joint, Interagency, and Multinational Lens: Perspectives on the Operational Environment, Volume II. This scholarly work was written by members of the Department of Joint, Interagency, and Multinational Operations (DJIMO) faculty at the U.S. Army Command and General Staff College. The chapters in this volume focus on topics that have a strategic-level planning flavor, are related to the instruments of national power, and will be of interest to all security professionals. The informative commentaries are written in the context of current challenges that our national level leaders face today. It is our hope that the readers will enjoy the authors’ diverse opinions that often challenge conventional ideas while contributing to their understanding of our complex security environment. Download this publication here.
The methodology behind Hacking for Defense is tested and validated. The concepts in the Lean Launchpad course have been taught at Stanford University for over ten years, because they continue to be validated by students who have become successful entrepreneurs. The Innovation Corps curriculum is taught at over a dozen universities, and hundreds of scientists have completed courses. Hacking for Defense was adapted from the successes and lessons learned in each of these courses, and it is designed to address the Army’s most significant requirements development challenge.
The need for changes to prevent further failures in the Army acquisition enterprise is undeniable. The system in this enterprise most in need of change is the requirements process governed by JCIDS. The core problem with how the Army implements JCIDS is the lack of a process and culture that values hypothesis testing and evidence over positional power and experience. The value of an evidence-based requirements process is demonstrated by multiple authors and entrepreneurs. The Hacking for Defense class demonstrates that proven successful business principles can be adapted into a comprehensive system to address the Army’s requirements challenges.
- Ben Fitzgerald, “The DIUx Is Dead. Long Live the DIUx,” Defense One online, 12 May 2016, accessed 8 September 2017, http://www.defenseone.com/ideas/2016/05/ash-carter-diux-pentagon-technology-innovation/128254/?oref=search_The%20DIUx%20Is%20Dead.%20Long%20Live%20the%20DIUx.
- Joe Gould, “McHugh: Army Acquisitions ‘Tale of Failure,’” Defense News online, 19 March 2015, accessed 8 September 2017, https://www.defensenews.com/land/2015/03/19/mchugh-army-acquisitions-tale-of-failure/.
- Christopher G. Pernin et al., “Lessons from the Army’s Future Combat Systems Program” (monograph, Santa Monica, CA: RAND Arroyo Center, 2012), http://www.rand.org/content/dam/rand/pubs/monographs/2012/RAND_MG1206.pdf.
- William M. Robare, “Army’s Failed Programs: Moral Imperative for Change” (Carlisle Barracks, PA: U.S. Army War College, November 2011), http://www.dtic.mil/dtic/tr/fulltext/u2/a565223.pdf.
- C. Todd Lopez, “How Long for a New Pistol?” Army.mil, 21 March 2016, accessed 8 September 2017, https://www.army.mil/article/164604.
- Office of Management and Budget (OMB), OMB Circular A-11 (Washington, DC: The White House, July 2017), https://www.whitehouse.gov/omb/circulars_a11_current_year_a11_toc.
- Chairman of the Joint Chiefs of Staff Instruction 3170.01I, Joint Capabilities Integration and Development System (Washington, DC: U.S. Government Publishing Office, 23 January 2015), accessed 15 September 2017, http://www.jcs.mil/Portals/36/Documents/Library/Instructions/3170_01a.pdf?ver=2016-02-05-175022-720.
- J. Roland Fox, Defense Acquisition Reform, 1960-2009: An Elusive Goal (Washington, DC: U.S. Army Center of Military History, 2011), http://www.history.army.mil/html/books/051/51-3-1/CMH_Pub_51-3-1.pdf.
- Goldwater-Nichols Department of Defense Reorganization Act of 1986, Pub. L. 99-433, 100 Stat. 992 (1986); National Defense Authorization Act for Fiscal Year 2017, Pub. L. 114-328, 130 Stat. 2000 (2016).
- Weapons System Acquisition Reform Act of 2009, Pub. L. 111-23, 123 Stat. 1704 (2009).
- Defense Acquisition Workforce, 10 U.S.C. § 87 (2013).
- The Clinger-Cohen Act, Pub. L. 104-106 110 Stat. 186 (1996); The Federal Information Technology Acquisition Reform Act, Pub. L. 113-291, 128 Stat. 291 (2015).
- Army Capabilities Integration Center, “Army Warfighting Challenges,” last updated 31 January 2017, accessed 2 October 2017, http://www.arcic.army.mil/App_Documents/AWFC-Current.pdf.
- Sydney J. Freedberg Jr., “Army Changing How it Does Requirements: McMaster,” Breaking Defense online, 19 February, 2015, accessed 11 September 2017, http://breakingdefense.com/2015/02/army-changing-how-it-does-requirements-mcmaster/.
- National Defense Authorization Act for Fiscal Year 1997, Pub. L. 104-201, 110 Stat. 2422 (1996).
- Clayton M. Christensen, Competing Against Luck: The Story of Innovation and Customer Choice (New York: HarperCollins, 2016).
- Steve Blank, “The Mission Model Canvas—An Adapted Business Model Canvas for Mission-Driven Organizations,” 23 February 2016, https://steveblank.com/2016/02/23/the-mission-model-canvas-an-adapted-business-model-canvas-for-mission-driven-organizations/; Alexander Osterwalder et al., Business Model Generation: A Handbook for Visionaries, Game Changers, and Challengers (Hoboken, NJ: John Wiley & Sons, 2010).
- Steve Blank, The Startup Owner’s Manual: The Step-by-Step Guide for Building a Great Company (Pescadero, CA: K&S Ranch, 2012).
Lt. Col. Thomas “Bull” Holland, U.S. Army, is the deputy director for support, Security Assistance Directorate, Office of Security Cooperation-Iraq. He holds a BS from Virginia Tech, and an MS and a PhD from the University of Missouri–Rolla. His assignments include service as an assistant professor at the United States Air Force Academy and staff officer on the Department of the Army Staff.