• Online Exclusive Article Archive

AI Integration for Scenario Development

Training the Whole-of-Force

Maj. Robert A. Coombs, U.S. Army

 

Download the PDF

 

Where We Are Now (Scenario, Thesis, Challenges, Model)

Imagine an indigenous hostile insurgency took over Indonesia’s government within three months supported by a first-world foreign government. The insurgents are equipped with cutting-edge weaponry, superior electronic warfare equipment, littoral defense weapons, and support for space-based attacks on U.S. assets. The contested environment is highly urbanized and requires Department of State involvement with assigned political advisors to U.S. brigade combat teams (BCT). The insurgents developed multiple commercially made unmanned underwater vehicles and unmanned ground vehicles with direct-fire capabilities. The Department of Defense (DOD) orders the Combat Training Centers Directorate (CTCD) to develop a fully established training scenario and employ it within twenty days to prepare units for combat operations within the archipelago. The scenario should address the new weaponry, the country’s multitude of cultures, various civil considerations, space-based warfare, electronic warfare, and unconventional warfare in a littoral combat zone. In this fictitious situation, the military would be hard-pressed to rapidly develop a scenario with the appropriate depth and coverage to facilitate an immersive, whole-of-government environment scenario capable of preparing soldiers for the evolving threat.

Artificial intelligence (AI) allows military planners the potential to rapidly adjust training scenarios in support of evolving developments and changes on the battlefields. One of the greatest challenges in the current operating environment is to keep pace with evolving technology. High technology capabilities such as hypersonic weaponry and space-domain warfare lead to a plethora of theories of how warfare might play out in the future. Policy think tanks such as the Brookings Institution propose technologies such as loitering munitions, AI-driven unmanned underwater vehicles, and AI drone swarms will become commonplace in our next large-scale conflict. However, Ivanka Barzashka, director of the Wargaming Network at King’s College, posits that many scholars are currently not working on developing AI-driven wargaming and epistemology required for modeling advancements in the next generation of warfare.¹ While we might be able to wage the next war using AI, we may be missing a critical opportunity to train with the same technology to create a whole-of-force training environment.

The military finds itself in a situation where it must increase its flexibility and the speed with which it is able to train forces in an evolving battlefield environment. Adoption of AI-based capabilities to develop scenarios in support of the CTCD can increase those training centers’ abilities to adapt to changes in foreign threats while allowing for scenarios to be adapted to BCT performance metrics “on the fly” during a rotation. The DOD’s opportunity to increase the flexibility of scenario development at pace with operations and changing geopolitical conditions can be realized with the application of AI assistance.

Over the previous two decades, the military’s task of training forces for operations focused on the low-technology opponents in either Iraq or Afghanistan. Soldiers trained for patrol-based operations focused on individual targeting at the company level. The counterinsurgency (COIN) environment lasted over two decades and required a relatively similar scenario for soldiers in both environments. Soldiers focused on similar cultures and asymmetric threats, targeting isolated population centers within large maneuver areas. Doctrine and COIN operations slowly evolved in parallel with the training environment, with scenario development complete in 2004, counterinsurgency manuals developed in 2006, and integration of a training strategy for COIN operations in 2008.² Scenario development and design for the changing environment took approximately seven years from the beginning of the Global War on Terrorism to replicate the battlefield’s asymmetric nature. The CTCD effectively adapted the scenario over the years for a low-technology opponent but might not have that same luxury during the next phase of warfare, which will be more complex, more joint, and more likely to involve large urban areas.

Following Russia’s invasion of Ukraine and the DOD’s growing interest in littoral combat in the Pacific, scenario development to prepare forces for America’s next combat environment is gaining the attention of military decision-makers. Maj. Gen. Curtis Taylor stated the next battlefield will be “bigger, faster, and more complicated,” than our previous COIN fights.³ The National Training Center’s (NTC) professional opposition force, the 11th Armored Cavalry Regiment, expanded proficiencies into using low-earth orbit satellites, drone-based swarm techniques, and social media to create dilemmas on training forces.⁴ To rapidly include these capabilities into the scenario, multiple documents such as city studies, cultural studies, economic assessments, government studies, and social studies require adjustment and development. The ability to rapidly adjust the scenarios with adequate depth to provide immersive experiences and train forces is a significant challenge to overcome due to its substantial required manpower.

Thomas C. Greenwood, Terry Heuring, and Alec Wahlman of the Institute for Defense Analysis posit that the next training revolution will require “an innovative and rigorous campaign of training and experimentation in order to become competent at conducting joint/combined all domain operations at scale.”⁵ The training scenarios must become more robust to facilitate political and military approaches to combat and competition as the United States pivots to great power competition. Great power competition training inherently requires joint approaches to conflict, robust diplomatic activity, and scenarios that merge the political-military environments. CTCD’s approach must remain flexible to allow for exploitation of small advantages under various battlefield conditions and environments. It should allow for rapid application of cyber, space, information warfare, and consolidation operations in concert with other governmental elements of power. But most of all, its scenario development needs to be rapidly created with depth and richness for soldiers to immerse themselves.

The most likely solution to facilitate future CTCD scenario changes is the adoption of AI as a cornerstone of operations. Any approach to AI’s application must conform with national security strategies, be nested with CTCD training capabilities, fix current limitations, and be a viable approach for AI by 2030. This article suggests the DOD is well poised to take advantage of the growing capability of AI for scenario development not only for combat training centers but for employment in the whole force. The commercial sector is already harnessing AI’s ability to provide scenario-based training, and it will be an inevitable addition to the DOD’s arsenal.

National Strategic Guidance

AI has become a buzzword in the modern DOD lexicon, and its application will continue to be a large driving force in managing immense and ever-growing datasets associated with a technology-driven military force. The 2022 National Security Strategy posits that the DOD’s investment in AI for “cyber and space domains, missile defeat capabilities, trusted artificial intelligence, and quantum systems” will be required to ensure overmatch of battlefield capabilities with our competitors.⁶ The DOD’s 2023 Data, Analytics, and Artificial Intelligence Adoption Strategy proposes the government’s desired outcomes are “battlespace awareness and understanding,” along with “adaptive force planning and application.”⁷ The strategy states AI technology will become ubiquitous in the DOD in the upcoming generations, with new soldiers employing AI-based capabilities as part of daily activities.

The use of AI specifically in support of training objectives is not specified in the National Security Strategy nor the National Defense Strategy for the military. The Artificial Intelligence Integration Center, part of the Army technology transfer program, is spearheading the integration of AI into the U.S. Army and primarily focusing its use on capabilities such as long-range precision fires and soldier lethality. Much of the attention of current AI efforts remains primarily within the autonomous vehicle and logistics improvement fields. Out of the ten AI modernization portfolios of the Artificial Intelligence Integration Center, development for training integration does not appear to be a core priority. While focusing on the forward edge of combat should be the rightful priority of the integration of AI, there appears to be a missed opportunity on its integration in training development.

Guidance on the employment of AI is still in its infancy. There appears to be growing interest by Congress to regulate AI in areas of copyright infringement, ethical use of the technology, and appropriate integration.⁸ Elon Musk and other AI program developers programs have discussed the benefits and dangers to Congress regarding these emerging technologies.⁹ But, with the median age of Congress resting at 58.9 years old and 65.3 years old for the Senate, the chance for effective military policy guidance and discussion on emergent AI out of the legislative branch may be delayed until the next decade until the use of these technologies becomes more commonplace.¹⁰

Current NTC and Training Environment

Over the course of three months, the NTC developed two new scenarios for combat force rotations, creating a fightable scenario at the NTC for large-scale combat operations (LSCO). The scenarios are effective adjustments to the training of maneuver warfare for the various BCTs, whose primary focus in LSCO is to close with and destroy the enemy through maneuver. Current scenarios apply to the common mission essential tasks (MET) we would expect a maneuver BCT to accomplish during isolated force-on-force conflict and defensive operations generally devoid of diplomatic, informational, and economic considerations. Scenario challenges begin to arise when questions regarding civic and governmental activities become entwined with a unit’s maneuver plan. Considerations such as local security forces in cities, local medical care facilities, and electrical power disruptions during combat operations are not significantly important to BCTs on the offense, and the time required to import these considerations in the scenario is tedious.

Creating the required depth of scenario to adequately immerse soldiers is a monotonous and time-consuming task, and typically not in the general skill set of individuals currently in the military or retired in a general schedule position. The scope of knowledge on intelligence functions, civil affairs, military police, and public affairs, along with a plethora of nonmilitary information required for a scenario developer, is generally triaged to allow a BCT to achieve their METs within force-on-force operations. Scenario writers put in significant work to develop these environments for force-on-force operations, but the data required to create a fully developed scenario is notwithstanding a daunting task for these individuals.

To illustrate the point, during a recent rotation at the NTC, observers, coaches, and trainers oversaw the surrender and tactical questioning of enemy prisoners on the battlefield. What was rapidly apparent was the enemy prisoners of war and the rotational unit soldiers had little knowledge about the political and social constructs of the scenario. To the infantry soldier, these factors didn’t affect the MET of closing with-and-destroying the enemy. Yet, when the BCT began asking for clarification from the forces, specifically to answer command priority intelligence requirements, the opposition force soldiers were unable to provide the requisite data owning to the lack of depth of the scenario. The second- and third-order effects to this lack of information stunted public affairs’ ability to produce holding statements for the commanders, confusion on identifying if the prisoners of war were civilians or military, lack of intelligence for assessment of enemy capabilities, and a lack of understanding of the enemy’s morale. The scenario lost the opportunity to become a whole-of-force training event.

Previous attempts to conduct scenario development at the NTC required a large workforce comprised of experts along with the time to analyze and update events. Depth of scenario became richer over time in the early 2000s due to the focus on a single environment for two decades. These training events evolved to cover patrol base operations and focused primarily on improvised explosive devices and ambush operations. Technologically, the warfare focused on an asymmetric opponent and took over three years to develop. With a refocus on great power competition and more advanced opponents, scenario development must be more rapid and robust.

NTC Initially Tests AI-Assisted Scenario Development

In 2023, the NTC collaborated with the Scale Donovan AI platform to explore integrating AI into NTC training operations. The initial conceptional solutions for AI integration ranged from querying supply chain management to supporting intelligence operations. Donovan demonstrated three unique capabilities that were relevant to our research: the ability to leverage multiple language models, the ability to create isolated searchable databases up to controlled unclassified information on a government approved system, and the ability to create prefabricated data forms for repeated queries under similar constraints. Donovan demonstrated the base requirements to augment scenario development and increase workload throughput with our limited on-hand resources.

The initial phase of testing began with using unclassified scenario data to automate scenario development and support research-based psychological operations series documentation. We rapidly identified that the scenario documentation was robust enough to facilitate a BCT to conduct offensive operations in a LSCO environment in the base dataset, but the scenario was not data-rich enough for generative AI to accomplish its tasks. To fill in gaps in the training data, we queried the database using a repository of doctrine as the AI’s base of reference material to identify gaps within the scenario. In one instance, AI informed us there was a lack of depth of information in the areas of sewage, water, electricity, academics, trash, medical, safety within the scenario’s towns for units to conduct civil affairs, and consolidation activities following combat operations. AI’s ability to reference isolated data sets was based on Army doctrine. It identified limitations that allowed us to target gaps of information. The uniqueness of Donovan to accomplish this task demonstrated a future capability to identify limitations in scenario development and propose datasets and scenario creation to fill in the information.

The second capability we tested was generative project design form that allowed the user to set the parameters of the request in a replicative manner. The model allows the user to set guidelines for querying a specific topic, develop a research methodology, and provide a refined response through AI-generated reports. To exercise this capability, we set forth to achieve an 80 percent solution generated through AI for the development of a psychological operations target audience assessment worksheet. This social science document is comprised of ten subsections that assess elements such as a target audience susceptibility, vulnerability, accessibility, and measures of effectiveness for charting an individual’s behavioral change. After identifying the proper lexicon for querying the databases, we identified that AI was capable of finding a 70 percent solution for our requests. The AI language model’s limitations included challenges in defining social science terminology and its inability to understand certain contextual understandings within doctrinal documents such as Field Manual 3-53, Military Information Support Operations, and Technical Manual 3-53.11, Influence Process Activity: Target Audience Analysis.¹¹ We also identified that the database we used was not robust enough and current language models are not capable at this time to exercise social science models for speculative research. Regardless of the limitation, the AI demonstrated an ability to reduce a task by an estimated ten hours of workload and increase efficiencies to the researchers.

Collaboration on the use of AI at NTC will continue to be of interest to the force. If the current DOD’s guidance and projections remain consistent, its capability will eventually be integrated into scenario development. AI will eventually support the force’s ability to tailor scenarios for individual BCTs and adjust them to geographic and diplomatic changes. We can even posit this capability will eventually proliferate down to BCTs themselves, providing units the ability to create scenarios unique to companies during situational training exercises. This capability will become especially profitable to the special operations community, who routinely work in various shades of diplomacy, information, military, and economic environments at the detachment level. Units such as military police, civil affairs, chaplains, judge advocates, and cyber soldiers will especially benefit from the richness of scenario creation that is currently unrealized in many unit training exercises due to the required manpower to create scenarios that support both force-on-force maneuver and other martial activities.

Current Limitations and Opportunities

The phrase “garbage in, garbage out” accurately depicts the current status of many AI generative programs. Remedial attempts to use AI generative imagery platforms demonstrates these programs have challenges in writing words and uniquely with the drawing of hands. In text-based use, outputs commonly lack the depth and clarity one would expect of human-produced products. Various attempts to create Army written documents without a robust dataset generally leads to an underwhelming output. These documents may not fit proper formats, even when asking the program to use Army Regulation 25-50, Preparing and Managing Correspondence, as the base of instructions.¹² Additionally, when testing various AI language models, it demonstrated limitations in developing information that could be called “creative,” as the capability only allows replicative creation of reference information. For all the potential use of AI systems, we are witnessing a capability still within its infancy.

AI currently produces what I will refer to as constrained and unconstrained coding. Constrained coding refers to systematic modeling used to truncate information down to a specific answer; this is similar in concept to the text-to-model prompting. This type of coding is exemplified in the use of AI to identify armored vehicles in large data sets of satellite imagery. The goal of constrained coding is to decrease ambiguity to provide a specific response. The second format is unconstrained coding, or the use of human linguistics to query a database with increased ambiguity of response. Unconstrained coding is applied in systems like ChatGPT to query, “create a fictitious city scenario in the city of Radesna and the required data for a SOF civil affairs unit to conduct civil reconnaissance.” AI has the capability to identify the components of a civil reconnaissance but is limited by what data it can provide on made up groups like civil resistance organizations and local nonprofits. Unconstrained coding requires Socratic querying and multiple developed research models. A lack of a robust database of examples and AI’s creativity challenge currently pose issues when attempting to leverage AI to produce hypothetical questions or scenarios in which it must combine desperate data through contextual clues.

The lack of creative ability and understanding of contextual clues within current AI models provides an opportunity for library science professionals to become a critical element to the military’s application of AI. Some AI companies are already identifying the need for librarians as “prompt engineers,” or individuals well-trained in linguistic modeling and database systems capable of identifying proper database “prompt” requests.¹³ Like the movie Office Space highlights, engineers and the users of the technology have a communication barrier between them, and someone needs to be able to translate between the two. Prompt engineers identify the proper research model, lexicon, and chain of prompts required for AI to arrive at the proper response. Individuals with degrees in library science are uniquely poised to fill the military’s future need for AI use and is a potential recruitment option to bridge AI platforms with capability requirements.

Conclusion: The Future of AI-Assisted Scenario Development

As AI becomes more common, we will move from the era of the “digital native” to that of the “artificial intelligence native.” Currently, technology and language models are still in their infancy but demonstrate a potential for application in scenario development and real-time scenario manipulation. Researchers have been working on AI-facilitated Dungeons and Dragons scenario campaign design with role players able to immerse themselves in a world that provides storylines and fills in missing gaps of information as the players attempt novel strategies.¹⁴ This is no different than soldiers asking for unique and creative information to solve complex issues that may appear to be outside current doctrine.

The future of AI in training scenario development provides significant opportunity to rapidly adapt rich exercise opportunities for the force. Once significant data sets are developed, AI will be a formidable training aid that not only supports the CTCD operations but can also provide a capability to use in schoolhouses to train new soldiers. Leaders can create robust environments to link BCT operations to political and social environment conditions. AI can provide opportunities for exercise directors to change conditions without the fear that the scenario isn’t robust enough to adapt to changing conditions.

Developing scenarios that improve with evolving technology will be critical to ensure the United States maintains a relative advantage in the global environment. The ability of CTCD to develop robust scenarios for a multitude of rotations will allow the flexibility and personalization that a BCT desires. For example, if a BCT is training for rotations to Africa, AI-supported scenario development can identify the parameters of the partner force, opposing force, civil considerations, and basic demographics to define an appropriate scenario that allows for unique aspects of that unit’s mission. Through AI-assisted development, the labor will transition skillsets to becoming prompt engineers and scenario quality control managers.

As researchers testing AI’s ability to construct the Dungeons and Dragons scenarios noted, current technology is capable of scenario development, “yet, more work must be done before we deploy them to our users.”¹⁵ The current limitations of AI should not slow down the military’s adoption of the technology but demonstrate the need of the DOD to invest more into the technology. Similar to how the Army promotes our soldiers on potential, we should also promote AI into our training scenario development on its potential to improve the training experience of our soldiers.

 

---

Notes

1. Ivanka Barzashka, “Wargames and AI: A Dangerous Mix that Needs Ethical Oversight,” Bulletin of the Atomic Scientists, 4 December 2023, https://thebulletin.org/2023/12/wargames-and-ai-a-dangerous-mix-that-needs-ethical-oversight/Ivanka.
2. Thomas C. Greenwood, Terry Heuring, and Alec Wahlman, “The ‘Next Training Revolution’: Readying the Joint Force for Great Power Competition and Conflict,” Joint Force Quarterly 100 (1st Quarter, January 2021), https://ndupress.ndu.edu/Media/News/News-Article-View/Article/2497134/the-next-training-revolution-readying-the-joint-force-for-great-power-competiti/.
3. Todd South, “New Challenges Await Army Units at the National Training Center,” Army Times (website), 29 March 2022, https://www.armytimes.com/news/your-army/2022/03/29/new-challenges-await-units-at-the-national-training-center/.
4. Ibid.
5. Greenwood, Heuring, and Wahlman, “Next Training Revolution.”
6. The White House, National Security Strategy (Washington, DC: The White House, October 2022), https://www.whitehouse.gov/wp-content/uploads/2022/11/8-November-Combined-PDF-for-Upload.pdf.
7. U.S. Department of Defense, Data, Analytics, and Artificial Intelligence Adoption Strategy (Washington, DC: U.S. Department of Defense, 27 June 2023), https://media.defense.gov/2023/Nov/02/2003333300/-1/-1/1/DOD_DATA_ANALYTICS_AI_ADOPTION_STRATEGY.PDF.
8. Christopher Zirpoli, “Generative Artificial Intelligence and Copyright Law,” CRS Legal Sidebar LSB10922 (Washington, DC: U.S. Congressional Research Service [CRS], 2023), https://crsreports.congress.gov/product/pdf/LSB/LSB10922.
9. Claudia Grisales, “The Who’s Who of the Tech World Meet with Senators to Debate Plan to Regulate AI,” NPR, 13 September 2024, https://www.npr.org/2023/09/13/1198994746/top-tech-leaders-are-to-meet-with-u-s-senators-on-the-future-of-ai-regulation.
10. Drew Desilver, “House Gets Younger, Senate Gets Older: A Look at the Age and Generation of Lawmakers in the 118th Congress,” Pew Research Center, https://www.pewresearch.org/short-reads/2023/01/30/house-gets-younger-senate-gets-older-a-look-at-the-age-and-generation-of-lawmakers-in-the-118th-congress/.
11. Field Manual 3-53, Military Information Support Operations (Washington, DC: U.S. Government Publishing Office [GPO], 2023).
12. Technical Manual 3-53.11, Influence Process Activity: Target Audience Analysis (Washington, DC: U.S. GPO, 2024).
13. Army Regulation 25-50, Preparing and Managing Correspondence (Washington, DC: U.S. GPO, 2020).
14. Nora Quiroz, “A New Profile for Librarians Brought by AI,” Libraries (blog), Sage, 16 May 2023, https://group.sagepub.com/blog/a-new-profile-for-librarians-brought-by-ai.
15. Jose Santiago et al., “Rolling the Dice: Imaging Gernative AI as a Dungeons & Dragons Storytelling Companion” (Preprint, submitted in 2023), https://www.researchgate.net/publication/369792636_Rolling_the_Dice_Imagining_Generative_AI_as_a_Dungeons_Dragons_Storytelling_Companion.

 

 

 

Back to Top