Adapting the Art of Design

A PME Game Design Framework

Download the PDF

Abstract

Matrix games are becoming increasingly popular in professional military education (PME). Jesse Schell’s (2020) The Art of Game Design: A Book of Lenses provides a proven framework for designing and evaluating matrix games in PME. I adapt this framework to examine three matrix games used at the U.S. Army War College to develop warfighting skills. These matrix games can be effective methods to assess student learning and develop student skills if properly designed and executed, whether in residence or online.

 

The use of matrix games in professional military education (PME) as a form of experiential learning can provide an effective way to help students “develop practical warfighting skills,” which is one of the critical tasks listed in the recent PME guidance Developing Today’s Joint Officers for Tomorrow’s Ways of War: The Joint Chiefs of Staff Vision and Guidance for Professional Military Education & Talent Management (Joint Chiefs of Staff, 2020).

PME schools must incorporate active and experiential learning to develop the practical and critical thinking skills our warfighters require. These methodologies include use of case studies grounded in history to help students develop judgment, analysis, and problem-solving skills, which can then be applied to contemporary challenges, including war, deterrence, and measures short of armed conflict. Curricula should leverage live, virtual, constructive, and gaming methodologies with wargames and exercises involving multiple sets and repetitions to develop deeper insight and ingenuity. We must resource and develop a library of case studies, colloquia, games, and exercises for use across the PME enterprise and incentivize collaboration and synergy between schools. (Joint Chiefs of Staff, 2020, p. 6)

The use of wargames goes back at least to the 19th century and the Prussian Kriegsspiel (Curry, 2020, p. 34). In wargames, the “sequence of events affects and is, in turn, affected by the decisions made by the players representing the opposing sides” (Perla, 1990, p. 263). Matrix games, originally developed by Chris Engel, are a type of wargame that is facilitated, uses role playing, and relies primarily on player arguments and an element of chance to “determine the success or failure of player actions” (Bae et al., 2019, p. xxv). Matrix games are flexible, scalable, and adaptable, characteristics that provide advantages within the PME environment.

Matrix games are used to develop warfighting skills at the U.S. Army War College (USAWC) in three very different approaches: small scale (seminars of 16 students), large scale (multiple seminars), and large scale (online). I provide a matrix game design framework that can create a more immersive learning experience and better develop those practical warfighting skills called for by the Joint Chiefs of Staff.

A Framework for Designing and Evaluating Matrix Games

In his award-winning book The Art of Game Design: A Book of Lenses, Jesse Schell (2020), a game designer and distinguished professor at Carnegie Mellon’s Entertainment Technology Center, described four basic elements of a game: story, aesthetics, technology, and mechanics. The first three elements have been adapted for application in designing matrix games for PME: scenario (for story), experience (for aesthetics), and resources (for technology). The following sections further clarify the adaptation of Schell’s elements for this framework.

Scenario

Scenarios provide the warfighting context for the game. For most matrix games, this means a narrative scenario describing a region, problem set, or set of players in the strategic, operational, or tactical environment. The scenario can be fictional or historical and can be rooted in the past, present, or future. Because students at USAWC are mostly senior military leaders, they are often skeptical of games if the scenario is not believable or does not resonate with their experiences or studies. Well-designed games often generate changes to the scenario through player interactions that add to game dynamics and opportunities for cooperation and conflict. This adds to the immersive experience of the player.

Experience

The most critical aspect of gaming in the PME context is the learning experience and how effective it is in developing warfighting skills. Skillful game design within PME combines immersive aesthetic aspects with equal considerations of effective learning methodologies.

For Schell, aesthetics refers to atmosphere, or how the game “looks, sounds, smells, tastes and feels” (Schell, 2020, p. 54). Aesthetics can draw a player into a game, create realism in the game, and enhance the player experience (Schell, 2020, pp. 10, 429). Creating a realistic, competitive learning experience along with a believable scenario can also assuage the skepticism military students can have of games and make the student take the game more seriously.

According to educational theorists Alice and David Kolb (2009), experiential learning is a “process whereby knowledge is created through the transformation of experience” (p. 298). Games provide an experience in which students hone their warfighting skills. Games involve “human players or actors making decisions in an artificial environment and then living with the consequences” (Bae et al., 2019, p. 5). Some scholars claim that games can “engage players in higher order cognitive learning outcomes such as problem solving, analysis, and decision-making” (Dabbagh et al., 2019, p. 66); these are the very skills sought after by the Joint Chiefs of Staff.

Resources

In Schell’s (2020) game design construct, “the technology is essentially the medium in which the aesthetics take place” (p. 54). Within PME, it is important to consider all resource requirements to include technology. One resource consideration for matrix games is the physical and/or online environment. Matrix games may require a space that can be secured from interruption with nearby breakout rooms where teams can go to develop strategy and negotiate. The physical medium itself can be as simple as a game board on a table with dice, playing pieces, and scorecards.

Online environment considerations include the ability to facilitate asynchronous learning of information needed to play the game and synchronous interactions to form teams and strategize or negotiate with other players and teams. Screen and video sharing may be required to facilitate game play in a live, virtual environment. The medium for matrix games may even include highly interactive, online environments and various automated tools. Faculty skills are required so they are expert in all areas of the game, the assessment procedures, and even troubleshooting in the online delivery platforms.

Mechanics

The immersive warfighting environment of PME matrix games pushes the mechanics of how the game is played closer to experiencing real-world rules of engagement. This builds on Schell’s (2020) concept for designing a game where “you have to choose the mechanics that will both strengthen that story and let that story emerge” (p. 54).

Game mechanics refer to “the procedures and rules [of] your game. Mechanics describe the goal of your game, how the players can and cannot try to achieve it, and what happens when they try” (Schell, 2020, p. 53). Developing effective game mechanics requires a balance between accuracy and complexity. The basic elements of mechanics (Schell, 2020, pp. 165–210) include space (discrete or continuous), time (discrete or continuous), objects (items in the space), actions (what players can do), rules (how the previous items interact), skills required (physical, mental, or social), and chance (role of uncertainty).

How well the game is facilitated is a critical learning experience factor that takes matrix game design beyond game mechanics to a more immersive, credible warfighting context. A skilled facilitator prompts the players to sharpen their arguments, clearly articulate their objectives, and reflect on their actions. Whether to include a role-playing element, which can force students to view an issue from multiple frames of reference, is another consideration.

The following section looks at the use of matrix games under three conditions: small scale (seminar), large scale (multiple seminars), and large scale (online). The article reviews each game using the four elements of this PME Game Design Framework: (1) scenario, (2) experience, (3) resources, and (4) mechanics. Additional analysis is provided on student assessment and evaluation methods for each game.

Kaliningrad Game

Background

The Kaliningrad game was developed by the USAWC Strategic Simulations Division, Center for Strategic Leadership. I first incorporated it into the USAWC curriculum in 2016 as part of the graduate seminar Security in Europe: NATO and the EU with 16 distance education students (see Angert & Barsness, 2016). In that first iteration, I assessed that students did not fully comprehend the limitations on both the European Union (EU) and the North Atlantic Treaty Organization (NATO) to take collective action. The course has since been revised to address this deficiency to include the state secretary of the Ministry of Defense of Latvia (and graduate of the USAWC) speaking to the graduate seminar and a resident panel of regional experts added on the Baltic States, the EU, and NATO. The game continues to be revised using scenario, experience, resources, and mechanics to improve the student learning experience.

Scenario

The Kaliningrad game depicts a fictional situation with the potential for conflict between Russia and countries neighboring the Kaliningrad Oblast. This conflict threatens to bring in the EU and NATO, including the United States, in defense of the Baltic States and Poland. The actions take place at the strategic level with each player employing the various instruments of national power to further their interests. The scenario is based on real events and set six months in the future; this not only provides realism to the game but also reduces scripting requirements for the faculty instructor. Students come into the class having previously studied the European region. As national security professionals, they routinely follow key developments in Europe. Therefore, the scenario focuses on updating the current environment to reflect possible changes instead of having to recreate an entire timeline for the region. Increased realism and credibility of the scenario are advantages of this design.

Experience

For a game to be successful within PME, one of the most important aspects is creating an immersive learning experience. Scenario immersion begins with the description where a competitive tone is set for the game:

A crisis is brewing in Russia’s Kaliningrad Oblast. You and your teammates will find yourselves engaged in a contest of “international wills” and “policymaking skills,” as you seek to promote interests without provoking a major war among nuclear powers. (Hillison, 2018)

The game space contributes to player immersion and represents a strategic-level headquarters or embassy. The game board is placed in the center of the large space where students gather around the board standing closest to the location on the board of the country or organization they represent. The facilitator stands at the top of the board.

The tabletop board (map) depicts the immediate area around Kaliningrad, areas outside of the region in which actions might take place, and key features related to the scenario (e.g., areas of ethnic Russian concentrations). Strategically important details (e.g., ports and roads) lend additional credibility to the experience.

Visual artifacts such as team placards (e.g., NATO Headquarters) are placed in each team’s workspace and students are given badges (e.g., EU High Representative) with flags to identify their role in the game. When a team takes an action, it places tokens with graphic symbols on the map nearest to where the action will take place. This gives the players a visual cue and a spatial context for the action.

Subject-matter experts are assigned to each team to provide insights on the unique point of view of that country or organization. Players are given formal invitation cards to request diplomatic negotiations and replicate the formality of diplomatic negotiations. These activities are designed to support experiential learning methods for students to further examine the roles and functions of the EU and NATO and how the U.S. works with them to further mutual interests.

Resources

A large space allows all teams to gather around the board. Nearby team workspaces replicate individual team embassies and organizational headquarters (e.g., NATO HQ). Human resources are a key component of the game. Faculty members assume one of three roles: facilitator, faculty instructor, or subject-matter expert. The facilitator overseas the mechanics of the game. The faculty instructor is responsible for assessing student learning and evaluating the game. The subject-matter expert provides contextual expertise.

Mechanics

Decision-making processes are added for the EU and NATO teams to reflect consensus procedures within those organizations and recurring meetings (e.g., the NATO–Russia Council meetings) to replicate structured dialogue within and between organizations and other countries.

Multiplayer teams are organized to represent key players in the region: Russia, the EU, NATO, the United States, the Baltic States, and Poland. Students are assigned to teams to distribute experience of unique individual backgrounds (e.g., assignment to NATO) or expertise (e.g., foreign area officer). For example, students who took the Russia regional studies course are assigned to the Russian team. This is key because accurate representation of Russian interests and strategic outlook is essential to creating a realistic atmosphere and understanding how their actions might impact U.S. interests.

Pre-Game Learning

Prior to playing the game, students have three lessons where they study national interests, challenges, and opportunities in the European region. They also learn about the roles, functions, and capabilities of both the EU and NATO through individual readings and seminar discussions. This allows them to better play their roles during the game.

As homework, students read the rules of the game and watch a demonstration video. The faculty instructor conducts an in-class practice round of the game to familiarize students with the mechanics of the game. This saves time for actual game play and gives students time to reflect upon their actions and resulting outcomes in the practice round.

Phases of the Game

Each round, or game turn, represents two weeks and is divided into three phases: planning, negotiations, and execution. During the planning phase, teams determine what actions to take in pursuit of their assigned goals. During the negotiations phase, teams conduct diplomatic negotiations with other players. After the negotiations phase, players take their positions around the game board for the execution phase (see Table 1).

Order of Play

The map board indicates the order of play, which remains the same throughout the game.

Player Actions

During its turn, each team presents its argument. The argument consists of three main parts:

• what action that team is taking
• why that team thinks the action will be successful (e.g., sufficient resources, past success, etc.)
• the desired outcome

Players are constrained in that they can only use one instrument of national power per turn (e.g., diplomatic, informational, military, or economic). This arrangement is designed to force them to prioritize instruments and to consider the impact of sequencing different instruments. For example, a military action might be more successful if it has been preceded by a diplomatic effort to elicit allied support and an information campaign that supported the desired end state of the military action.

The facilitator adjudicates the outcomes of each action. Each action starts with a 58% probability of success, requiring a dice roll of seven or greater. The facilitator adjusts the dice roll based on the degree of difficulty of the action, the strength of the argument, the strength of the counterarguments, and the impact of environmental trackers (see Table 2).

The dice roll instills an element of chance and friction into the outcome that replicates reality and adds to the experience of competition. A skilled facilitator explains the result by adding to the story line, rather than just giving the result of the roll. By describing the outcome in terms that could plausibly account for the result (e.g., an unseasonable winter storm thwarting a military exercise), the facilitator adds to the immersive nature of the student experience.

The dice roll provides a feedback loop opportunity. For example, highly successful rolls not only achieve the desired outcome but also change the environment (e.g., world opinion) and thus increase the probability of success in subsequent rounds (see Table 3).

Victory Conditions Assessment

After each round, the subject-matter expert assigned to each group provides an assessment to his or her team based on the following questions:

• Did the students demonstrate an understanding of how to effectively use the instruments of power?
• Did they understand the linkage between their actions and changes in the strategic environment?
• Did they demonstrate an understanding of the roles and capabilities of the EU and NATO?

At the end of the game, the faculty instructor, facilitator, and students collectively assess team and student performance. The faculty instructor guides this reflection by asking probing questions about team actions, instruments of power used, and outcomes using player team journals (see Table 4). Students explain their goals, their strategies to achieve those goals, and then determine whether they have achieved them and why. Students also examine how they dealt with any threats or opportunities that surface during the game. Finally, students contribute what they learned during the exercise. Through self-assessment, students take ownership of their actions and become accountable for the effectiveness of their judgment, analysis, and problem-solving skills.

Evaluation

The faculty instructor and facilitator collectively evaluate the game and review student surveys to modify the game as necessary. Faculty evaluations and narrative comments on the surveys reflect that the game contributes to student cognitive ability to analyze the strategic environment, develop strategies, and make appropriate decisions.

Scaling Up–The South China Sea Capstone Exercise

Background

Based on the success of Kaliningrad, the Distance Education Department added matrix games to the resident courses. A matrix game was added to the First Resident Course at the end of the first year as a formative assessment; a modified version was added to the Second Resident Course, which takes place at the end of the final year as a summative assessment. These games contribute to the assessment of outcomes with regards to strategy, instruments of power, and evaluation of the environment. The game is also used to assess the student’s ability to communicate clearly, persuasively, and candidly.

Scenario

The South China Sea (SCS) scenario depicts the competition in the SCS area where China and other nations have competing sovereignty claims. The United States also has security and economic interests in the SCS. The SCS provides a strategic-level environment for the game based on historical information, and like Kaliningrad, it is set six months in the future.

Experience

Each year, the USAWC updates the scenario based on current events. To enhance the realism of the experience, a simulated newscast video provides details on the situation prior to the game.

To scale up the game from one seminar to 23, some aesthetic qualities are sacrificed to provide sufficient space and facilitators. For example, game play is conducted in the seminar rooms that cannot accommodate separate team workspaces. Teams end up having to conduct negotiations in the corners of the room, in the hallway, or in breakout areas near the seminar room. While this works, it detracts from the immersive experience of the game.

Scenario injects are used at the end of each round to change the environment in which the teams compete. In his book Learning by Doing, e-learning analyst and simulation designer Clark Aldrich (2005) notes that students rarely get to experience conflict in role-playing scenarios (p. 104). Injects (e.g., a pilot shot down by another country) enable the facilitators to increase tension in the scenario (and thus the need for military action), or to deescalate tension (when things are spiraling out of control).

Resources

Expanding the game from one seminar to the entire class requires significant additional resources: 23 seminar rooms, 23 game sets, 23 faculty instructors, and 23 facilitators. Each seminar requires one faculty instructor to assess student learning and one facilitator to run the game. Expert facilitators are brought in from other schools, such as the National Defense University, to assist in executing the game.

The game designers create two different maps to accommodate the different objectives of the two courses. They depict the overlapping economic exclusion zones (territorial claims) of the various players, key geographic features (e.g., disputed islands), and resource-rich areas containing oil and gas fields.

The course director is responsible for training the faculty instructors and facilitators and provides students with a reference booklet for use during the game. The booklet includes a short narrative overview and a list of student interests and policy goals they use to create their strategies, prioritize their objectives, and help structure their arguments and responses to other players. It also provides examples of how the different instruments of power might be used to achieve their desired outcomes.

Mechanics

Most of the mechanics remain the same as those in the Kaliningrad game with the following exceptions: for the SCS scenario, the teams represent China, Indonesia, Philippines, Vietnam, Malaysia, and the United States. Also, students rotate through the “spokesperson” role so that every student’s communication skills can be assessed during the argumentation phase.

Assessment

The course director provides a rubric and tracking sheets for recording assessments by faculty instructors. Faculty instructors use these to conduct both formative and summative assessments.

Evaluation

The SCS matrix game has proven effective at achieving the desired course learning outcomes. As with Kaliningrad, each seminar’s faculty instructor, facilitator, and students collectively evaluate the game in terms of meeting the learning objectives and creating a realistic experience. Feedback is collected during the end of course hot wash and used for game revisions. For example, seminars may fail to get through the full spectrum of competition during the game, and modification of injects may enable facilitators to modulate tensions in the game scenario.

Student surveys have yielded similar positive results as with Kaliningrad. One area identified for improvement is the need for workshops to further develop instructors’ and facilitators’ skills in creating an immersive experiential learning experience.

The after action reviews and course hot wash provided rich qualitative insights into student learning. During the games, students who made alliances or coordinated with other teams tended to achieve better results if their goals were aligned. This reinforced insights on collective action and the value of cooperation. Students learned that the sequence of player actions matter. For example, using diplomatic, economic, and informational influence to set up military actions often leads to better outcomes. This reinforces the benefits of the whole-of-government, or in EU terms, the comprehensive approach to security issues. Students also learned how to adapt their strategies if their approaches were not working. For example, failed military actions were often followed up with less aggressive actions using other instruments of national power.

Reacting to COVID-19 Matrix Game Goes Online

About two months prior to execution in 2020, the USAWC commandant made the decision to conduct the resident courses online due to the COVID-19 pandemic. Only the Second Resident Course included a matrix game due to limited adjustment time for online delivery. The scenario and mechanics of the game required only minor updates and the assessment was largely the same, but the other elements had to be tailored for online delivery. Re-creating an immersive, online learning experience in three months’ time was a challenge. The following is not comprehensive but is illustrative of some of the key design considerations for running this matrix game online.

Scenario

The online game updates the SCS scenario to add an Australian team to reflect that country’s increasingly important role in the region. The timeline is set further in the future, 2023, to portray heightened tensions and stimulate more competition between the teams.

Experience

The choice of medium impacts the student experience. The course director choses a video-teleconferencing program that both students and faculty are familiar with. Students create team-specific profile pictures which enhance team identity and promote easy recognition.

In some ways, the online platform allows for a more immersive experience than the in-residence game. Conducting the game online reduces the physical space requirement. Each team has a private area to conduct an analysis of the environment, to discuss its strategy, and to negotiate with other teams. Separate conference rooms are added to provide neutral meeting areas for negotiations.

Resources

While the physical requirements are reduced, the human resources remain the same: faculty instructors and facilitators for each seminar. Students and faculty require a computer, internet access, a microphone, and ideally, a webcam. The game board and tokens are created online. Conducting the game online also requires training so that all participants master the skills required to participate in the online platform.

Mechanics

The rules are the same as in-resident, with some modifications. During the pre-learning phase, students submit their individual strategies to their faculty instructor prior to meeting as a team. This allows the instructor to assess how well each student understands the strategy formulation framework and to provide individual feedback to each student. Once their strategies are submitted, students meet online in their teams to plan their collective team strategy. Faculty instructors meet with their facilitators in advance to determine how they will communicate with each other during and between rounds. Students also require additional time to develop their counterarguments online.

Evaluation

It is harder for faculty and students to process oral arguments online. Having students submit a written summary of their moves in the chat box prior to oral arguments seems to improve the processing and recording of actions. While it is still too early to evaluate the success of this online matrix game, one consideration may be the use of an online virtual campus.

Matrix Games: Flexible and Scalable

These three examples demonstrate the flexibility and scalability of matrix games. They can be effective for a single seminar or for multiple seminars. They can be conducted in residence or online. They can be played in a few hours, an entire day, or over several days as an experiential learning activity to meet learning outcomes.

In his 2019 report On Wargaming: How Wargames Have Shaped History and How They May Shape the Future, Matthew Caffrey, a former professor of wargaming and campaign planning at the Air Command and Staff College, argues that wargames can save lives and lead to victory in actual warfare. They do this by developing the skills of leaders and organizations, providing a venue to experiment with strategy and tactics, and increasing the player’s familiarity with “the environments in which they will operate” (Caffrey, 2019, p. 339).

Of course, wargames are not a panacea. The article “Wargaming has a Place” offers an array of experiential learning activities used at the Air War College and cautions against overemphasizing the value of games (Lee & Lewis, 2019). The authors argue that games often suffer from oversimplification and complex adjudication procedures and that other activities, such as staff rides and simulations, can better achieve desired learning objectives. Even proponents of wargames, such as Peter Perla and Ed McGrady, caution against poorly designed games having negative impacts based on incorrect information, over or understated risks, and the failure to account for chance and friction in game narratives (Perla & McGrady, 2011, p. 123). Finally, not all games are effective educational tools. If a game is ineffective, “usually the culprit is that the focus has drifted too far from the learning objective” (Weinstein, 2016, p. 47).

Conclusion: Game On!

The evaluation of matrix games at the USAWC demonstrates that games can be effective methods of assessing student learning and developing student warfighting skills if properly designed and executed. Effective games require a commitment to significant planning, rehearsal, and faculty development. Further use of the four elements of this PME Game Design Framework, (1) scenario, (2) experience, (3) resources, and (4) mechanics, should yield even richer collaborations among PME institutions on use of games to develop warfighters.

---

References

Aldrich, C. (2005). Learning by doing: A comprehensive guide to simulations, computer games, and pedagogy in e-learning and other educational experiences. Pfeiffer.

Anderson, J. (1980). Cognitive psychology and its implications. Freeman.

Angert, M., & Barsness, D. (2016, August 15). Kaliningrad 2017 matrix game at the US Army War College. PAXsims. https://paxsims.wordpress.com/2016/08/15/kaliningrad-2017-matrix-game-at-the-us-army-war-college/

Bae, S., Bartels, E., Smith, B., & Wong, Y. (2019). Next-generation wargaming for the U.S. Marine Corps: Recommended courses of action (RR-2227-USMC). RAND Corporation. https://doi.org/10.7249/RR2227

Caffrey M. (2019). On wargaming: How wargames have shaped history and how they may shape the future. Naval War College Press.

Curry, J. (2020). The utility of narrative matrix games–A Baltic example. Naval War College Review, 73(2), 33–52. https://digital-commons.usnwc.edu/nwc-review/vol73/iss2/6/

Dabbagh, N., Marra, R., & Howland, J. (2019). Meaningful online learning: Integrating strategies, activities, and learning technologies for effective designs. Routledge.

Hillison, J., (2018). Course Description, DE5540: Security in Europe - NATO and the EU.

Joint Chiefs of Staff. (2020, May 1). Developing today’s joint officers for tomorrow’s ways of war: The joint chiefs of staff vision and guidance for professional military education & talent management. Department of Defense. https://www.jcs.mil/Portals/36/Documents/Doctrine/education/jcs_pme_tm_vision.pdf?ver=2020-05-15-102429-817

Kolb, A., & Kolb, D. (2009). The learning way: Meta-cognitive aspects of experiential learning. Simulation and Gaming, 40(4), 313–305.

Lee, C., & Lewis, B. (2019). Wargaming has a place, but is no panacea for professional military education. War on the Rocks. https://warontherocks.com/2019/08/wargaming-has-a-place-but-is-no-panacea-for-professional-military-education/

Perla, P, & McGrady, E. (2011). Why wargaming works. Naval War College Review, 64(3), 111–130.

Schell, J. (2020). The art of game design: A book of lenses (3rd ed.). CRC Press.

Weinstein, M. (2016). Are you game for learning? Training, 53(5), 44–47. http://pubs.royle.com/publication/?m=20617&i=336393&p=46

Wunische, Adam. (2019). Lecture versus simulation: Testing the long-term effects. Journal of Political Science Education, 15(1), 37–48. https://doi.org/10.1080/15512169.2018.1492416

Back to Top

October 2020

QUICK LINKS