• Online Exclusive Article Archive

Take Ownership of Your Formation’s Data Literacy

 

Capt. Orlando Nieves III, U.S. Army

1st Lt. John Boyer, U.S. Army

Sgt. Feihrren Calhoun, U.S. Army

 

Download the PDF

 

Data will bring clarity to the fog of war—if it is collected and handled properly. As our Nation’s adversaries—China, Russia, Iran, and North Korea—adopt increasingly advanced technologies that allow their forces to operate at unprecedented speeds on the battlefield, we must match their intensity by focusing on our soldiers’ and civilians’ skill sets. This can be achieved by not only training U.S. forces to use the new technology that the United States has iteratively acquired, but by also upskilling formations to make sense of the overwhelming amounts of data that Americans are exposed to daily. It is critical that units do not wait for the Department of Defense (DOD), service headquarters, and senior leaders to order the participation in generalized data literacy programs. Rather, each unit must take charge of its own literacy in a way that is customized to its unique mission requirements. It will otherwise be too late to be competitive if we are confronted with war and have no ability to keep up with the pace of battle. In response, the Army Intelligence Data Strategy 2022–2025, specifically Line of Effort (LOE) 4: Cultivate Data Literacy, makes clear the requirement that the Army Military Intelligence Corps will advance the data-related knowledge and skills of the workforce by strengthening education, training, and professional development opportunities.¹ LOE 4 lays out requirements for the Army military intelligence community to reach data centricity, including developing and implementing a data literacy plan, increasing data literacy through investment in data training, and promoting smart data practices. Fortunately, there are tested methods to start the data literacy cultivation journey, all of which require unit buy-in first.

In 2022, the 513th Military Intelligence Brigade (513th MIB[T]), the Intelligence and Security Command’s (INSCOM) theater intelligence brigade aligned to Army Central Command (ARCENT), adopted Innovation, Modernization, and Experimentation (INMODEX) as an official LOE by integrating it in all aspects of the brigade’s day-to-day operations and transforming the unit culture to remove barriers to implementation. The brigade launched the INMODEX LOE by focusing first on four foundational elements: culture, data and process literacy, cloud, and governance.² Data and process literacy was selected to address two of the brigade’s pain points and associated requirements. First, the 513th MIB(T) recognized that technological capabilities are only as powerful as their users (soldiers and civilians). As technology advances, so must we. The DOD frequently expresses the need for data science integration, common intelligence picture development that requires connected data sources, and the incorporation of artificial intelligence and machine learning algorithms. Yet upon looking closely at those technical efforts, the brigade recognizes that the users must be able to effectively communicate their own data requirements and feed them into the systems. Would this require everybody to be a data scientist or data analyst? Absolutely not. But we must bring our people along so they can use the newly integrated technology. Second, the brigade identified deficiencies in its processes, such as archaic analytic processes in the 513th Theater Ground Intelligence Center that limited the brigade’s support to ARCENT, as well as command and staff systems that relied on hand-jamming data into static (not cloud-based) “databases,” such as PowerPoint slides and Excel sheets.

The 513th MIB(T) needed to change the way it operated in these realms, which was based on inefficient data practices. As a result, the brigade charged forward on growing data literacy in its formation to optimize how it provides timely and actionable intelligence and empowers well-informed decision-making. Specifically, 513th MIB(T) has placed over four hundred individuals through a course known as Data Literacy 101 (DL101) by partnering with the Center for Data Analysis and Statistics at the U.S. Military Academy (USMA) and the Office of the Deputy Chief of Staff for Intelligence (G-2) Intelligence Community Information Management (DAMI-IM) Directorate, followed by Army Cyber Command (ARCYBER) and INSCOM. The 513th MIB(T) aims to have 100 percent of the brigade attend DL101 by 2024. The brigade has also established a data literacy task force that will tailor the DL101 curriculum to individual sections in the 513th MIB(T), implement process changes that enable data optimization at the user level, and weaponize the force with technical training and resources. Members who work within the brigade’s formation of intelligence analysts, collectors, technical operators, and support staff comprise the task force. The 513th MIB(T) has made data literacy a unit-wide effort by setting the expectation that it is critical for everybody to contribute to make tangible progress. The brigade is now calling on other units to take similar approaches, especially by taking advantage of preexisting resources such as DL101.

Col. Nicholas Clark, program director of applied statistics and data science at USMA, created DL101 because of real operational problems that he observed while serving on an operational experience as the chief data scientist for Joint Special Operations Command. Clark realized that data is not always empowered to serve its purpose. For example, the consequences of data knowledge gaps were visibly evident at the tactical level as U.S. forces withdrew from Afghanistan in 2021. “During the turning point, some data taken off mission objectives was such poor quality that it wasn’t usable,” stated Clark while referencing data collection opportunities that were wasted solely due to poor quality data acquired on the frontlines.³ “There were things that could have happened at the objectives that weren’t occurring,” he continued.⁴ This was not a negligence issue, but rather a literacy gap. Some operators and soldiers serving as collectors did not know what constituted “good” versus “bad” data, which stunted follow-on analysis and decision-making. There were also no pre-mission planning processes in which specific data requirements were passed from experienced data subject-matter experts to men and women collecting on the frontlines. This situation alerted leadership that even tactical elements must be prepared to work with data or we will continue to lose the ability to conduct informed analysis. In addition, urgency to solve this problem grows when we factor in the difference between the past and future; if data quality mattered during a counterinsurgency fight, we can only imagine how important it would be during rapid large-scale combat operations that rely on the Army to fulfill its strategic role of providing multidomain operations to joint force commanders.⁵ While searching for a solution, Clark recognized that we will not have data scientists or data analysts to run data collection on the “X” but recalled that certain skills are taught standard across the Army. “Everybody is expected to undergo training for marksmanship, combat lifesaver, and basic communication equipment operations. But what about a combat lifesaver equivalent for data literacy?” he asked.⁶

The result? Clark developed a sixteen-hour course that covers basics in reading data, working with data, analyzing data, and communicating effectively from data. The course is designed to be taught over four morning sessions but can be condensed into two days if operational requirements so dictate. It tackles the following student outcomes that target all job populations:

• Understand what questions can and cannot be answered with data
• Become critical consumers of data and statistics
• Predict and interpret graphical displays
• Communicate the significance of data and analysis
• Understand the strengths and weaknesses of common software

These student outcomes are intended to provide Army soldiers and civilians a push in the right direction of properly formulating questions that data can answer, understanding how to properly format data, and communicating results to tell convincing data stories by using the tools available to them. It also enables leaders to understand the questions that we have right now that we could use effective data practices to answer. The 513th MIB(T) immediately saw potential in the course to answer its requirements, and during summer 2022, reached out to Clark to bring the program to Fort Gordon, Georgia.

The 513th MIB(T) hosted its first iteration of DL101 on 17 May 2022 at Fort Gordon by inviting Center for Data Analysis and Statistics director Col. Jim Starling to teach nineteen soldiers and civilians. The brigade has since hosted the course monthly and worked with various instructors from ARCYBER, INSCOM, and Network Enterprise Technology Command (NETCOM) to iteratively adjust the course with lessons learned over time (see figure 1). These lessons learned can be categorized into culture, justification, partners, environment, material, real problem experimentation, and assessment and follow-up (see figure 2).

Culture

Our culture must be shifted to normalize bottom-up problem-solving, especially when dealing with data issues. Leadership must reward soldiers and civilians who creatively approach their unique data problems with specifically tailored solutions. It must be communicated in all directions that this course is intended to provide tools that will improve soldiers’ processes.

It is necessary that leaders of the organization provide buy-in to data literacy through action from the very beginning. In the 513th MIB(T), all command sergeants major and lieutenant colonels have attended DL101. This demonstrates urgency by ensuring that all members of an organization attend the course, and it reveals that if leadership can prioritize their own attendance, then so can everyone else.

Justification

Ensure that everybody attending DL101 understands why they are there. Many personnel may instinctively approach the course as a check-the-box requirement because attending the course is required, and they would therefore have a lower likelihood of putting in the effort to adopt DL101’s practices long term. We as leaders need to prevent that by providing justifications that convince people to invest more of their time into the program.

Here is a brief example justification to provide on the front-end (e.g., course invite), followed by continuous reiteration: “Technological capabilities are only as powerful as its users, and as technology advances, so must we. We will not make everybody a data scientist or a data analyst—but we have to build our skills so we can use new capabilities. Several problems we run into as intelligence professionals and enablers can be solved by optimizing our approach to working with data. For example, some analysts will spend more time sifting through pools of data to find the information to analyze, rather than analyzing the data itself.”⁷ This creates a relatable anecdote to attendees that can be combined with the previously mentioned DL101 learning objectives to make it apparent why we are moving in this direction.

Partners

Based on interest from neighboring organizations, it was recognized that the 513th MIB(T) is not the only unit on Fort Gordon that will benefit from data literacy training. For example, the U.S. Army Signal School and 116th Military Intelligence Brigade were actively seeking opportunities to upskill and were willing to collaborate on resource requirements to host courses. In addition, including members from both units increased Fort Gordon and INSCOM senior leader awareness about the need for data training at all units throughout the Army. This is amplified when inviting members from higher headquarters to attend (e.g., INSCOM and ARCENT), as they can communicate requirements and associated positive effects at their levels.

The ARCYBER Technical Warfare Center (TWC) proved to be an extremely valuable partner during 513th’s second iteration of DL101 in June 2022 and onwards. The TWC consists of military and civilian developers that focus on conducting development, security, and operations in support of ARCYBER’s mission. It is manned with expertise that supports technical learning and access to industry partners, such as the Cyber Fusion Innovation Center, which is collocated with Georgia Cyber Center in downtown Augusta, Georgia. The TWC also provided an officer to teach the 513th’s third iteration of DL101.

Environment

Courses hosted offsite have proven to be much more valuable learning experiences than courses hosted on base. By utilizing locations away from the flagpole, soldiers are exposed to fresh environments that psychologically empower learning new, traditionally nonmilitary concepts. Small adjustments to facilitate learning, such as allowing students to wear business casual attire to class, enable classes to exhibit a more academic atmosphere. In addition, removing personnel from work minimizes the chances of work-related distractions and interruptions.

The 513th MIB(T) utilizes its partnership with the TWC to acquire DL101 classroom space at the Cyber Fusion Innovation Center and Georgia Cyber Center in downtown Augusta. This is not only useful for removing students from Fort Gordon, but it also places the students in an environment where they are surrounded by cyber professionals and Augusta University academics who use data to solve real problems every day.

Material

Information is best retained if it is provided in a language that people understand. DL101 traditionally uses example data sets and anecdotes that apply to a broad set of professions, such as noncombatant evacuation operations and Ranger Assessment and Selection Program retention data. For many students who work in intelligence, these examples do not always facilitate the understanding of the course’s statistical and mathematical methods because the concepts are new to them, and the examples are not relatable.

By partnering with Maj. Daniel Baller, an operations research and systems analysis officer and data scientist at INSCOM, to teach the 513th’s fourth and sixth iterations of DL101, it was recognized through course feedback that intelligence soldiers will better understand DL101’s concepts if associated examples are related to the intelligence profession. Baller has been able to incorporate INSCOM-related examples and states that the next step will be to create and utilize data sets in the curriculum that are related to intelligence methods, which will lead to students more effectively absorbing the course’s lessons. It reportedly benefitted students’ understanding by simply tying back the importance of detailed data requirements to priority intelligence requirements. Example intelligence data sets can be further customized for courses tailored to a specific military occupational specialty, whether conducted in a schoolhouse for professional military education or in a unit’s single-source intelligence cell.

Real-Problem Experimentation

Learning new skills requires implementation of those skills on real problems. According to Baller, “We can talk data literacy training all we want, but if we don’t turn around and make it a conscious part of our everyday lives, then it doesn’t really take hold. Not only do we have to get our people trained with formal education, but we have to start incorporating it into everything we do. You have to make it a habit.”⁸ Instructors can support this with a few strategies, including implementing DL101-required projects during class that incorporate the solving of student-provided problems while unit representatives can conduct follow-up over time. It is also valuable to suggest tools (e.g., software) that specific populations have access to in their work environments. For example, Army365 provides access to several power platform tools such as Power Business Intelligence, Power Apps, and Power Automate. These low-code/no-code tools are relatively easy to learn and allow for data visualization and automation that can streamline processes on both unclassified and classified networks. The 513th MIB(T) began demonstrating these programs in class on example data sets during its sixth iteration of DL101. Tools like Power Business Intelligence have proven extremely useful in the 513th MIB(T)’s staff modernization strategy. By enabling staff sections to replace standard PowerPoint slides with Power Business Intelligence data visualizations, dashboards are semi-live. They only require unmodified Excel sheets to be exported from CAC-enabled websites (e.g., Global Combat Support System–Army, Medical Protection System) and dropped in linked data source folders, where they are immediately ingested and displayed on cloud-based servers via Power Business Intelligence. The manual export and file drop process is a temporary workaround in lieu of the inability to build application programming interfaces that automate the data links.

In addition, units can require data collection plans to be created before operations and exercises, in which the following questions are answered and shared:

• What are the questions we want to answer on the back end of this operation?
• What is the data we want to collect?
• How do we want to store the data?
• What tools do we have at our disposal to conduct analysis?

Specifically to leadership, Baller suggests “setting requirements to your formation in a way that requires them to use data to conduct analysis.”⁹ He consistently explains that commanders and intelligence officers must ask questions that data can then inform, which is covered by the course when describing how to ask a good data-driven question.

Assessment and Follow-Up

Pushing for DL101 feedback from students is necessary to gauge the success of the various experiments ongoing during the course’s development, especially if seeking to modify the curriculum to match specific populations. This is like how Baller was able to tailor the course to the INSCOM population.

Riding the momentum by immediately and consistently engaging with students following DL101 ensures the newly taught concepts are not lost. Many attendees will be visibly interested in the course material and would put in the effort to learn more if provided the resources. According to Army Intelligence Data Officer Erika Vargas, the DAMI-IM data maturity assessment revealed that people want to be made aware of training opportunities but are underinformed. “The problem isn’t that there’s not enough training—it’s just that they aren’t aware of what’s available,” she stated.¹⁰ This requires upskilling pipelines to be accessible and widely communicated.

Following up with students one to three months after their DL101 course to inquire if they are using any concepts learned in the course is important for gauging the effectiveness of the course and its effect on the unit. If none of the concepts are employed, this signals it may be time to modify the curriculum.

Finally, having an assessment plan in place to evaluate how much students have learned is important to tracking effectiveness and giving quantifiable data in support of the DL101 initiatives.

As successful as the first few iterations of data literacy were in garnering initial feedback and lessons learned, the 513th MIB(T), INSCOM G-3’s operations research and systems analysis team, and the USMA Center for Data Analysis and Statistics all recognized a growing problem: continually sending a few experts (Starling, Baller, etc.) to teach these courses to our formations is unsustainable if we desire to implement DL101 across the Army. For this to truly be successful, organizations across the Army need to have the ability to teach and tailor the DL101 course in-house. The 513th MIB(T) looked to external organizations to adapt approaches that addressed this problem, such as Gulf Bank of Kuwait. Mai Alowaish, chief data and innovation officer at Gulf Bank, recognized that for a new initiative to be successful, it requires ownership at all levels. Thus, the 513th MIB(T) replicated Alowaish’s Gulf Bank data champion program through the creation of the 513th Data Literacy Task Force (DLTF).¹¹ By designating a dozen representatives throughout the formation to serve in the DLTF, the 513th MIB(T) is empowering soldiers who directly experience and understand their data issues to create highly tailored solutions. Their three primary duties include creating and teaching DL101 curriculums that fit their varying intelligence disciplines and support roles, to be held at a rate that meets 100 percent DL101 attendance in their formations by 2024; implement processes that revamp how their formation manages data, conducts analytics, increases workflow efficiency, and uses available software to tell convincing data stories; and serving as cultural change catalysts, consistently communicating the need to change how we work with data through engagement with peers and junior leaders.

After a few months, however, the 513th recognized that simply sending the DLTF through the DL101 course was not sufficient to make its members feel comfortable enough with the material to tailor their own courses and teach members of the brigade. In addition, the 513th MIB(T) was not alone in its desire to have in-house educators.

To address the requirement for an instructor course, Clark and the USMA hosted the Data Literacy 101 Train the Educator Conference 5–9 June 2023 at West Point. Over the course of five days, roughly one hundred military representatives from the United States and Europe took the DL101 course and were then given the opportunity to study, tailor, and teach the material to randomized classrooms of fellow data literacy educators with the help of author and data literacy expert Jordan Morrow. Each small group had a West Point instructor assigned to facilitate the presentations, provide pedagogical feedback, and help individuals tailor their courses to their specific audiences. Clark also taught a “Student Outcome” and “Student Indicator” assessment framework that the 513th MIB(T) has since adopted, encompassing the following:

• Student outcomes are broad, encompassing statements that aim to convey what we want the student to take away from DL101. These are not statements that are supposed to be measurable.
• Student indicators are nested within student outcomes and are meant to be narrow and measurable statements.
• Nested within the student indicator is an assessment question or method aimed at providing measurable data against the student indicator.
• Example student outcome: “Predict and interpret graphical displays.”
• Example student indicator: “Students can readily identify issues with manipulated bar graphs.”
• Example assessment question: Student is presented a bar graph with manipulated dates on the x-axis, changing intervals on the y-axis, and hard-to-read colors. “What are two things that should make you skeptical about this graph?”

To implement this assessment model, provide a check on student progress, and track change over time, the 513th MIB(T) is implementing a pretest, a posttest, a one-month follow-up survey, and a six-month follow-up survey. The pre- and posttests are aimed at collecting quantitative data and are digitally administered immediately before and after the course. The tests ask the same types of questions but use different wording and data. The one-month and six-month follow-up surveys are more qualitative in nature and aim at gaining insight into how lessons learned in DL101 have influenced day-to-day operations. With all conference attendees developing the same type of DL101 course and assessment model, organizations across INSCOM and the Army are now better equipped to start teaching their own tailored variations of DL101.

Building on the momentum of the Train the Educator Conference, the DLTF representatives, titled battalion data leaders and company/section data liaisons, are provided additional training via upskilling pipelines maintained by the brigade data representative. The goal of the upskilling program is to be realistic. The intent is to only include courses that teach representatives data skills that they can apply at work so they can iterate on real problems, which requires taking into consideration the software that each section has available to them. For example, tools that a signals intelligence section has available to them on NSAnet will differ from what a staff section has on NIPR Army365. This variation requires that each DLTF representative is deliberate in suggesting upskilling pipelines for their formations. To encourage other formations to implement similar upskilling series, the 513th MIB(T) opened the upskilling courses already scheduled to everyone in attendance at the Train the Educator conference.

The DLTF is just one example of how to approach organizational data literacy requirements, but it demonstrates that becoming “data literate” requires teamwork across the unit. The 513th MIB(T) is still on its way to becoming a 100 percent DL101-trained organization as it experiments with the implementation of new data concepts. Even when it reaches its brigade goal, which will be enduring due to personnel and technology turnover, it will only address the 513th’s culture and literacy. What about other units? The Train the Educator Conference was a good start toward standardization, but Army Intelligence Deputy Chief Data Officer Chandra Donelson states, “Data is key to all of our initiatives. Therefore, we must accelerate and expand our ability to provide data literacy training to the entire force. Ensuring people—our greatest asset—can read, write, communicate, and analyze data will be pivotal to becoming more data-centric, operationalizing innovation, and adopting AI-enabled capabilities. And the best time to start that is today.”¹²

To transform the DOD’s culture, buy-in is needed from all organizations across the department. Therefore, the 513th MIB(T) is calling on organizations to jump start their own data literacy programs by using USMA’s proven DL101 curriculum and designating specific team members to power the initiative. Do not wait for a different organization to launch it for you. It is imperative that nontechnical junior leaders lean forward on growing their own data literacy programs and that they receive senior leader support in doing so. Junior leaders have a direct influence on their soldiers and noncommissioned officers and can effectively prepare their own formations to accept this required cultural and technical change. The resources are there—we just have to experiment with them.

---

Notes

1. Office of the Deputy Chief of Staff, G-2, Army Intelligence Data Strategy 2022–2025 (Fort Belvoir, VA: Headquarters, Department of the Army, G-2, 2023).
2. Orlando Nieves III, Innovation, Modernization, and Experimentation (INMODEX) Guide, version 1 (Fort Gordon, GA: 513th Military Intelligence Brigade [Theater], 19 January 2023), 13, accessed 14 July 2023, https://armyeitaas.sharepointmil.us/:b:/t/513th_Innovation/EWiy5PLgA_5MiX2sVTOvC1MB4b2XsbsOnUKxlrz4b2LsYQ?e=H1VhEW.
3. Nicholas Clark (program director, Applied Statistics and Data Science, U.S. Military Academy), in conversation with the author, 3 January 2023.
4. Ibid.
5. Field Manual 3-0, Operations (Washington, DC: U.S. Government Publishing Office, 2022), 1-1.
6. Nicholas Clark (program director, Applied Statistics and Data Science, U.S. Military Academy), in conversation with the author, 3 January 2023.
7. Nieves, Innovation, Modernization, and Experimentation, 19.
8. Daniel Baller (data scientist, U.S. Army Intelligence and Security Command [INSCOM]), in conversation with the author, 4 January 2023.
9. Ibid.
10. Erika Vargas (intelligence data officer, U.S. Army INSCOM), in conversation with the author, 11 January 2023.
11. Mai Alowaish, “Building Data Champions and Scaling Internal Talent,” Army Military Intelligence Data Talk (unpublished meeting summary, Fort Belvoir, VA: Intelligence Community Information Management Directorate, Office of the Deputy Chief of Staff, G-2, last modified 25 October 2022), accessed 14 July 2023, https://armyeitaas.sharepoint-mil.us/:b:/t/513th_Innovation/Ef2wc4-xd1ZJqc6SdWAEB8gBGuV2X9MBBUVqyy_7Y21gGQ?e=uX931h.
12. Chandra Donelson (Army intelligence deputy chief data officer, U.S. Central Command), in conversation with the author, 10 January 2023.

 

 

Back to Top