Learning Engineering at a Glance

Based on the iFEST Poster (Winner of Best Poster Design)

Jim Goodell¹, Aaron Kessler², and Sae Schatz³

 

¹ Quality Information Partners (QIP), Fairfax, Virginia, United States

² Learning Sciences and Teaching for Open Learning, Massachusetts Institute of Technology

³ Advanced Distributed Learning Initiative, Orlando, Florida, United States

 

 

Download the PDF

Abstract

Engineering has been successfully applied to many complex human-centered challenges. This article proposes learning engineering to transform military learning at the pace and scale needed to respond to the growing complexity of the global security environment. It is based on the award-winning “Learning Engineering at a Glance” poster presented at the Innovation, Instruction, and Implementation in Federal E-Learning Science & Technology Conference, the premier conference on distributed learning.

 

Do you feel uncomfortable when the words “learning” and “engineering” are used together? If you do and your background is in education, education research, instructional design, or one of the disciplines investigating the science of learning, then you have peers who have similar feelings. However, this should not be the case because these communities of inquiry and practice are not at odds with the emerging process and practice called learning engineering. Learning scientists and instructional designers alike have questioned whether “learning engineering” is just a new label on what they already do. Many people are uneasy because they associate the word “engineering” with work that is cold and mechanical compared to the very human process of learning, which happens uniquely within the mind of a learner, not something done to learners.

The IEEE IC Industry Consortium on Learning Engineering (ICICLE) attempts to take a more learner-focused approach by defining learning engineering as follows:

Learning engineering is a process and practice that applies the learning sciences using human-centered engineering design methodologies and data informed decision-making to support learners and their development. (IEEE ICICLE, 2019)

 

Furthermore, ICICLE recognizes learning engineering as an iterative process that requires multiple cycles of creation, implementation, and investigation, most often by an interdisciplinary team rather than an independent learning engineer.

The purpose of this article is to raise levels of comfort with the notion of learning engineering, and levels of understanding about why engineering for learning is needed in military education and training. The book Learning Engineering Toolkit: Evidence-Based Practices from the Learning Sciences, Instructional Design, and Beyond (Goodell & Kolodner, 2023) makes the case for learning engineering as a distinct, professional practice that complements related professions and fields of study. The Learning Engineering Toolkit uses a story from another distinctly human field, medicine, to make this case. The following is a paraphrase of that story (Goodell, 2022): The lives of countless soldiers were saved during and since World War II based on the 1928 discovery of penicillin. Throughout history, the major killer in wars was infection rather than battlefield injuries. In World War I, the death rate from bacterial pneumonia was 18 percent; in World War II, it fell to less than 1 percent. However, at the beginning of the war there wasn’t enough penicillin to fully treat a single patient (Wood, 2010). It took the work of chemical engineers like Margaret Hutchinson Rousseau and Jasper H. Kane to create a deep-tank fermentation process that made it possible for the United States to produce 2.3 million doses in time for the invasion of Normandy in the spring of 1944. As illustrated in Figure 1 and described in Learning Engineering Toolkit (Goodell & Kolodner, 2022), this story is a victory for both science and engineering.

This engineered process to scale production of penicillin is credited with helping to win World War II. Another kind of engineering, learning engineering, may prove critical for future victories.

The goals of science and engineering are different. The goal of science is to discover the truth about the world as it is. The goal of engineering is to create scalable solutions to problems using science as one tool in that endeavor.

Just like the need to scale the production of penicillin, there are learning sciences discoveries yet to be enabled at scale. The capability of our military personnel is at a disadvantage if the learning sciences cannot be applied at the requisite breath, pace, and efficiency. Learning engineering is needed.

From Science History Institute, Wikimedia Commons (https://commons.wikimedia.org/wiki/File:Penicillin_poster_5.40.tif). In the public domain.

 

Figure 2 shows “Learning Engineering at a Glance,” a poster presented at the Advanced Distributed Learning’s Innovation, Instruction, and Implementation in Federal E-Learning Science & Technology Conference in August 2022; it defines learning engineering and its foundational concepts.

By QIP with contributions from J. Goodell, L. Huster, A. Kessler, B. Redd, S. Schatz, K.P. Thai, and M. Yang. Adapted from the work of IEEE IC Consortium on Learning Engineering, Kurt VanLehn et al.

 

The following sections each briefly describe a core characteristic of learning engineering, from its disciplined and iterative process to its inclusion of the learning sciences and data-driven methods. These features are what distinguish learning engineering from more traditional instructional design approaches.

Learning Engineering Is a Process

Learning engineering is more of a verb than a noun; it is more about what interdisciplinary teams of people do than a job title for one person.

A process defines how work is done. Processes have inputs, process steps, and outputs. The learning engineering process, as shown in Figure 3, can be generalized: it starts with understanding the challenge within a context, and then it includes cycles of creation, implementation, and investigation (Kessler et al., 2022) often considered concurrently. The process is iterative and includes multiple passes. Design of learning content or solutions that do not involve iterative cycles of improvement, guided by insights from data, cannot be considered learning engineering.

Challenges that need to be solved using the learning engineering process are often complex and require a multifaceted learning engineering team to address them.

After initially defining the challenge, the next step may be creation, implementation, or investigation. The problem to be solved in context may call for creation of a new learning experience, adjustments to the implementation of an existing learning solution, or additional data analyses as part of the investigation phase. The creation phase often involves mini iterations of creation, user testing, and investigation for the development of content, learning solutions, learning experiences, learning conditions (such as changes to an environment for more optimal outcomes), instrumentation (data collection infrastructure), and/or implementation plans.

While the generalized process is consistent, specific learning engineering processes used by a large team may vary from those used by a small team and the processes for developing one kind of experience (e.g., a field training exercise) may be different from those used for designing a different activity (such as a university curriculum).

Learning engineering challenges, as well as the processes used to address them, often have subchallenges or require subprocesses that need to be considered concurrently. Processing, analyzing, and interpreting the data from an implementation of a learning experience is necessary to inform the next iterative cycle of the learning engineering process.

Learning Engineering Applies the Sciences of Learning

Herb Simon (1967), who coined the term “learning engineering,” wrote, “learning is a complex psychological process, and it would be naïve to think that anyone can design an effective learning environment and an effective program of learning experiences for students without a mastery of what is known, scientifically and practically, about that process” (p. 73).

Many different branches of science explore foundational understanding of learning and include studies having to do with how the brain and nervous system work from physiological and psychological perspectives along cognitive, behavioral, and motivational dimensions.

While it is beyond the scope of this article to attempt to cover the range of remarkable recent discoveries about how people learn (National Research Council, 2000; National Academies of Sciences, Engineering and Medicine, 2018), we recognize that with many of these discoveries—like the discovery of penicillin—a decade or more can pass without scaled application. The twenty-first century has been called the golden age for brain research (Chopra & Tanzi, 2021), based in part on the availability of technologies such as functional magnetic resonance imaging (fMRI). The opportunity to apply the findings from this golden age of research to military learning may depend on how well we can develop the practice of learning engineering.

Learning Engineering Is Human-Centered

Learning engineering requires a human-centered focus. Human-centered design starts by understanding the challenge from the learners’ perspectives and then creating solutions through research-based iterative design. Learning engineering’s human-centered perspective has its roots in several fields including human-centered design, design thinking, universal design for learning, learning experience design, and design-based research (Thai et al., 2022).

Human-centered design is an iterative process that relies on data and data-driven decisions. For learning, human-centered design typically involves understanding learner variability; for example, developing personas to refer to during design, making ideas concrete with prototypes, and testing ideas iteratively with representatives of the learner population. The process can include the following (Thai et al., 2022):

• empathy development via observing or interacting with learners;
• codesign, or participation of learners in the design process;
• translation of ideas to rapid prototypes (e.g., design cards);
• testing ideas quickly with learners (for instance, to ensure the learners are meaningfully engaged, for sustainment of motivation and engagement, for feelings of being invited into the process, and for moving learning forward);
• iterative refinement with simple prototypes, which become more complex and developed with continued user input; and
• iterative refinement during implementation with continued user input.

A learning engineering team should be collectively focused on producing the outcomes from a learning experience by designing an effective set of conditions for learning rather than focusing on content or technology as their main product.

From “Learning Engineering Is Engineering in Learning,” by J. Goodell, J. Kolodner, and A. Kessler, Learning Engineering Toolkit: Evidence-Based Practices from the Learning Sciences, Instructional Design, and Beyond, p. 138, 2022 (https://doi.org/10.4324/9781003276579-6) . Routledge.

 

Learning Engineering Is Engineering

Engineering is the application of creativity and science to solve problems. Learning engineering is the application of the learning sciences to creatively solve problems for learners and learning. Engineering domains differ in the problems to be solved and in the science to be applied in solving them. Mechanical and chemical engineering apply sciences such as physics, material science, and chemistry, whereas learning engineering applies, for example, cognitive, sociocultural, behavioral, and motivational sciences.

Each engineering domain is unique. There are, however, some overarching principles that apply to any kind of engineering. An engineering mindset is a systems mindset. Systems, whether learning systems, telecommunication systems, or pharmaceutical production processing systems, are designed using models of various degrees of fidelity. Scalability of complex systems is achieved in part by breaking them into modules, with interfaces between those modules. Interoperability is improved by using standard interfaces. Parts of a system have design constraints and tolerances (Barr et al., 2022).

Engineering control theory has relevance for learning engineering, as illustrated in Figure 4. Often, education systems are designed with open loop control systems that fail when the controller has an incomplete model of the learner, learning process, learning conditions or other factors.

In a closed-loop system, the controller compares feedback from the output to adjust the input. In learning engineering, the outputs (e.g., how well personnel are performing at a task) must be compared to the desired performance of the task. Faster, more frequent, and richer feedback is generally better, especially for dynamic systems, and can even compensate for less-than-ideal conditions in other parts of the system. Control systems use filters and dampers to prevent overcorrecting. Engineered systems and human learning work best with multiple feedback loops. Subsystems may have their own feedback control loops.

Learning Engineering Uses Data

Data-informed decision-making is an integral part of learning engineering. Without it, you may be doing learning design, but you are not doing learning engineering (Czerwinski, Goodell, et al., 2022). Data-informed decision-making has two parts: instrumentation and analytics.

Instrumentation is the part of the learning engineering process responsible for designing, developing, and implementing the data collection used within the learning solution to inform iterative improvements to the learning solution.

Analytics is the part of learning engineering responsible for analysis and use of data within the learning solution to inform iterative improvements to the learning solution.

Instrumentation uses sensors to capture data along with software and hardware to process and store data for subsequent analysis, as illustrated in Figure 5. Sensors are human-computer or environment-computer interfaces that capture data.

Data-informed design decisions are needed because human intuitions about what helps people learn are often wrong. Looking at data from simple experimental trials can keep the work focused on the most important design features and avoid costly diversions. Experiments within learning engineering focus on uncovering highly contextualized findings within specific learning experiences, under specific conditions, or for specific populations.

Data analysis for learning engineering is a team sport, often requiring collaboration among psychometricians, learning designers, content and assessment developers, implementation consultants, user interface/user experience (UI/UX) designers, data scientists, software engineers, and product managers. Together, these teams often enable the generation of big data, which are extremely large data sets that may be analyzed computationally to reveal patterns, trends, and associations, especially relating to human behavior and interactions.

Of course, quantitative data does not always tell the whole story. Often, answers to qualitative questions about the conditions and context reveal what is happening. Learning is contextually situated and so are the results of any experimental data. So, learning engineering also embraces qualitative research approaches.

In learning engineering, data are sometimes used to model things about the learner, to predict a learner’s state or behavior under given conditions, and to predict how well a given activity will bring about a particular learning outcome. Data can be used to question existing practices and assumptions.

The Learning Engineering Toolkit includes tools for data analysis such as the learning analytics process model (Czerwinski, Domadia, et al., 2022) shown in Figure 6.

Tools of the Trade

The learning engineering process calls for sets of tools and practices that themselves should be optimized through iterative and data-informed cycles of improvement. An initial set of tools in the Learning Engineering Toolkit address the following:

• tools for understanding the challenge
• tools from the learning sciences
• tools for teaming
• lean-agile development tools
• human-centered design tools
• data instrumentation tools
• software and technology standards as tools
• tools for learner motivation
• implementation tools
• ethical decision-making tools
• data analysis tools

Learning Engineering for Military Learning

Learning engineering is already underway in limited contexts within the U.S. military; for example, the Army’s Synthetic Training Environment (STE) is a virtual training environment that brings together live and virtual training environments, aiming to deliver accessible exercises that mimic the full complexity of the physical world (Stone, 2021). STE development and implementation follows an iterative, multidisciplinary, learning engineering process that applies learning sciences using human-centered engineering design methodologies and data-informed decision-making. The STE with Experiential Learning for Readiness extends the Army’s STE capability with persistent tracking of individual and team performance data to infer proficiency levels, identify strengths and weaknesses, and adaptively tailor coaching and remediation (Goldberg et al., 2021). The complexity of this system of systems is managed by applying principles of engineering, modularization with reusable components and standardized interfaces between systems and components.

Conclusion

The capability of our military personnel is at a disadvantage if the wealth of discoveries from the learning sciences cannot be applied at scale using human-centered engineering methodologies and optimized through a process of data-informed decision-making. “The future learning ecosystem—a holistic, lifelong, personalized learning paradigm—represents a contrast to the Industrial Age model of time-focused, one-size-fits-all learning” (Walcutt & Schatz, 2019, p. 5). This new paradigm requires a complex continuum of data-driven, task-embedded, personalized, lifelong anywhere, anytime, learning. Engineering has been successfully applied to many other complex human-centered challenges. Learning engineering is a new strategic weapon in force readiness. Just as chemical engineering helped win World War II through scaled production of penicillin, learning engineering may help win the next war by optimizing the collective capacity of our forces.

---

References

Barr, A., Dargue, B., Goodell, J., & Redd, B. (2022). Learning engineering is engineering. In J. Goodell & J. Kolodner (Eds.), Learning engineering toolkit: Evidence-based practices from the learning sciences, instructional design, and beyond (pp. 125–152). Routledge. https://doi.org/10.4324/9781003276579-8

Chopra, D., Tanzi, R. (2021). The brain needs reinventing—Here’s how. https://www.deepakchopra.com/articles/the-brain-needs-reinventing-heres-how/

Czerwinski, E., Domadia, T., Craig, S. D., Goodell, J., & Ritter, S. (2022). Data analysis tools. In J. Goodell & J. Kolodner (Eds.), Learning engineering toolkit: Evidence-based practices from the learning sciences, instructional design, and beyond (pp. 365–378). Routledge. https://doi.org/10.4324/9781003276579-23

Czerwinski, E., Goodell, J., Ritter, S., Sottilare, R., Thai, K. P., & Jacobs, D. (2022). Learning engineering uses data (Part 1): Instrumentation. In J. Goodell & J. Kolodner (Eds.), Learning engineering toolkit: Evidence-based practices from the learning sciences, instructional design, and beyond (pp. 153–174). Routledge. https://doi.org/10.4324/9781003276579-9

Goldberg, B., Owens, K., Gupton, K., Hellman, K., Robson, R., Blake-Plock, S., & Hoffman, M. (2021, November 29–December 3). Forging competency and proficiency through the synthetic training environment with an experiential learning for readiness strategy [Paper presentation]. Interservice/Industry Training, Simulation, and Education 2021 Conference (I/ITSEC), Orlando, FL, United States. https://www.xcdsystem.com/iitsec/proceedings/index.cfm?Year=2021&AbID=97151&CID=862#View

Goodell, J. (2022). Introduction. In J. Goodell & J. Kolodner (Eds.), Learning engineering toolkit: Evidence-based practices from the learning sciences, instructional design, and beyond (pp. 5–25). Routledge. https://doi.org/10.4324/9781003276579-3

Goodell, J., Kolodner, J., & Kessler, A. (2022). Learning engineering applies the learning sciences. In J. Goodell & J. Kolodner (Eds.), Learning engineering toolkit: Evidence-based practices from the learning sciences, instructional design, and beyond (pp. 47–82). Routledge. https://doi.org/10.4324/9781003276579-6

IEEE IC Industry Consortium on Learning Engineering. (2019). What is ICICLE? https://sagroups.ieee.org/icicle/

Kessler, A., Craig, S. D., Goodell, J., Kurzweil, D., & Greenwald, S. W. (2022). Learning engineering is a process. In J. Goodell & J. Kolodner (Eds.). Learning engineering toolkit: Evidence-based practices from the learning sciences, instructional design, and beyond (pp. 29–46). Routledge. https://doi.org/10.4324/9781003276579-5

National Academies of Sciences, Engineering, and Medicine. (2018). How people learn II: Learners, contexts, and cultures. National Academies Press. https://doi.org/10.17226/24783

National Research Council. (2000). How people learn: Brain, mind, experience, and school: Expanded edition. National Academies Press. https://doi.org/10.17226/9853

Simon, H. A. (1967). The job of a college president. The Educational Record, 48, 68–78. http://iiif.library.cmu.edu/file/Simon_box00065_fld04978_bdl0001_doc0001/Simon_box00065_fld04978_bdl0001_doc0001.pdf

Stone, A. (2021, September 30). US Army makes headway on synthetic training environment. Defense News. https://www.defensenews.com/training-sim/2021/09/30/us-army-makes-headway-on-synthetic-training-environment/

Thai, K. P., Craig, S. D., Goodell, J., Lis, J., Schoenherr, J. R., & Kolodner, J. (2022). Learning engineering is human-centered. In J. Goodell & J. Kolodner (Eds.), Learning engineering toolkit: Evidence-based practices from the learning sciences, instructional design, and beyond (pp. 83–123). Routledge. https://doi.org/10.4324/9781003276579-7

Walcutt, J. J., & Schatz, S. (Eds.). (2019). Modernizing learning: Building the future learning ecosystem. U.S. Government Publishing Office.

Wood, J. (2010, July 16). Penicillin: The Oxford story. Oxford News Blog. https://www.ox.ac.uk/news/science-blog/penicillin-oxford-story

Back to Top

Conference Edition 2023

QUICK LINKS