The Institutional Research and Assessment Division of Army University

Research About and for the U.S. Army Training and Doctrine Command

Alice J. (Sena) Garven and
Wade R. Elmore

Army University

Download the PDF

Abstract

Institutional research provides invaluable feedback about an organization’s products and performance. Although there is a wealth of institutional research occurring throughout U.S. Army Professional Military Education and Training (PMET), this research largely remains in stovepipes, thus limiting the visibility of insights and innovations identified. To fill this need, the Army University (ArmyU) has created the Institutional Research and Assessment Division (IRAD) to conduct research, manage research programs, and provide expert technical assistance to the U.S. Army Training and Doctrine Command (TRADOC). IRAD has developed the ArmyU’s institutional research plan (AIRP) to fill the gap in enterprise-level institutional research in PMET. The AIRP describes research activities on a continuum with research about ArmyU on one end and research for ArmyU on the other. IRAD has developed two lines of effort (LOEs): LOE 1 is to work with those conducting research activities to coordinate, synchronize, and integrate research that is currently being conducted; and LOE 2 is to investigate independent research questions to benefit PMET and the Army. The level of success for both LOEs depends on cooperation and collaboration throughout PMET, and it will require IRAD to develop a federated network of organizations that value research and can contribute to an enterprise level of understanding, insight, and innovation.

Why Does Army University Need an Institutional Research and Assessment Division?

Institutional research is a vital resource to traditional colleges and universities as it provides valuable research expertise and feedback to the institution in order to improve, innovate, and adapt education to meet the needs of the students, and in the case of Army University (ArmyU), the Army. While the term institutional research covers a breadth of topics, and there is no standard set of research conducted by an institutional research department or division, there are several forms of feedback that these units can provide to their institutions. Evaluations can provide actionable feedback on courses, classes, curriculum, instructors, programs, and technologies or techniques to both identify what is unsuccessful and needs to be modified as well as what is successful and should be promulgated throughout the institution. Reporting of student success also provides an overall metric of the success of an institution.

As Army Professional Military Education and Training (PMET) has developed and evolved to meet the needs of the Army, the number of schools and centers of excellence responsible for education and training has increased and become geographically dispersed. While the training and education has remained world class-integration, synchronization, and innovation across the enterprise has not been fostered. ArmyU was created to modernize PMET to better prepare soldiers and Army civilians for the complex 21st-century security environment. To achieve this goal, ArmyU’s mission is to increase academic rigor and relevance; increase competence, character, and commitment of soldiers, Army civilians, and leaders; expand the prestige of Army learning organizations; identify and promulgate best practices in education and management; and increase the agility of PMET to adapt to the changes needed by the operational force (Army University, 2017, p. 2). ArmyU connects PMET across cohorts within a unified educational system organized like the best colleges and universities in the United States. This organization allows for synchronization of education and training while sharing resources across the learning enterprise and cultivating innovation through the sharing of information. The creation of ArmyU also allows for the development of the first enterprise-wide institutional research division to synchronize research and assessment being performed across the enterprise and to help to decrease the stovepipes of research that exist within the schools and centers.

What Is the Institutional Research and Assessment Division?

The Institutional Research and Assessment Division (IRAD) was established in 2017, as ArmyU’s primary staff section to conduct research, manage research programs, and provide technical assistance to the U.S. Army Training and Doctrine Command (TRADOC) to support decision-making for innovative learning, leading to improved tactical and technical expertise and increased readiness of our soldiers. The research IRAD supports exists on a continuum anchored by what is described as research about ArmyU on one end and research for ArmyU on the other, as depicted in the figure. Research efforts on the about end of the continuum are more descriptive in nature and include programmatic efforts to maintain and improve existing systems. Research for ArmyU is more exploratory in nature and is undertaken with the intent to discover and test innovative solutions, which may be promulgated throughout PMET. Just like ArmyU is not designed to perform education throughout PMET, but rather to support and provide technical expertise to the schools and centers, IRAD is not intended to perform all of the institutional research done within the schools and centers but to synchronize, coordinate, and empower schools and centers with the tools to effectively conduct their own research. IRAD will initiate and retain control of much of the research done for the ArmyU end of the continuum, and success will largely rely on partnerships throughout the institutional and operational Army. The work being done on the about ArmyU end of the continuum is largely already being done throughout PMET, and IRAD will serve as the synchronizing and unifying role, which will require collaboration with the organizations within ArmyU and TRADOC who are collecting the institutional data. This model is generally referred to as a federated network model and gives the Army an advantage over the traditional model by preserving the autonomy and authority of the individual organizations (Swing & Ross, 2016, p. 8). This is a break from the traditional model used in most colleges and universities, where institutional research is centralized into a single department or division responsible for conducting all institutional research. The traditional model would present many difficulties to ArmyU and ultimately would not fit within its mission of supporting and assisting the schools and centers. Additionally, the federated network model fosters a climate of collaboration and mutual responsibility toward the goal of improving the enterprise.

What Gap Does IRAD Fill for ArmyU and the Army?

Institutional research is currently happening throughout PMET with each school or center relying on their quality assurance office, curriculum development staff, faculty development staff, or other staff to gather feedback internally. This is a valuable resource for the individual schools and centers to sustain, improve, and adapt PMET to the needs of their soldiers, but it does not provide insights institution wide. In addition to the research performed at the schools and centers, there are several organizations throughout TRADOC that collect valuable data to provide the schools and centers feedback on their compliance with the regulations and standards set for education and training. Furthermore, there are additional units within TRADOC that provide research and analysis to their organization with institutional research as a small subset of their duties. There are still other organizations within TRADOC that perform institutional research and analysis to evaluate programs, techniques, and technologies by request as an additional duty. Unlike traditional colleges and universities—the breadth of institutional research is occurring in PMET without a coordinating body.

The level of institutional research being conducted throughout PMET demonstrates there is value for the feedback it generates, but the innovation and insights generated by this research are limited to those who are aware of it. The gap that IRAD fills is lack of a unifying organization to help share that information across schools, centers, and other organizations to facilitate adaptation and innovation at the organizational level. In the same way that ArmyU was designed to modernize, synchronize, and integrate PMET, IRAD is uniquely suited to coordinate, synchronize, and integrate institutional research throughout PMET, filling the identified gap. The existing information being collected could be standardized, compiled, and synchronized across the enterprise by IRAD to feed into a Learning Common Operating Picture providing decision information for all levels of leadership throughout the PMET system. IRAD also has the technical expertise to help build the capacity of the schools and centers to gather feedback from sources that may be more difficult to reach, such as the operational force. IRAD has been staffed to have the expertise to both draw together the institutional research currently conducted throughout the enterprise to answer larger questions about the enterprise, as well as conduct and coordinate enterprise-level research to evaluate techniques and technologies.

What Is IRAD Going to Do?

IRAD has developed a plan to coordinate, synchronize and integrate the research occurring throughout PMET. As IRAD laid out the plan for bringing together the information being collected throughout PMET, it kept in mind the principles of the mission command philosophy and the service and support orientation of ArmyU (U.S. Department of the Army, 2012). While it is clear that IRAD is not in command of any of the organizations they support, IRAD does hope to provide leadership in the efforts to coordinate, synchronize, and integrate research across PMET. While developing the ArmyU institutional research plan (AIRP), IRAD was presented with the opportunity—and frankly, the necessity—to break from the traditional model for institutional research found in most colleges and universities. The traditional model relies on a centralized institutional research division that controls the research and feeds the results and recommendations to the subordinate organization and the organizational leadership. The break from the traditional model is a shift away from a centralized model toward a decentralized model with IRAD supporting the independent units conducting their own research while IRAD performs the duties of synchronization, compiling, analyzing, and reporting the findings at the enterprise level. IRAD respects the autonomy and expertise of the schools and centers to identify what feedback would be most useful to them while maximizing the benefits of creating an enterprise-wide feedback system.

The AIRP can be broken down into two major lines of effort (LOE) that lay on different ends of the IRAD research continuum, as seen in the figure (on page 76). LOE 1 exists within the about ArmyU end of the IRAD research continuum and includes the coordination, synchronization, and integration of the institutional research conducted by the independent institutions within PMET. The tasks within this line are focused on working with organizations throughout PMET to bring together the relevant information they are collecting to be aggregated up to the enterprise level. LOE 2 is IRAD-conducted research, which falls on the for ArmyU end of the IRAD research continuum. The efforts in LOE 2 are internally and externally driven research focusing on innovation and evaluation of techniques and technologies relevant to PMET.

LOE 1, in the AIRP, fills the gap in PMET enterprise-level institutional research by assigning IRAD the duties of coordinating, synchronizing, and integrating research across PMET. In this role, IRAD will identify all of the sources of institutionally relevant data and begin a dialogue with those organizations. The existing information being collected could be compiled and synchronized across the enterprise by IRAD to feed into a Learning Common Operating Picture providing decision information for all levels of leadership throughout the PMET system. Communication with each organization may vary depending on what IRAD can offer them as well as the type and amount of institutional research they conduct, but collaboration for mutually beneficial outcomes will always be the goal.

Aligned with our mission to support ArmyU and the organizations it supports, the major goal for AIRP is to assist schools and centers in increasing the quality and utility of information gathered within their internal feedback loops, such as faculty, course, and curriculum evaluations, while simultaneously improving the standardization enterprise-wide so that the information being collected can be compiled and analyzed for enterprise-level feedback. The plan is to leverage the expertise within IRAD to improve the quality of information being collected and to ensure that this information is most useful to those collecting it. This assistance could take the form of identifying questions of interest, reviewing items used, or incorporating new survey technology available to streamline information collection and expand the reach beyond those who recently completed the education or training. IRAD can also leverage its enterprise-wide viewpoint to look for commonalities in the questions of interest across organizations so those questions can be standardized providing consistent information at the enterprise level. As LOE 1 matures, IRAD will be able to measure and report enterprise-level advancement and identify schools and centers that excel in certain areas to look for best practices.

In AIRP, LOE 2 encapsulates the research efforts by IRAD to evaluate and investigate techniques, theories, and technologies that might be valuable to the PMET. These efforts will be driven by many factors, but they will generally fall within three major categories. The first major effort will investigate research questions generated internally within IRAD. These will be enterprise relevant questions based on observations while interacting with both the operational and institutional Army. These projects will generally not require any outside resources and will be conducted concurrently with the LOEs. The second major effort will be answering questions posed by the Learning Sciences Committee (LScC) and the Army Learning Coordination Council. IRAD will likely not have the capacity to answer all of the questions posed by these committees, and thus will need to prioritize and possibly seek additional resources to support this effort on a case-by-case basis. The third major effort will be an ongoing effort working with the schools and centers throughout PMET to identify techniques, tactics, and procedures (TTPs) developed at their institutions. The TTPs collected will be analyzed and tested to identify which meet the criteria of a best practice. The identified best practices will then be disseminated throughout the enterprise.

How Is IRAD Going to Do It?

The implementation of the AIRP is tied to building relationships throughout PMET as the federated network relies on the independent organizations throughout PMET collaborating and sharing information with IRAD. The first step to building relationships is bringing people to the table. IRAD has begun this process by leveraging its role in the LScC by inviting stakeholders and collaborators to be members, where IRAD hopes to work to build a shared understanding of the value of institutional research to the Army learning enterprise, and how the members can contribute and collaborate to build the network. IRAD hopes that this will create momentum from those on the committee to all of PMET through personal and professional connections.

Implementation of LOE 1 from AIRP will rely on creating a network of organizations that see the value of standardized and comparable feedback across PMET, which can be aggregated to the enterprise level. IRAD plans to build this network largely through the relationships established in the LScC, by working with committee members to identify data sources throughout PMET. IRAD has already begun this process by inviting those identified as stakeholders and data sources to be a part of the committee. IRAD will actively work to expand the network as other stakeholder organizations are identified and invited to join the network. IRAD also hopes that the network will expand organically and as members spread the word that participation has benefits.

Another avenue IRAD is using to build relationships and expand the network is through the Learning Enterprise Assistance Program (LEAP), an ArmyU program developed to provide expert assistance from ArmyU to the learning enterprise. IRAD’s contribution to LEAP is principally assisting the schools and centers build their capacity to collect actionable feedback about their products and services. This is a win-win-win situation. The schools and centers win through an increased capacity to do institutional research and gather feedback both from students and faculty and from the operational force. IRAD wins by building the network required for coordinated, synchronized, and integrated enterprise-level institutional research. The enterprise wins by gaining access to the innovation and insights generated throughout PMET.

Progress has been made on all three major efforts within LOE 2. IRAD has progressed within the first major effort by establishing a process for identifying, vetting, approving, and conducting research within IRAD. IRAD is currently working through the process with several research ideas identified in briefings from senior leaders and conversations with other divisions within ArmyU. IRAD expects that two independent research projects will kick off in fiscal year 2019. As IRAD reaches full operational capacity, it should be able to increase the amount of internally initiated research, but this will always need to be balanced with externally generated research, which will generally take precedent over internal research. The second major effort within LOE 2 will be influenced by IRAD’s exposure to the organizations that it supports. To fully implement this, IRAD will need to advertise its mission and capabilities within ArmyU, the Combined Arms Center, and TRADOC so organizations will reach out to IRAD to sponsor or suggest research. The first meetings of the LScC have occurred, and there has been good participation with many stakeholders already attending. The third major effort within LOE 2 is also underway and will rely on the same relationships as all of IRAD’s other research efforts. As the goal is to establish a system to identify education and training TTPs that rise to the level of a best practice through analysis, which can then be disseminated throughout the enterprise, IRAD will need to work with the schools and centers to create a system that provides information with minimal additional work.

The successful implementation of the AIRP is clearly contingent on the participation of those at the schools and centers. IRAD hopes to play a leadership role in the efforts to collect, analyze, and report enterprise-wide institutional research while helping the schools and centers streamline and standardize their internal institutional research processes. IRAD believes that a focus on developing a federated network model with emphasis on collaboration and productive discourse will benefit all involved.

Conclusion

Institutional research is an extremely valuable way to gain insight and feedback. There is a wealth of institutional research being done throughout PMET, but it is mostly stovepiped within the schools, centers, and other organizations doing the research. This creates a lack of visibility for successful innovation and adaptation resulting in missed opportunities to save time, effort, and resources. As the enterprise-level institutional research organization, IRAD has developed a plan to coordinate, synchronize, and integrate research from across PMET, while also providing enterprise-level research and support. The key component to the ArmyU institutional research plan is the federated network model for institutional research, which relies on highly autonomous members working together to improve PMET. By implementing a federated network, IRAD can fill the data information gap for ArmyU and TRADOC leadership while facilitating individual components and schools to fulfill their obligations and responsibilities to their specific chain of command.

IRAD will need the schools, centers of excellence, and all other TRADOC organizations conducting research activities to become collaborators in an enterprise-level institutional research federated network. With the understanding that this effort will fail without the participation of the units throughout PMET, IRAD hopes to build trust through transparency and demonstrate the value of the federated network through successful collaborations. This article is the first effort by IRAD to build a shared understanding of institutional research among stakeholders, explain the PMET intent, and establish the purpose, goals, and status of enterprise-wide institutional research in PMET. Enterprise-level institutional research will support those who develop curriculum, faculty, and systems that keep PMET in the U.S. Army relevant and the warfighters ready to win in a complex world.

---

References

Army University. (2017). Army learning strategy. Retrieved from https://usacac.army.mil/node/1720

Swing, R. L., & Ross, L. E. (2016). A new vision for institutional research. Change: The Magazine of Higher Learning, 48(2), 6–13. doi:10.1080/00091383.2016.1163132

U.S. Department of the Army. (2014). Mission command (Army Doctrine Publication 6-0). Washington, DC: U.S. Government Publishing Office.

Back to Top

April 2019