Journal of Military Learning

Developing an Inclusive Outcomes Statement Peer-Review

Adapting the Degree Qualifications
Profile to a Military Context

Erin N. O’Reilly

Defense Language Institute Foreign Language Center

Download the PDF Download the PDF


This article explores the application of the Degree Qualifications Profile (DQP) framework at the Defense Language Institute Foreign Language Center to revise an institutional learning statement for inclusiveness, sufficiency, and distinctiveness appropriate to the institutional mission. Using a structured Tuning process, which Jankowski and Marshall (2017) define as a “faculty-driven process of determining in a specific field of study what a student should know and be able to so upon completion of a degree” (p. 7), a matrixed team of stakeholders rewrote an outcomes statement suitable for diverse external audiences. The results of the process suggest that adapting the DQP framework to military training and education contexts has merit.


Articulating the value of training and education programs for diverse stakeholders has become a necessary undertaking for both higher education and the U.S. Army. For the former, a variety of factors have shifted public perceptions of college education, including the increase in the cost for degrees and concerns about student preparedness (Jankowski & Marshall, 2017; Jones & Kleiner, 2015). Similarly, while the military has a strong history of outcomes-based training and education programs, only in recent years has Army University launched a concerted effort to capture the learning experiences across a service member’s career through formal certification, licensure, and educational credentials in partnership with academia and industry (Army University, 2017; Kem & Hotaling, 2017). The parallel evolution of higher education and the Army training mission has resulted in a common need to effectively communicate our stories of learning to diverse audiences, including institutional partners, future employers, and students. Academia has turned to qualifications frameworks, or outcomes rubrics, to communicate value (Jankowski & Provezis, 2011; Jones & Kleiner, 2015). These frameworks, if adapted, may be suitable tools to help define program quality in a military training context.

This article explores the application of the Degree Qualifications Profile (DQP) framework (Lumina Foundation for Education, 2011) to articulate the program learning outcomes for military service members who graduate from the Defense Language Institute Foreign Language Center (DLIFLC), Monterey, California, with an Associate of Arts degree in foreign language. The article then explores the application process and the challenges in working with the framework. Finally, it closes with recommendations for programs concerned with defining the value of military training and education for audiences external to the Department of Defense (DOD), including academia and future industry employers of program graduates.

Qualifications Frameworks

In the early 2000s, leaders in the higher-education arena identified accountability as a critical issue surrounding the public’s perceptions of quality. This was an unsurprising find considering the majority of assessment reviews were internally oriented using specialized language not intended for public audiences (American Council on Education, 2004). Since that time, differences in vocabulary and assessment processes have continued to be problematic for stakeholders, opinion makers, and the public in their attempts to define and assess quality in higher education (Gaston, 2014; Suskie, 2015). Qualifications rubrics, or frameworks, evolved in response to these challenges and have received increased attention in academia in their deployment at the local, state, and national levels over the past decade (Adelman, Ewell, Gaston, & Geary Schneider, 2014; Markle, Brenneman, Jackson, Burrus, & Robbins, 2013).

These frameworks describe the knowledge, skills, and abilities (KSAs) of a training and education program and their interlinkage based on a qualification, or qualification level, within a system. At the national level, frameworks may be quality assured by governments, accountability offices, or education nonprofits. A framework’s purpose is twofold: to instill learner confidence that their learning will be recognized and to assure employers of the learners’ skill sets. Frameworks can link together to form overarching systems, as with the European Qualifications Framework and the Qualifications Framework for the European Higher Education Area (“How Does the EQF Work?,” 2019). These interlinking systems accommodate learners who actively move between programs.

Frameworks offer several advantages for institutes working on outcomes (“Connecting Credentials Framework,” 2016). First, they provide a common reference point to compare levels of KSAs that are integrated into degrees, certificates, or licensures within academia and industry. Additionally, frameworks can link knowledge at discrete points across diverse credentials by using a common language. This facilitates translating learning from one credential to subsequent learning programs to ensure continuity of learning and potentially to reduce completion time in a given program.

In 2011, the Lumina Foundation published the DQP, subsequently updated in 2014 (Adelman et al., 2014). The developers of the DQP designed the tool to be flexible enough for any context and to serve “as a universal translator, allowing various groups to talk with each other” (Jankowski & Marshall, 2017, p. 8). Within the United States, the DQP framework has emerged as one of the primary tools to form a consensus on a public definition of quality in higher education at three distinct levels: associate, baccalaureate, and master’s (Jones & Kleiner, 2015). Without a shared definition of quality, higher education cannot self-assess and provide accountability metrics understood by diverse audiences.

The DQP functions as a tool to focus conversations on appropriate outcomes and levels of rigor (Ewell, 2016; Suskie, 2015). Since 2011, more than 780 institutions have used the DQP for a range of purposes, including to assess the connection between general education and major course sequences; to reorient the mission statement and curriculum following a DQP review; to engage in discussions with stakeholders, including employers, about current and future needs; and to perform a gap analysis when reviewing learning statements (Jankowski & Marshall, 2017). This article explores the application of the DQP as a tool to review the inclusiveness of a programmatic outcomes statement. Here, inclusiveness refers to the extent to which the outcomes statement can be understood by diverse audiences, including professionals in higher education outside of the DOD, program graduates, and industry employers.

A statement’s appropriacy can be assessed through a Tuning process. Faculty within the European Union developed Tuning, a “faculty-driven process of determining in a specific field of study what a student should know and be able to so upon completion of a degree” (Jankowski & Marshall, 2017, p. 7), as a means to define learning competencies in such a manner that students moving between countries during their higher education would be prepared regardless of where they had previously studied. Marshall, Kalina, and Dane (2010) describe Tuning as answering the fundamental question: When students complete a specific program, what should they know, understand, and be able to do? In U.S. higher education, Tuning consists of five constituent elements: (a) identifying core competencies for a discipline, (b) identifying possible career pathways, (c) gathering stakeholder input, (d) revising core competencies based on input, and (e) implementing results at the local level. While the exact method of Tuning can take on different forms, broad stakeholder involvement serves as a cornerstone of Tuning, which fundamentally encourages faculty to transition thinking from the local to the discipline level (Institute for Evidence-Based Change [IEBC], 2012). This stakeholder involvement, in turn, promotes integrative thinking about the students’ learning experiences between field-specific knowledge and general education and also across levels (e.g., associate to bachelor’s degrees). Finally, those engaged in Tuning have a greater understanding of how other types of institutions approach learning and the description of learning as they work to identify core competencies within a discipline.

Tuning and the DQP are complementary (IEBC, 2012). While Tuning encourages programs to consider discipline-specific expertise across levels, the DQP offers a general framework for five areas of knowledge that may fall outside of a given discipline. These areas include (a) specialized knowledge, (b) broad/integrative knowledge, (c) intellectual skills, (d) applied learning, and (e) civic learning (Adelman et al., 2014). Identifying these five proficiency areas of learning promotes a comprehensive review of the intellectual skills learners develop over a sequence of instruction.

The first area, specialized knowledge, encompasses the outcomes specific to a particular program. This accounts for those concepts, theories, and bodies of knowledge considered fundamental for a given area of study (i.e., field or major). The framework does not prescribe pedagogy but rather a reference point for the level of learning. Second, broad and integrative knowledge refers to learning that fosters global, cultural, and democratic perspectives. While broad and integrative knowledge typically comes from a general education course sequence, the DQP framework allows learning to be articulated at all levels, thus providing support and context for specialized studies. Next, intellectual skills capture learners’ fluency in communication (both oral and written) as well as analytical inquiry. Applied learning, the fourth area, is defined as the ability to recall information from prior learning and to combine it with new information in novel situations. The fifth area, civic learning, captures the students’ ability to engage with diverse perspectives and to form their own responses to social challenges.

An additional sixth area, institution-specific outcomes, allows institutions to identify learning outcomes unique to their missions (Adelman et al., 2014). Institution-specific outcomes are appropriate for programs that have incorporated mission-based outcomes in their curricula (e.g., faith-based or military programs). This final area allows for increased flexibility in adapting the DQP framework to different contexts.

Program Overview

Since 2002, DLIFLC has awarded Associate of Arts (AA) degrees in foreign language to program graduates through its regional accreditation (Defense Language Institute Foreign Language Center DLIFLC, 2019). The institute currently offers 17 credit-bearing language programs that share standardized learning outcomes related to foreign language proficiency.

As a small, nontraditional federal degree-granting program, one ongoing challenge for the institute involves communicating the value of its credit-bearing courses to audiences external to the DOD, including partnerships with civilian educational institutions through formal articulation agreements, and through the credit-transfer process for individual graduates.

In a recent initiative to review the course catalog documents, the institute convened an accreditation working committee. The catalog’s contents meet several accreditation requirements set forth by the accreditor, one of which is the inclusion of a concise statement on programmatic learning outcomes. Clear outcomes statements using accessible language appropriate for a range of audiences serve to bridge the gap for service members interested in continuing their higher education or seeking employment in industry. The original statement was written as follows:

At the end of the DLIFLC language program, students will be able to demonstrate and utilize speaking, reading, listening and writing skills of the language along with the cultural and ethical knowledge of the country and language they are learning. These skills are measured through assessment processes such as Oral Proficiency Interview (OPI), Defense Language Proficiency Test 5 (DLPT5), student learning behavior, and immersion. [DLIFLC], 2017, p. 26)


The previous statement had two limitations. First, it conveyed limited information about the actual KSAs of the program’s graduates. The field of foreign language has rich, discipline-specific outcomes that were not included in the original statement (American Council on the Teaching of Foreign Languages, 2015). Second, the statement referenced DOD assessment processes that would be unfamiliar to stakeholders outside of the government. Synthesizing the value of any military training or education in a manner that corresponds to a parallel program in higher education or industry licensing body can support service members throughout their military and civilian careers. As such, the committee adapted the DQP framework to develop an inclusive outcomes statement for the institute’s AA degree program.


The objective of this applied project was to explore the suitability of adapting the DQP framework in the development of an inclusive outcomes statement for a military foreign language training and education program.


Participants. Using the Tuning approach, a matrixed committee with 13 members participated in the statement revision process. Participants were selected in consultation with academic leadership to ensure broad input from across the institute, as well as expertise in higher education. Led by the institute’s accreditation cochairs, the team consisted of administrative representatives from senior academic leadership, shared governance representation, including members of the faculty and Academic Senate, staff representation from the Office of the Registrar and the academic library, and military representation from past graduates serving in administrative appointments.

Process. The statement-review process involved a series of four one-hour meetings spanning six weeks. Prior to the first meeting, the accreditation cochairs benchmarked outcomes statements at peer institutions in higher education. Several programs had applied the DQP framework to their foreign language majors. The cochairs provided these statements, copies of the DQP framework, and a handout of DQP definitions to the working group during the first meeting. The cochairs took notes independently during meetings by annotating participants’ comments and input and then cross-checked the notes immediately following each meeting.

In the first meeting, the committee cochairs introduced the DQP tool and framework. Participants were asked to brainstorm graduates’ KSAs related to each of the six areas in the framework independently. During the second meeting, the members shared examples through discussion and began filling in the framework. Next, the committee chairs consolidated the comments and drafted a statement. The committee reviewed the draft statement and refined the language with collective input at the third meeting. Between the third and fourth meetings, the draft statement was shared with the institute’s Academic Senate for feedback. The committee incorporated this input and completed the statement in the fourth and final meeting.


Application process. The DQP framework facilitated a discussion that mirrored conversations at peer institutions in academia, namely by understanding that the end product of education, regardless of major or focus area, is individuals capable of thinking critically, communicating clearly, and adapting specialized knowledge needed in the workforce of tomorrow (Jones & Kleiner, 2015). Prior to coming to a consensus about outcomes, however, the group needed to develop a shared understanding of the DQP framework terminology. Below, the revised statement is broken down into three sections, each preceded by a discussion of how the group applied the DQP terminology for that section: specialized knowledge and applied learning, broad and integrative learning, and civic learning.

Specialized knowledge and applied learning. First, the committee interpreted specialized knowledge as those foreign language proficiency outcomes taught at peer institutions in academia. This involved benchmarking similar civilian degree programs and mapping outcomes referencing commonly accepted standards and practice for the level of instruction (i.e., two-year foreign language degrees).

Added to these outcomes were institution-specific emphases, covering skills unique to military foreign language professionals outside of the scope of foreign language programs offered at civilian colleges. This specialized knowledge included regional security issues, U.S. foreign policy, and military linguists’ job-specific skills.

Next, the committee discussed applied learning and intellectual skills. These areas required a holistic consideration of the learning taking place over the program of instruction (e.g., the graduates’ ability to function in diverse groups or synthesizing information when faced with unknowns). Outcomes in applied learning and intellectual skills may be found in general education courses or discipline-specific courses. The faculty members led a discussion focused on classroom and formal coursework experiences that required students to demonstrate content mastery, including problem-solving, dealing with ambiguity, and integrating background knowledge. This discussion covered examples of how these skills develop throughout the program of instruction.

Beginning with the areas of specialized knowledge, and applied learning and intellectual skills, the new statement was expanded as follows:

Service members who graduate DLIFLC with an AA degree achieve a minimum functional and working proficiency in listening and reading (receptive skills) at the Advanced Low Level (2) and proficiency in speaking (productive skill) at the Intermediate High Level (1+) commensurate with the Interagency Language Roundtable guidelines. They have an understanding of the linguistic components and lexicon of the target language; a fundamental knowledge pertaining to the cultural institutions, patterns of behavior, history and geography of the target culture(s) and how these affect values and traditions; and a demonstrated respect, understanding and sensitivity for the cultural norms and values, contributions, social issues, and political institutions of the language’s native speakers. Graduates demonstrate problem-solving skills and the ability to deal with knowledge gaps on the job through the application of their education, training, skills, and abilities in the foreign language. They have basic knowledge and awareness of security issues of the target language region and have demonstrated the foundational skills of a military linguist, including transcription, translation, and interpretation. (DLIFLC, 2019, p. 27)


Broad and integrative learning. For the second section, the team identified broad and integrative knowledge outcomes through the students’ specialized studies and general education coursework. The DQP emphasizes that broad and integrative learning can happen at all levels to support specialized learning (Adelman et al., 2014). A predisposition to focus on the narrow training mission (foreign language acquisition as a technical skill) challenged the team’s ability to move beyond field-specific outcomes, which had been included in the first part of the statement. Reorienting the discussion around a shared definition of broad and integrative knowledge led to a statement that more accurately reflected the rich learning that occurs throughout the degree program:

Service members who hold an AA degree in foreign language from DLIFLC possess broad integrative knowledge, skills, and perspectives supportive of the military linguist mission. This knowledge promotes life-long learning in a wide range of human interests and is considered foundational to critically engage with personal, cultural, moral, civic, and societal issues. (DLIFLC, 2019, p. 27)


Civic learning. Finally, the committee considered several elements for the civic learning knowledge area, including (a) the service members’ participation in a diverse, cross-cultural training context; (b) general education and specialized military studies; and (c) military experiences that are a part of the indoctrination process but take place outside of the classroom during the service members’ time at the institute. Combined, these experiences support the development of global citizens prepared for civic responsibilities, as articulated in the final part of the statement:

DLIFLC graduates represent the U.S. as global citizens through their civic responsibilities. At the AA level, graduates develop as global citizens through their educational and military experiences that promote awareness of and respect for complex cross-cultural interactions with individuals who have diverse religions, socio-economic backgrounds, and linguistic perspectives. (DLIFLC, 2019, p. 27)


Overall, the committee’s experience in applying the DQP framework was similar to that of other colleges. According to Lederman (2014), one of the strengths of the DQP is that it encourages deeper conversations among faculty members and administrators about what is happening in the classroom, what students should be learning, and what students are learning. The committee discussed student assignments, work products, and assessments as evidence during the statement revision process, further supporting the finding that the DQP framework promotes dialogue around learning outcomes (Ewell, 2013). Additionally, the DQP encouraged the committee to broaden its consideration of learning outcomes beyond the technical skills focus and across the graduates’ experiences at the institute. Examining the nexus between the technical learning and the holistic training experience resulted in a richer and more accurate description of what service members have achieved upon degree conferral. The logical next step in the DQP process would involve further mapping and assessment of the degree-level learning outcomes across the training program.

Barriers to Successful Implementation

Adapting the DQP to a nontraditional education program presented unique challenges. Different audiences possess different levels of understanding and distinctive ways of speaking, a truism for any military training program. Added to this, every field has its own technical language to describe what it does but at the risk that “audiences may take that language as jargon that interferes with their understanding” (Jankowski & Marshall, 2017, p. 156). Through Tuning, faculty develop a greater awareness of how colleagues at other institutions describe learning. When crafting a holistic outcomes statement, the language needed to be inclusive and accessible for the diverse audience of professionals in higher education and private industry. As a military training institution with a well-defined mission, however, committee members had difficulty moving beyond the specialized knowledge and terminology used within the DOD to describe language learning and language proficiency. For example, while the DOD uses the Interagency Language Roundtable scale to measure foreign language proficiency, the institute’s academic peers rely on the American Council on Foreign Language Teaching scale. The final statement includes descriptors for both scales. Similarly, common student-learning outcomes descriptors, which were understood within the working group, needed to be refined for an external audience. To that end, including representatives from the Office of the Registrar and faculty representatives with experience in U.S. higher education on the working team was crucial to the process’s success.

A separate challenge was presenting the DQP framework in a manner so that group members understood its purpose. The DQP framework is not aspirational nor is it designed to capture the ideal outcomes of a program. Rather, the framework articulates what every learner can do at the end of the instructional sequence (Ewell, 2013). This can be a paradigm shift for trainers and educators working with a range of learner capabilities, as the tendency is to focus on the exemplars (Adelman et al., 2014).

A final administrative challenge involved workload. Tuning’s intentional involvement of diverse stakeholders, including faculty representatives, administrators, student support services, and program graduates, resulted in a coherent narrative statement. However, these same individuals had conflicting time demands. While participation was viewed as a positive and valued service to the institute, the members could not dedicate time outside of the structured meetings to work on the project. Reviewing the framework, discussing as a group, and then coming back to continue brainstorming proved more effective than asking individuals to work independently.


The DQP served as a tool to dialogue collectively about the learning experiences during a program and, arguably, across a lifetime of professional training and education. Every program serves a unique mission, population, and curricular model. These will shape how a program chooses to engage with outcomes frameworks. Several recommendations can be made for others considering adapting the DQP. First, cross-matrixed working groups with representative stakeholders (i.e., Tuning) will ensure an inclusive end product and raise the collective consciousness of the holistic learner experience. Second, sufficient time needs to be dedicated to norm the group on a shared definition of the DQP terminology. Third, the group will need access to outcomes statements from peer programs in academia or licensing and credentialing bodies for benchmarking purposes (Ewell, 2013). Likewise, selecting programs that have used the DQP themselves further facilitates the process by providing working examples. This step brings attention to the common language used in a given field or sector outside of the military. Finally, teams will want to consider participants’ workload to manage the project effectively.

A training program’s breadth may determine the relevancy of each of the six DQP knowledge areas. The application of the DQP to the DLIFLC military training and education context was unique because the institute has a comprehensive degree program, including a general education component. As such, the committee addressed each area of the framework with the institute-specific category serving to capture the learning outcomes distinctive to the military mission. While those leading programs with narrower training missions might find only certain areas of the DQP relevant, they may also discover general-education outcomes embedded in the curricula. Reviewing statements from peer institutions and including team members with experience in higher education can foster this broader conversation.


Institutions bear the responsibility for defining the value of a training or education program in a manner that clearly conveys learning outcomes to the program’s stakeholders. Producing documents tailored to express the purpose, goals, and substance of a program to diverse groups promote inclusiveness by facilitating communication (Jankowski & Marshall, 2017). The designers of the DQP developed an adaptable tool capable of identifying and assessing learning outcomes using a common language. This article examined how the DQP framework can be employed to evaluate an institutional learning outcomes statement’s sufficiency and strength within a military training and education context.

Military programs aspiring to translate their courses for civilian credits face a challenge familiar to higher education: a sector deeply committed to articulating the value of certifications and degrees to diverse stakeholders. The DQP may help administrators and instructors to think broadly about the learning competencies that military professionals acquire throughout their careers in different training settings and to define those competencies in an inclusive manner that enables academia and industry to better understand the value of the military learning experience.

Disclaimer. This article has been approved for public release by the Defense Language Institute Foreign Language Center’s Public Affairs Office. Contents of this article are not necessarily the official views of the Defense Language Institute Foreign Language Center nor are they endorsed by the Department of the Army, the Department of Defense, or the U.S. government. All third-party products/materials featured in the article remain the intellectual property of their respective authors/originators. Use of outside materials is done under the fair use copyright principle, for educational purposes only. The content of this article is the sole responsibility of the author.


Adelman, C., Ewell, P., Gaston, P., & Geary Schneider, C. (2014). The degree qualifications profile. Indianapolis, IN: Lumina Foundation. Retrieved from

American Council on Education. (2004). Public accountability for student learning in higher education: Issues and options. Washington, DC: Business Higher Education Forum.

American Council on the Teaching of Foreign Languages (ACTFL). (2015). ACTFL performance descriptors for language learners. Alexandria, VA: Author. Retrieved from

Army University. (2017). The Army University – Army learning strategy. Fort Leavenworth, KS: Author. Retrieved from

Connecting credentials framework. (2016). Retrieved from

Defense Language Institute Foreign Language Center. (2017). General catalog 2017-2018. [Catalog]. Monterey, CA: Author.

Defense Language Institute Foreign Language Center. (2019). General catalog 2019-2020. [Catalog]. Monterey, CA: Author.

Ewell, P. T. (2013, January). The Lumina degree qualifications profile (DQP): Implications for assessment (National Institute for Learning Outcomes Assessment [NILOA] Occasional Paper No. 16). University of Illinois at Urbana-Champaign, NILOA. Retrieved from

Ewell, P. T. (2016, 7 April). Improving with age. Inside Higher Ed. Retrieved from

Gaston, P. L. (2014). Higher education accreditation: How it’s changing, why it must. Sterling, VA: Stylus.

How does the EQF work? (2019). Retrieved from

Institute for Evidence-Based Change. (2012). Tuning American higher education: The process. Retrieved from

Jankowski, N., & Marshall, D. (2017). Degrees that matter: Moving higher education to a learning systems paradigm. Sterling, VA: Stylus.

Jankowski, N. A., & Provezis, S. J. (2011). Making student learning evidence transparent: The state of the art. University of Illinois at Urbana-Champaign, NILOA. Retrieved from

Jones, N., & Kleiner, H. (2015, 27 March). Professors should define student success. Retrieved from

Kem, J., & Hotaling, A. (2017). What is Army University supposed to do and how is it going so far? Journal of Military Learning, 1(1). Retrieved from

Lederman, D. (2014, 8 October). A conversation starter. Retrieved from

Lumina Foundation for Education. (2011). The degree qualifications profile. Indianapolis, IN: Author.

Markle, R. E., Brenneman, M., Jackson, T., Burrus, J., & Robbins, S. (2013). Synthesizing frameworks of higher education learning outcomes (Educational Testing Service [ETS] Research Report No. RR-13-22). Princeton, NJ: ETS. Retrieved from

Marshall, D., Jankowski, N., & Vaughn, T. (2017). Tuning impact study: Developing faculty consensus to strengthen student learning. University of Illinois at Urbana-Champaign, NILOA.

Marshall, D., Kalina, M., & Dane, W. (2010). Tuning educational structures: A guide to the process (Version 1.0). Retrieved from

Suskie, L. (2015). Five dimensions of quality: A common sense guide to accreditation and accountability. San Francisco, CA: Jossey-Bass.

Erin N. O’Reilly serves as the quality assurance officer and accreditation liaison officer for the Defense Language Institute Foreign Language Center, Monterey, California. O’Reilly holds a PhD in education from Northcentral University and an MA in English as a second language from Arizona State University. She is a graduate of the Defense Language Institute Foreign Language Center and served one tour overseas with the Multinational Forces and Observers and the U.S. Army. Her interests include faculty development, institutional assessment, and language program administration.

Back to Top

October 2019