Journal of Military Learning

Using Q-Methodology to Understand Student Learning Preferences

Darrell W. Driver

U.S. Army War College

Download the PDF Download the PDF


The purpose of this article is to demonstrate the utility of Q-Methodology in understanding student learning preferences. Q-Method is a research approach that uses a statement sorting exercise to understand a respondent’s subjective and holistic view of a particular issue. In this case, students from the U.S. Army War College’s Academic Year 2021 were asked to rank order a set of 28 statements related to the recent debate on professional military education (PME) reform that culminated in the release of the 2020 Joint Chiefs of Staff Vision and Guidance for Professional Military Education and Talent Management. The application of principal component analysis to this data revealed the emergence of three different perspectives related to the central topics of curriculum design and instructional preferences. Labeled here as autonomous, classical, and adaptive learners, an explanation of each view is provided and implications for PME are discussed. The Q-Method instrument can be adapted to address other PME-related issues, including toward developing assessment-informed educational experiences.


The 2018 National Defense Strategy’s observation that professional military education (PME) in the U.S. “has stagnated” (Office of the Secretary of Defense [OSD], 2018) ignited widespread debate on the future of PME. Had PME become “focused more on the accomplishment of mandatory credit at the expense of lethality and ingenuity” (OSD, 2018)? If so, what was to be done about it? If not, why had the man who led the development of the National Defense Strategy, Secretary of Defense James Mattis, arrived at the conclusion? For several years, civilian and military scholars alike have traded articles purporting to diagnose the true nature of the problems that plagued PME in order to offer preferred solutions. Everything from instructional method to curriculum content and design have been examined. This debate culminated with the May 2020 publication of The Joint Chiefs of Staff Vision and Guidance for Professional Military Education and Talent Management (U.S. Joint Chiefs of Staff, 2020).

The reform debate presented the opportunity to demonstrate the applicability of Q-Method as an alternative to surveys in exploring student attitudes and preferences toward student learning experiences. Q-Method does this by using a statement-sorting exercise to understand a respondent’s holistic and subjective view of a particular issue, what Q-Method refers to as a communication concourse (Brown, 1980; Stephenson, 1953). Rather than a focus on independent responses to separate survey questions, Q-method studies are focused on understanding the holistic points of view present in a community on a given issue area. The unit of analysis becomes the individual’s view represented by the individual’s preference rankings. How many points of view are there? Which individuals subscribe to which points of view? And which statements were the basis of the consensus in each point of view?

Q-Method offers a unique approach for understanding student learning preferences and expectations on questions like curriculum design and instructional approach, which can help inform a more assessment-based and tailorable PME experience. To demonstrate the approach, a Q-Method study involving 53 students was conducted at the U.S. Army War College from February to March 2021. The statements students were asked to sort were drawn from the PME debates that unfolded in the two years between the publication of the 2018 National Defense Strategy and the 2020 Joint Staff Vision and Guidance.

The resulting data indicate that when given the opportunity to offer their views on the central issues of curriculum design and instructional approaches, students prefer a variety of solutions that have been offered in recent PME debates. Nevertheless, student views on these topics generally coalesced around three primary perspectives: (1) the autonomous learner, which seeks much more curriculum flexibility and self-guided learning that involves opportunities to learn by doing; (2) the classical learner, which values a guided learning experience with a prescribed core curriculum and a learning model similar to a typical civilian graduate program; and (3) the adaptive learner, which in many ways represents a middle ground between the previous two groups, expressing the desire for both structure and choice when it comes to curricula and self-authored learning experiences. It was the adaptive learner’s perspective that explained the largest amount of variance in the data and had the most students associated with it.

After a brief review of some of the recent PME reform arguments, a more detailed explanation of the Q-Method instrument is provided before turning to the findings and some of the potential limitations with a study like this. Whatever the reader’s view on the PME debates themselves, the author’s chief goal is to demonstrate the utility of Q-Method as a means of lending structure and rigor to the study of subjectivity. It is a method that might be employed to understand the range of perspectives on any number of topics in PME or its constituent disciplinary fields.

Debating PME Reform

Mattis’s call to reinvigorate PME sparked many responses about how to achieve that goal (OSD, 2018). This debate generally centered on two related questions regarding educational content and instructional methods. Is PME focusing on the wrong material, or is it delivering that material in ways that do not meet the needs of military professionals?

The curriculum content debate focused on two related concerns. The first had to do with the relative priority that should be placed on military-related topics as compared to broader theoretical or policy-related concerns found in a civilian security studies classroom. The second issue is the amount of freedom and flexibility students should be afforded to chart their own unique educational experiences. Thornhill (2018) has taken up the first issue by arguing that the education of professional military leaders had grown too similar to that which might be found in civilian security studies programs. Instead, Thornhill contends that these PME programs should focus on providing students with the practical skills they will need to become future senior commanders and higher headquarters staff officers. Mittelstadt (2018) and Morgan-Owen (2018) argue, conversely, that the complex security and decision-making environment of the future calls for precisely the kind of analytic tools and problem-solving approaches civilian graduate education is best prepared to deliver. Mittelstadt (2018) sums up this prescription with the call to put more college in the war colleges.

As to the second issue of student choice and curriculum flexibility, the question has centered on how much of a standard curriculum should be common to all and how much would be left to the students to chart a tailored program of study. Those arguing for a common core contend that effective literacy at the highest levels of foreign and security policy requires exposure to the widely shared language, concepts, ideas, and theories that define discussion and debate in this epistemic community (Biddle, 2020). This suggests the need to expose all of those making the transition to this strategic level to a common body of knowledge curated by experienced practitioners and academics. Conversely, others have argued that a more flexible curriculum would allow students the space for innovation and the ability to focus on more real-world problems. In this approach, Duncan and Yang (2018) contend the PME institutions would focus students more on the need for creative thinking about future problems rather than indoctrination into a security studies canon.

In addition to this debate over curriculum content, there is a debate over the instructional approaches best suited for PME education. This discussion is characterized by a continuum of views centered on the degree to which PME learning is best achieved as a student-led vice an instructor-led endeavor. The standard model of the PME is a seminar classroom in which instructor-facilitated discussion and dialogue are the primary means of learning (Leonard, 1991). In a varied version of this approach, Gudmundsson (2018) has argued that historical case studies provide more effective bases for classroom dialogues and discussions because the instructional strategy allows military professionals to see how key concepts were considered and employed by leaders dealing with actual challenges in complex environments.

In a somewhat more significant departure from the seminar dialogue model, Lacey (2016) has argued that wargaming, whereby students are put in complex problem-filled environments from which they must reason their way to defensible decisions, is an even more effective means of instruction for military professionals. Perez (2018) has taken a similar position on the issue of how to develop effective strategists and security policy leaders, arguing that future security leaders will not get better at developing effective policies and strategies unless they are able to practice it in their educational experiences. For Perez (2018) and others, this problem-based learning approach to “strategy as performance” holds the most hope because it turns adult learners into the authors of their own learning experience (Hennessey, 2019, 2020).

Such calls for increased use of problem-based learning approaches are also in line with the Joint Staff’s Vision and Guidance, which directs the incorporation of “active and experiential learning to develop the practical and critical-thinking skills our warfighters require” (U.S. Joint Chiefs of Staff, 2020, p. 6). This can be even more important for adult learners who, according to Knowles (1984, 1988), prefer more autonomy in the learning process to explore approaches commensurate with their learning preferences. Similarly, Kolb and Kolb (2011) argue that the education, career choice, and jobs one has held can heavily impact one’s approach to learning. This results from the pressure of learning under specific career-related environmental demands, a point stressed in the Joint Staff’s Vision and Guidance (U.S. Joint Chiefs of Staff, 2020).

Thus, the above literature suggests that understanding the learning preferences of the adult learners in question can be a useful first step in evaluating the various approaches championed in the PME curriculum and instructional approach debates. This is not to suggest that military professionals share a similar “learning style” that is stable over time. The literature on the validity of coherent learning styles is mixed, and the relationship between learning styles and educational outcomes is equally uncertain (Donggun & Carr, 2017; Hickox, 1995; Willingham et al., 2015). However, as the Army People Strategy makes clear (Grinston et al., 2019), it can be important to understand the talents, experiences, and preferences of adult military learners in order to design effective instructional approaches. This is where Q-Methodology can offer an alternative approach to traditional survey and interview instruments.



Q-Method was developed by the Oxford physicist William Stephenson (1953) as a method for the scientific study of subjectivity in the social and psychological sciences. It involves a Q-sample of statements taken from an issue area. These statements are collectively referred to as the communication concourse and represent the range of views that have been advanced on the topic. Respondents are asked to first sort these statements into three groups: agree, neutral, and disagree. After which, respondents further sort the statements from strongly agree to strongly disagree according to a normal distribution (see Figure). This sorting represents the respondents’ perspective on the variety of views measured, not independently like typical surveys, but against other views on the topic. This required prioritization forces respondents to make choices about what they value and offers a holistic account of the respondents’ views on the defined body of discussion and debate. Either factor analysis or principal component analysis is applied to the set of respondents’ sorts to determine where there are clusters of similarities among the individual perspectives. The result is a set of factors or composite sorts that are emergent on a particular issue, in this case student perspectives on making learning in PME more effective. Individual sorts or responses are evaluated for the degree to which they are similar to each of these composite perspectives to a given level of significance. In this way, Q-Method offers a quantitative method for developing a grounded understanding of the views held about an issue.

The Q-Sample and the Person Sample

To carry out a Q-Study of this sort, one needs to identify a Q-Concourse of statements and a sample of respondents best positioned to provide insights on the issue at hand. Statements gathered for this study (see Table 1) are drawn from the public debates on PME reform that have emerged since the 2018 call for PME reform from the secretary of defense. The debates over curriculum design and instructional approaches was the focus of the statements used in this study.

Example Q-Sort Configuration for 28 Statements

Students in the U.S. Army War College Academic Year (AY) 2021 Europe, South Asia, and Americas regional studies programs were invited to complete the exercise online using the “Q-Method Software” service. The 155 students in these three regional studies courses represent about 40% of the AY21 class, and of the 155 students who were invited to participate in the study, 53 completed the exercise (34% response rate or 14.5% of the resident class). Of these 53 respondents, 34 were U.S. military, 13 were international fellows, and six were U.S. civilians.


The Q-Sorts of all 53 respondents were correlated using principal components analysis, and the emerging factors were rotated using varimax criteria. Various numbers of factors were considered for analysis. The three-factor solution yielded the best result. Adding additional factors picked up fewer and fewer additional students with a significant loading, and the first three factors cumulatively explained 36.8% of the total variance in the data. Most importantly, however, reviewing the composite sorts of factors beyond factor three revealed few new learning preference insights, only finer variations on the first three ideal types. Each of these factors are summarized below, starting with the third factor and counting down to the first factor, which has the highest number of significant loadings and explains the most variance in the data.

Factor #3: The Autonomous Learner

Nine of the students in the sample loaded significantly on this factor at the .05 significance level, and this factor’s composite sort explained 7.3% of the total variance in the data. Starting with the autonomous learner provides useful context for understanding the other two learning perspectives, as these learners desire the most independence to chart their own educational paths and are most willing to value a student-directed approach to learning. See Tables 1 and 2 for how each statement was prioritized in this factor’s composite sort.

The autonomous learners favored statements that gave them more control over their course selection and learning experience. They were the most willing to agree that “senior service college students should be able to craft their own program of study” (statement 12). They were disapproving of statements like “the War College needs more college and less war” (statement 15) or the War College “should take further steps toward emulating higher education” (statement 17). They were, however, not ready to agree that this flexibility might be used to “collaborate on real world problems with organizations and researchers outside of the War College” (statement 11). This suggests a desire to maintain freedom to pursue their own interests rather than remaining subject to the research requirements of others.

Autonomous learners took a similar independent approach to their preferences for instructional approaches. This group was the most interested in applied, problem-based learning approaches in which students had more control over the learning process. Autonomous learners agreed more than the other groups with statements like “it would be better for students to take more ownership of learning by doing” (statement 10) or “to get better at developing effective security policies and strategies one actually needs to practice developing policy and strategy” (statement 27). This translated into the view that longer papers were not useful for strategic mindedness (statement 26), and exams do not “force one to absorb key program themes and ideas” (statement 20). Instead, this group agreed that it was “ultimately the student’s responsibility to understand and draw meaning from the concepts and theories in the course” (statement 2).

Figure 1. Instructional Approach Questions (continued)
Figure 1. Instructional Approach Questions (continued)
Figure 1. Instructional Approach Questions (continued)
Figure 1. Instructional Approach Questions (continued)

Despite these differences with the other perspectives, there was a significant amount of agreement between autonomous learners and the other two groups. Perhaps for different reasons than the others, this group placed a high degree of value on “hearing from a recognized expert on a topic” (statement 6) and engaging in seminar discussion and debate as a useful means of exploring key themes and testing assumptions (statements 7 and 23). Like the other perspectives, autonomous learners also agreed that “regional and local knowledge of political, cultural, and historical factors are critical to crafting good security policy” (statement 28).

Factor #2: The Classical Learner

Thirteen of the students in the sample loaded significantly on this factor at the .05 significance level, and this factor’s composite sort explained 8.3% of the total variance in the data. Though still sharing many important characteristics with the other groups, classical learners were the most distinct from autonomous learners on several key issues. More than the other two perspectives, the classical learners value a traditional graduate school model, centered on the instructor-led, or at least facilitated, instruction and a structured curriculum. The term classical learners was chosen for the greater emphasis this group places on the role of the instructor and for the desire expressed in this group to look toward civilian higher education as a model for senior PME. See Tables 1 and 2 for how each statement was prioritized in this factor’s composite sort.

The classical learners favored statements that highlighted the importance of the instructor in the learning experience. This included agreement that hearing from experts was a great way to compliment the reading and gain an appreciation for the strategic environment (statements 6 and 21), which were both sentiments also favored by the other two perspectives, and “it’s the instructor’s role to ensure everyone grasps the key concepts, theories, and themes, no matter what instructional method is employed” (statement 1), which was a statement not favored by the other two perspectives. This view extended to curriculum choice where classical learners agreed that “faculty should ensure every student is exposed to the broadly studied core security and defense concepts” (statement 13).

Classical learners not only emphasized the role of the instructor in learning, but they also looked toward civilian higher education as a model for senior PME. This was the issue that distinguished this group the most from the other two, as classical learners strongly agreed that “if the War College is to achieve its educational aims, it ought to take further steps toward emulating higher education institutions” (statement 17), a sentiment that was strongly unfavorable for the other two groups.

Nevertheless, there were limits to the classical learners’ willingness to conflate senior PME and civilian education, as they joined the other two groups in strongly rejecting the statement that the “War College needs more college and less war to prepare the students for the complexity of the strategic level” (statement 15). Classical learners also were not wholly ready to absolve themselves from a more attenuated version of the problem-based learning model, expressing a strong interest in the historical case study approach and agreeing that “working one’s way through historical cases and asking ‘what would I do’ is a more effective means of coming to terms with concepts and theories of war and strategy” (statement 8).

Factor #1: The Adaptive Learner

Sixteen of the students in the sample loaded significantly on this factor at the .05 significance level, and this factor’s composite sort explained 21.6% of the total variance in the data This made it the factor that explained the largest amount of variance and had the highest number of significant sorts. This perspective is presented last because it in many ways represents middle ground between some of the most distinguishing statements of the previous two groups, but also has some of its own unique characteristics. See Tables 1 and 2 for how each statement was prioritized in this factor’s composite sort.

Adaptive learners agreed with autonomous learners on the value of a variety of student-centered, problem-based learning approaches. This group agreed that “to truly understand a complex concept or theory, one has to apply it” (statement 24), and that “to get better at developing effective security policies one actually needs to practice developing policy and strategy; reading and talking about an issue only gets one so far” (statement 27). In one area, the adaptive learners expressed a desire for even more independence than the autonomous group, preferring the idea of developing an appreciation of the strategic environment through “personal research and reading on the complexity of issues involved” (statement 22), rather than the approach favored by autonomous learners to “take more ownership of learning by doing more research in teams to research and develop potential solutions to real world problems” (statement 10).

Adaptive learners also sided with autonomous learners on the need for curriculum flexibility and choice. However, adaptive learners were dismissive of the autonomous learners’ view that this flexibility was because “who else knows more about one’s own educational needs than the student” (statement 12). Instead, adaptive learners agreed that one benefit of a flexible curriculum was the ability “to innovate and collaborate on real world problems with organizations and researchers outside of the institution” (statement 11).

Despite these points of difference, this group was similar to the other two in the value it placed on seminar interaction and hearing from experts on complex topics. It also joins the other two in dismissing the call for more college and less war in the War College (statement 15), breaking also with classical learners on the idea that the War College ought to seek to emulate civilian high education institutions (statement 17).

Finally, there were also some unique views in the adaptive learners’ perspective. These learners embraced the value of test preparation over the shorter policy papers preferred by the other two groups (statement 20). Adaptive learners embraced wargaming over the historical case study approach favored by classical learners, which can be compared to the autonomous learners’ inclination to reject both of these learning approaches.


With this data organized into groups, what is one to make of these PME students’ viewpoints on these debates? First, there is clear support from two of the perspectives for moving toward more curriculum flexibility and more problem-based instructional approaches. However, the classical learners model indicates that there is still a constituency for more curriculum certainty and planned programmatic structure. There are similar mixed results when it comes to instructional approaches. None of the groups were ready to dismiss the value of the traditional expert lecture or dialogue and debate-based seminar learning, even as adaptive and autonomous learners were especially keen to move toward more problem-based learning approaches (see Table 2). In short, the move toward greater curriculum flexibility and more problem-based instruction should continue to involve a variety of instructional approaches and preserve avenues for more structure for the students preferring that approach.


This insight can also be found in the unfolding PME debate, which suggests that problem-based learning approaches without an appropriate structure and foundation could confuse more than they clarify. In his description of a problem-based approach to learning, which Perez (2018) refers to as “strategy as performance,” Perez argues that educators must first “impart to their students the skill of researching and ‘seeing’ the strategic environment” before students can then cultivate and hone those skills in a performative way. For Perez, this includes exposure to habits of research in fields like history and the social sciences, but it also includes the exposition of visualization exercises like the practice of graphically depicting complex causal relationships. This all requires early, intensive instructor involvement.

When it comes to the debate over curriculum content, this also is not a zero-sum game. First, if curricula are going to be flexible and tailorable, one might recognize that students will prefer varying levels of structure. Options that include more pre-charted paths may prove especially interesting for learners who relate more to the classical model. Similarly, even for those seeking a more customized approach, established track options may prove useful. With such changes, the role of the faculty advisor in student decision-making will become more, not less, important. More curriculum flexibility, like more student-centered approaches to learning, may not necessarily signal less structure. This flexibility does require a different kind of structure to support student preferences.


Before closing, it is also useful to point out a few limitations to a Q-Method investigation of this sort. First, this study focused on Army War College students at the senior PME level. It is not clear if mid-career PME students would respond similarly. For instance, the preferences for curriculum flexibility and problem-based approaches to learning in two of the factors may have been less pronounced in more junior PME institutions where students have less military and academic experience. A similar Q-study in such institutions would be needed to compare the findings. The same is true for the six civilians and 13 international fellows in the sample. One of the U.S. civilians loaded significantly on the classical learners’ factor, the remainder had no significant loadings. The international fellows, by contrast, resembled their U.S. counterparts in the breakdown, with four fellows loading significantly on the adaptive learner factor, three for on the classical learner factor, and one for autonomous learner. Q-Method PME studies in their home institutions and countries could offer some useful comparisons.

Second, the goal of Q-Methodology is not to use a smaller sample to estimate the proportion of individuals in a larger population who hold one view or the other, as is the case with traditional survey instruments. Instead, with a reasonably representative small sample, Q-Method purports to reveal the range of holistic perspectives that exist on an issue in the broader community. This difference is important. This means that, while there is no reason to believe that the proportion of those loading significantly on these three factors would change if the remainder of the students in the AY21 class completed the exercise, generalization to that conclusion is not the goal of the method. Third, from a set of Q-Method responses alone, it is difficult to deduce why respondents made the ordering decisions they did. An important limitation of this particular study is that it did not include follow-on interviews with the respondents. Such interviews can be valuable in determining the rational or motivations behind the expressed preferences and help the researcher understand how the respondent is interpreting each of the statements. Where time and resources permit it, such interviews are preferred.


In summary, this study indicates that there are important learning preference similarities in this group of Army War College students, but there are also important differences in preferences and expectations that might be considered in curriculum design and instructional decision-making. A Q-Method study of the sort presented here may prove useful for other institutions confronting such questions. Though it is the Department of Defense, rather than the students themselves that are the primary stakeholders in the department’s PME institutions, understanding student attitudes and preferences toward learning remain important for achieving the desired learning outcomes. At a minimum, instructors may want to address student preferences and expectations that contrast with the planned approach. As PME educator Hamilton (2019) has observed, “Military learners (like other adult students) carry their ideas, concerns, and experiences to class with them. Faculty can ignore this dynamic but often at the cost of ‘losing’ students and leaving learning outcomes unfulfilled” (p. 3).


Biddle, T. (2020). Coercion theory: A basic introduction for practitioners. Texas National Security Review, 3(2), 94–109.

Brown, S. R. (1980). Political subjectivity: Applications of Q methodology in political science. Yale University Press.

Donggun, A., & Carr, M. (2017). Learning styles theory fails to explain learning and achievement: Recommendations for alternative approaches. Personality and Individual Differences, 116, 410–416.

Duncan, A., & Yang, A. (2018, July 19). Exploiting the wellspring: Professional military education and grassroots innovation. War on the Rocks.

Grinston, M., McConville, J., & McCarthy, R. (2019). The Army people strategy. U.S. Department of the Army.

Gudmundsson, B. (2018, October 2). A fourth way in professional military education. War on the Rocks.

Hamilton, M. (2019). Prioritizing active learning in the classroom reflections for professional military education. Journal of Military Learning, 3(2), 3–17.

Hennessey, M. (2019, March 26). Reframing rigor for senior service colleges. War on the Rocks.

Hennessey, M. (2020, August 3). How can we know if professional military education works? War on the Rocks.

Hickox, L. (1995). Learning styles: A survey of adult learning style inventories. In R. R. Sims & S. J. Sims (Eds.), The importance of learning styles: Understanding the implications for learning, course design, and education (pp. 25–47). Greenwood Press.

Knowles, M. (1984). Andragogy in action: Applying modern principles of adult learning. Jossey-Bass.

Knowles, M. (1988). The modern practice of adult education: From pedagogy to andragogy. Cambridge.

Kolb, A., & Kolb, D. (2011). The Kolb learning style inventory 4.0: A comprehensive guide to the theory, psychometrics, research on validity, and educational applications. Institute for Experiential Learning.

Lacey, J. (2016, April 19). Wargaming in the classroom: An odyssey. War on the Rocks.

Leonard, H. (1991). With open ears: Listening and the art of discussion learning. In C. R. Christensen, D. A. Garvin, & A. Sweet (Eds.), Education for judgment: The artistry of discussion leadership (pp. 137–152). Harvard Business School Press.

Mittelstadt, J. (2018, June 20). Too much war: Not enough college. War Room-U.S. Army War College.

Morgan-Owen, D. (2018, July 25). Approaching a fork in the road: Professional education and military learning. War on the Rocks.

Office of the Secretary of Defense. (2018). Summary of the 2018 national defense strategy of the United States of America: Sharpening the American military’s competitive edge. U.S. Government Publishing Office.

Perez, C. (2018, September 7). What military education forgets: Strategy is performance. War on the Rocks.

Stephenson, W. (1953). The study of behavior: Q-technique and its methodology. University of Chicago Press.

Thornhill, P. (2018). To produce strategists, focus on staffing senior leaders. War on the Rocks.

U.S. Joint Chiefs of Staff. (2020). Developing today’s joint officers for tomorrow’s ways of war: The joint chiefs of staff vision and guidance for professional military education and talent management.

Willingham, D. T., Hughes, E. M., & Dobolyi, D. G. (2015). The scientific status of learning styles theories. Teaching of Psychology, 42(3), 266–271.

Col. Darrell Driver, U.S. Army, is an associate professor and director of the Regional Studies Program at the U.S. Army War College. He teaches courses in national security policy and strategy, and European studies. His research focus includes transatlantic security topics and civil-military relations. His 2008 book Sparta in Babylon: Case Studies in the Public Philosophy of Soldiers and Civilians used a Q-Method approach to understand the public philosophies of U.S. Army officers. He holds a bachelor’s degree from the University of Notre Dame and master’s and doctorate degrees from Syracuse University.

Back to Top

October 2021