Senior Service College Students' Sense of Belonging in a Problem-Based Learning Environment
Brandy Jenner
University of Baltimore
Megan J. Hennessey
Air University
Download the PDF 
Abstract
This study explores students' sense of belonging in problem-based learning (PBL) environments at the senior service college level in professional military education. Two seminars of students from the resident education program at the United States Army War College participated in a PBL intervention in the school's five-day introductory course. Seeking to explore the influence of problem-based learning on individual student experiences, the inclusion of a sense of belonging measure was one part of this intervention. Adapted from Walton and Cohen's (2007) measure of sense of social and academic fit, the Sense of Belonging measure recorded students' attitudinal reactions to the PBL intervention in the context of their feelings of inclusion and cognitive conformity within their respective seminar groups. Overall, the implementation of a problem-based learning intervention does not appear to have had an adverse effect on the treatment group's sense of belonging, comfort, or agency in the course when compared to the control group.
With its emphasis on student-generated research, the integration of theory and practice, and application of knowledge and skills to realistic problems, problem-based learning (PBL) is an ideal instructional strategy for postgraduate and executive education environments (Savery, 2006). Recently, the U.S. Army War College (USAWC), the senior service college of the U.S. Army and a professional military education institution regionally accredited to award graduate degrees, has explored PBL in curriculum design and facilitation as a way to develop and measure students' ability to translate their knowledge of strategy into the performance of strategic activities (Perez, 2018). To this end, two seminars of students from the resident education program at the USAWC were selected to participate in a PBL intervention in the school's Introduction to Strategic Studies course. Seeking to explore the influence of PBL on individual student experiences, the inclusion of a Sense of Belonging measure was one part of this intervention. Adapted from Walton and Cohen's (2007) measure of sense of social and academic fit, the Sense of Belonging measure recorded students' attitudinal reactions to the PBL intervention—one they had never experienced before—in the context of their feelings of inclusion and cognitive conformity within their respective seminar groups.
The Seminar Environment at the U.S. Army War College
The USAWC's 10-month resident education program features courses in the theory of war and strategy, strategic leadership, and military strategy and campaigning, among others, culminating in the completion of a master's degree in strategic studies. Most of the instruction in this program occurs in small seminars of 15–16 students each that run concurrently during the school day. Each seminar is taught by a faculty team charged to "establish a climate of innovation, tolerance, cooperation, and respect" (Hennessey, 2018, p. 25). An entire lesson in the Introduction to Strategic Studies course, the first required course of the academic year, is dedicated to establishing social and behavioral norms in seminar settings. The learning outcomes for that lesson are to "examine concepts associated with listening, discourse types, team learning, and reflection that influence interactions and enhance learning within the seminar" and "develop a set of seminar norms for the upcoming academic year" (Meinhart, 2018, p. 10). Such relationship building and social connectedness among students can predict favorable learning and workplace outcomes (Walton & Cohen, 2007). In fact, the value of seminar learning was recently emphasized again in the revised Officer Professional Military Education Policy (Chairman of the Joint Chiefs of Staff [CJCS], 2020).
Feelings of social exclusion can lead to deficits in cognitive processing and logical reasoning in a way that nonsocial obstacles cannot (Baumeister et al., 2002). Ostensibly, in PBL environments wherein logical reasoning is key to the attainment of learning outcomes, students' sense of belonging and feelings of social inclusion may be foundational to student success. As Mason (2009) explains, cohort formation becomes extremely influential for learning in the context of complex problem-solving, and students' success is tied to the diverse perspectives shared within that cohort. Likewise, peer support, a natural element of belonging within a community, contributes to individual students' stress reduction, and ultimately, their persistence in PBL (Bédard et al., 2012).
With the established understanding that sense of belonging could positively affect student experiences in PBL environments and that "specifics of the environment play a crucial role" (McGann & De Jaegher, 2007, p. 418) in self-perception of experience, this study sought to explore the inverse relationship: How does engagement in problem-based learning influence students' sense of belonging in a seminar environment? This was especially germane to the USAWC student experience in that the PBL intervention described earlier was the first curricular event of the academic year and could therefore set the tone for seminar cohort and individual student experiences in the remaining 10 months of the program of instruction. If found to have a negative effect on students' sense of belonging, the value and placement of these PBL exercises would require reconsideration in order to preserve the inclusive experience of seminar learning.
Method
To assess how engagement in PBL influenced students' sense of belonging in a seminar environment, a pre- and posttest model was applied before and after completion of the five-day Introduction to Strategic Studies course. The instrument for both the pre- and posttest was a Sense of Belonging measure adapted from Walton and Cohen's (2007) instrument and included 17 questions on a Likert scale that assessed students' self-perception of their inclusion in the seminar learning environment as well as their feelings of cognitive conformity with their classmates (see Appendix A). Two seminars (one treatment seminar and one control seminar) of 15-16 students each were purposefully selected so that each included two women students—who are underrepresented at the institution—and at least three international students (the maximum amount per seminar assignment policies at the time) who each had scored no lower than the intermediate skill range on the Test of English as a Foreign Language (TOEFL) exam before coming to the college. Four out of the 24 seminars met these inclusion criteria. The research team approached these seminars in numerical order (e.g., Seminar 1, Seminar 2) until two seminars' teaching teams consented to participate in the study.
Findings
The small sample size of 31 total students who took both pre- and posttests in the two seminars dictated descriptive statistics as the appropriate method to illuminate trends in the data. Means and standard deviations were computed and then compared using a two-tailed t-test to discern differences in pre- and posttest means within each of the treatment and control groups, as well as to check for differences between the pretest scores of both groups and the posttest scores of both groups.
Comparing the pretests of the treatment and control groups revealed little of significance, with one exception: treatment group students initially rated the phrase "I get along well with people in my seminar" significantly (p < 0.1) lower than the control group. Comparison of the posttests of the treatment and control groups yielded nothing of significance. Next, we compared pre- and posttests within both the treatment and control groups. In the control group, there was a mildly significant decrease (p < 0.1) between the pre- and posttests on the following statements: "People in my seminar accept me" and "If I wanted to, I could potentially do very well in my seminar." In the treatment group, between the pre- and posttests, we found a mildly significant increase (p < 0.1) in student self-reports on the statement, "I am similar to the kind of people who succeed in my seminar." This indicates that these senior service college students may do one of two things: (1) they initially overestimate their belonging within seminar and appropriately correct downward in the first week of class, or (2) they correctly estimate their initial sense of belonging and experience something causing their belonging to decrease. In either case, PBL seems to mitigate the decrease in some aspects of sense of belonging and slightly increase other aspects of belonging among students. For complete statistical results, see Appendix B.
Overall, the implementation of a PBL intervention does not appear to have had an adverse effect on the treatment group's sense of belonging, comfort, or agency in the course when compared to the control group. In fact, evidence points slightly to the contrary. Consistent with the PBL literature, we saw a small decrease in student level of comfort and a small increase in student sense of ambiguity reported by the treatment group (Jonassen, 2007, 2011). However, as evidenced by student performance on the summative assessment of the Introduction to Strategic Studies course and on final oral comprehensive exams, these effects did not compromise the attainment of course or lesson learning outcomes.
Concluding Discussion
The Sense of Belonging measure has not previously been used in combination with problem-based learning interventions and so represents a novel approach to assessing student integration and experience within PBL environments, specifically within a professional military education context. It is vital to examine student experiences within transformative educational interventions, in addition to academic outcomes, to ensure students receive the intended effects of the intervention with limited risk or disadvantages.
A limitation of this study included the small sample size, a byproduct of the pilot nature of the PBL intervention. Future iterations of the study could use the same Sense of Belonging instrument adapted from Walter and Cohen (2007) across multiple test and control seminars, bolstered by qualitative data from semistructured interviews or focus groups that explore student experience of inclusion even further.
Future studies might also draw on the work of Lohman and Finkelstein (2000) to further explore self-directedness in conjunction with sense of belonging in PBL environments and in various sizes of learning groups. While Lohman and Finkelstein found that medium-sized groups of around six students are the most effective for gains in overall learning transfer in PBL environments, more research is needed. Findings of such research could inform the sizes of future seminars in professional military education institutions and at senior service colleges. Finally, the connection between students' sense of belonging and instructor immediacy behaviors is a natural next step for investigation and could have actionable effects on faculty development in PBL environments (Arbaugh, 2001; Mehrabian, 1966; Richmond et al., 2003).
The renewed focus on problem-based learning as an aspect of outcomes-based military education seen in the Officer Professional Military Education Policy (CJCS, 2020) necessitates more clarity and empirical data regarding student experiences in PBL environments. Students' sense of belonging in seminar is one such data point, and one that informs the way educators and student peers in these environments can interact and encourage each other's mutual learning.
The opinions expressed here do not represent those of the U.S. Army War College, Air University, the Department of Defense, or any part of the U.S. government.
References
Arbaugh, J. B. (2001). How instructor immediacy behaviors affect student satisfaction and learning in web-based courses. Business Communication Quarterly, 64(4), 42–54. https://doi.org/10.1177%2F108056990106400405
Baumeister, R. F., Nuss, C. K., & Twenge, J. N. (2002). Effects of social exclusion on cognitive processes: Anticipated aloneness reduces intelligent thought. Journal of Personality and Social Psychology, 83(4), 817–827. https://doi.org/10.1037//0022-3514.83.4.817
Bédard, D., Lison, C., Dalle, D., Côté, D., & Boutin, N. (2012). Problem-based and project-based learning in engineering and medicine: Determinants of students' engagement and persistence. The Interdisciplinary Journal of Problem-based Learning, 6(2), 7–30. https://doi.org/10.7771/1541-5015.1355
Chairman of the Joint Chiefs of Staff. (2020). Officer professional military education policy (Chairman of the Joint Chiefs of Staff Instruction 1800.01F). Department of Defense. https://www.jcs.mil/Portals/36/Documents/Doctrine/education/cjcsi_1800_01f.pdf?ver=2020-05-15-102430-580
Hennessey, M. J. (2018). New faculty orientation handbook: Academic year 2019. U.S. Army War College.
Jonassen, D. H. (1997). Instructional design models for well-structured and ill-structured problem-solving learning outcomes. Educational Technology Research and Development, 45(91), 65–94. https://doi.org/10.1007/BF02299613
Jonassen, D. H. (2011). Learning to solve problems: A handbook for designing problem-solving learning environments. Routledge.
Lohman, M., & Finkelstein, M. (2000). Designing groups in problem-based learning to promote problem-solving skill and self-directedness. Instructional Science, 28(4), 291–307. https://doi.org/10.1023/A:1003927228005
Mason, M. (2009). Making educational development and change sustainable: Insights from complexity theory. International Journal of Educational Development, 29(2), 117–124. https://doi.org/10.1016/j.ijedudev.2008.09.005
McGann, M., & De Jaegher, H. (2007). Enactive theorists do it on purpose: Toward an enactive account of goals and goal-directedness. Phenomenology and the Cognitive Sciences, 6(4), 463–483. https://doi.org/10.1007/s11097-007-9074-y
Mehrabian, A. (1966). Immediacy: An indicator of attitude in linguistic communication. Journal of Personality,34(1), 213–217.https://doi.org/10.1111/j.1467-6494.1966.tb01696.x
Meinhart, R. (2018). Seminar learning. In M. J. Hennessey & B. Weigle (Eds.), Introduction to strategic studies course directive (pp. 10–11). U.S. Army War College.
Perez, C. (2018, September 7). What military education forgets: Strategy is performance. War on the Rocks. https://warontherocks.com/2018/09/what-military-education-forgets-strategy-is-performance/
Richmond, V. P., McCroskey, J. C., & Johnson, A. D. (2003). Development of the nonverbal immediacy scale (NIS): Measures of self- and other-perceived nonverbal immediacy. Communication Quarterly, 51(4), 502–515. https://doi.org/10.1080/01463370309370170
Savery, J. (2006). Overview of problem-based learning: Definitions and distinctions. The Interdisciplinary Journal of Problem-based Learning, 1(1), 8–20. https://doi.org/10.7771/1541-5015.1002
Walton, G. M., & Cohen, G. L. (2007). A question of belonging: Race, social fit, and achievement. Journal of Personality and Social Psychology, 92(1), 82–96. https://doi.org/10.1037/0022-3514.92.1.82
Brandy Jenner, PhD, is the associate director for teaching and learning excellence at the University of Baltimore. She holds a doctorate in education from University of California-Irvine and is a former postdoctoral fellow in educational methodology at the U.S. Army War College.
Megan J. Hennessey, PhD, is the director of the Teaching and Learning Center at Air University. She holds a doctorate in higher education from George Mason University and has held multiple faculty and instructional systems designer positions in professional military education and other government settings.
Back to Top