The Psychological Intangibles of Soldier Readiness
Part III: Intangible Measures Development and Field Test
By Dr. Christopher Vowels
U.S. Army Research Institute for the Behavioral and Social Sciences
&
Dr. Steven Aude
Contributor
December 13, 2019
* All 3 articles are located in the PDF below
Download the PDF
The goal of this phase of research was to develop valid measures of intangible psychological concepts and to test those measures in a live training environment. For the purposes of the research, the authors used the term intangible to describe psychological constructs that contribute to Soldier mission readiness. Soldier mission readiness describes Soldiers' preparedness for a wide range of missions (e.g., disaster relief, short-term contingency operations, long-term deployments, counterinsurgency operations, full spectrum operations, etc.). There were three phases of this research; the field test is the third. Below, we provide brief summaries of the first two phases and introduce Phase III.
Phase I
The purpose of Phase I of this research was the identification of psychological constructs critical for Soldier mission readiness. To support the research objectives, a comprehensive literature review including academic and military sources was conducted and a combined total of approximately 100 Soldiers and leaders were either inter- viewed or participated in focus groups. Several constructs were identified, each of which had multiple sub-constructs embedded within them. Data collection with Soldiers and leaders assisted in the development of a concise list of four key intangibles deemed most critical to mission readiness, namely: hardiness, grit, will, and initiative.
Phase II
Phase II of the research consisted primarily of data collection focused on the measurement development for hardiness, grit, will and initiative, as well as the identification of what types of training/learning environments are most conducive to observe Soldiers displaying these four intangible constructs. During data collection, interviews were conducted with a combined total of approximately 50 Soldiers and leaders. Results indicated that a number of training environments are potentially effective environments for the type of experiential and realistic conditions needed to train intangibles (e.g. Combat Training Centers). Interviewees suggested that any given Soldier's immediate superior would provide the most accurate assessment of intangible psychological constructs displayed by them. To provide the basis for such assessments, performance indicators for each of the intangible constructs were identified. The end result of this phase of the research provided the necessary data to move towards the development of actual measures that could be used in a live training environment.
Phase III: Current Research
Phase III of this research involved two primary tasks, each consisting of several subtasks. In this phase, emphasis was on developing and establishing the validity of selected intangible measures. Likewise, psychometric analyses of the measures were conducted to determine if construct validity and reliability statistics were acceptable and to refine the measures accordingly. In the final task, the measures were evaluated in a field training environment. Soldiers used the measures in live training and reported on their general acceptance. This approach allowed for an evaluation of the overall effectiveness and efficiency of the measures in a field training environment.
In sum, the present research builds on the previous phases of intangible construct content and measurement literature reviews, together with Soldier data collections, to establish the validity of Soldier measures of intangibles critical to mission readiness. Furthermore, the field test demonstrates Soldiers can effectively and efficiently utilize the instruments in a realistic training environment. Note that a more extensive technical report of this study was previously published and available online (Aude, Nicely, Lodato, & Vowels, 2015).
Establish the validity of perseverance and initiative measures
After multiple iterations of discussion by the research team and confirming what the empirical data was indicating, two measures were developed for two intangible constructs: initiative and perseverance (which consisted of a combination of hardiness, grit, and will). Given that the final intent of the measures were that they could be used by NCOs in a field environment, we chose a format that would be field-expedient in terms of scoring, easily understood by the end-user, and still capture primary elements of each intangible, thereby creating a tool for use by both Soldiers and researchers.
In order to check the validity, or provide statistical evidence that the measures did indeed measure what they are proposed to do, we asked approximately 150 Soldiers (largely sergeants and staff sergeants) to provide ratings while thinking about a specific Soldier performing a specific task. We also asked questions with regard to whether the measures of initiative and perseverance were easy to understand and if any items were unclear.
Through statistical analyses, the evidence provided by Soldiers indicated that each measure was capturing each intangible effectively and that the items for each measure were statistically reliable. Statistical reliability indicates that scores from a test or measure are accurate and would be consistently reproduced across different administrations of the measure. Given these findings, we were able to move to the final step, using the measures in a live training environment. This allowed us to get candid feedback from the end-users that the measures appear to measure what they are supposed to.
Field Test of Measures
The culminating event of this research was to develop measures that could be used by Soldiers in a training environment with little train-up and that provided a means of capturing critical intangible data not readily available in existing Army measures. In order to field test the measures, we partnered with the Medical Simulation Training Center (MSTC). Specifically, it was important to ensure that both measures were viewed as important, of an appropriate length, and included clearly written instructions and items. Experienced Soldier participants (acting as squad leaders) at MSTC were chosen for the field test because its training puts Soldiers under conditions in which they are expected to exhibit initiative and perseverance. The MSTC Military Operations in Urban Terrain (MOUT) site provides a mentally and physically challenging environment that Soldiers must maneuver through as a squad. Furthermore, the squad leaders, by way of their Army and supervisory experience, met the rating criteria previously identified to accurately rate intangible constructs.
Participants
A total of 10 experienced squad leaders participated in the field test. Of these 10 Soldiers, nine indicated that their rank was “SGT” and one indicated “Other.” Nine of the participants indicated that their position was “Squad/Section/Team Leader” while one indicated “Other NCO Position.” The average time of service for the 10 participants was 63.1 months (just over five years). And the average number of deployments was 1.7.
Measures
The two 18-item measures developed for the field test were nearly identical to those created and tested during the construct validation (described above). The format and instructions slightly differed and the evaluation items that followed the measures were expanded and revised (six items plus an open-ended question for additional comments). These adjustments were made to account for the change in use of the measures from simply thinking about a Soldier and rating them (construction validation), to the actual observation and subsequent rating of that same Soldier (field test/face validation). For each of the six evaluation items, participants responded according to a 7-point Likert scale (1=Strongly Disagree, 7 = Strongly Agree). Full versions of both measures used during the field test are available in Appendix B & C of the ARI technical report located at:
http://www.dtic.mil/dtic/tr/fulltext/u2/a616373.pdf
Procedure
The procedure for the field test involved two main steps. In the first step, squad leaders were oriented to the two measures, the benefits of this research were explained to them, and informed consent was obtained. Squad leaders were told to pay particular attention to Soldiers in their squad who exhibited signs of initiative and perseverance (or a lack thereof). They were then shown the two measures they would be expected to rate a Soldier on and asked to quickly scan and indicate their understanding of the items. The squad leaders were then reminded that they would rate one Soldier on each construct measure at the conclusion of the MSTC exercise.
In the second step, the squad leaders met back with the research team after the MSTC After Action Review. During this meeting, they were asked to recall one or more Soldiers who displayed initiative and perseverance during the MSTC exercises. Each participant was then provided with instructions for the two measures, and asked to rate one Soldier using the initiative measure and either the same or a different Soldier using the perseverance measure. Soldiers were also instructed to complete the brief evaluation (six items and one open-ended question) for each measure. After completing both measures and the evaluations, squad leaders were thanked for their participation and contribution to Army understanding of Soldier perseverance and initiative.
Results
The focus of field data analysis was on the initiative and perseverance measure evaluation items and open-ended question that participants completed after having rated Soldiers on these intangibles. These items were designed to confirm the display of the intangible during the training exercise and obtain user feedback on the effective use of the measures in a field environment. The means and standard deviations for these six items, for initiative and perseverance, are provided in Table 9.
The results presented in Table 9 provide support for both the appropriateness of the training venue selected as well as the effective field use of the two measures. The MSTC training venue clearly provided the opportunity for Soldiers to display initiative and perseverance. Furthermore, while previous research had indicated that the content comprising these two constructs was important, it was good confirmation to hear that both of the named constructs, initiative and perseverance, were viewed by these 10 NCOs as important aspects of training (Mean = 6.60 out of a 7.0 scale). The squad leaders tended to disagree with the statement “there were too many questions in this measure” confirming that the 18-item length was about right. Ease of understanding of measure instructions and the items themselves was also a positive finding.
Additional comments on the measures included one participant suggesting that the measures ought to be used by leaders with their own platoons and squads. The MSTC participants come from a variety of Fort Hood units and squad leaders are assigned their role for the purposes of completing the MOUT lane. This participant went on to say that a leader who knows their Soldiers very well could use the measures more effectively as a tool. This comment lends support for one of the primary users of the measures to be by the immediate supervisors of Soldiers.
Discussion
This study provided support for the construct validity and reliability of the initiative and perseverance measures. In conducting validation steps, statistical support was found for retaining 18-item measures for both initiative and perseverance. Results from a statistical approach indicate that each measure was measuring just one intangible, as intended. While initiative appears to be measuring an overall action orientation, perseverance is measuring a motivation to achieve in spite of obstacles and setbacks. This suggests that it was effective to assess each of the constructs with a separate 18-item measure and that no subscales were necessary. Additionally, the use of bipolar response scales was effective at avoiding common measurement problems such as response range restriction, skewness, and a lack of variability in responses. Bipolar scales also contributed to ease of use by raters.
This study also found support for the field utility of the instruments for measuring intangibles during realistic training. Findings from the field test indicated that squad leaders who completed the measures felt that they assessed an important aspect of training, that the measures did not include too many items, and that instructions and items themselves were clear. These findings support the general face validity of the two measures and that they represent constructs important to Soldier mission readiness.
Future Research
The current research provides valid and reliable measures of Soldier initiative and perseverance. Yet beyond providing Soldiers, or their leaders, with a scaled rating of each construct, there are no suggestions or recommendations as to how a Soldier who is rated low, for example, might improve. Earlier phases of this research identified the type of training within which the intangible constructs are best observed and experienced. Yet the mere experience of situations requiring the display of initiative and/or perseverance are not known to improve a Soldier's long-term display of either intangible.
Specific types of training or targeted tasks (e.g. confidence-building exercises) might prove helpful toward the development of initiative and perseverance. While it is beyond the scope of this research to identify means of intangible development, future research might focus on some of the following ideas.
It may be the case that the mere administration and use of appropriate feedback methods with Soldiers would make them more aware of the behavioral indicators of initiative and perseverance. Consequently, feedback on their ratings may lead to greater exhibition and demonstration of behaviors associated with initiative and perseverance. So, too, a leader's use of reinforcement (e.g. praise, encouragement) associated with the behaviors hold promise for increasing the frequency of their display by Soldiers.
Further questions remain that could bolster home station training effectiveness and advance the science including: As performance fluctuates, does the display of intangibles track with that performance, and, if so, can we predict one from the other?
Perhaps with further research, we may be able to address questions regarding whether measuring intangibles in a live training environment is not only possible, but fosters recent calls to, for instance, encourage better understanding of self-awareness and self-discovery in Soldiers and leaders (Department of the Army, 2015a).
Then again, it may be that initiative and perseverance are more trait-like and not easily subject to development and change. In other words, initiative and perseverance may be human characteristics that are developed and fixed early in life. Consequently, it would be difficult for a Soldier, or those who supervise them, to change the level of initiative or perseverance an individual exhibits during a single unit assignment. Thus, research that determines the extent to which the intangibles are fixed, learned over time, or subject to immediate change and development, would be helpful toward shaping unit level training and expectations. Specifically, it would help Soldiers and their leaders to know to what degree they can and should expect these intangibles to change over time. Additionally, such research might also lead to the use of perseverance and initiative measures as key measures for job selection.
Lastly, it would be helpful to engage Army leaders in a discussion of the potential applications of this research. For example, an important next step might be to conduct a working group with senior unit leaders, training officers and NCOs, to identify where and how the measures should be used in relation to the units' overall training strategy. Such an exercise might also guide and focus the future research agenda herein.
To read the first article in this series, click here: Part I: The Concept
To read the second article in this series, click here: Part II: Concept Criticality and Training
References
Aude, S. N., Nicely, K., Lodato, M. & Vowels, C. L. (2015). Preparing Brigade Combat Team (BCT) Soldiers for mission readiness through research on intangible psychological constructs and their applications: Validation and Pilot. (Technical Report 1348). Retrieved
from http://www.dtic.mil/dtic/tr/fulltext/u2/a616373.pdf
Dr. Steven Aude is an experienced behavioral scientist with over twenty years of experience conducting applied research to enhance human performance. He is currently leading a comprehensive study to conceptualize Army unit culture. He was previously an Army officer, served in the 101st Airborne Division (Air Assault) as a battalion and brigade executive officer. He was a Special Forces qualified officer and led a Special Forces A Team in 1/10th Special Forces, Germany, as well as infantry platoons at Fort Lewis, WA. Aude also taught leadership and human resources at the United States Military Academy.
Dr. Christopher Vowels is an applied experimental psychologist with over 10 years of experience working directly with operational units. He is currently a team leader with the ARI Fort Hood Research Unit located at Fort Hood, Texas. Most recently, he has been conducting research to improve unit performance measurement, particularly for live training environments. Vowels received a Ph.D. in Psychology with an emphasis on Cognition and Judgment and Decision-Making from Kansas State University. While attending graduate school, he served as a Consortium Research Fellow with ARI at Fort Leavenworth conducting research on leader development and improving course instruction.
Back to Top