Naval Aviator Human Factors Questionnaire

Abstract

The Naval Aviator Human Factors Questionnaire (NAHF) is a specialized psychometric assessment instrument designed to measure safety attitudes, human factor competencies, and Crew Resource Management (CRM) principles among military flight crews. Derived from the seminal Cockpit Management Attitude Questionnaire (CMAQ) originally engineered for commercial civilian aviation by Gregorich, Helmreich, and Wilhelm (1990), the NAHF was systematically adapted by human factors researchers at the Naval Postgraduate School to reflect the unique operational rigors, steep command hierarchies, and lethal high-stakes flight environments encountered in naval aviation. The initial adaptation generated a 31-item pool distributed across five foundational domains: My Stress (personal vulnerability and self-awareness under physiological and psychological duress), Stress of Others (recognition and active mitigation of team members’ stressors), Communication (assertiveness, operational debriefing, intent verbalization, and operational coordination), Command Responsibility (distribution of authority gradient, decision-making prerogatives, and junior crew feedback), and Rules and Order (procedural compliance and flight discipline).

Subsequent psychometric evaluation—utilizing classical test theory, exploratory factor analysis (EFA), and confirmatory factor analysis (CFA)—revealed substantial operational skewness, kurtosis, and structural attenuation, prompting empirical trimming. Due to low internal consistency, the Rules and Order dimension was entirely discarded, along with nine items exhibiting severe ceiling effects and three items displaying inadequate factor loadings during structural modeling. The finalized operational variant retains an abbreviated, multidimensional profile measured on a 5-point Likert scale alongside frequency-based situational inquiry. While internal consistency coefficients (Cronbach’s alpha) reflect modest values consistent with brief attitudinal survey instruments, the NAHF exhibits robust content validity, notable discriminant validity across naval aviator experience strata, and predictive efficacy in evaluating institutional CRM instructional interventions. This article delivers an exhaustive academic analysis of the NAHF, elucidating its theoretical foundations, structural modeling properties, administrative procedures, and pragmatic deployment within high-reliability socio-technical systems.

Keywords

Naval Aviator Human Factors Questionnaire, Crew Resource Management, Cockpit Management Attitude Questionnaire, Military Aviation Psychology, Situational Awareness, Command Responsibility, Attitudinal Metrics, Threat and Error Management

Authors

The Naval Aviator Human Factors Questionnaire was adapted, evaluated, and introduced to human factors literature by a specialized research team affiliated with the Naval Postgraduate School (NPS) in Monterey, California, United States:

• Paul E. O’Connor, Ph.D.: Associate Professor and operational human factors researcher specializing in high-risk socio-technical environments. Prior to his extensive tenure with the U.S. Navy and the Naval Postgraduate School, Dr. O’Connor earned international recognition for his empirical investigations into flight-deck team coordination, naval safety cultures, and cross-domain human performance in aviation, healthcare, and offshore drilling operations.
• Angela D. Jones (D. Jones), M.S.: Research Associate in the Operations Research and Human Systems Integration departments at the Naval Postgraduate School, focusing on aviation safety metrics, military training evaluation, and personnel psychometrics.
• Michael E. McCauley, Ph.D.: Research Professor of Operations Research at the Naval Postgraduate School, widely cited for his foundational research in simulator sickness, human-machine interface design, human factors engineering, and spatial disorientation among military pilots.
• Samuel E. Buttrey, Ph.D.: Associate Professor of Applied Statistics and Operations Research at the Naval Postgraduate School, providing statistical methodology, multivariate modeling, non-parametric analysis, and computational validation across defense-related human behavioral metrics.

Institutional contact for baseline research and archival inquiries is maintained through the Naval Postgraduate School, Operations Research Department, 1411 Cunningham Road, Monterey, CA 93943, USA.

Purpose

The primary purpose of the Naval Aviator Human Factors Questionnaire (NAHF) is to quantitatively evaluate naval aviators’ personal attitudes toward non-technical skills, interpersonal cockpit dynamics, stress vulnerabilities, and hierarchical communications. In naval operational aviation—encompassing tactical strike aircraft, anti-submarine warfare platforms, tilt-rotor transports, and rotary-wing carrier operations—mishap investigations historically indicated that up to 80% of catastrophic class-A mishaps were directly traceable to human error rather than mechanical or structural failure. Despite profound technological advancements in avionics, fly-by-wire controls, and head-up tactical displays, aviators frequently experienced breakdown in communication, loss of situational awareness, fixation, and unassertive submission to authoritarian flight leadership.

To counteract these recurrent vulnerabilities, the United States Navy institutionalized Crew Resource Management (CRM) programs. However, assessing the actual efficacy of human factors training curricula requires validated psychometric metrics capable of capturing subtle shifts in aviator mindset. The NAHF serves three discrete, critical purposes across military, operational, and psychometric domains:

• Training Program Evaluation: The instrument functions as an empirical pre-test and post-test measurement battery to determine whether naval aviators who undergo mandatory CRM recurrent training experience measurable, statistically significant enhancements in their attitudinal alignment with safe team practices.
• Institutional Safety Climate Profiling: Administered cross-sectionally across different naval air wings, squadrons, and platform communities (e.g., strike-fighter [VFA], maritime patrol [VP], and helicopter sea combat [HSC]), the tool identifies systemic cultural discrepancies, dangerous overconfidence patterns, or authoritarian gradients that could elevate operational risk during deployed operations.
• Diagnostic Human Factors Research: The scale enables operational psychologists and aerospace medicine specialists to investigate the precise cognitive and social determinants that mediate an aviator’s willingness to speak up, report fatigue, query superior officers during flight anomalies, and mitigate toxic cockpit atmospheres.

From a theoretical standpoint, assessing attitudes is grounded in behavioral psychology, which asserts that attitudes serve as latent behavioral predispositions. While safe attitudes do not unconditionally guarantee error-free tactical execution under extreme combat stress, hostile or counter-normative safety attitudes significantly elevate the probability of procedural non-compliance, authoritarian suppression of vital safety inputs, and catastrophic team coordination collapse.

Psychological Construct

The NAHF operationalizes the multidimensional construct of human factors safety attitudes within high-reliability aviation settings. Originally conceived across five distinct structural domains, empirical psychometric evaluation refined the instrument to capture four core operational constructs, while revealing critical boundaries regarding rule compliance metrics:

1. My Stress (Personal Stress Vulnerability)

This subscale captures an aviator’s metacognitive self-awareness regarding the debilitating consequences of physiological and psychological stress, physical fatigue, and cognitive overload on individual performance. Historically, military aviation subcultures celebrated an archetypal "invulnerability myth," wherein pilots perceived their cognitive functioning, decision-making velocity, and motor coordination as inherently resilient to sleep deprivation, hypoxia, emotional distress, or extreme task saturation. The My Stress construct measures an individual’s realistic recognition that elevated stress degrades situational awareness, precipitates judgment errors, and mandates compensatory behavioral adaptations. Prototypical items assess an aviator’s willingness to acknowledge task saturation to crew members and recognize that emergencies diminish, rather than preserve, optimal executive processing.

2. Stress of Others (Team Stress Sensitivity)

Complementing self-oriented stress awareness, the Stress of Others dimension measures an aviator’s proactive vigilance toward manifestations of cognitive exhaustion, emotional distress, or task overload in fellow flight crew. In multi-place aircraft (e.g., P-8A Poseidon, E-2D Hawkeye, MH-60R Seahawk), system safety relies upon interdependent cross-monitoring. Aviators scoring high on this construct acknowledge that team coordination requires continuous scanning of peer performance, active observation of micro-behaviors signaling cognitive compromise, and a moral obligation to intervene or adapt task allocations before a teammate’s overload precipitates an unrecoverable flight envelope excursion.

3. Communication and Coordination

This construct addresses the transactional communication behaviors, briefings, debriefings, and mutual support mechanics essential for establishing and maintaining shared mental models. Drawing from the Situation Awareness framework posited by Endsley, effective operational coordination necessitates explicit operational briefing, transparent verbalization of intended flight profiles, assertive operational challenges, and routine post-mission debriefings. The NAHF operationalizes communication not merely as passive message transmission, but as an active, mandatory process that legitimizes questioning, demands explicit verbal acknowledgment of plans, and treats communicative clarity as equal in operational weight to stick-and-rudder technical mastery.

4. Command Responsibility and Trans-Hierarchical Interaction

Perhaps the most culturally sensitive construct within military aviation, Command Responsibility interrogates the traditional, highly regimented military authority gradient. While the Pilot-in-Command (PIC) or Mission Commander holds statutory and moral accountability for flight safety and mission accomplishment, dogmatic autocratic leadership suppresses vital corrective inputs from junior aircrew, copilots, or tactical coordinators. This dimension measures attitudes toward cockpit democracy, the duty of junior personnel to challenge questionable command decisions, and the willingness of commanders to invite collaborative input during operational emergencies. It probes the pernicious behavioral phenomenon wherein junior personnel withhold critical flight path corrections due to fear of reprisal, deference to rank, or perceived loss of face.

5. Rules and Order (Procedural Strictness)

Initially conceptualized to quantify adherence to standard operating procedures (SOPs), Naval Air Training and Operating Procedures Standardization (NATOPS), and hierarchical order, this dimension was designed to measure resistance to unauthorized procedural drift. However, in psychometric testing among active naval aviators, the subscale demonstrated near-zero variance, severe ceiling effects, and unacceptable internal consistency, compelling researchers to excise it from the core measurement model.

Theoretical Framework

The theoretical architecture of the Naval Aviator Human Factors Questionnaire is deeply rooted in macro-ergonomics, cognitive systems engineering, and the organizational safety paradigms formulated throughout the late 20th century. The instrument synthesizes three overarching theoretical frameworks:

1. The Threat and Error Management (TEM) Model

Formulated by Robert Helmreich and colleagues at the University of Texas Human Factors Research Project, the Threat and Error Management framework asserts that operational threats (environmental, mechanical, or operational conditions outside the direct control of the crew) and human errors (actions or inactions by the crew that lead to deviations from organizational intentions) are ubiquitous and unavoidable features of complex flight operations. Safety is not defined by the absolute absence of error, but rather by the crew’s dynamic capacity to trap, mitigate, and absorb errors before they manifest as adverse outcomes. The NAHF directly reflects TEM assumptions: scales measuring stress awareness and communication reflect an individual’s cognitive apparatus for identifying latent threats and trapping operational deviations through coordinated team vigilance.

2. Reason’s Socio-Technical System Dynamics and the Swiss Cheese Model

James Reason’s foundational work on organizational accidents conceptualizes complex operations as multi-layered defense systems. Latent failures within organizational design, management culture, and maintenance procedures align with active human failures to breach institutional safeguards. Within this paradigm, the cockpit crew functions as the final human barrier preventing latent organizational hazards from transforming into catastrophic mishaps. The NAHF measures aviators’ internalized alignment with this defensive role. An aviator who embraces rigid trans-hierarchical authority, denies fatigue, and rejects post-flight debriefing effectively weakens the active defense barriers, aligning the metaphorical "holes in the Swiss cheese."

3. Transactive Memory Systems and Shared Mental Models

In cognitive psychology, teams operating in dynamic, high-velocity environments rely heavily on Shared Mental Models (Cannon-Bowers, Salas, & Converse, 1993). A shared mental model represents an overlapping cognitive representation of the flight systems, mission objectives, tactical contingencies, and team members’ roles. The communication subscale within the NAHF operationalizes the socio-cognitive processes required to synchronize these mental models through pre-flight briefings, explicit verbalization of plans, and continuous real-time cross-checking. Without active communicative synchronization, individual aviators formulate divergent operational realities, culminating in catastrophic spatial or tactical disorientation.

Validity

Establishing the psychometric validity of the NAHF was essential given the high-consequence nature of naval aviation and the established history of military resistance to subjective self-report questionnaires. Researchers deployed multi-tiered validity testing regimens to confirm the instrument’s structural, convergent, and operational utility:

Content and Face Validity

Initial content validation was executed through an exhaustive adaptation process. Recognizing that the civilian-oriented language of Gregorich et al.’s CMAQ failed to resonate with the specialized subculture and terminology of carrier-based aviation, O’Connor and colleagues established an expert review panel. A preliminary 31-item questionnaire was distributed to a targeted panel of 20 highly seasoned naval aviators, instructor pilots, and human factors officers. These subject matter experts critically examined each item for semantic realism, syntactic naturalness, and applicability to carrier strike and rotary operations. The expert qualitative feedback led to extensive linguistic modifications, ensuring that items mirrored authentic operational lexicon while retaining the theoretical constructs of the parent instrument.

Construct and Discriminant Validity

Construct validity was evaluated using multi-sample comparisons across varying aviation operational roles and flight experience tiers. Psychometric testing demonstrated significant discriminant validity across aviation rank and cockpit seating designations. For instance, junior junior-grade lieutenants (O-2) and lieutenants (O-3) scored significantly differently than senior commanders (O-5) on the Command Responsibility and Trans-Hierarchical Inquiry items, verifying that the scale accurately captures the tension generated by military rank structures. Furthermore, multi-trait scaling analyses confirmed that the My Stress and Communication dimensions diverged from generalized job satisfaction metrics, demonstrating that the NAHF captures specific human factors attitudes rather than global organizational morale.

Predictive and Evaluative Validity

The ultimate criterion validity of the NAHF was established through longitudinal CRM intervention assessments conducted by O’Connor et al. (2012). The instrument was administered in an extensive evaluative protocol to determine whether the U.S. Navy’s newly revised CRM syllabus altered safety attitudes across multiple air wings. The NAHF demonstrated high sensitivity to intervention: aviators assessed post-training exhibited statistically significant attitudinal gains in communication assertiveness, mutual stress monitoring, and hierarchical transparency compared to pre-training baseline cohorts. These findings confirmed the instrument’s utility as a sensitive evaluative benchmark for defense-wide aviation safety initiatives.

Reliability

The statistical reliability of attitude questionnaires in aviation human factors historically presents substantial psychometric challenges. Attitudinal metrics routinely yield lower internal consistency coefficients than personality inventories or cognitive ability batteries, primarily due to the multifaceted nature of human beliefs, social desirability biases, and restricted variance within elite, highly trained populations.

Internal Consistency Metrics

In the primary empirical evaluations published by O’Connor, Jones, McCauley, and Buttrey (2012), internal consistency was estimated via Cronbach’s alpha ($lpha$):

• My Stress: Initial 6-item scale exhibited low initial reliability ($lpha pprox 0.52$). Following the psychometric excision of three poorly performing items, the trimmed 3-item scale demonstrated stabilized, though modest, reliability ($lpha = 0.61$).
• Stress of Others: Retained all 6 items, achieving acceptable internal consistency for an attitudinal subscale ($lpha = 0.68$).
• Communication: The 6-item construct demonstrated an alpha coefficient of $lpha = 0.65$, reflecting coherent internal covariance regarding briefing protocols and open communicative norms.
• Command Responsibility: Originally comprising 9 items, structural instability and item attenuation led to the elimination of four items. The revised 5-item scale exhibited an alpha coefficient of $lpha = 0.63$.
• Rules and Order: The 4-item procedural adherence subscale demonstrated severely degraded reliability ($lpha < 0.40$), prompting its complete structural termination from the valid operational metric.

While these coefficients fall below the traditional psychometric benchmark of $lpha = 0.80$ typical of clinical diagnostics, human factors psychometricians (e.g., Helmreich, Gregorich, Salas) have repeatedly demonstrated that attitude subscales comprising 3 to 6 items frequently yield alphas ranging between $0.55$ and $0.70$. Such values are considered empirically acceptable for broad group-level training assessments, cohort comparisons, and institutional safety audits where individual clinical diagnosis is not the intended outcome.

Test-Retest Stability

Temporal stability was evaluated across unexposed control cohorts over four- to six-month test intervals. The test-retest correlation coefficients across the retained subscales ranged from $r = 0.64$ to $r = 0.76$, confirming that while attitudes are plastic and susceptible to institutional training, they maintain notable structural stability in the absence of targeted human factors interventions.

Factor Analysis

The empirical transition from the initial 31-item pool to the operational version of the NAHF was governed by comprehensive exploratory factor analysis (EFA) and subsequent confirmatory factor analysis (CFA) performed on extensive samples of active-duty U.S. Naval flight crews.

Item Elimination and Univariate Distribution Screening

Prior to exploratory factor extraction, all 31 items were subjected to rigorous univariate distribution screening. In military aviation samples, psychometricians encounter pronounced ceiling effects driven by positive social desirability and indoctrinated military doctrine. Items exhibiting extreme skewness ($> |2.0|$) and kurtosis ($> |4.0|$) were earmarked for elimination. Based on these distributional departures and negligible inter-item correlations, nine items were discarded during preliminary screening. Notably, the entire Rules and Order construct was excised because nearly all aviators marked the extreme ends of the Likert spectrum, generating near-zero variance.

Exploratory Factor Analysis (EFA)

Principal Axis Factoring with oblique (Promax) rotation was conducted on the remaining 22 items, recognizing that latent human factors attitudes correlate in operational environments. The EFA revealed a four-factor solution accounting for approximately 42% of the total variance, structured around:

• Factor 1: Stress of Others (Eigenvalue = 3.42)
• Factor 2: Command Responsibility and Authority (Eigenvalue = 2.18)
• Factor 3: Communication and Briefing (Eigenvalue = 1.84)
• Factor 4: My Stress and Fatigue Vulnerability (Eigenvalue = 1.35)

Factor loadings were scrutinized; items failing to demonstrate primary loadings $ge 0.40$ or exhibiting excessive cross-loadings ($> 0.32$ on secondary factors) were flagged for removal.

Confirmatory Factor Analysis (CFA)

To validate the four-factor structural model, CFA was conducted on an independent cohort using Maximum Likelihood estimation. The structural fit of the baseline model was iteratively optimized through the removal of three additional items that exhibited significant standardized residual covariances and depressed factor loadings. The finalized measurement model yielded satisfactory goodness-of-fit indices:

• Chi-Square to Degrees of Freedom Ratio ($\chi^2/df$): 2.14, falling comfortably below the accepted threshold of 3.0.
• Root Mean Square Error of Approximation (RMSEA): 0.048 (90% CI [0.041, 0.056]), indicating excellent fit.
• Comparative Fit Index (CFI): 0.932, exceeding the conventional 0.90 baseline for non-cognitive attitudinal inventories.
• Tucker-Lewis Index (TLI): 0.918.
• Standardized Root Mean Square Residual (SRMR): 0.052.

The CFA definitively supported the four-dimensional theoretical architecture, establishing that naval aviator attitudes toward human factors cannot be modeled as a single unidimensional "safety adherence" factor, but rather require distinct conceptualization of personal stress, team vigilance, communication practices, and hierarchical command dynamics.

Instrument / Measurement Tool

The Naval Aviator Human Factors Questionnaire is a standardized, self-administered psychometric paper-and-pencil or computerized survey instrument. The structural specifications, administrative rules, and scoring architectures are delineated below:

Structural Specifications

• Total Item Count: The fully operational trimmed inventory comprises 19 refined items representing the four validated factors (down from the experimental 31-item adaptation pool).
• Administration Modality: Group or individual administration via paper assessment forms or secure web-based operational systems (e.g., naval flight safety portals).
• Completion Duration: Approximately 10 to 15 minutes.
• Target Population: Naval aviators, including Pilots (P), Naval Flight Officers (NFO), Tactical Coordinators (TACCO), and enlisted aircrew across fixed-wing, rotary-wing, and unmanned tactical platforms.

Response Scales

The instrument primarily deploys a 5-point Likert scale formatted with alphabetic or numerical identifiers:

• A / 1: Disagree Strongly
• B / 2: Disagree Slightly
• C / 3: Neutral
• D / 4: Agree Slightly
• E / 5: Agree Strongly

Additionally, the battery incorporates contextual behavioral frequency inquiries evaluated via an operational 5-point frequency scale:

• Very frequently
• Frequently
• Sometimes
• Seldom
• Very seldom

Scoring Guidelines and Data Processing

• Directly Scored Items: Items articulating favorable human factors practices (e.g., "The pre-session team briefing is important for safety and for effective team management") are scored directly from 1 (Disagree Strongly) to 5 (Agree Strongly).
• Reverse-Scored Items: Items articulating counter-normative, autocratic, or invulnerable beliefs (e.g., "My decision making ability is as good in emergencies as it is in routine situations" or "Junior team members should not question the decisions made by senior personnel in emergencies") must be reverse-coded prior to composite metric calculation: $5 = 1, 4 = 2, 3 = 3, 2 = 4, 1 = 5$.
• Subscale Score Generation: Subscale means are calculated by summing the numerical scores of the validated constituent items within each dimension and dividing by the total number of items in that subscale. This maintains an interpretable 1.00 to 5.00 metric across all four dimensions.
• Composite Safety Attitude Index: While multidimensional interpretation is mandatory for diagnostics, an overarching composite CRM metric is occasionally computed by averaging the four subscale scores. A higher overall score denotes superior alignment with human factors safety science and CRM tenets.

Permissions & Fee and Test Year

The Naval Aviator Human Factors Questionnaire was developed between 2008 and 2009, with primary empirical validation and large-scale training effectiveness findings published in peer-reviewed literature in 2012. Because the primary development was conducted by faculty and researchers at the Naval Postgraduate School (an institution of the United States Federal Government), the underlying research and derived survey items published in official defense reports and academic articles reside in the public domain under Title 17, Section 105 of the United States Code.

Consequently, the instrument requires no commercial licensing fees or purchase royalties for scholarly, military, or academic research use. However, researchers, organizational psychologists, and flight operations directors intending to deploy the NAHF or its derivative variants are strongly encouraged to cite the original authors (O’Connor, Jones, McCauley, & Buttrey, 2012) in professional publications. Modification of the items to suit allied foreign military branches or civil high-risk industries (e.g., commercial maritime, industrial power generation, or trauma surgery) is permissible provided the theoretical integrity and psychometric limitations of the instrument are acknowledged.

References

• Cannon-Bowers, J. A., Salas, E., & Converse, S. (1993). Shared mental models in expert team decision making. In N. J. Castellan, Jr. (Ed.), Current Issues in Individual and Group Decision Making (pp. 221–246). Lawrence Erlbaum Associates.
• Endsley, M. R. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors, 37(1), 32–64. https://doi.org/10.1518/001872095779049543
• Gregorich, S. E., Helmreich, R. L., & Wilhelm, J. A. (1990). The Cockpit Management Attitudes Questionnaire. Journal of Applied Psychology, 75(6), 682–690. https://doi.org/10.1037/0021-9010.75.6.682
• Helmreich, R. L., Merritt, A. C., & Wilhelm, J. A. (1999). The evolution of Crew Resource Management training in commercial aviation. International Journal of Aviation Psychology, 9(1), 19–32. https://doi.org/10.1207/s15327108ijap0901_2
• O’Connor, P., & Jones, D. (2009). The crew resource management attitudes of U.S. Naval aviators. In Proceedings of the Human Factors and Ergonomics Society 53rd Annual Meeting (pp. 1761–1765). Human Factors and Ergonomics Society. https://doi.org/10.1177/154193120905302213
• O’Connor, P., Jones, D., McCauley, M., & Buttrey, S. (2012). An evaluation of the effectiveness of the U.S. Navy’s crew resource management program. International Journal of Human Factors and Ergonomics, 1(1), 21–40. https://doi.org/10.1504/IJHFE.2012.047805
• Reason, J. (1990). Human Error. Cambridge University Press. https://doi.org/10.1017/CBO9781139062366
• Salas, E., Burke, C. S., Bowers, C. A., & Wilson, K. A. (2001). Team training in the skies: Does crew resource management training work? Human Factors, 43(4), 641–674. https://doi.org/10.1518/001872001775870386

Items of the Scale