Command Safety Assessment Survey (CSAS)

1. Abstract

The Command Safety Assessment Survey (CSAS) is an organizational safety climate measurement instrument developed by psychometric and human factors researchers at the Naval Postgraduate School (NPS) in Monterey, California. Designed specifically for military operational environments, the CSAS is completed periodically via a secure web-based infrastructure by United States Naval Aviators, flight crew members, and supporting aviation personnel across United States Navy and Marine Corps operational squadrons. The theoretical infrastructure of the survey is grounded in the Model of Organizational Safety Effectiveness (MOSE), a conceptual architecture designed to capture systemic risk-mitigation dimensions within High Reliability Organizations (HROs). The original comprehensive instrument comprises 61 items mapping across five core organizational domains: Process Auditing, Reward System, Quality Assurance, Risk Management, and Command and Control.

Respondents evaluate individual items utilizing an authentic 5-point Likert response scale ranging from 1 (“Strongly Disagree”) to 5 (“Strongly Agree”), with an additional “Not Applicable / Don’t Know” option. Substantive psychometric evaluations conducted on vast empirical cohorts of operational aviators (e.g., Buttrey et al., 2010; O’Connor et al., 2011) have rigorously investigated the structural integrity, internal consistency, and criterion-related validity of the instrument. While the full 61-item operational deployment yielded variable factor determinacy and raised challenges regarding discriminant validity when predicting prospective class-A through class-C aviation mishaps, refined exploratory and confirmatory factor analyses isolated a robust, psychometrically validated 12-item core subset demonstrating exceptional construct validity and internal consistency reliability. This article provides a comprehensive academic review of the CSAS, delineating its historical origin, latent psychometric parameters, structural models, diagnostic utility, and operational considerations within defense aviation psychology.

2. Keywords

Command Safety Assessment Survey, CSAS, safety climate, safety culture, High Reliability Organizations, Naval Postgraduate School, Naval Aviation, Model of Organizational Safety Effectiveness, risk management, human factors

3. Authors

The Command Safety Assessment Survey was developed and psychometrically evaluated by a multidisciplinary team of military behavioral scientists, operations research methodologists, and human factors ergonomists associated with the Naval Postgraduate School (Department of Operations Research and School of Operational and Information Sciences) in collaboration with the Naval Safety Center (now the Naval Safety Command), located in Norfolk, Virginia.

• Dr. Paul O’Connor, Ph.D. — Professor of Human Factors and Patient Safety; formerly Assistant Professor, Department of Operations Research, Naval Postgraduate School, Monterey, CA. Expert in organizational safety climate, human reliability, and team resource management.
• Dr. Samuel E. Buttrey, Ph.D. — Associate Professor of Operations Research, Department of Operations Research, Naval Postgraduate School, Monterey, CA. Specializes in statistical modeling, computational data analysis, and predictive psychometric analytics.
• Dr. Angela O’Dea, Ph.D. — Research Associate, Department of Operations Research, Naval Postgraduate School, Monterey, CA. Focuses on occupational psychology, organizational culture, and aviation safety analysis.
• Dr. Quinn Kennedy, Ph.D. — Research Assistant Professor, Department of Operations Research, Naval Postgraduate School, Monterey, CA. Specializes in cognitive aging, performance under extreme environmental stress, and military human systems integration.

4. Purpose

The overarching purpose of the Command Safety Assessment Survey (CSAS) is to evaluate, monitor, and diagnostically benchmark the organizational safety climate within operational aviation squadrons across the U.S. Navy and U.S. Marine Corps. Operating at the boundary of technological capability and extreme environmental risk, military aviation represents an archetypal High Reliability Organization where procedural errors, system breakdowns, or lapses in risk perception can precipitate catastrophic loss of life and multi-million-dollar asset destruction.

Historically, safety management protocols in naval aviation relied primarily on lagging indicators—namely, the retrospective epidemiological analysis of aviation mishaps, flight incidents, hazard reports (HAZREPs), and material failures. Although forensic investigations yield valuable localized interventions, lagging metrics inherently reflect systemic breakdown after harm has already transpired. The CSAS was systematically engineered to supply proactive leading indicators of operational risk by quantifying the workforce’s aggregate perceptions of the organizational policies, procedural integrity, supervisory practices, and command behaviors that define the daily operational climate.

At the command level, the operational purpose of the CSAS is diagnostic and preventative. The survey is deployed across routine operational cycles, pre-deployment workups, and following transitions in squadron leadership (such as the change of command between incoming and outgoing Commanding Officers). Aggregated, anonymized survey outputs furnish commanding officers, aviation safety officers (ASOs), and executive leadership with an empirical profile of unit vulnerabilities, such as emerging complacency, perceived production pressure, failure to report near-misses, or inconsistent enforcement of standard operating procedures (SOPs). In applied military psychology and organizational research, the CSAS serves as a benchmark tool to model the mechanisms through which command leadership attitudes translate into front-line compliance and proactive hazard identification.

5. Psychological Construct

The psychological construct assessed by the CSAS is organizational safety climate, specifically operationalized within naval aviation commands. In psychometric literature, safety climate is conceptualized as the shared, surface-level perceptions held by employees regarding the safety policies, practices, and procedures that are actively rewarded, supported, and expected within their immediate working environment (Zohar, 1980, 2010). Unlike the broader, deeper, and more immutable construct of safety culture—which embodies deeply held subconscious values, core beliefs, and historical organizational paradigms—safety climate reflects a temporally sensitive, measurable psychological state that fluctuates in response to operational demands, resource shifts, and leadership behaviors.

Under the Model of Organizational Safety Effectiveness (MOSE), the CSAS dissects this global climate construct into five discrete operational dimensions:

• Process Auditing: The psychological and structural assessment of internal checks, standardized evaluations, and quality oversight mechanisms designed to detect latent hazardous conditions before catastrophic failures occur. This involves active surveillance of maintenance protocols, flight readiness examinations, and Naval Air Training and Operating Procedures Standardization (NATOPS) compliance.
• Reward System: The perceived social reinforcement, institutional recognition, or administrative accountability administered by leadership. This dimension assesses whether personnel perceive that demonstrating safety compliance and proactive hazard reporting are genuinely incentivized, or whether production schedules, rapid aircraft turnarounds, and mission completions are informally rewarded at the expense of safety discipline.
• Quality Assurance: Shared perceptions concerning the stringency, integrity, and non-negotiability of procedural standards, maintenance documentation, and technical expertise. It encapsulates the refusal to allow “gun decking” (falsifying inspection paperwork) or unauthorized operational shortcuts under severe tempo constraints.
• Risk Management: The structural and cognitive incorporation of systematic hazard identification techniques, most notably formal Operational Risk Management (ORM) and Crew Resource Management (CRM). This dimension gauges whether risk assessment is viewed as an interactive, life-saving decision architecture or dismissed as a bureaucratic, check-in-the-box exercise.
• Command and Control: The perceived bidirectional communication channel between front-line operators (aviators, aircrew, maintainers) and executive squadron leadership (Commanding Officer, Executive Officer, Command Master Chief). It measures psychological safety—specifically the degree to which junior personnel feel empowered to voice critical safety alerts, exercise “knock-it-off” or abort authority, and contest unsafe orders without fear of professional reprisal.

6. Theoretical Framework

The architectural foundation of the CSAS is rooted in the convergence of two foundational theoretical bodies: High Reliability Theory and Social Cognitive Safety Climate Theory.

High Reliability Theory (Weick et al., 1999; Roberts, 1990) posits that certain complex organizational systems operate under conditions of extreme operational hazard and continuous environmental volatility yet maintain remarkably low rates of catastrophic failure over prolonged temporal spans. Such organizations—including nuclear aircraft carriers, spaceflight missions, and military flight operations—exhibit five systemic cognitive-operational characteristics: preoccupation with failure, reluctance to simplify interpretations, sensitivity to operations, commitment to resilience, and deference to expertise. The CSAS was engineered to quantify these exact HRO properties. Specifically, items assessing whether aircrew members challenge unsafe instructions regardless of rank directly measure the deference to expertise and non-hierarchical problem-solving that characterize high-reliability organizational dynamics.

Simultaneously, the survey aligns with Dov Zohar’s multilevel model of safety climate (Zohar, 2000, 2010). Zohar postulated that organizations possess competing operational objectives—predominantly the tension between production efficiency (in the military context, operational mission generation and combat readiness) and safety preservation. While formal doctrine invariably declares safety as paramount, workers discern the true organizational priorities by observing supervisory behaviors, budget allocations, and disciplinary reactions during operational crises. When leaders tolerate deviations from SOPs to make flight schedules, front-line personnel infer that production outranks safety. The CSAS captures this psychological calculus by evaluating the extent to which command leadership models adherence, provides adequate rest cycles, and equitably supports mission abort decisions.

Furthermore, the Model of Organizational Safety Effectiveness (MOSE), developed by Libuser (1994) and adapted for military aviation contexts by researchers at the Naval Postgraduate School, synthesizes these sociological frameworks into a pragmatic diagnostic typology. MOSE asserts that accidents within high-risk sociotechnical systems are rarely the exclusive consequence of isolated individual human error (Reason, 1990). Rather, individual cognitive lapses or active failures are facilitated and amplified by upstream organizational vulnerabilities within auditing, reward mechanisms, quality enforcement, risk assessment methodologies, and command communication conduits.

7. Validity

The psychometric validity of the CSAS has undergone extensive empirical examination across large-scale cohorts of operational naval aviation squadrons, yielding nuanced findings that have fueled critical discourse in military psychometrics.

Construct and Structural Validity

Construct validity reflects the degree to which an instrument accurately operationalizes its theoretical target construct. In early developmental evaluations of the CSAS, researchers encountered structural complexities. In a seminal construct validity evaluation analyzing responses from over 2,000 naval aviators, Buttrey, O’Dea, O’Connor, and Kennedy (2010) identified that although the 61-item scale exhibited high facial and content validity among domain experts and operational pilots, the empirical inter-item correlation matrix exhibited pervasive multicollinearity. Responses across all 61 items were heavily skewed toward positive endorsements, reflecting a systemic ceiling effect common to military units with high esprit de corps and strong institutional loyalty.

Through iterative exploratory and confirmatory factor modeling, Buttrey et al. (2010) demonstrated that the theoretical five-factor MOSE topology did not cleanly replicate in unconstrained factor rotations. Instead, many items loaded diffusely across a monolithic “general command leadership and safety commitment” factor. Nevertheless, through systematic item reduction, the researchers successfully established that a focused 12-item subset exhibited exceptional structural construct validity, retaining robust factor determinacy and distinct parameter loadings representing core command leadership, compliance, and procedural fidelity.

Predictive and Discriminant Validity

The ultimate benchmark of an applied military safety climate instrument is its criterion-related predictive validity: does a lower safety climate score statistically forecast a higher likelihood of subsequent aircraft mishaps? O’Connor, Buttrey, O’Dea, and Kennedy (2011) executed an extensive longitudinal investigation assessing the relationship between squadron CSAS scores and prospective mishap records (Class A, B, and C flight and ground mishaps) cataloged by the Naval Safety Center over multi-year windows.

Their findings revealed critical psychometric limitations regarding the survey’s prospective discriminant validity. Specifically, the authors found insufficient empirical evidence to conclude that aggregate squadron safety climate scores derived from the CSAS reliably predicted future aviation mishap risk. The statistical models demonstrated that squadrons that subsequently suffered Class A flight mishaps (involving fatalities or aircraft destruction exceeding $2 million) did not exhibit significantly lower historical CSAS scores compared to mishap-free squadrons. Several systemic psychometric and epidemiological phenomena account for this limitation:

• Extreme Low Base-Rate of Mishaps: Major aviation accidents are statistically rare events, creating severe epidemiological distribution skewness that restricts statistical power and attenuates predictive correlation coefficients.
• Ceiling Effects and Response Compression: Most aviators rate their units highly favorable (means often exceeding 4.2 on a 5-point scale), yielding constrained variance across squadrons.
• Temporal Lag: Safety climate represents a transient organizational snapshot, whereas mishaps frequently occur months or years post-survey administration, during which personnel turnover (typically 30–50% per annum in operational squadrons) fundamentally alters the command climate.

Consequently, psychometric consensus indicates that while the CSAS possesses strong diagnostic and face validity for localized organizational development, its utility as an autonomous, predictive actuarial instrument for mishap forecasting is limited (O’Connor et al., 2011).

8. Reliability

The internal consistency reliability of the Command Safety Assessment Survey has been extensively validated across multiple operational samples. Reliability indices differ markedly depending on whether the full 61-item survey or the refined 12-item core scale is evaluated.

For the comprehensive 61-item instrument, omnibus internal consistency is exceptionally high. Multiple independent empirical assessments (e.g., Buttrey et al., 2010; O’Connor et al., 2011) have documented an overall Cronbach’s alpha ($lpha$) ranging between .95 and .97 across naval aviation cohorts. Subscale-level internal consistency estimates across the five theoretical MOSE domains generally satisfy standard psychometric criteria:

• Process Auditing: $lpha = .81 – .86$
• Reward System: $lpha = .78 – .84$
• Quality Assurance: $lpha = .83 – .88$
• Risk Management: $lpha = .85 – .89$
• Command and Control: $lpha = .87 – .91$

However, psychometricians have noted that an omnibus alpha exceeding .95 strongly indicates structural item redundancy—where multiple items assess nearly indistinguishable cognitive and behavioral facets. For the psychometrically refined 12-item scale developed by Buttrey et al. (2010), internal consistency remained robust at $lpha = .86$, indicating an optimal balance of item efficiency, construct coverage, and minimal parameter redundancy.

Because the CSAS is completed anonymously to foster candor in reporting command deficiencies, classical test-retest reliability across individual respondents cannot be tracked longitudinally over time. However, squadron-level intraclass correlation coefficients ($ICC[1]$ and $ICC[2]$) have been computed to ascertain the degree of within-unit agreement and between-unit differentiation. Squadron aggregation metrics typically demonstrate moderate within-group agreement ($r_{wg(j)} > .75$), justifying the aggregation of individual aviator perceptions into a composite unit-level safety climate score.

9. Factor Analysis

The structural dimensionality of the CSAS has been the subject of extensive psychometric investigation, primarily utilizing Exploratory Factor Analysis (EFA) followed by Confirmatory Factor Analysis (CFA) to evaluate the tenability of the five-factor MOSE taxonomy.

Exploratory Factor Analysis Findings

In the comprehensive evaluations conducted by Buttrey, O’Dea, O’Connor, and Kennedy (2010), an EFA was conducted on a primary calibration sample of operational aviator responses ($N > 1,500$) using principal axis factoring with promax (oblique) rotation, reflecting the theoretical assumption that organizational safety climate sub-dimensions are inherently intercorrelated. The unconstrained scree plot and eigenvalue analysis (Kaiser-Guttman criterion) revealed a massive primary eigenvalue accounting for over 45% of the total variance, followed by a precipitous drop-off to minor secondary factors that each accounted for less than 4% of additional variance.

When force-extracted to the five theoretical MOSE factors, substantial cross-loadings (> .35) emerged across dozens of items. In particular, items ostensibly designed to measure Quality Assurance and Process Auditing loaded heavily onto a dominant general factor alongside Command and Control. This indicated that front-line operational personnel do not conceptually distinguish technical auditing processes from general supervisory leadership oversight.

Confirmatory Factor Analysis & Structural Optimization

To resolve the empirical misfit of the 61-item architecture, Buttrey et al. (2010) executed CFA modeling across an independent validation sample. As summarized in the structural comparison table above, the initial 61-item five-factor theoretical model displayed inadequate goodness-of-fit indices: the Comparative Fit Index (CFI) of .81 and Tucker-Lewis Index (TLI) of .80 fell considerably short of the standard .90/.95 thresholds for acceptable psychometric fit. The Root Mean Square Error of Approximation (RMSEA) was elevated at .068.

Through systematic modification index evaluation, residual covariance inspection, and theoretical content retention, Buttrey and colleagues isolated a parsimonious 12-item core scale. Confirmatory factor analysis of this 12-item specification demonstrated superior model fit: $\chi^2 / df = 2.11$, $ ext{CFI} = .96$,$ ext{TLI} = .95$,$ ext{RMSEA} = .038$ ($90%\text{ CI } [.031, .045]$), and $ ext{SRMR} = .031$. Standardized factor loadings across all 12 retained items ranged solidly between .58 and .82, confirming that a concise, psychometrically robust latent factor structure underlies the broader operational inventory.

10. Instrument / Measurement Tool

• Test Type: Organizational Safety Climate Assessment / Diagnostic Survey
• Administration Format: Web-based, self-administered survey (historically administered via secure military servers through the Naval Postgraduate School and Naval Safety Center/Command)
• Target Population: Military aviation personnel, including active-duty and reserve naval aviators, naval flight officers (NFOs), enlisted aircrew, aviation maintenance technicians, and squadron leadership
• Item Count: 61 comprehensive operational items (with a psychometrically validated 12-item core research subset)
• Response Format: 5-point Likert scale: 1 = Strongly Disagree, 2 = Disagree, 3 = Neutral, 4 = Agree, 5 = Strongly Agree (plus ‘Not Applicable / Don’t Know’)
• Theoretical Structure: Structured around five domains from the Model of Organizational Safety Effectiveness (MOSE):
  • Process Auditing (e.g., Items 1, 6, 20, 48)
  • Reward System (e.g., Items 4, 12, 21, 55)
  • Quality Assurance (e.g., Items 7, 16, 32, 36)
  • Risk Management (e.g., Items 14, 17, 41, 56)
  • Command and Control (e.g., Items 5, 9, 18, 26, 45)
• Scoring and Directionality: Positively phrased items are scored from 1 (Strongly Disagree) to 5 (Strongly Agree). Negatively phrased items reflecting dysfunctional safety practices are reverse scored prior to calculating subscale and aggregate command means. High scores represent a positive, proactive, and resilient organizational safety climate. “Not Applicable / Don’t Know” responses are coded as missing data and excluded from mean computations.
• Average Completion Time: 15 to 25 minutes for the full 61-item assessment.

11. Permissions & Fee and Test Year

The Command Safety Assessment Survey (CSAS) was conceptualized, drafted, and refined through technical studies published between 2000 and 2011 by researchers at the Naval Postgraduate School in Monterey, California, under work sponsored by the United States Navy and the Naval Safety Center (now the Naval Safety Command).

As a work prepared by military personnel and federal civilian researchers within the scope of their official employment, the survey items, associated technical reports, and validation monographs exist in the public domain under the provisions of Title 17, Section 105 of the United States Code. There are no licensing fees, copyright royalties, or purchase costs associated with the utilization of the instrument for non-commercial academic, scientific, or organizational safety research.

While the full survey instrument is in the public domain, institutional use within operational military branches requires formal administrative authorization through the appropriate service safety center or commanding authorities. Independent researchers seeking to deploy the scale in external defense, high-reliability commercial, or aerospace environments are advised to cite the primary psychometric development technical reports and journal articles authored by the Naval Postgraduate School research team (e.g., Buttrey et al., 2010; O’Connor et al., 2011).

12. References

• Buttrey, S. E., O’Dea, A., O’Connor, P., & Kennedy, Q. (2010). An evaluation of the construct validity of the command safety assessment survey (Technical Report NPS-OR-10-004). Naval Postgraduate School. https://edocs.nps.edu/npspubs/scholarly/TR/2010/NPS-OR-10-004.pdf
• Libuser, C. (1994). Organizational structure and risk: An operational model of organizational safety effectiveness (Doctoral dissertation, University of California, Los Angeles). ProQuest Dissertations Publishing.
• O’Connor, P., Buttrey, S. E., O’Dea, A., & Kennedy, Q. (2011). An assessment of the relationship between safety climate and mishap risk in U.S. Naval aviation (Technical Report NPS-OR-11-004). Naval Postgraduate School. https://edocs.nps.edu/npspubs/scholarly/TR/2011/NPS-OR-11-004.pdf
• O’Connor, P., Buttrey, S. E., O’Dea, A., & Kennedy, Q. (2011). Identifying and addressing the limitations of safety climate surveys. Journal of Safety Research, 42(4), 259–265. https://doi.org/10.1016/j.jsr.2011.06.002
• Reason, J. (1990). Human error. Cambridge University Press. https://doi.org/10.1017/CBO9781139062367
• Roberts, K. H. (1990). Some characteristics of one type of high reliability organization. Organization Science, 1(2), 160–176. https://doi.org/10.1287/orsc.1.2.160
• Weick, K. E., Sutcliffe, K. M., & Obstfeld, D. (1999). Organizing for high reliability: Processes of collective mindfulness. Research in Organizational Behavior, 21, 81–123.
• Zohar, D. (1980). Safety climate in industrial organizations: Theoretical and applied implications. Journal of Applied Psychology, 65(1), 96–102. https://doi.org/10.1037/0021-9010.65.1.96
• Zohar, D. (2000). A group-level model of safety climate: Testing the effect of supervisory practices on subgroup damage. Journal of Applied Psychology, 85(4), 587–596. https://doi.org/10.1037/0021-9010.85.4.587
• Zohar, D. (2010). Thirty years of safety climate research: Reflections and future directions. Accident Analysis & Prevention, 42(5), 1517–1522. https://doi.org/10.1016/j.aap.2009.12.019

13. Items of the Scale