Feedback Tool for Rescue Force Development Examinations (FIRE-P)

Abstract

The Feedback Tool for Rescue Force Development Examinations (known by its German acronym FIRE-P, Feedback-Instrument für Rettungskräfte-Entwicklungsprüfungen) is a specialized psychometric assessment instrument engineered to evaluate the perceived quality, instructional alignment, psychometric design, and perceived cognitive and psychological burden of written examinations administered in emergency response and fire brigade leadership training programs. Developed through a cooperative research initiative between the Department of Organizational and Business Psychology at the University of Münster (Westfälische Wilhelms-Universität Münster) and the State Fire Service Institute of North Rhine-Westphalia (Institut der Feuerwehr Nordrhein-Westfalen [IdF NRW]), the instrument adapts theoretical and methodological principles established in higher education course and examination evaluation—specifically the Münster Questionnaire for the Evaluation of Written Examinations (MFE-K)—to the high-stakes, operationally rigorous domain of emergency rescue leadership education. The development of FIRE-P followed a rigorous multi-phase psychometric design: qualitative exploratory interviews and content analysis with subject matter experts, examiners, and course candidates (N = 61; Study I); cognitive pre-testing and item comprehensibility assessment (N = 22; Study II); exploratory factor analysis (EFA) across seventeen incident commander cohorts (N = 428; Study III); and confirmatory factor analysis (CFA) paired with extensive construct validation across sixteen leadership training courses (N = 301; Study IV). The core measurement model comprises 12 mandatory items structured across three primary dimensions: Preparation (Vorbereitung; 4 items), Design (Gestaltung; 5 items), and Perceived Load (Belastung; 3 items), complemented by 12 optional contextual, bias, and qualitative feedback items (Items 13–24). Responses on the primary subscales are recorded on a 7-point Likert-type agreement scale ranging from 1 (“strongly disagree”) to 7 (“strongly agree”), with an additional discrete option denoting that an item cannot be answered meaningfully. Psychometric evaluations substantiate robust internal consistency across all subscales (Cronbach’s α = .76 to .86), excellent structural validity confirming a stable three-factor latent architecture, and coherent patterns of convergent, discriminant, and criterion-related validity with training performance, emotional state, perceived examination stress, and individual personality traits.

Keywords

FIRE-P, examination evaluation, fire brigade leadership training, emergency services education, test quality, psychometrics, course evaluation, test burden, instructional alignment, IdF NRW

Authors

The Feedback Tool for Rescue Force Development Examinations (FIRE-P) was conceptualized, operationalized, and psychometrically validated through an academic-practitioner collaboration between the University of Münster and the State Fire Service Institute of North Rhine-Westphalia:

• Nadine Busjan, M.Sc. — Department of Psychology, Organizational and Business Psychology, Westfälische Wilhelms-Universität Münster, Münster, Germany. Primary researcher on examination evaluation criteria and exploratory psychometrics (Study I and Study III).
• Lukas Frerichs, M.Sc. — Department of Psychology, Organizational and Business Psychology, Westfälische Wilhelms-Universität Münster, Münster, Germany. Lead psychometrician on structural equation modeling, confirmatory factor validation, and nomological network assessment (Study IV).
• Dr. Meinald T. Thielsch — Professor and Senior Researcher, Department of Psychology, Organizational and Business Psychology, Westfälische Wilhelms-Universität Münster, Münster, Germany. Corresponding scientific lead and co-developer of examination evaluation methodology in academic and operational contexts. (Contact: [email protected]).
• In cooperation with the Institut der Feuerwehr Nordrhein-Westfalen (IdF NRW), Münster, Germany — The central public safety and fire service training college for the federal state of North Rhine-Westphalia, responsible for curricular governance, examiner coordination, and structural qualification frameworks for fire service commanders.

Purpose

Emergency rescue services and municipal fire departments operate in volatile, uncertain, complex, and high-threat environments where operational miscalculations can result in immediate loss of civilian life, severe physical injury to crew personnel, and massive infrastructural destruction. Consequently, tactical leaders—specifically group commanders (Gruppenführer) and platoon commanders (Zugführer)—bear extraordinary command, operational, and legal responsibilities under statutory frameworks such as Feuerwehr-Dienstvorschrift 100 (FwDV 100, “Leadership and Command in Operations”) and Feuerwehr-Dienstvorschrift 3 (FwDV 3). To guarantee that field commanders possess the requisite cognitive mastery, tactical comprehension, decision-making capabilities, and hazard-mitigation competencies, public safety academies rely on high-stakes qualification examinations. These examinations serve as institutional gatekeepers, determining whether candidates obtain legal certification to command operational units in real-world crisis events.

Despite the critical importance of rescue force education for public safety, systematic quality assurance methodologies tailored specifically to emergency service examinations were historically nonexistent. While quality management in vocational and higher education expanded rapidly over recent decades, fire service examinations frequently relied on traditional assessment customs, without standardized psychometric checks on item construction, perceived examination fairness, transparency of grading criteria, or constructive alignment between classroom pedagogy and summative assessments. Poorly constructed examinations introduce construct-irrelevant difficulty, ambiguous prompt phrasing, unrealistic time pressures, and arbitrary grading metrics—factors that distort test scores, trigger maladaptive exam anxiety, and impair test fairness without enhancing diagnostic precision.

The primary purpose of the FIRE-P is to remediate this institutional deficit by providing fire training academies, course designers, and institutional quality management bodies with a psychometrically validated, standardized diagnostic instrument to continuously monitor, evaluate, and optimize written certification examinations. Administered immediately following the completion of an examination, FIRE-P captures candidates’ direct, authentic appraisal of instructional preparation, test construction quality, and psychophysiological burden. In applied training operations, the tool delivers actionable empirical feedback to faculty and syllabus committees, pinpointing whether discrepancies in examination outcomes stem from curriculum gaps (e.g., inadequate preparatory mock tests), syntactic ambiguities in exam questions, or excessive cognitive overloading. In scientific and educational research, FIRE-P serves as an empirical framework to examine how instructional design interacts with candidate characteristics, performance anxiety, and competency development under high-stakes vocational training conditions.

Psychological Construct

The FIRE-P conceptualizes examination quality not as a unidimensional global rating of student satisfaction, but as an interactive, multidimensional educational construct reflecting the degree to which a formal assessment instrument fulfills pedagogical, psychometric, and psychological standards. Grounded in the taxonomy of educational evaluation and psychometric testing, the instrument delineates three mandatory core dimensions alongside secondary contextual and affective moderators.

1. Preparation (Vorbereitung)

The Preparation dimension evaluates the degree of constructive alignment between pre-examination instructional activities and the ultimate summative evaluation. This construct reflects candidate perceptions regarding the transparency of performance standards, the pedagogical efficacy of instructional delivery, the availability and quality of pre-exam learning resources (such as practice tests and sample exercises), and the authentic applicability of acquired knowledge within the test itself. A well-prepared cohort exhibits high scores on this dimension, indicating that the curriculum established clear expectations and equipped candidates with the cognitive scaffolds necessary to navigate the assessment. Exemplar items assess whether the candidate felt thoroughly prepared during the course, whether adequate mock exams were supplied, whether performance expectations were transparent prior to testing, and whether the examination allowed for genuine application of the instruction provided.

2. Design (Gestaltung)

The Design dimension captures the formal, structural, and psychometric clarity of the examination instrument itself. Rooted in classical test theory and instructional syntax, this construct encompasses the linguistic precision of item phrasing, the unequivocal nature of procedural instructions, the ergonomics of the chosen response format (e.g., structured open answers, diagrammatic analysis, multiple-choice), the logical structural sequencing of the exam components, and the perceived operational relevance of the content to actual field command practice. A high score on this factor reflects an examination that minimizes construct-irrelevant variance; candidates can demonstrate their operational competency unhindered by syntactical vagueness, misleading stems, or poorly formatted response sheets.

3. Perceived Load (Belastung)

The Perceived Load dimension measures the subjective psychological, cognitive, and systemic strain experienced by candidates as a direct consequence of the examination and its surrounding curricular structure. Distinct from healthy cognitive engagement, this dimension identifies test over-difficulty, excessive breadth and volume of examinable material, and the cumulative exhaustion produced by sequential examination batteries within intense, condensed residential leadership curricula. Crucially, in the FIRE-P measurement framework, Load items are constructed such that agreement indicates high perceived stress, material overload, and excessive difficulty. Thus, in institutional quality assurance, low to moderately low values on this scale are considered optimal, reflecting an appropriately challenging assessment that avoids debilitating distress.

Secondary Contextual and Bias Dimensions

In addition to the three latent dimensions, the FIRE-P incorporates optional modular indicators designed to capture external physical testing environments (Rahmenbedingungen; e.g., ambient noise, examiner rapport, physical materials, procedural opportunity to ask clarification questions), potential perceptual biases (Bias-Variablen; e.g., candidate course interest, subjective career dependence, minimum-effort passing motivation, and retrospective solution satisfaction), and overall global evaluation (Globalurteil; recorded via the standardized German six-point academic grading scale).

Theoretical Framework

The theoretical architecture of the FIRE-P integrates foundational concepts from educational psychology, cognitive load theory, and the psychometric principles of formative course evaluation. The instrument is directly informed by three complementary frameworks:

Constructive Alignment and Instructional Design Theory

First and foremost, the instrument operationalizes John Biggs’ theory of Constructive Alignment. Biggs posited that effective educational systems require systematic coherence between intended learning outcomes (ILOs), the teaching and learning activities deployed in instruction, and the assessment tasks used to evaluate candidate mastery. In fire brigade leadership education, constructive alignment dictates that tactical decision-making skills taught in the classroom must match the cognitive operations demanded in written examinations. When alignment fails—for instance, when teaching emphasizes conceptual doctrine while exams penalize candidates on obscure municipal administrative minutiae—candidates experience heightened disorientation and perceived unfairness. FIRE-P’s Preparation scale directly monitors the degree to which candidates perceive that the instructional phase structurally anticipates and transparently reflects the evaluative challenge.

Cognitive Load Theory

The conceptualization of the Load (Belastung) and Design (Gestaltung) subscales is strongly underpinned by Sweller’s Cognitive Load Theory. In testing situations, total cognitive load consists of intrinsic load (the inherent complexity of the tactical scenario), germane load (mental processes devoted to schema acquisition and problem resolution), and extraneous load (mental effort wasted on navigating poorly phrased items, ambiguous formatting, or erratic page layouts). FIRE-P assumes that high-quality examinations maximize germane cognitive processing while systematically suppressing extraneous cognitive load. Items assessing linguistic ambiguity, formatting ergonomics, and informational clutter directly operationalize the reduction of extraneous load. Conversely, items assessing material volume and excessive difficulty distinguish between manageable intrinsic challenge and cognitive overload that impairs fair assessment.

Higher Education Examination Evaluation Paradigms

Methodologically, the FIRE-P builds directly upon the psychometric paradigms established by Froncek, Hirschfeld, and Thielsch (2014) in their development of the Münster Questionnaire for the Evaluation of Written Examinations (MFE-K). The MFE-K established that student evaluations of written assessments in tertiary education conform to a reliable multi-factor structure encompassing student burden, transparency, and exam layout. The creators of FIRE-P translated and expanded these constructs for adult vocational emergency personnel, accounting for the unique sociodemographic profile of fire service members, the strict paramilitary hierarchy of command courses, and the profound operational significance of the subject matter.

Validity

The validity of the FIRE-P has been substantiated through a multi-tiered validation program adhering to the Standards for Educational and Psychological Testing, incorporating content, construct, convergent, discriminant, and criterion-related empirical investigations.

Content Validity

Content validity was established during Study I and Study II through qualitative and pre-test empirical methodologies. In Study I, semi-structured interviews were conducted with n = 11 key stakeholders (6 course participants, 5 certified examiners and senior lecturers at the IdF NRW), supplemented by structured written qualitative inquiries with n = 50 active candidates in group and platoon commander courses (total N = 61; all male; candidate age M = 34.0, SD = 7.8; lecturer age M = 42.2, SD = 8.8). Inductive qualitative content analysis extracted eight foundational quality categories: Preparation, Transparency, Practical Relevance, Design, Feedback, Load, Fairness, and Testing Framework. In Study II, an initial 37-item pre-test battery was administered to N = 22 group commander candidates directly after testing. Candidates evaluated item importance and linguistic clarity on 4-point rating scales, confirming that the generated items exhibited direct practical relevance and unambiguous phrasing, leading to the targeted removal of three non-discriminating items.

Construct and Structural Validity

Structural validity was confirmed via cross-validation across independent cohorts. Exploratory factor analysis in Study III (N = 428) identified that the underlying item pool coalesced into three robust factors accounting for 54.33% of the total variance across retained items. Subsequent Confirmatory Factor Analysis in Study IV (N = 301) verified that the three-factor model provided an acceptable-to-good fit to the empirical data across multiple structural equation indices, confirming the latent differentiation between Preparation, Design, and Perceived Load.

Convergent, Criterion, and Discriminant Validity

Criterion and construct validity were verified in Study III and Study IV against proximal and distal psychological criteria. In Study III (N = 428), the FIRE-P subscales correlated significantly in expected directions with candidate affective state at the time of evaluation (assessed via a 5-point Smiley mood scale) and perceived situational stress during the examination (measured with four standardized stress items). Specifically, candidates reporting higher exam stress exhibited significantly higher scores on the FIRE-P Belastung scale and significantly lower scores on Vorbereitung and Gestaltung.

In Study IV (N = 268 complete validation datasets), candidate ratings were systematically correlated with an extensive nomological battery of validated psychological instruments:

• Global Examination Grade (Item 24): Strong convergent associations were established with the overall exam grade. The Preparation (r = .62) and Design (r = .86) scales demonstrated strong positive correlations with favorable examination grades, whereas the Load scale exhibited a substantial inverse correlation (higher load correlating with poorer overall evaluations).
• Personality Traits: As theoretically hypothesized, Conscientiousness (measured via the BFI-S; Schupp & Gerlitz, 2014) and General Self-Efficacy (measured via the ASKU; Beierlein et al., 2014) exhibited positive associations with the Preparation scale. Highly conscientious candidates demonstrated greater proactive study engagement, yielding higher appraisals of instructional preparedness.
• Affective Organizational Commitment: Measured using the COBB commitment inventory (Felfe et al., 2014), organizational commitment among fire service members positively predicted Preparation and Design ratings, reflecting high intrinsic engagement with professional standards.
• Locus of Control and Optimism/Pessimism: Internal locus of control and dispositional optimism (SOP2; Kemper et al., 2014) correlated positively with perceived design clarity and decision adequacy, whereas external locus of control and dispositional pessimism correlated positively with perceived Load and inversely with Preparation.
• Test Anxiety and Resilience: Debilitating test anxiety (Modick, 2014) correlated moderately to strongly with the FIRE-P Load scale, confirming that the subscale effectively registers cognitive and emotional distress, whereas psychological resilience (CD-RISC; Sarubin et al., 2015) buffered against elevated load perceptions.

Reliability

The reliability of the FIRE-P was rigorously evaluated across two independent, large-scale empirical samples representing fire service leadership courses. Internal consistency was estimated via Cronbach’s alpha (α) for each of the three latent subscales across Study III and Study IV.

In Study III (N = 428 valid examination evaluations across 17 distinct group commander and platoon commander courses), the internal consistency coefficients demonstrated solid psychometric precision:

• Preparation (Vorbereitung, 4 items): Cronbach’s α = .84
• Design (Gestaltung, 5 items): Cronbach’s α = .81
• Perceived Load (Belastung, 3 items): Cronbach’s α = .79

In Study IV (N = 301 valid evaluations across 16 leadership courses comprising nine distinct non-technical qualification pathways, including advanced incident command and hazardous materials [CBRN/ABC] management), the internal consistency estimates remained highly stable:

• Preparation (Vorbereitung, 4 items): Cronbach’s α = .82
• Design (Gestaltung, 5 items): Cronbach’s α = .86
• Perceived Load (Belastung, 3 items): Cronbach’s α = .76

Across both operational samples, the reliability metrics fall squarely within the acceptable-to-good range (α between .76 and .86). Psychometric standards dictate that for instruments whose operational interpretation occurs predominantly at the aggregated cohort or course level (rather than for high-stakes individual diagnostics), reliability coefficients exceeding .75 are fully sufficient to ensure stable, replicable quality evaluations across recurring examination cycles.

Factor Analysis

The dimensional structure of the FIRE-P was established through consecutive exploratory and confirmatory factor analytic workflows adhering to modern latent variable modeling standards.

Exploratory Factor Analysis (Study III)

In Study III, 27 candidate items were administered to N = 428 participants. Preliminary item screening eliminated items exhibiting non-response or “not answerable” rates exceeding 5%, extreme skewness (> |2|) or kurtosis (> |7|), item-intercorrelations exceeding .85, or ceiling-effect difficulty parameters. Principal axis factoring (PAF) with oblique Promax rotation was conducted on the remaining 24 items. Multiple extraction criteria—including the Kaiser-Guttman eigenvalue rule (> 1.0), Cattell’s scree test, and Velicer’s Minimum Average Partial (MAP) test—converged on an initial four-factor extraction:

• Factor 1: Eigenvalue = 9.35 (accounting for 38.95% of total variance)
• Factor 2: Eigenvalue = 2.15 (accounting for 8.96% of total variance)
• Factor 3: Eigenvalue = 1.54 (accounting for 6.42% of total variance)
• Factor 4: Eigenvalue = 1.49 (accounting for 6.19% of total variance)

Inspection of the factor pattern matrix revealed that Factor 4 suffered from substantial structural instability; all associated items except one exhibited significant cross-loadings (> .30) on earlier factors or failed to demonstrate conceptual distinctiveness. When cross-loading items were systematically removed based on statistical and theoretical criteria, only a single isolated item remained on Factor 4. Consequently, the fourth factor was discarded, and a streamlined 15-item, three-factor structure was retained for cross-validation.

Confirmatory Factor Analysis (Study IV)

In Study IV, the 15-item provisional scale was submitted to Confirmatory Factor Analysis (CFA) using maximum likelihood estimation in an independent sample of N = 301 emergency service commanders. During initial model specification, three items exhibited inadequate standardized factor loadings (λ ≤ 0.50) and elevated unique error variances (ε > 0.70). These three underperforming items were pruned, yielding the definitive 12-item FIRE-P instrument.

The resulting 12-item three-factor latent model demonstrated an acceptable-to-good fit to the empirical data across global and incremental goodness-of-fit indices, confirming that the latent variables Preparation, Design, and Load appropriately represent the multi-faceted evaluation construct. Standardized factor loadings (λ) and item-total correlations across the validation studies confirmed robust item-trait relationships, as summarized in the psychometric tables below.

Factor Loadings and Psychometric Properties

The statistical characteristics of the 12 core items across Study III (EFA, N = 428) and Study IV (CFA, N = 301) demonstrate consistent factor saturation:

• Preparation Subscale (F1):
  • Item 1: Study III M = 4.97, SD = 1.45, r_it = .77, λ = .81; Study IV M = 5.82, SD = 1.24, λ = .79.
  • Item 2: Study III M = 3.96, SD = 1.82, r_it = .60, λ = .77; Study IV M = 4.75, SD = 1.73, λ = .62.
  • Item 3: Study III M = 4.50, SD = 1.52, r_it = .70, λ = .79; Study IV M = 5.46, SD = 1.36, λ = .77.
  • Item 8: Study III M = 5.17, SD = 1.30, r_it = .70, λ = .51; Study IV M = 5.86, SD = 1.09, λ = .80.
• Design Subscale (F2):
  • Item 4: Study III M = 3.66, SD = 1.78, r_it = .67, λ = .68; Study IV M = 5.00, SD = 1.56, λ = .78.
  • Item 5: Study III M = 5.01, SD = 1.54, r_it = .58, λ = .64; Study IV M = 5.46, SD = 1.38, λ = .79.
  • Item 6: Study III M = 4.36, SD = 1.70, r_it = .76, λ = .81; Study IV M = 5.19, SD = 1.38, λ = .75.
  • Item 7: Study III M = 5.55, SD = 1.16, r_it = .56, λ = .61; Study IV M = 5.76, SD = 1.16, λ = .80.
  • Item 9: Study III M = 5.21, SD = 1.28, r_it = .49, λ = .47; Study IV M = 5.60, SD = 1.37, λ = .64.
• Load Subscale (F3):
  • Item 10: Study III M = 4.46, SD = 1.44, r_it = .59, λ = .72; Study IV M = 2.95, SD = 1.48, λ = .76.
  • Item 11: Study III M = 4.40, SD = 1.45, r_it = .66, λ = .78; Study IV M = 1.05, SD = 1.50, λ = .74.
  • Item 12: Study III M = 4.76, SD = 1.57, r_it = .61, λ = .67; Study IV M = 2.71, SD = 1.57, λ = .66.

Inter-Scale Correlations

The latent factors exhibit meaningful inter-correlations. In Study III (above diagonal, N = 428) and Study IV (below diagonal, N = 301), with Load items recoded such that higher scores represent lower subjective burden:

• Preparation – Design: r = .62 (Study III), r = .86 (Study IV), p < .01.
• Preparation – Load (recoded): r = .48 (Study III), r = .56 (Study IV), p < .01.
• Design – Load (recoded): r = .47 (Study III), r = .62 (Study IV), p < .01.

Instrument / Measurement Tool

The FIRE-P is a standardized paper-and-pencil or digital evaluation questionnaire designed for post-examination administration. Its structural specifications are organized as follows:

• Test Type: Multi-dimensional post-examination course evaluation inventory / institutional quality assurance tool.
• Target Population: Candidates and officers completing formal written qualification examinations in emergency medical, rescue, and fire service training academies (specifically group commanders, platoon commanders, and related incident leadership echelons).
• Item Inventory:
  • Mandatory Core Inventory: 12 closed-format rating statements measuring three validated latent dimensions.
  • Optional Contextual Inventory: 12 supplementary items (Items 13–24) capturing environmental framework conditions (5 items), bias-control variables (4 items), qualitative feedback/post-exam debriefing needs (2 items), and a global evaluative grade (1 item).
• Response Formats:
  • Core Items (1–12) & Optional Items (13–15, 18–22): Siebenstufiges Antwortformat mit den Optionen 1 = stimme gar nicht zu, 2 = stimme nicht zu, 3 = stimme eher nicht zu, 4 = neutral, 5 = stimme eher zu, 6 = stimme zu, 7 = stimme vollkommen zu. Ein zusätzliches Feld bietet die Möglichkeit anzugeben, dass das jeweilige Item nicht sinnvoll beantwortbar sei.
  • Percentage Estimation (Item 16): Open numerical percentage estimate (0%–100%).
  • Dichotomous Query (Item 17): Binary response format (yes / no [ja / nein]).
  • Qualitative Free-Text (Item 23): Open qualitative response box for constructive commentary on strengths and specific areas for instructional/test improvement.
  • Global Mark (Item 24): Standard German six-point academic grading scale: 1 = sehr gut (very good), 2 = gut (good), 3 = befriedigend (satisfactory), 4 = ausreichend (sufficient/pass), 5 = mangelhaft (deficient/fail), 6 = ungenügend (insufficient/complete fail).
• Scoring and Computational Rules:
  • Subscale scores are calculated as the arithmetic mean of all completed items within that dimension:
  • Scale 1: Preparation (Vorbereitung): Mean of Items 1, 2, 3, and 8. Higher scores (approaching 7.0) indicate superior alignment, high instructional clarity, and outstanding test preparation.
  • Scale 2: Design (Gestaltung): Mean of Items 4, 5, 6, 7, and 9. Higher scores (approaching 7.0) indicate high syntactic precision, intuitive formatting, and practical operational relevance.
  • Scale 3: Perceived Load (Belastung): Mean of Items 10, 11, and 12. In the raw non-recoded format, higher values indicate excessive difficulty and cognitive overload; optimal educational practice targets low to moderate values (e.g., ≤ 3.5). For correlational or factor analytic comparisons, items may be reverse-coded (8 − score) such that higher scores represent lower subjective burden.
  • Missing Value Protocols: Responses marked as “nicht sinnvoll beantwortbar” (“not answerable in a meaningful manner”) are coded as missing data. In digital surveys, Items 1–12 should be administered as mandatory fields. In paper-and-pencil surveys, any respondent protocol with three or more missing items across the 12 core items should be excluded from formal psychometric analyses.
  • Optional Items Interpretation: Items 13 through 24 are evaluated descriptively as individual diagnostic checklist markers rather than aggregated latent scales. Qualitative comments from Item 23 should be reviewed systematically via inductive categorization when recurring themes emerge across candidates.

Permissions & Fee and Test Year

The FIRE-P scale development was conducted between 2016 and 2017 through research by Busjan (2016) and Frerichs (2017) under the academic supervision of Prof. Dr. Meinald T. Thielsch at the University of Münster in institutional partnership with the Institut der Feuerwehr Nordrhein-Westfalen. The tool was officially validated and published as an open-access scientific and organizational diagnostic instrument in 2017.

Licensing and Accessibility: The scale items, scoring rubrics, and structural parameters are published for non-commercial educational, scientific, and quality-assurance purposes. Emergency service training institutions, fire academies, and public safety organizations are permitted to adapt and administer the instrument free of charge, provided appropriate academic attribution and citation are given to the original researchers and developing institutions. Deployment of the instrument within proprietary commercial software systems or non-academic consulting frameworks requires explicit written authorization from the authors.

References

Below is the scholarly literature documenting the development, conceptual foundation, and empirical validation of the FIRE-P and its related evaluation methodologies:

• Bechler, K., & Thielsch, M. T. (2012). Evaluation von Studium und Lehre: Ein Praxisleitfaden zur Erstellung von Fragebögen [Evaluation of studies and teaching: A practical guide for creating questionnaires]. Münster: WWU Münster.
• Beierlein, C., Kovaleva, A., Kemper, C. J., & Rammstedt, B. (2014). Ein Messinstrument zur Erfassung subjektiver Kompetenzerwartungen: Allgemeine Selbstwirksamkeit Kurzskala (ASKU) [An instrument for measuring subjective competence expectations: General Self-Efficacy Short Scale]. GESIS Working Papers, 2014/17. Mannheim: GESIS. https://doi.org/10.12758/mda.2014.011
• Busjan, N. (2016). Entwicklung und Validierung eines Fragebogens zur Evaluation schriftlicher Prüfungen an der Feuerwehrschule [Development and validation of a questionnaire for evaluating written examinations at the fire service academy] (Unpublished master’s thesis). Westfälische Wilhelms-Universität Münster, Germany.
• Colquitt, J. A., LePine, J. A., & Noe, R. A. (2000). Toward an integrative theory of training motivation: A meta-analytic path analysis of 20 years of research. Journal of Applied Psychology, 85(5), 678–707. https://doi.org/10.1037/0021-9010.85.5.678
• Colquitt, J. A., & Simmering, M. J. (1998). Conscientiousness, goal orientation, and motivation to learn during the learning process: A longitudinal-study. Journal of Applied Psychology, 83(4), 654–665. https://doi.org/10.1037/0021-9010.83.4.654
• Felfe, J., Six, B., Schmook, R., & Knorz, C. (2014). Commitment Organisation, Beruf und Beschäftigungsform (COBB) [Commitment to organization, profession, and employment form]. In D. Danner & A. Glöckner-Rist (Eds.), Zusammenstellung sozialwissenschaftlicher Items und Skalen (ZIS). https://doi.org/10.6102/zis52
• Feuerwehr-Dienstvorschrift 100. (1999). Führung und Leitung im Einsatz (FwDV 100) [Leadership and command in operations]. Ausschuss Feuerwehrangelegenheiten, Katastrophenschutz und zivile Verteidigung (AFKzV). Stuttgart: Kohlhammer.
• Feuerwehr-Dienstvorschrift 3. (2008). Einheiten im Lösch- und Hilfeleistungseinsatz (FwDV 3) [Units in firefighting and rescue operations]. Ausschuss Feuerwehrangelegenheiten, Katastrophenschutz und zivile Verteidigung (AFKzV). Stuttgart: Kohlhammer.
• Frerichs, L. (2017). Konstruktvalidierung eines Instruments zur Evaluation von schriftlichen Prüfungen im Bereich der Rettungskräfteentwicklung [Construct validation of an instrument for the evaluation of written examinations in the field of rescue force development] (Unpublished master’s thesis). Westfälische Wilhelms-Universität Münster, Germany.
• Froncek, B., Hirschfeld, G., & Thielsch, M. T. (2014). Münsteraner Fragebogen zur Evaluation von Klausuren (MFE-K) [Münster questionnaire for the evaluation of written examinations]. Zeitschrift für Hochschulentwicklung, 9(2), 115–138. https://doi.org/10.3217/zfhe-9-02/10
• Froncek, B., & Thielsch, M. T. (2014). Qualität von Prüfungen: Was zeichnet eine gute Klausur aus? [Quality of examinations: What characterizes a good written exam?]. Das Hochschulwesen, 62(1+2), 48–54.
• Hagemann, V. (2011). Teamtraining in High-Risk-Organisationen [Team training in high-risk organizations]. Frankfurt am Main: Peter Lang.
• Jäger, R. S. (2004). Von der Beobachtung zur Notengebung: Ein Lehrbuch der schulischen Leistungsbeurteilung [From observation to grading: A textbook on academic performance assessment] (4th ed.). Landau: Verlag Empirische Pädagogik.
• Jakoby, N., & Jacob, R. (2014). Messung von internen und externen Kontrollüberzeugungen [Measurement of internal and external locus of control]. In D. Danner & A. Glöckner-Rist (Eds.), Zusammenstellung sozialwissenschaftlicher Items und Skalen (ZIS). https://doi.org/10.6102/zis63
• Kemper, C. J., Beierlein, C., Kovaleva, A., & Rammstedt, B. (2014). Skala Optimismus-Pessimismus-2 (SOP2) [Optimism-Pessimism Scale-2]. GESIS Working Papers, 2014/15. Mannheim: GESIS. https://doi.org/10.12758/mda.2013.003
• Lee, S. H., & Olshfski, D. (2002). Employee commitment and performance: Testing a pledge-to-perform model with fire fighters. Public Productivity & Management Review, 25(3), 269–284. https://doi.org/10.2307/3381203
• Martocchio, J. J., & Judge, T. A. (1997). Relationship between conscientiousness and self-deception and self-efficacy during a training program. Journal of Applied Psychology, 82(5), 764–773. https://doi.org/10.1037/0021-9010.82.5.764
• Modick, H. (2014). Leistungshemmende und leistungsfördernde Prüfungsangst [Performance-inhibiting and performance-enhancing test anxiety]. In D. Danner & A. Glöckner-Rist (Eds.), Zusammenstellung sozialwissenschaftlicher Items und Skalen (ZIS). https://doi.org/10.6102/zis88
• Mummendey, H. D., & Grau, I. (2014). Die Fragebogen-Methode [The questionnaire method] (6th ed.). Göttingen: Hogrefe.
• Noe, R. A. (1986). Trainees’ attributes and attitudes: Neglected influences on training effectiveness. Academy of Management Review, 11(4), 736–749. https://doi.org/10.5465/amr.1986.4283922
• Richardson, M., Abraham, C., & Bond, R. (2012). Psychological correlates of university students’ academic performance: A systematic review and meta-analysis. Psychological Bulletin, 138(2), 353–387. https://doi.org/10.1037/a0026838
• Sarubin, N., Gutt, D., Giegling, I., Bühner, M., Hermisson, I., Köhler, A., & Rujescu, D. (2015). Erste Überprüfung der psychometrischen Eigenschaften der deutschsprachigen Kurzversion der Connor-Davidson Resilience Scale (CD-RISC-10) [First evaluation of the psychometric properties of the German short version of the Connor-Davidson Resilience Scale (CD-RISC-10)]. Zeitschrift für Klinische Psychologie und Psychotherapie, 44(3), 209–221. https://doi.org/10.1026/1616-3443/a000323
• Scheier, M. F., & Carver, C. S. (1985). Optimism, coping, and health: Assessment and implications of generalized outcome expectancies. Health Psychology, 4(3), 219–247. https://doi.org/10.1037/0278-6133.4.3.219
• Schuler, H., & Prochaska, M. (2001). Leistungsmotivationsinventar (LMI) [Achievement Motivation Inventory]. Göttingen: Hogrefe.
• Schulte, F. P., & Thielsch, M. T. (in press). Kompetenzen im Katastrophenschutz: Anforderungen, Ausbildung und Prüfungen [Competencies in civil protection: Requirements, training, and examinations]. Bevölkerungsschutz.
• Schupp, J., & Gerlitz, J.-Y. (2014). Big Five Inventory-SOEP (BFI-S). In D. Danner & A. Glöckner-Rist (Eds.), Zusammenstellung sozialwissenschaftlicher Items und Skalen (ZIS). https://doi.org/10.6102/zis54
• Smith, T. D., & Dyal, M. A. (2016). Stressors, work-family conflict, and burnout in emergency medical technicians and paramedics: A cross-sectional study. International Journal of Stress Management, 23(4), 384–404. https://doi.org/10.1037/str0000019
• Spooren, P., Brockx, B., & Mortelmans, D. (2013). On the validity of student evaluation of teaching: The state of the art. Review of Educational Research, 83(4), 598–642. https://doi.org/10.3102/0034654313496870
• Wachtel, H. K. (1998). Student evaluation of college teaching effectiveness: A brief review. Assessment & Evaluation in Higher Education, 23(2), 191–212. https://doi.org/10.1080/0260293980230207
• West, S. G., Finch, J. F., & Curran, P. J. (1995). Structural equation models with nonnormal variables: Problems and remedies. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 56–75). Thousand Oaks, CA: Sage Publications.
• Young, K. M., & Cooper, C. L. (1995). Occupational stress in the ambulance service: A review of the literature. Stress Medicine, 11(2), 105–113. https://doi.org/10.1002/smi.2460110204

Items of the Scale