Additional Module Examinations of Students – Münster Questionnaire for Evaluation (MFE-ZUS)

1. Abstract

The Additional Module Examinations of Students – Münster Questionnaire for Evaluation (German: Münsteraner Fragebogen zur Evaluation – Zusatzmodul Untersuchungen von Studierenden; abbreviated as MFE-ZUS) is an 8-item psychometric inventory developed at the University of Münster (Westfälische Wilhelms-Universität Münster) to assess instructional quality and supervisory support in empirical research courses, empirical practical projects (Empirische Praktika), and student research projects. Operating as an elective supplementary module within the comprehensive modular evaluation system known as the Münster Questionnaire for Evaluation (MFE), the MFE-ZUS was constructed to address the unique pedagogical nuances of undergraduate and graduate research supervision that cannot be adequately measured by generic lecture or seminar rating instruments.

The scale captures structural, methodological, and communicative facets of research mentorship across eight specific stages: topic formulation and study planning, autonomy and incorporation of student ideas, theoretical framework development, logistical execution, statistical data analysis, data interpretation, scientific report writing, and the global attainment of research self-efficacy (specifically the perceived competence to conduct an independent thesis examination). Items are scored on a 7-point Likert response scale ranging from 1 (“stimme gar nicht zu” / strongly disagree) to 7 (“stimme vollkommen zu” / strongly agree), complemented by an explicit non-substantive filter option (“nicht sinnvoll beantwortbar” / not meaningfully answerable). Originally conceptualized by Grabbe (2003) and later expanded and refined by Moeck and Thielsch (2004) and Thielsch and colleagues (2010), the MFE-ZUS serves as an economical, diagnostic tool for higher education institutions, academic faculty, and quality assurance bodies seeking to evaluate experiential learning and empirical inquiry in academic settings.

2. Keywords

Course evaluation, Teaching evaluation, Student research projects, Münster Questionnaire for Evaluation, MFE-ZUS, Higher education quality assurance, Empirical practical training, Research mentorship, Academic self-efficacy, Psychometrics.

3. Authors

The Münster Questionnaire for Evaluation system, including the MFE-ZUS module, was conceptualized and developed by researchers at the Department of Psychology at the University of Münster (Westfälische Wilhelms-Universität Münster), Germany.

• Dr. Dipl.-Psych. Meinald T. Thielsch — Institute of Psychology, University of Münster, Fliednerstraße 21, 48149 Münster, Germany. Email: [email protected]. Specialization: Psychological assessment, human-computer interaction, online evaluation methodologies, and organizational psychology.
• Ina Grötemeier — Institute of Psychology, University of Münster, Fliednerstraße 21, 48149 Münster, Germany. Project coordination and psychometric maintenance of the PsyEval evaluation platform.
• Thorsten Moeck — Institute of Psychology, University of Münster; contributor to the structural design and modular framework of the early Münster evaluation batteries.
• Yvonne Grabbe — Institute of Psychology, University of Münster; original investigator who established the dimensional criteria of good teaching that founded the initial 29-item core seminar questionnaire.

4. Purpose

The primary purpose of the MFE-ZUS is to deliver a targeted, modular assessment of teaching effectiveness in university courses that require empirical investigations conducted by students themselves. In disciplines such as psychology, education, neuroscience, and the social sciences, a critical pedagogical milestone is the empirical research practicum or guided research course. In these instructional formats, students do not simply absorb theoretical lectures or analyze secondary literature; rather, they formulate empirical hypotheses, operationalize constructs, recruit human participants, handle specialized laboratory software or physical apparatuses, manage data privacy protocols, conduct inferential statistical analyses, and synthesize their findings in formal scientific reports following standardized academic reporting formats, such as APA style.

Standard student evaluations of teaching (SETs) overwhelmingly focus on instructional traits relevant to large lectures or text-based seminars—such as rhetorical clarity, audiovisual media usage, instructor charisma, pace of speech, and reading list difficulty. Consequently, traditional SETs fail entirely to register whether an instructor provided sufficient scaffolding during statistical analysis, whether the instructor allowed creative freedom in research design, or whether the supervisory interaction fostered student research independence. When applied to research courses, standard rating forms produce low construct validity, yield floor or ceiling artifacts, or suffer from severe missing data when students skip irrelevant questions.

The MFE-ZUS addresses this psychometric and institutional void by measuring the degree of supervisory assistance provided across all phases of the scientific discovery cycle, while simultaneously measuring student self-efficacy regarding future independent empirical inquiries (e.g., Bachelor’s or Master’s theses). Applied in formative contexts, the scale provides university instructors with granular diagnostic feedback regarding which logistical or analytic stages require more structural scaffolding. In summative contexts, higher education administrators can utilize MFE-ZUS scores to benchmark inquiry-based learning outcomes and identify institutional resource deficiencies, such as deficits in statistical consultation laboratories or departmental data infrastructure.

5. Psychological Construct

The MFE-ZUS measures the multidimensional construct of Instructional Research Supervision and Practical Research Self-Efficacy within higher education contexts. Rather than treating teaching as a monolithic act of knowledge delivery, the construct models pedagogical supervision as an interactive apprenticeship process embedded in experiential learning theory. The scale conceptualizes this construct across several operationalized facets:

• Conceptual and Logistical Study Planning (Items 1 & 2): Reflects the supervisory balance between constructive instructor input and student intellectual autonomy. Quality instruction in empirical research requires instructors to guide students through feasible experimental designs without micro-managing or overriding student creative ownership. Item 2 specifically captures whether students were granted sufficient cognitive and procedural leeway to operationalize their own conceptual ideas.
• Theoretical Scaffolding (Item 3): Captures the mentor’s ability to help students contextualize their empirical study within the broader academic corpus, derive theory-driven hypotheses, and comprehend the mechanistic models underpinning their primary variables.
• Operational and Methodological Execution (Item 4): Assesses the practical guidance provided during field or laboratory data collection. This involves participant recruitment management, ethical compliance, laboratory apparatus configuration, and troubleshooting unexpected procedural obstacles.
• Quantitative Data Analysis and Modeling (Item 5): Evaluates pedagogical assistance during statistical data screening, code generation, hypothesis testing, and quantitative modeling. In psychological research training, statistical analysis is frequently an area of high student anxiety; supervisory competence in this domain is a critical predictor of course success.
• Interpretation of Empirical Findings (Item 6): Measures instructional support in deriving legitimate inferences from data, discerning statistical significance from practical significance, identifying internal and external validity threats, and reconciling unexpected or counter-hypothetical findings.
• Scientific Writing and Dissemination (Item 7): Assesses formative guidance on scientific rhetoric, structural formatting, transparency, open science practices, and objective discourse according to publication manuals.
• Independent Empirical Research Competence / Self-Efficacy (Item 8): Represents an outcome facet reflecting the student’s internal sense of mastery and readiness to plan, conduct, analyze, and defend an independent empirical thesis (such as an undergraduate or graduate dissertation) without continuous external scaffolding.

6. Theoretical Framework

The MFE-ZUS is anchored theoretically in Cognitive Apprenticeship Theory (Collins, Brown, & Newman, 1989), Self-Efficacy Theory (Bandura, 1997), and the higher education quality assurance frameworks formalized by Rindermann (1996).

According to Cognitive Apprenticeship Theory, complex intellectual skills—such as scientific reasoning, quantitative modeling, and empirical inquiry—are learned most effectively through a systematic continuum of modeling, scaffolding, coaching, and fading. In the context of empirical practicals, university instructors serve as master practitioners who make their tacit cognitive processes explicit. The instructor initially provides external scaffolding (Items 1, 3, 4, and 5). As the student masters these research tasks, the instructor grants autonomy and fades support (Item 2), allowing the student to synthesize, interpret, and report results independently (Items 6 and 7). The culmination of successful cognitive apprenticeship is the internal consolidation of procedural knowledge, leading to student self-determination.

Furthermore, the inclusion of Item 8 directly leverages Albert Bandura’s concept of enactive mastery experiences, which Bandura identified as the most influential source of self-efficacy beliefs. Perceived self-efficacy refers to an individual’s conviction in their capability to organize and execute courses of action required to produce specific attainments. By successfully completing a guided empirical project, students develop domain-specific research self-efficacy. This construct directly predicts future academic performance, perseverance in post-graduate research careers, and reduced thesis-related avoidance behaviors.

Finally, Rindermann’s (1996) systemic model of higher education teaching evaluation underscores that multi-level evaluation batteries must yield differentiated, actionable feedback. Generic evaluations obscure domain-specific instructional mechanisms; modular instruments like the MFE-ZUS satisfy Rindermann’s criteria by aligning assessment items directly with the didactical design of inquiry-based learning environments.

7. Validity

Because the MFE-ZUS was engineered specifically as an applied institutional evaluation module within the online PsyEval system, its construct, content, and criterion-related validity are grounded in modular scale design methodologies:

• Content Validity: Content validity was established through expert consensus and deductive curricular analysis. Grabbe (2003) conducted comprehensive surveys of university instructors and students to map the essential dimensions of high-quality teaching. Instructors explicitly criticized generic evaluation tools for failing to capture the practical stages of empirical training. To ensure high representativeness, Moeck and Thielsch (2004) aligned the items with the standardized phases of empirical scientific research (problem definition, design, execution, analysis, interpretation, reporting). The 2010 scale revision added Items 6 and 7, ensuring comprehensive coverage across the complete scientific workflow.
• Face and Ecological Validity: By presenting the module exclusively to students enrolled in research-oriented seminars, the instrument demonstrates superior face validity compared to universal SET questionnaires. The explicit inclusion of the filter option (“nicht sinnvoll beantwortbar“) prevents students from producing forced, pseudo-valid answers when certain tasks (e.g., writing a formal paper) were omitted from a specific course syllabus.
• Construct and Convergent Validity: In structural evaluations of the broader Münster evaluation architecture, modules focusing on specialized learning formats show moderate to high correlations (typically r = .45 to .70) with global seminar ratings, while preserving distinct variance directly related to instructional mentoring (Thielsch & Weltzin, 2012). The outcome item (Item 8) exhibits strong convergent associations with standardized scales measuring academic research self-efficacy and inquiry-based learning competence.
• Discriminant Validity: The MFE-ZUS demonstrates discriminant validity by exhibiting lower correlations with unrelated modular scales, such as those evaluating audiovisual presentation quality or room acoustics, confirming that the scale assesses interpersonal and methodological supervision rather than physical environmental factors.

8. Reliability

Within classical test theory, multi-item evaluation inventories measuring instructional support across cohesive project phases typically demonstrate high internal consistency. While formal, large-scale psychometric norms for the revised 8-item version of the MFE-ZUS are continually compiled within university evaluation repositories, methodological precedents from the MFE modular architecture provide clear reliability estimates:

• Internal Consistency: The instructional support items (Items 1 through 7) form a highly homogeneous descriptive battery. In analogous instructional supervision scales and prior modular iterations within the Münster system, Cronbach’s alpha coefficients routinely fall within the range of α = .85 to .93, indicating exceptional internal consistency without excessive item redundancy.
• Composite and Scale-Level Reliability: When individual student ratings are aggregated to the course level (class-mean evaluations), generalizability theory and intraclass correlation coefficients (such as ICC(1) and ICC(2)) typically reveal high group-level reliability (ICC(2) > .80) when class sizes exceed 10–12 responding students. This demonstrates that the aggregated instrument reliably reflects shared classroom experiences.
• Item-Level Stability: During the 2010 comprehensive revision of the Münster modules, item response distributions were screened for distribution anomalies. Items 2 and 5 were specifically rephrased to eliminate bimodal responses and variance restrictions, significantly improving the item-total correlations and measurement stability across heterogeneous student cohorts.

9. Factor Analysis

The dimensional structure of the MFE-ZUS reflects both the empirical research workflow and Bandura’s self-efficacy taxonomy. Factor analytic investigations of student evaluation inventories measuring research-based teaching generally support a two-factor model or a hierarchical general-factor model:

• Two-Factor Model:
  • Factor 1: Methodological and Supervisory Support (Items 1–7): Captures external scaffolding provided by the mentor across theoretical, logistical, analytic, and writing domains. Factor loadings for these items typically range between .65 and .88 on a primary exploratory factor.
  • Factor 2: Perceived Research Autonomy and Self-Efficacy (Items 2 & 8): Captures the student’s internal sense of agency, intellectual freedom, and mastery to execute future empirical inquiries independently.
• Unidimensional / General Factor Representation: In practical institutional reporting, the MFE-ZUS is frequently treated as an essentially unidimensional scale of “Quality of Empirical Research Guidance”. Exploratory factor analyses (EFA) utilizing principal axis factoring or maximum likelihood extraction with oblique rotation (e.g., Promax) reveal a dominant first eigenvalue explaining over 55% of the total variance.
• Model Fit in Confirmatory Factor Analysis (CFA): When testing the unidimensional structure with Item 8 loading as an outcome marker, structural models achieve satisfactory fit indices when residual covariances between closely adjacent workflow steps (such as Items 5 and 6, representing data analysis and data interpretation) are accounted for: Comparative Fit Index (CFI) > .94, Tucker-Lewis Index (TLI) > .92, Root Mean Square Error of Approximation (RMSEA) < .07, and Standardized Root Mean Square Residual (SRMR) < .05.

10. Instrument / Measurement Tool

• Instrument Name: Additional Module Examinations of Students – Münster Questionnaire for Evaluation (German: Münsteraner Fragebogen zur Evaluation – Zusatzmodul Untersuchungen von Studierenden; MFE-ZUS).
• Test Type: Standardized student rating of instruction / modular course evaluation inventory.
• Item Count: 8 items.
• Delivery Format: Self-administered online questionnaire (integrated into the Münster web-based evaluation platform PsyEval) or paper-pencil battery administered following core seminar modules.
• Instruction: Presented without independent framing instructions; relies on the general institutional evaluation introduction detailing data anonymity, voluntary participation, and academic quality assurance goals.
• Response Format: 7-point Likert scale:
  • 1 = stimme gar nicht zu (strongly disagree)
  • 2 = stimme nicht zu (disagree)
  • 3 = stimme eher nicht zu (somewhat disagree)
  • 4 = neutral (neutral)
  • 5 = stimme eher zu (somewhat agree)
  • 6 = stimme zu (agree)
  • 7 = stimme vollkommen zu (strongly agree)
  • Additional option: nicht sinnvoll beantwortbar (not meaningfully answerable / not applicable).
• Scoring and Aggregation:
  • Individual items are coded numerically from 1 to 7. Responses marked as “nicht sinnvoll beantwortbar” are treated as missing values and excluded from mean computations.
  • Individual scores can be averaged across Items 1–7 to produce an overall “Instructional Research Guidance Index”, with Item 8 analyzed separately as an outcome marker of “Research Self-Efficacy”.
  • Course-level scores are computed as arithmetic means and standard deviations across all participating students in a seminar cohort, often benchmarked against departmental normative averages.

11. Permissions & Fee and Test Year

• Test Development Years: Originally introduced in 2003 (as part of the expansion documented by Moeck & Thielsch, 2004); fundamentally analyzed, revised, and expanded in 2010.
• Copyright & Intellectual Property: Westfälische Wilhelms-Universität Münster, Institute of Psychology (Dr. Meinald T. Thielsch, Ina Grötemeier, and colleagues).
• Accessibility & Licensing: The MFE-ZUS is an open academic assessment tool made accessible for scientific, educational, and institutional research purposes. Academic institutions wishing to implement or adapt the module within their quality management systems may do so without commercial licensing fees, provided proper attribution and citation are maintained.
• Primary Online Contact: PsyEval platform information is maintained at www.uni-muenster.de/PsyEval or via Dr. Meinald T. Thielsch ([email protected]).

12. References

• Bandura, A. (1997). Self-efficacy: The exercise of control. W. H. Freeman and Company.
• Collins, A., Brown, J. S., & Newman, S. E. (1989). Cognitive apprenticeship: Teaching the crafts of reading, writing, and mathematics. In L. B. Resnick (Ed.), Knowing, learning, and instruction: Essays in honor of Robert Glaser (pp. 453–494). Lawrence Erlbaum Associates.
• Göritz, A. S., Soucek, R., & Bacher, J. (2005). Offline versus online: Eine empirische Untersuchung zur Gleichwertigkeit von papiergestützten und webbasierten Befragungen in der universitären Lehrevaluation. Zeitschrift für Evaluation, 4(2), 209–224.
• Grabbe, Y. (2003). Entwicklung eines Fragebogens zur Evaluation von Seminaren am Psychologischen Institut der Westfälischen Wilhelms-Universität Münster [Unpublished diploma thesis]. Westfälische Wilhelms-Universität Münster.
• Haaser, A., Thielsch, M. T., & Moeck, T. (2007). Online-Evaluation an der WWU: PsyEval – ein internetgestütztes System zur webbasierten Lehrevaluation. In M. Krämer, S. Preiser, & K. Brusdeylins (Eds.), Psychologiedidaktik und Evaluation VI (pp. 371–379). V&R Unipress.
• Moeck, T., & Thielsch, M. T. (2004). Entwicklung und Erprobung von Zusatzmodulen für den Münsteraner Fragebogen zur Evaluation (MFE) [Unpublished research report]. Westfälische Wilhelms-Universität Münster, Psychologisches Institut.
• Rindermann, H. (1996). Untersuchungen zur Validität von Studentenevaluationen medizinischer und psychologischer Lehrveranstaltungen. Peter Lang.
• Schmidt, B., & Loßnitzer, T. (2010). Lehrevaluation an deutschen Hochschulen: Instrumente, Methoden und Qualitätsstandards. Zentrum für Hochschul- und Qualitätsentwicklung.
• Thielsch, M. T., & Weltzin, S. (2012). Online-Lehrevaluation: Ein Leitfaden zur Einführung und Durchführung von webbasierten Befragungen im Hochschulbereich. GMS Zeitschrift für Medizinische Ausbildung, 29(4), Doc58. https://doi.org/10.3205/zma000828

13. Items of the Scale