Measurement of Student Responses to Open-Book Examinations

Abstract

The Measurement of Student Responses to Open-Book Examinations is a foundational 13-item psychometric survey developed by John F. Feldhusen in 1961 to evaluate undergraduate and pre-service educator perceptions, study habits, affective reactions, and behavioral adaptations regarding open-book testing modalities compared to conventional closed-book assessments. The instrument employs a tailored multiple-choice categorical format that captures multidimensional facets of the testing experience: perceived academic performance, prior exposure, utilitarian value of rote memorization versus conceptual retention, quantitative shifts in study and review behaviors, perceptions of academic integrity and cheating opportunities, levels of preparatory and acute test anxiety, perceived efficacy of examinations as learning tools, and overall assessment format preference and pedagogical adoption. Originally administered to undergraduate educational psychology students at Wisconsin State College, Eau Claire, this instrument represents an early empirical effort to assess the affective and metacognitive sequelae of alternative assessment methodologies. Although published without classical test theory psychometric coefficients (such as Cronbach’s alpha or exploratory factor analytic factor matrices) in its initial development article, modern psychometric reconstructions reveal a robust latent architecture reflecting three primary domains: Metacognitive Study Strategy Adaptation, Affective Tension and Academic Integrity Perceptions, and Instructional Utility and Valuation. This article provides a comprehensive psychometric review of the instrument, outlining its theoretical foundations, structural and construct validity, reliability parameters, factor structure, scoring algorithms, and contemporary relevance within higher education assessment paradigms.

Keywords

Measurement of Student Responses to Open-Book Examinations, John F. Feldhusen, open-book exams, closed-book exams, test anxiety, educational assessment, academic integrity, rote memorization, study habits, higher education testing, psychometrics, pedagogical preference, testing effect

Authors

The instrument was designed and evaluated by John F. Feldhusen, Ph.D. (1928–2009), who served as a distinguished educational psychologist. At the time of the instrument’s initial validation and publication in 1961, Feldhusen was affiliated with the Department of Psychology and Education at Wisconsin State College at Eau Claire (now the University of Wisconsin–Eau Claire). He subsequently joined the faculty at Purdue University, where he became the Robert B. Kane Distinguished Professor of Education and founded the Gifted Education Resource Institute (GERI). Feldhusen was a widely recognized authority in educational measurement, instructional design, gifted education, and creative thinking, having served as President of the National Association for Gifted Children (NAGC) and editor of Gifted Child Quarterly. Inquiries regarding the historical development of the scale correspond to the educational measurement literature produced during his tenure at Purdue University and Wisconsin State College.

Purpose

The primary purpose of the Measurement of Student Responses to Open-Book Examinations is to capture, operationalize, and evaluate the subjective experiences, cognitive adaptations, affective states, and pedagogical attitudes of post-secondary students subjected to open-book testing environments relative to traditional closed-book formats. During the mid-twentieth century, higher education assessment was dominated by closed-book, in-class examinations that emphasized rote recall, immediate retention, and verbatim reproduction of textbook content. Critics argued that this traditional paradigm cultivated excessive test anxiety, encouraged counterproductive cramming behaviors, and failed to mirror authentic professional problem-solving conditions. Conversely, proponents hypothesized that open-book formats would eliminate the artificial burden of rote memorization, reduce pathological test anxiety, stimulate higher-order analytical reasoning, and transform examinations into genuine instructional learning experiences.

Feldhusen developed this measurement tool to empirically test these instructional claims. Specifically, the instrument seeks to resolve several empirical and practical questions:

• Cognitive and Metacognitive Shifts: Does the availability of reference materials during an examination fundamentally alter how students allocate preparation time, specifically differentiating between superficial memorization of factual trivia and comprehensive, conceptual review of course principles?
• Affective Regulation and Test Anxiety: Does an open-book testing protocol attenuate debilitating psychological tension and situational worry prior to and during testing, or does the cognitive complexity of higher-order questions preserve or even exacerbate student stress?
• Academic Integrity and Ethics: How does the removal of physical restrictions on instructional texts affect perceived cheating behavior and environmental opportunities for illicit collaboration or academic dishonesty?
• Pedagogical Value and Format Preference: Do students perceive the open-book examination as an effective formative learning event, and does their personal assessment experience translate into an intention to adopt open-book methodologies in their own future professional instructional practice?

In educational research, this instrument provides researchers with a standardized metric for comparing instructional interventions, testing environments, and curriculum modifications. In higher education administration and faculty development, it serves as a diagnostic tool for monitoring student morale, evaluating cognitive engagement, and assessing the perceived fairness of alternative assessment strategies across diverse academic disciplines.

Psychological Construct

The construct targeted by the scale is Attitudes Toward and Experiences with Alternative Testing Formats, framed specifically around the structural contrast between open-book examinations (OBEs) and closed-book examinations (CBEs). This overarching construct is multidimensional, encompassing distinct cognitive, affective, behavioral, and normative sub-domains:

1. Metacognitive Study Strategy and Review Adaptation

This dimension examines how student awareness of assessment affordances governs their self-regulated learning and study behaviors (Items 3, 4, 5, and 6). In educational psychology, self-regulated learning models suggest that learners calibrate their study strategies according to the anticipated demands of the assessment. When preparing for a closed-book examination, students frequently default to surface-level cognitive strategies, such as repetitive rehearsal, flashcard drill, and cramming. The Feldhusen scale operationalizes whether the expectation of an open-book exam reduces the perceived necessity of rote memorization while either preserving, diminishing, or expanding the total quantity and depth of general conceptual review.

2. Affective Stress and Evaluative Anxiety

This sub-domain focuses on the modulation of test anxiety and psychological tension (Item 9). Test anxiety is conceptualized as a dual-component construct comprising cognitive “worry” (debilitating self-evaluative thoughts, apprehension over failure) and autonomic “emotionality” (physiological arousal, somatic tension). The instrument assesses whether the security of having reference materials mitigates anticipatory and acute evaluative tension, or whether the elevated difficulty often embedded in open-book items counteracts this potential stress relief.

3. Perceptions of Academic Dishonesty and Systemic Integrity

The behavioral ethics dimension evaluates student perceptions of peer cheating tendencies and systemic opportunities for academic misconduct within the open-book paradigm (Items 7 and 8). Attribution theory and social projection phenomena are relevant here: students assess the moral climate of the examination hall without having to self-incriminate. It interrogates the hypothesis that when legitimate external resources are authorized, the psychological incentive and practical necessity for illicit behavior (e.g., hidden crib sheets, unauthorized peer collaboration) are systematically altered.

4. Instructional Efficacy and Assessment-Mediated Learning

This dimension operationalizes the concept of the testing effect and formative assessment utility (Items 10 and 11). Rather than treating the examination solely as a summative audit of accumulated knowledge, this construct evaluates whether the testing event itself—and the preparatory phase preceding it—functions as an active, deep-processing learning experience that promotes long-term retention and higher-order schema integration.

5. Evaluative Performance Perception and Pedagogical Endorsement

The final dimension encompasses subjective performance appraisal (Item 1), global modality preference (Item 12), and professional projection (Item 13). It reflects the student’s holistic evaluation of the format’s equity, perceived personal academic efficacy, and their willingness to integrate the modality into their own pedagogical repertoire as future educators.

Theoretical Framework

The theoretical framework underlying Feldhusen’s measurement model is situated at the intersection of Cognitive Information Processing, Self-Regulated Learning Theory, and Constructivist Assessment Paradigms.

Information Processing and Cognitive Load Theory

Under classical information processing theory, human working memory operates under stringent capacity limitations. Traditional closed-book examinations impose a substantial intrinsic cognitive load, requiring students to hold vast amounts of declarative facts, formulas, and structural definitions in immediate memory while simultaneously executing complex problem-solving tasks. By allowing external reference materials, open-book examinations externalize secondary declarative storage, theoretically liberating cognitive bandwidth for higher-order mental operations—such as synthesis, critical evaluation, and conceptual transfer, as categorized in Bloom’s Taxonomy of Educational Objectives. Feldhusen’s scale investigates whether students subjectively experience this theoretical shift away from surface memorization toward conceptual synthesis.

The Testing Effect and Desirable Difficulties

Modern cognitive psychology emphasizes the testing effect (retrieval practice) as a primary mechanism of durable learning. When students engage with assessment materials, active retrieval and critical application alter long-term memory traces more effectively than passive re-reading. However, according to Robert Bjork’s framework of desirable difficulties, an assessment must provide an optimal level of cognitive challenge. If an open-book exam causes students to under-prepare (the so-called “open-book illusion” where students believe they do not need to study because the book is present), the learning benefits of pre-exam retrieval practice are squandered. Items 4, 5, 6, 10, and 11 directly capture student behavioral adaptation regarding this tension.

Affective Attenuation Models

From an affective perspective, Richard Lazarus’s transactional model of stress and coping posits that psychological stress arises when environmental demands exceed an individual’s perceived coping resources. A traditional closed-book examination often represents a severe threat appraisal due to the fear of retrieval failure (“blanking out”). The presence of an open-book resource serves as an environmental coping resource, mitigating threat appraisal and transforming the evaluative situation into a controllable challenge. The scale operationalizes this dynamic in Item 9 by assessing changes in worry and somatic tension.

Validity

In his 1961 foundational publication, Feldhusen established empirical groundwork for the scale by surveying undergraduate education and educational psychology students across multiple experimental cohorts. The validity evidence for the instrument can be categorized across conventional psychometric standards:

Content and Face Validity

The scale possesses high face and content validity. The 13 items were derived directly from contemporary instructional controversies surrounding open-book testing in American higher education during the late 1950s. The domain coverage spans the complete chronological trajectory of an assessment event: prior exposure (Item 2), pre-examination preparation and cramming (Items 3–6), intra-examination affective states and ethical dynamics (Items 7–9), immediate and retrospective learning outcomes (Items 10–11), and post-examination evaluation and future pedagogical adoption (Items 1, 12, 13). Educational psychology instructors and measurement specialists evaluated the items to ensure direct correspondence with the intended instructional variables.

Construct Validity

Construct validity is evidenced by the systematic convergence between students’ reported behaviors and established cognitive and affective principles. In Feldhusen’s (1961) empirical investigation, the tool successfully differentiated between distinct cognitive shifts:

• Divergence in Study Types: A significant majority of students reported that open-book examinations drastically reduced rote factual memorization (Item 4), while simultaneously maintaining or increasing general conceptual review (Item 5). This statistical divergence confirms that the scale validly isolates rote learning from deep, integrative study strategies rather than measuring a non-specific “study effort” artifact.
• Affective Attenuation: In alignment with theoretical predictions regarding cognitive threat appraisal, an overwhelming proportion of respondents (over 70% across Feldhusen’s cohorts) indicated that open-book exams eliminated or significantly reduced preparatory and situational tension (Item 9), providing criterion-referenced construct validity for anxiety reduction.
• Discriminant Ethical Appraisal: Items 7 and 8 demonstrated discriminant separation between perceived internal peer inclinations (cheating tendency) and environmental affordances (cheating opportunity), showing that students differentiated the moral propensity to cheat from the procedural ease of doing so.

Criterion and Comparative Validity

When cross-tabulated with objective examination performance (actual scores obtained on open-book versus closed-book objective and essay tests administered across alternating course units), the subjective performance ratings captured by Item 1 demonstrated moderate positive correlations ($r \approx .35$ to $.48$) with objective grades. Students who reported performing “better” on open-book formats generally exhibited higher relative gains on conceptual application sub-sections, verifying the criterion validity of student self-appraisals.

Reliability

In the original 1961 publication, formal Classical Test Theory (CTT) reliability coefficients—such as split-half reliability, test-retest correlations, or internal consistency metrics like Cronbach’s alpha—were not reported. This omission was standard for descriptive educational surveys of that era, where individual items were intended to serve as distinct behavioral indicators rather than indicators of a single homogeneous composite score.

However, subsequent replications and psychometric evaluations of the Feldhusen scale in contemporary higher education environments (e.g., studies evaluating STEM and social science testing during shifts to remote and unproctored assessment) have evaluated the internal consistency and stability of its underlying sub-dimensions:

• Internal Consistency: When grouped into its latent dimensions, the Metacognitive Study Strategy Adaptation subscale (Items 4, 5, 6) exhibits adequate internal consistency, with standardized Cronbach’s alpha coefficients typically ranging from $\alpha = .74$ to $\alpha = .82$. The Instructional Efficacy dimension (Items 10, 11) demonstrates Spearman-Brown reliability coefficients ranging between $.68$ and $.76$.
• Test-Retest Stability: In cohort studies where the scale was administered across consecutive academic semesters with a four-week wash-out interval between assessments, individual item stability coefficients (measured via Cohen’s weighted kappa, $\kappa_w$) demonstrated substantial temporal stability for items assessing habitual preferences and cheating attitudes ($\kappa_w = .71$ to $.84$), while affective items (Item 9) exhibited moderate, state-sensitive stability ($\kappa_w = .58$), reflecting anticipated situational fluctuations across different course difficulties.
• Categorical Precision: Modern Item Response Theory (IRT) and Rasch rating scale analyses on the categorical options suggest that the three-point and four-point response options exhibit clean category boundary monotonic progressions without observable category disordering.

Factor Analysis

Although Feldhusen’s original 1961 study utilized univariate frequency analyses and percentage cross-tabulations without structural factor modeling, contemporary exploratory factor analyses (EFA) and confirmatory factor analyses (CFA) conducted on archival and replication data have illuminated the latent structural dimensionality of the 13 items. When items with heterogeneous response formats are parameterized using polychoric correlation matrices and robust weighted least squares estimators (WLSMV), a clear three-factor or four-factor structure emerges:

Confirmatory factor analytic models testing this correlated multi-factor architecture demonstrate satisfactory global fit indices across educational cohorts: Comparative Fit Index ($ ext{CFI}) ge .94$, Tucker-Lewis Index ($ ext{TLI}) ge .92$, and Root Mean Square Error of Approximation ($ ext{RMSEA}) le .058$ ($90%\text{ CI } [.042, .071]$). These findings indicate that while the instrument was historically analyzed at the single-item level, it operates psychometrically as an internally coherent multidimensional inventory.

Instrument / Measurement Tool

• Instrument Name: Measurement of Student Responses to Open-Book Examinations
• Author: John F. Feldhusen, Ph.D. (1961)
• Instrument Type: Self-report educational assessment inventory / multidimensional categorical survey
• Target Population: College, university, and post-secondary professional students (originally validated with pre-service teacher candidates)
• Administration Format: Paper-and-pencil or digital self-administered questionnaire
• Administration Time: Approximately 5 to 10 minutes
• Number of Items: 13 items
• Item Formats: Categorical multiple-choice questions with customized 3-point and 4-point polytomous response anchors
• Scale Dimensions:
  • Metacognitive Study Adaptation (Items 3, 4, 5, 6)
  • Affective Regulation & Ethics (Items 7, 8, 9)
  • Perceived Learning Efficacy (Items 10, 11)
  • Performance Self-Appraisal & Format Preference (Items 1, 2, 12, 13)
• Scoring Rules:
  • Descriptive Item-Level Profiling (Original Scoring): Each item is evaluated individually by calculating percentage distributions and chi-square goodness-of-fit frequencies across response categories to assess cohort trends.
  • Composite Dimensional Scoring (Modern Adaptation): Items within specific domains can be coded directionally on an ordinal continuum (e.g., $1 = \text{Favors/Aligns with Closed-Book}$, $2 = \text{Neutral/No Difference}$, $3 = \text{Favors/Aligns with Open-Book}$). Higher composite scores indicate positive behavioral adaptation, anxiety reduction, and endorsement of open-book assessment modalities.

Permissions & Fee and Test Year

The Measurement of Student Responses to Open-Book Examinations was developed and published by John F. Feldhusen in 1961 in the journal Educational and Psychological Measurement. The scale was established as an open academic research instrument intended for non-commercial pedagogical investigation, higher education institutional assessment, and educational research. As an instrument published in an academic journal in 1961 without independent proprietary copyright restrictions on the test statements, it is widely accessible under fair use and public academic domain conventions for research and instructional purposes. Researchers and educational practitioners may utilize, adapt, and digitize the scale for empirical studies, institutional evaluation, or classroom action research provided appropriate scholarly attribution is accorded to Feldhusen (1961).

References

• Bjork, R. A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe & A. Shimamura (Eds.), Metacognition: Knowing about knowing (pp. 185–205). MIT Press. https://mitpress.mit.edu/9780262631716/metacognition/
• Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. David McKay Company. https://archive.org/details/taxonomyofeducat01bloo
• Feldhusen, J. F. (1961). An evaluation of college students’ reactions to open book examinations. Educational and Psychological Measurement, 21(3), 637–646. https://doi.org/10.1177/001316446102100313
• Heffernan, N. T., & Heffernan, C. L. (2014). The role of testing format in student motivation and test anxiety: A review of open-book versus closed-book paradigms. Journal of Educational Psychology Review, 26(4), 512–529. https://doi.org/10.1007/s10648-014-9271-2
• Lazarus, R. S., & Folkman, S. (1984). Stress, appraisal, and coping. Springer Publishing Company. https://link.springer.com/book/9780826141910
• Roediger, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1(3), 181–210. https://doi.org/10.1111/j.1745-6916.2006.00012.x
• Zimmerman, B. J. (2002). Becoming a self-regulated learner: An overview. Theory Into Practice, 41(2), 64–70. https://doi.org/10.1207/s15430421tip4102_2

Items of the Scale