Jacob Cohen – 1923 1998

Biography

In the intellectual history of twentieth-century behavioral science, few figures have exerted as profound and enduring an influence upon the methodology, philosophy, and practical execution of psychological research as Jacob Cohen. Operating at the confluence of clinical psychometrics, mathematical statistics, and epistemology, Cohen functioned for nearly half a century as the methodological conscience of the social sciences. At a time when psychological research was swept up in the triumphant post-war expansion of empirical quantification, Cohen recognized that the discipline’s empirical edifice rested upon perilous foundations. Investigators routinely deployed complex inferential mechanics without grasping their mathematical constraints, mistook ritualized significance testing for substantive scientific discovery, and systematically designed experiments that were statistically incapable of detecting the very phenomena they sought to investigate.

Cohen’s career was defined by a relentless, elegant, and often wry crusade against methodological dogma. Best known for his foundational work in establishing the theory and practice of statistical power analysis, the formulation of standardized effect size metrics such as Cohen’s d, the invention of chance-corrected inter-rater reliability through Cohen’s kappa, and the mathematical unification of analysis of variance within the general linear model through multiple regression, Cohen fundamentally reconstructed how behavioral researchers operationalize uncertainty. His scholarly output was characterized not by detached mathematical abstraction, but by a deeply humanistic and pedagogical commitment to rescuing working scientists from their own statistical illusions. He possessed an uncanny ability to translate intricate probability theory into lucid, intuitive frameworks that practicing psychologists, psychiatrists, sociologists, and medical researchers could immediately apply to their substantive inquiries.

This comprehensive monograph charts the life, intellectual evolution, and scientific legacy of Jacob Cohen from his early years in working-class Brooklyn and his formative service during the Second World War, through his clinical immersion in the Veterans Administration, to his decades of leadership at New York University. It details his seminal theoretical contributions, culminating in his famous 1994 polemic, “The Earth Is Round (p < .05),” and examines how his visionary critiques presaged by half a century the contemporary replication crisis and the modern open science movement. In tracing Cohen’s biographical and academic journey, one encounters not merely the history of a quantitative psychologist, but the evolutionary trajectory of modern quantitative methodology itself.

1. Early Life, Family Background, and World War II Military Service (1923–1947)

1.1 Childhood and Formative Years in New York City

Jacob Cohen was born on April 20, 1923, in Brooklyn, New York, to Jewish immigrant parents who had joined the vast migration of Eastern European families seeking refuge and economic opportunity in the United States during the early decades of the twentieth century. Raised in a dense, culturally vibrant, and socioeconomically modest urban enclave, Cohen was steeped from his earliest days in an environment that balanced severe material scarcity with an intense, near-sacred reverence for secular education, intellectual disputation, and cultural self-improvement. His parents, though possessing limited formal schooling themselves, instilled within the household a deep respect for analytical clarity, ethical responsibility, and academic diligence, recognizing that academic achievement represented the singular reliable conduit for generational advancement within American society.

Cohen’s elementary and secondary education took place entirely within the New York City public school system, an institutional ecosystem that was, during the interwar era, experiencing an unprecedented golden age of pedagogical rigor. Navigating these rigorous urban classrooms, Cohen distinguished himself through an extraordinary early aptitude for mathematical computation, verbal argument, and logical puzzles. The public schools of New York served as an intellectual crucible where first-generation immigrant children were exposed to classical humanities, physical sciences, and modern mathematics by teachers who were themselves frequently scholars of substantial caliber. Cohen’s formative intellectual environment was defined by constant debate, competitive academic discourse, and an early exposure to systematic, disciplined problem-solving methodologies.

Beyond the classroom, the bustling urban reality of Depression-era New York provided Cohen with an informal education in human diversity, social complexity, and observational psychology. The vibrant political debates of the Brooklyn streets, combined with the acute economic anxieties of the Great Depression, nurtured in the young Cohen a keen awareness of how environmental conditions shape human behavior and psychological well-being. Rather than retreating into purely abstract mathematical contemplation, Cohen displayed an early orientation toward applying analytical reasoning to real-world, human-centered problems. This early synthesis of rigorous quantitative curiosity and grounded, humane observation would ultimately define his unique trajectory as a quantitative clinical psychologist.

1.2 Military Service in the United States Army During World War II

The peaceful trajectory of Cohen’s early intellectual pursuits was decisively interrupted by the geopolitical cataclysm of the Second World War. Following the United States’ entry into the global conflict, Cohen answered the call to service, enlisting in the United States Army. His service coincided with the climactic operations of the European Theater of Operations, where the American military was engaged in massive logistical, combat, and administrative mobilizations unprecedented in scale and organizational complexity. As an enlisted soldier, Cohen was thrust into a world far removed from the academic debates of New York, confronting firsthand the stark realities of warfare, international logistics, and military organization.

Recognizing his exceptional analytical faculties, computational precision, and intellectual maturity, the military command assigned Cohen to specialized technical units charged with communications, logistics, and personnel administration. During his deployment in the European Theater, Cohen became directly involved with the administrative mechanisms used by the military to evaluate, classify, and allocate human personnel across diverse operational roles. The United States Armed Forces were, at that historical juncture, the largest consumer and innovator of psychometric and aptitude testing protocols in the world, utilizing tools derived from the Army Alpha and Beta examinations to rapidly categorize millions of recruits based on mechanical aptitude, general intelligence, and psychological stability.

This wartime immersion in large-scale human assessment protocols exerted a profound, lasting impact on Cohen’s emerging intellectual identity. He witnessed how high-stakes organizational decisions were routinely predicated on psychometric instruments of varying quality, observing both the immense utility of standardized measurement and the catastrophic human consequences of measurement error, diagnostic misclassification, and flawed statistical assumptions. The military experience stripped quantitative testing of any sterile academic innocence; for Cohen, testing, reliability, and validity became immediate matters of life, efficiency, and human dignity. This crucible of wartime operational testing provided the practical bedrock upon which his subsequent critiques of psychological assessment tools would be constructed.

1.3 Post-War Reintegration and Undergraduate Education

Following the cessation of hostilities in 1945 and his honorable discharge from the Army, Cohen returned to a radically transformed New York City. Like hundreds of thousands of his fellow veterans, he stood as a beneficiary of the Servicemen’s Readjustment Act of 1944, universally known as the G.I. Bill. This landmark legislation democratized American higher education, unlocking elite academic pathways for working-class veterans who would otherwise have been excluded from university life by financial constraints. For Cohen, the G.I. Bill was the transformative vehicle that permitted his total dedication to formal scholarship, freeing him from immediate economic survival and allowing him to immerse himself entirely in the post-war university environment.

Cohen matriculated at the City College of New York (CCNY), an institution celebrated as the “proletarian Harvard” and renowned for its fiercely competitive, intellectually hyperactive undergraduate culture. CCNY during the late 1940s was arguably one of the most intellectually electrifying campuses in the world, populated by brilliant, politically engaged, and intensely ambitious veterans who debated philosophy, economics, and psychology with relentless zeal in the famous campus alcoves. Within this demanding setting, Cohen initially pursued broad academic interests spanning the social sciences, philosophy of science, and mathematics, searching for a discipline that could harmonize his love for formal analytical precision with his abiding concern for empirical human welfare.

It was during his undergraduate tenure at CCNY that Cohen gravitated decisively toward empirical psychology. Under the influence of faculty who emphasized empirical methodology and scientific naturalism, Cohen recognized that psychology was poised at a critical historical juncture, attempting to emancipate itself from unfalsifiable psychoanalytic speculation through the adoption of quantitative experimental protocols. Cohen graduated with his Bachelor of Science degree in psychology, possessing an uncommon balance of comprehensive psychological theory and an advanced, intuitive command of statistical reasoning that set him distinctively apart from his academic peers.

2. Doctoral Training and Early Academic Foundations at New York University

2.1 Graduate Studies in Clinical Psychology

Immediately following the completion of his undergraduate degree at CCNY, Cohen sought entry into graduate education, securing admission to the Department of Psychology at New York University (NYU). The late 1940s represented a period of explosive growth and structural professionalization for American clinical psychology, largely catalyzed by the Veterans Administration and the United States Public Health Service, which sought to train thousands of clinical psychologists to treat the massive wave of psychologically traumatized veterans returning from global combat. NYU was at the forefront of this curricular transformation, constructing a graduate program that married rigorous psychoanalytic and psychiatric theory with rigorous psychometric testing and laboratory experimental design.

At NYU, Cohen embarked upon the demanding dual curriculum required of post-war doctoral students in clinical psychology. He was required to achieve clinical mastery over diagnostic interviewing, projective testing, and psychotherapy, while simultaneously completing an extensive sequence of courses in experimental methodology, psychophysical measurement, and inferential statistics. This dual training exposed Cohen to an epistemological tension that would animate his entire academic career: the profound chasm dividing the intuitive, qualitative clinical practitioner from the rigorous, skeptical quantitative experimentalist. Cohen quickly demonstrated an exceptional capacity to bridge these disparate worlds, earning his Master of Arts degree while formulating a research agenda focused on improving the empirical foundations of diagnostic assessment.

His advancement to doctoral candidacy marked the emergence of Cohen as an incisive methodological thinker. Rather than accepting traditional psychodiagnostic procedures on clinical faith, Cohen approached the clinical armamentarium with deep empiricism. He became intensely interested in the mechanisms through which clinicians arrived at diagnostic conclusions, the degree to which diagnostic classifications were reliable across independent practitioners, and the statistical justifications underlying clinical inference. His graduate work was marked by an unwavering demand for evidentiary rigor, positioning him as an emerging critical voice within NYU’s clinical training community.

2.2 Doctoral Research and Early Quantitative Investigations

Cohen completed his Doctor of Philosophy (Ph.D.) degree at New York University in 1950, presenting a doctoral dissertation centered on clinical evaluation metrics, diagnostic validity, and the psychometric behavior of personality assessment instruments. His doctoral research placed him at the heart of the fierce mid-century debates regarding the scientific status of projective techniques, most notably the Rorschach Inkblot Test and the Thematic Apperception Test (TAT), which then completely dominated diagnostic clinical psychology. While clinical orthodoxies asserted that these instruments provided direct, unfettered windows into the unconscious dynamics of patients, Cohen subjected them to rigorous empirical and psychometric scrutiny.

His doctoral investigations systematically revealed the acute diagnostic fragility and measurement limitations inherent in projective metrics. Cohen identified severe measurement error, pervasive scoring subjectivity, and unacceptably low levels of inter-examiner reliability in clinical evaluations. He demonstrated that clinicians interpreting identical projective protocols routinely produced radically divergent diagnostic classifications, effectively projecting their own cognitive idiosyncrasies onto the diagnostic data. These early investigations convinced Cohen that clinical psychology was operating in an epistemological danger zone, confusing subjective diagnostic confidence with objective statistical validity.

Furthermore, Cohen’s dissertation research illuminated critical statistical shortcomings in contemporary psychological tools. He observed that researchers and clinicians routinely employed inappropriate inferential tests, ignored violations of distributional assumptions, and operated with total disregard for the statistical power of their sample designs. Even at this nascent stage of his career, Cohen grasped that the problem was not merely the qualitative ambiguity of clinical psychology, but a fundamental, systemic failure of statistical education across the behavioral sciences. His doctoral work thus laid the programmatic groundwork for a lifetime dedicated to statistical pedagogy and methodological reform.

2.3 Mentorship and Methodological Milieu of Post-War Psychology

Cohen’s emergence as a quantitative thinker was deeply informed by the broader methodological milieu of post-war American psychology, an era marked by the uncritical codification of inferential statistics into behavioral curricula. During the late 1940s and early 1950s, American psychology departments were enthusiastically adopting the statistical frameworks developed in Great Britain during the interwar years, primarily the agrarian experimental designs and significance testing paradigms formulated by Sir Ronald A. Fisher, alongside the alternative mathematical models of hypothesis testing advanced by Jerzy Neyman and Egon Pearson.

However, as Cohen astutely recognized, American psychology had not embraced these systems with philosophical coherence; instead, textbook authors and university instructors had mashed together an unholy, hybrid amalgamation of Fisherian null-hypothesis testing and Neyman-Pearson decision theory. From Fisher, researchers inherited the fetishization of the 5% significance level (p < .05) as an absolute, qualitative dividing line between scientific truth and empirical void; from Neyman and Pearson, they adopted the terminology of Type I and Type II errors, while almost entirely discarding the concepts of statistical power, alternative hypotheses, and effect size. Cohen absorbed these mathematical developments under the guidance of quantitatively oriented faculty, but he did so with an acutely skeptical eye toward this intellectual confusion.

This post-war methodological environment led Cohen to his foundational intellectual conviction: that the misuse of statistical inference in behavioral and clinical research was not an isolated collection of minor arithmetic mistakes, but a deep, structural epistemological pathology. Researchers were treating statistical tests as a mechanical ritual—a form of mathematical sanctification that spared them the arduous labor of substantive theoretical reasoning and systematic measurement refinement. Cohen resolved to direct his intellectual energies toward dismantling this ritualized dogmatism, dedicating his scholarship to replacing mechanical significance testing with a rational, transparent, and empirically sound model of scientific inference.

3. Early Clinical Appointments and the Veterans Administration Experience

3.1 Research Clinical Psychologist at the Veterans Administration

Upon completing his doctorate in 1950, Cohen did not immediately retreat into the sheltered halls of an academic psychology department. Instead, he accepted appointments as a Research Clinical Psychologist within the United States Veterans Administration (VA) healthcare system, eventually rising to become Chief of the Clinical Psychology Service at the VA Hospital in the Bronx and later maintaining significant clinical affiliations with the VA Hospital in Montrose, New York. The VA hospital system at that time represented the premier clinical training and research environment for psychological disorders in the world, housing tens of thousands of combat veterans suffering from acute and chronic psychiatric conditions, ranging from traumatic neuroses to severe schizophrenic psychoses.

In his capacity as a chief clinical psychologist and research director, Cohen was immersed in the complex, high-stakes realities of large-scale inpatient psychiatric diagnostics and clinical psychopharmacological interventions. The 1950s bore witness to the dawn of the psychopharmacological revolution, characterized by the introduction of early neuroleptics such as chlorpromazine and early tricyclic antidepressants. Cohen was tasked with evaluating the institutional efficacy of these novel somatic treatments, alongside traditional psychotherapeutic regimens, within complex hospital ward environments. This required the continuous monitoring of patient symptom profiles, behavioral changes, and discharge prognoses across diverse psychiatric cohorts.

It was within this intensive clinical laboratory that Cohen directly encountered the crippling consequences of inadequate statistical methodology. Conducting therapeutic trials within psychiatric settings presented formidable practical challenges: patient attrition was rampant, diagnostic categories were notoriously imprecise, and sample sizes within specific diagnostic sub-types were frequently constrained by clinical availability. Cohen observed that institutional clinical researchers were conducting clinical trials with samples so small that their statistical power to detect genuine, moderate therapeutic improvements was practically non-existent. These underpowered trials were routinely leading investigators to abandon potentially life-saving somatic and behavioral treatments on the erroneous grounds that they had “failed to reach statistical significance,” an empirical catastrophe that deeply troubled Cohen’s humanitarian and scientific sensibilities.

3.2 Cross-Validation and Methodological Diagnostics in Clinical Assessment

Concurrently with his oversight of therapeutic clinical trials, Cohen directed his empirical gaze toward the internal mechanics of psychiatric diagnosis and psychological test administration. Working within the expansive wards of the VA hospitals, he launched extensive research projects designed to evaluate the systematic inter-rater subjectivity among practicing clinical psychologists, psychiatric residents, and senior diagnostic physicians. Cohen organized controlled diagnostic exercises wherein multiple experienced clinicians evaluated identical patient interviews, clinical histories, and psychometric profiles, subsequently recording their diagnostic classifications, prognostic forecasts, and therapeutic recommendations in total isolation from one another.

The empirical results of these investigations were deeply sobering. Cohen documented an alarming degree of diagnostic divergence: clinicians evaluating the very same patient reached wildly contradictory diagnostic determinations, with one clinician identifying paranoid schizophrenia where another diagnosed severe manic-depressive reaction or character neurosis. Cohen heavily critiqued the prevailing institutional practice of achieving “qualitative consensus” through clinical team conferences. He recognized that this so-called consensus was typically not an objective convergence of independent clinical judgments, but an artifact of institutional hierarchy, rhetorical intimidation, and social conformity, wherein the opinion of the chief of service or the most assertive clinician was uncritically rubber-stamped by subordinates.

Cohen demonstrated that the psychiatric establishment lacked any formalized, mathematically defensible metric for verifying statistical concordance among diagnostic raters. The field had relied on raw percentage agreement—a metric fundamentally contaminated by chance agreements driven by base rates—or had inappropriately adapted Pearson product-moment correlation coefficients to nominal diagnostic categories. Cohen’s rigorous documentation of these measurement breakdowns within the VA hospital system laid the foundational rationale for his subsequent development of formalized coefficients of observer agreement, demonstrating that clinical psychology could never attain scientific maturity until it could reliably measure its primary phenomena of interest beyond the illusions of chance.

3.3 Transition from Hospital Practice to Full-Time Academic Scholarship

By the late 1950s, Cohen reached a defining crossroads in his professional life. While he found deep fulfillment in his clinical service to veterans and commanded immense respect as a hospital administrator and diagnostic research director, he realized that his capacity to rectify the systemic methodological pathologies of behavioral science was fundamentally constrained within the hospital setting. The statistical illiteracy, measurement unreliability, and underpowered experimental designs he had documented in the VA were symptoms of a broader, structural crisis rooted in university graduate training programs. If the behavioral sciences were to be intellectually redeemed, the intervention had to occur at the pedagogical source: the academy.

In 1959, Cohen resigned his full-time clinical administration post to accept a professorship within the Department of Psychology at New York University, inaugurating a legendary four-decade tenure as a full-time academic scholar, educator, and quantitative innovator. Cohen’s transition to NYU was marked by an immediate institutional ambition: he set out to build an elite, dedicated curriculum in quantitative psychology that would transcend the mechanical, cookbook statistical courses then typical of American doctoral programs. He recognized that graduate students were being taught how to calculate formulas rather than how to think statistically, leaving them defenseless against interpretive fallacies when analyzing complex empirical data.

At NYU, Cohen formalized the Quantitative Psychology curriculum, structuring advanced sequences in multivariate analysis, experimental design, and psychometric theory. He positioned himself not as a distant, theoretical mathematician, but as a methodological translator who understood the messy, ambiguous realities of clinical and behavioral data from years of frontline experience. This unique clinical perspective endowed Cohen’s academic scholarship with an urgent practical relevance. He did not write for mathematical purists; he wrote for the empirical researcher struggling to make sense of flawed observations, elusive behavioral phenomena, and underfunded research environments, thereby beginning his transformative campaign to modernize the quantitative architecture of the social sciences.

4. Conceptualizing Statistical Power Analysis in the Behavioral Sciences

4.1 The Epistemic Neglect of Statistical Power Prior to the 1960s

To comprehend the revolutionary magnitude of Jacob Cohen’s contributions to behavioral statistics, one must examine the epistemological landscape that dominated psychology, sociology, and medicine prior to the 1960s. The post-war scientific consensus had elevated Fisher’s null-hypothesis significance testing (NHST) to an absolute orthodoxy. Within this framework, researchers operated under a singular, obsessive imperative: the control of the Type I error rate, denoted mathematically as alpha ($\alpha$). Investigators vigilantly ensured that the probability of falsely rejecting a true null hypothesis—declaring an effect when no effect existed in the population—was strictly constrained to an arbitrary, conventional threshold, typically fixed at $\alpha = .05$ or $\alpha = .01$.

In dramatic and catastrophic contrast, the scientific establishment almost totally ignored the companion concept formalized by Jerzy Neyman and Egon Pearson: the Type II error rate, designated as beta ($\beta$). Type II error represents the probability of failing to reject a false null hypothesis—that is, the failure to detect a true, substantive phenomenon that actually exists in the population. The complement of this error probability, defined as $1 – beta$, constitutes statistical power: the mathematical probability that an empirical test will yield statistically significant results given that the research hypothesis is true and an effect of a specified magnitude exists. Prior to Cohen’s intervention, behavioral researchers proceeded as if statistical power was an academic luxury, an arcane mathematical nuance with little practical bearing on the conduct of empirical science.

The epistemic consequences of this systematic neglect were devastating. Researchers routinely designed, executed, and published empirical investigations with completely inadequate sample sizes, oblivious to the fact that their studies were structurally blind to the effects they were investigating. Because scientific journals exhibited an overwhelming editorial bias toward publishing exclusively statistically significant outcomes (the infamous “file drawer problem”), the literature was systematically distorted. Investigators who conducted underpowered studies either abandoned genuine scientific discoveries because they failed to achieve the sanctified threshold of $p < .05$, or they inadvertently filled the academic journals with wildly inflated, unreplicable effect sizes resulting from rare, chance-driven statistical anomalies. Cohen saw this systemic neglect as a profound threat to the cumulative validity of psychological science.

4.2 The Seminal 1962 Survey of the Journal of Abnormal and Social Psychology

Recognizing that theoretical exhortations alone would not shatter the complacency of the academic community, Cohen resolved to conduct an uncompromising, comprehensive empirical audit of contemporary psychological scholarship. In 1962, he published a landmark, seismically disruptive paper in the Journal of Abnormal and Social Psychology titled “The Statistical Power of Abnormal-Social Psychological Research: A Review.” In this tour-de-force methodological survey, Cohen took an unprecedented empirical step: he systematically audited every single quantitative article published in the entire 1960 volume of the discipline’s flagship journal, meticulously calculating the statistical power of every reported statistical test across a total of seventy major empirical investigations.

Cohen established three heuristic benchmarks for the magnitude of population effects—small, medium, and large—and calculated the retrospective power of each study to detect these hypothetical effects at the standard $\alpha = .05$ significance criterion. The empirical revelations of his survey were staggering and exposed a structural crisis at the heart of psychological research:

• The average statistical power across all seventy empirical papers to detect a small effect size was an abysmal 0.18. This demonstrated that researchers investigating subtle psychological phenomena had less than a one-in-five chance of obtaining a statistically significant result, even when their substantive hypothesis was completely correct.
• For medium effect sizes—which Cohen identified as capturing the typical magnitude of psychological differences and therapeutic interventions—the average statistical power across the journal was precisely 0.48. This revealed that the typical published study in psychology’s premier journal had less than a coin-flip’s chance of detecting a genuine, realistic phenomenon.
• Only when investigating large effect sizes did the average statistical power reach an acceptable level, averaging 0.83; however, as Cohen vigorously noted, genuinely massive effects are exceptionally rare in the subtle, multi-determined realms of human psychology and social behavior.

The inescapable mathematical conclusion of Cohen’s 1962 audit was that the elite researchers of the era were squandering vast fortunes in intellectual energy, participant time, and institutional capital on experiments that were doomed to failure from their inception. Researchers were operating under the pervasive, tragic illusion that their tests were rigorous because they held alpha strictly to .05, totally unaware that their overwhelming methodological vulnerability was an uncontrolled, massive beta error rate hovering around 52%. Cohen’s audit made it impossible for the discipline to continue ignoring statistical power, providing the decisive empirical foundation for the modern statistical reform movement.

4.3 Core Mechanics of Statistical Power: Alpha, Sample Size, and Effect Size

To rescue the discipline from this methodological quagmire, Cohen formalized the conceptual and mathematical mechanics of statistical power, presenting behavioral scientists with an intuitive, fourfold interdependent statistical framework. Cohen demonstrated that in any statistical test of an inferential hypothesis, precisely four fundamental parameters exist in a locked, immutable mathematical relationship:

1. The Significance Criterion ($\alpha$): The probability threshold for rejecting the null hypothesis, representing the investigator’s tolerance for committing a Type I error. As $\alpha$ becomes more stringent (moving from .05 to .01 or .001), statistical power ($1 – beta$) inevitably decreases, demanding larger sample sizes to maintain identical detection sensitivity.
2. The Sample Size ($N$): The total number of observational units or experimental participants recruited into the investigation. Because sampling error is inversely proportional to the square root of sample size, increasing $N$ shrinks the standard error of the estimate, sharpening the probability distribution and dramatically inflating statistical power.
3. The Population Effect Size ($ES$): The true magnitude of the phenomenon, difference, or association existing within the population under study. Ceteris paribus, massive phenomena are trivial to detect even with small samples, whereas subtle, nuanced effects demand substantial statistical power to emerge cleanly from background sampling noise.
4. Statistical Power ($1 – beta$): The probability of successfully rejecting the null hypothesis given that it is false and that the specified population effect size genuinely exists.

Cohen demonstrated that these four parameters are so intimately linked that if any three are specified, the fourth is mathematically determined. From this insight, Cohen introduced the fundamental distinction between retrospective power analysis (the post-hoc evaluation of completed studies) and prospective power planning. Prospective power planning, Cohen argued, is an indispensable prerequisite for ethical and scientifically valid research design: investigators must determine in advance the smallest effect size of substantive or clinical interest, establish their acceptable alpha and power thresholds (which Cohen suggested should conventionally be targeted at power = .80, balancing a four-to-one ratio between Type II error at $\beta = .20$ and Type I error at $\alpha = .05$), and subsequently compute the precise sample size $N$ mathematically required to achieve that power.

Through this elegant formulation, Cohen elevated statistical power from an academic abstraction into a moral and economic imperative. He forcefully argued that conducting an underpowered experiment is fundamentally unethical. It wastes precious human subject participation, exposes clinical patients to experimental risks with near-zero probability of scientific illumination, squanders scarce grant funding, and pollutes the scientific record with false-negative findings that prematurely terminate productive avenues of scientific inquiry. Cohen had transformed power analysis into the premier tool of rational experimental design.

5. The 1969 Magnum Opus: Statistical Power Analysis for the Behavioral Sciences

5.1 Genesis, Structure, and Methodological Purpose of the Text

Despite the clarity and empirical force of his 1962 paper, Cohen quickly recognized an immense practical bottleneck: working behavioral researchers were not mathematicians. In the 1960s, prior to the widespread availability of desktop computers, statistical packages, or algorithmic calculators, determining non-central distributions—the complex non-central t, F, and chi-square distributions required to calculate exact statistical power—demanded formidable, highly specialized mathematical labor far beyond the training of doctoral psychologists. If researchers were to adopt prospective power planning, they required an accessible, non-mathematical lookup system that eliminated the need for manual calculus.

Driven by this urgent pedagogical imperative, Cohen spent years compiling an extraordinary mathematical reference manual, culminating in the 1969 publication of his magnum opus: Statistical Power Analysis for the Behavioral Sciences. The book was a monument of computational labor and pedagogical design. Cohen structured the volume around specific statistical tests, allocating dedicated, self-contained chapters to independent and paired-samples t-tests, product-moment correlations, tests of differences between proportions, chi-square tests of contingency tables, and one-way and factorial analyses of variance (ANOVA). Each chapter provided comprehensive, exhaustive look-up tables that permitted any researcher, without solving a single differential equation, to locate their required sample size or assess their prospective power by simply identifying their significance criterion, intended sample size, and anticipated effect size.

The impact of Cohen’s 1969 book was immediate, profound, and international. It dismantled the final practical excuse for the neglect of statistical power, landing like a revelation in graduate seminars and research laboratories across the globe. Researchers who had previously regarded power as an esoteric, impenetrable mystery suddenly possessed an intuitive desktop reference that illuminated the exact structural parameters of their experimental designs. Cohen’s manual became an indispensable, standard text in graduate statistical education and established him unequivocally as the foremost authority on experimental planning in the human sciences.

5.2 Operationalizing the ‘Medium Effect Size’ and Normative Benchmarks

A central, defining achievement of the 1969 manual was Cohen’s introduction of standardized operational definitions for effect size magnitudes. Cohen recognized that researchers embarking upon prospective power calculations routinely encountered an agonizing impasse: how could they specify an anticipated population effect size before they had even conducted the study to measure it? In the absence of prior empirical data, experimental planning threatened to grind to an absolute halt. To resolve this dilemma, Cohen conducted extensive reviews of the behavioral literature and proposed three universal, standardized benchmark tiers: small, medium, and large effect sizes across every major class of statistical test.

Cohen was meticulously careful in defining the philosophical rationale underlying these benchmarks, especially his conceptualization of the medium effect size. He defined a medium effect as a phenomenon of sufficient magnitude to be perceptible to the naked, unaided eye of a careful, experienced observer in the field. For instance, in terms of mean differences, Cohen noted that a medium effect corresponded to the difference in mean height between eighteen-year-old and sixteen-year-old girls, or the psychological difference observed between neurotic patients and general population controls on standardized personality inventories. A small effect was defined as a subtle, elusive phenomenon that required formal statistical machinery to detect cleanly, while a large effect represented an unmistakable, dramatic divergence, such as the difference in intelligence scores between college graduates and individuals with severe intellectual disabilities.

Crucially, however, Cohen issued explicit, repeated warnings against the unthinking, rigid reification of these heuristic benchmarks. He insisted with urgent prose that his small, medium, and large labels were arbitrary conventions intended exclusively as crude guidelines for novice researchers or exploratory fields lacking any prior empirical history. He pleaded with the scientific community not to transform these benchmarks into mechanical, uncritical standards, warning that what constitutes a “small” effect in a brief laboratory memory task might represent a gigantic, transformative “large” effect within population epidemiology or public health interventions. Despite Cohen’s vigorous caveats, the subsequent empirical literature would frequently fall prey to the very unthinking reification he feared, solidifying these benchmarks into rigid, algorithmic cutoffs across major academic disciplines.

5.3 The 1977 and 1988 Revised Editions and Extended Algorithmic Models

The overwhelming success of the 1969 text led to substantial revisions and major expansions in subsequent editions. In 1977, Cohen published a second edition that incorporated advanced methodologies, refined mathematical approximations, and substantially expanded lookup tables. The most significant theoretical and algorithmic expansion, however, arrived with the publication of the definitive third edition in 1988. In this monumental work, Cohen vastly modernized the mathematical foundations of power analysis, integrating comprehensive coverage of multiple linear regression, partial correlation, hierarchical modeling, and multivariate configurations of the General Linear Model.

The 1988 edition embraced the emerging dawn of personal computing. While retaining the beloved lookup tables for researchers without immediate software access, Cohen introduced sophisticated, generalized computational algorithms that could be easily translated into software scripts, spreadsheets, and early computer programs. He systematically formalized effect size derivations for complex interaction terms in multi-factorial ANOVA and developed the parameter $f^2$, unlocking prospective power planning for the most sophisticated multivariate regression designs utilized by contemporary social scientists.

Across these three decades of relentless book revisions, Cohen waged an unceasing intellectual campaign to institutionalize power analysis within the administrative apparatus of scientific funding. He engaged directly with federal grant agencies, most notably the National Institutes of Health (NIH) and the National Science Foundation (NSF), arguing that public and philanthropic funds should never be awarded to underpowered research proposals. Cohen’s persistent advocacy across these successive editions fundamentally restructured the grant review architecture of Western science, ultimately resulting in modern institutional mandates requiring prospective power calculations as an non-negotiable prerequisite for competitive research funding.

6. Standardizing Effect Size Metrics: Cohen’s d, f, w, and Beyond

6.1 Derivation and Mathematical Properties of Cohen’s d

Prior to Jacob Cohen’s systematic formalization of effect sizes, researchers evaluating group differences were entirely constrained by the raw units of their specific measurement instruments or, far worse, relied exclusively on sample-dependent test statistics like the t-value or its associated p-value. A researcher measuring depression using the Beck Depression Inventory could not meaningfully synthesize or compare their results with an investigator measuring the same construct via the Hamilton Rating Scale for Depression. Cohen recognized that scientific accumulation required a scale-free, unit-invariant metric of phenomenon magnitude. To resolve this universal challenge, Cohen introduced and formalized what is known throughout global science as Cohen’s d.

Cohen mathematically defined d as the difference between two population means divided by the common, pooled standard deviation of the populations:

d = (μ₁ – μ₂) / σ

In sample data, this parameter is estimated by taking the difference between the sample means of group one and group two and dividing this raw divergence by the pooled sample standard deviation, calculated as the square root of the weighted average of the two group variances:

s_pooled = √[ ((n₁ – 1)s₁² + (n₂ – 1)s₂²) / (n₁ + n₂ – 2) ]

Through this normalization, Cohen’s d expresses the magnitude of difference between two independent groups purely in units of standard deviation. A Cohen’s d of 0.50 signifies that the mean of the experimental group sits precisely one-half of a standard deviation above the mean of the control group, representing an overlap between the two distributions of approximately 80% and indicating that roughly 69% of the experimental group exceeds the mean of the control group.

Cohen’s formulation must be historically situated alongside other standardization efforts. Gene V. Glass introduced Glass’s Delta ($\Delta$), which normalizes the mean difference utilizing exclusively the standard deviation of the untreated control group, arguing that experimental manipulations may heterogeneously distort the variance of the treated group. Later, Larry Hedges formulated Hedges’ g, which applies an exact correction factor to Cohen’s d to eliminate the slight upward bias that occurs when calculating standardized mean differences in very small samples ($N < 20$). Despite these valid mathematical refinements, Cohen’s d remains the foundational, universally utilized bedrock metric for pairwise comparisons and stands as the primary currency for contemporary meta-analytic syntheses across medicine, education, and the behavioral sciences.

6.2 Effect Sizes for Multi-Group and Associational Designs (Cohen’s f, f², and q)

While Cohen’s d elegantly addressed the simple two-group design, the post-war behavioral sciences relied heavily on complex, multi-group experimental configurations evaluated through Analysis of Variance (ANOVA) and multi-variable associational matrices. Cohen recognized that these advanced designs required specialized effect size metrics capable of capturing non-directional variance among three or more groups, as well as unique variance accounted for in multivariate systems. To this end, Cohen developed a family of specialized effect size metrics: Cohen’s f, Cohen’s $f^2$, and Cohen’s q.

For one-way and factorial Analysis of Variance, Cohen introduced Cohen’s f, which he defined as the standard deviation of the standardized population means divided by the within-group standard deviation. While researchers were accustomed to reporting eta-squared ($\eta^2$) or partial eta-squared—metrics representing the proportion of total variance explained by the experimental factor—Cohen demonstrated that $\eta^2$ exhibits non-linear properties that make it cumbersome for prospective power calculations. Cohen’s f solved this directly, providing an operational metric that scales linearly with experimental divergence, with the mathematical conversion cleanly established as:

f = √( η² / (1 – η²) )

For multiple linear regression, hierarchical analysis, and generalized linear models, Cohen formulated what would become the gold-standard metric for localized effect magnitude: Cohen’s $f^2$. When evaluating the unique contribution of a set of independent variables added to a regression model, $f^2$ is computed as:

f² = (R²_AB – R²_A) / (1 – R²_AB)

where $R^2_A$ represents the variance explained by a base set of covariates, and $R^2_{AB}$ represents the total variance explained after adding the predictor set of substantive interest. Cohen’s $f^2$ directly operationalizes the signal-to-noise ratio of the unique predictor variance against the residual unexplained variance, establishing universal benchmark values of 0.02 (small), 0.15 (medium), and 0.35 (large).

Finally, recognizing that researchers routinely needed to determine whether two independent correlation coefficients ($r_1$ and $r_2$) were significantly different from one another, Cohen devised Cohen’s q. Recognizing that the raw difference between two Pearson correlations ($r_1 – r_2$) does not possess a constant distribution across the -1.0 to +1.0 spectrum—because moving from .80 to .90 represents a vastly greater divergence in explained variance than moving from .10 to .20—Cohen utilized Fisher’s logarithmic z transformation to derive q:

q = | z’₁ – z’₂ |

where $z’ = 0.5 \cdot \ln[(1 + r)/(1 – r)]$. Cohen’s q thus established a normalized, standardized metric for correlation comparisons, expanding the reach of rigorous power analysis across all standard associational research designs.

6.3 Contingency Table Effect Sizes: Cohen’s w and Goodness-of-Fit

Beyond continuous variables and linear relationships, a massive proportion of behavioral, psychiatric, and epidemiological data exists in the form of discrete, nominal categories evaluated through Pearson chi-square ($\chi^2$) tests of independence and goodness-of-fit models. Prior to Cohen’s work, researchers routinely evaluated chi-square tests entirely on the basis of whether the calculated test statistic exceeded the critical mathematical threshold for a given degrees of freedom, completely conflating sample size with substantive association magnitude. A large sample could produce an overwhelmingly significant $\chi^2$ test from an entirely trivial categorical departure, whereas an underpowered clinical sample could obscure a massive, clinically transformative categorical shift.

To establish a rigorous, scale-invariant effect size for discrete data, Cohen formulated Cohen’s w. Defined for both one-dimensional goodness-of-fit tests and multi-dimensional contingency tables, Cohen’s w is calculated as the square root of the sum of the squared differences between the observed proportion ($P_{1i}$) and the expected proportion ($P_{0i}$) under the null hypothesis, normalized by the expected proportion across each cell:

w = √[ ∑ ( (P₁ᵢ – P₀ᵢ)² / P₀ᵢ ) ]

Alternatively, in sample data, Cohen’s w can be calculated directly from the standard Pearson chi-square statistic by dividing $\chi^2$ by the total sample size $N$ and taking the square root:

w = √( χ² / N )

Cohen demonstrated that in the specific case of a 2×2 contingency table, Cohen’s w is mathematically identical to the absolute value of the classic phi ($phi$) coefficient, representing a fourfold point correlation coefficient. For larger, multi-category matrices, Cohen established the precise mathematical relationships between w and Cramér’s V (where $w = V \cdot \sqrt{k – 1}$, with $k$ representing the smaller number of rows or columns). Cohen set the operational benchmarks for w at 0.10 for small effects, 0.30 for medium effects, and 0.50 for large effects.

The philosophical and methodological implications of Cohen’s w were transformative. By decoupling the evaluation of categorical contingency tables from sample size, Cohen provided researchers with the conceptual tools needed to look past dichotomous significance. Researchers could now discern whether an observed distribution represented a trivial fluctuation amplified by an enormous cohort, or a substantive, clinically vital categorical disparity that demanded theoretical explanation and replicated intervention.

7. The Formulation of Cohen’s Kappa: Quantifying Inter-Rater Reliability

7.1 The 1960 Paper: A Coefficient of Agreement for Nominal Scales

While developing his foundational work on statistical power, Cohen simultaneously solved another pervasive measurement crisis that had long paralyzed clinical psychology, psychiatry, and behavioral observation. In 1960, he published a revolutionary paper in Educational and Psychological Measurement titled “A Coefficient of Agreement for Nominal Scales.” The paper addressed a fatal flaw in contemporary assessment methodology: the universal reliance on “raw percent agreement” to quantify the concordance between two independent clinicians, observers, or diagnostic coders.

Cohen demonstrated that raw percent agreement is a mathematically bankrupt metric because it completely ignores the agreement that occurs purely by chance. For instance, if two psychiatric evaluators are reviewing a clinical population wherein 90% of the patients suffer from major depression, both raters could assign the diagnosis of depression to nearly every patient almost at random or based entirely on their knowledge of the institutional base rate. The raw percentage agreement between them would approach 85% to 90%, creating a dangerous illusion of diagnostic precision when, in objective reality, their individual diagnostic classifications were completely uncoupled from the specific symptoms of individual patients. Raw agreement invariably overestimates reliability, particularly in skewed or highly prevalent diagnostic contexts.

To eliminate this distortion, Cohen derived Cohen’s Kappa ($kappa$). He mathematically isolated the observed proportion of agreement ($p_o$) from the proportion of agreement that would be expected entirely by chance ($p_c$), given the observed marginal distributions of the two independent raters. The mathematical formula for Cohen’s Kappa is established as:

κ = (pₒ – p꜀) / (1 – p꜀)

In this elegant formulation, the denominator $(1 – p_c)$ represents the maximum possible agreement achievable beyond chance, while the numerator $(p_o – p_c)$ represents the actual agreement achieved beyond chance expectation. When agreement matches chance expectation exactly, $kappa = 0$; when observed agreement is absolute and complete, $kappa = 1.0$; and when observed agreement is less than chance, $kappa$ yields a negative value, signaling systematic disagreement. Cohen’s 1960 paper permanently standardized inter-rater reliability measurement across global psychiatry, clinical trials, and behavioral ethology.

7.2 The 1968 Innovation: Weighted Kappa for Ordinal Measurement

Despite the brilliance of the 1960 nominal Kappa, Cohen recognized that the metric possessed an inherent clinical limitation: it treated all disagreements as qualitatively identical. In a nominal classification system (such as diagnosing Schizophrenia versus Bipolar Disorder versus Borderline Personality Disorder), any disagreement is mathematically binary; the diagnostic coders either agree or they do not. However, in routine clinical and educational practice, measurement scales are frequently ordinal, characterized by ordered severity levels, clinical staging models, or Likert rating metrics (for instance, rating patient depression as absent, mild, moderate, or severe).

Within an ordinal framework, treating all diagnostic disagreements as equal is psychometrically absurd. If Rater A classifies a patient as having “mild” depression and Rater B classifies the same patient as having “moderate” depression, this represents a minor clinical divergence. If, however, Rater A classifies that same patient as “absent” while Rater B classifies them as “severe,” this constitutes an acute diagnostic failure. The standard 1960 nominal Kappa penalized both disagreements with identical mathematical severity. To resolve this inadequacy, Cohen published another pioneering methodological paper in 1968 in Psychological Bulletin, introducing Weighted Kappa ($\kappa_w$).

Weighted Kappa introduced a mathematical partial credit system into inter-rater concordance by assigning weights ($w_{ij}$) to each cell in the contingency matrix, corresponding to the degree of agreement or disagreement between categories $i$ and $j$. The general formula for Weighted Kappa is formulated as:

κ_w = (pₒ(w) – p꜀(w)) / (1 – p꜀(w))

where $p_o(w)$ and $p_c(w)$ represent the weighted observed and weighted chance-expected proportions across the entire matrix. Cohen formulated two primary weighting schemas: linear weights, where the penalty increases linearly with category separation, and quadratic weights, where disagreement penalties increase exponentially with the square of the category divergence. Cohen’s Weighted Kappa with quadratic weighting achieved a profound mathematical triumph: it proved to be completely identical to the intraclass correlation coefficient (ICC) under specific two-way ANOVA assumptions, cleanly bridging non-parametric categorical concordance with classic parametric reliability theory.

7.3 Methodological Critiques, Paradoxes, and Refinements of Kappa

As Cohen’s Kappa established itself as the global standard for reliability assessment, intensive mathematical scrutiny revealed specific operational boundaries and statistical anomalies. Most prominent among these were the classic Kappa Paradoxes, first systematically documented by Feinstein and Cicchetti in 1990. The first paradox occurs when high observed agreement ($p_o$) is accompanied by an unexpectedly low, deflated value of $kappa$; this paradox emerges under conditions of extreme trait prevalence, where marginal distributions are heavily unbalanced, driving chance expected agreement ($p_c$) to astronomical levels and mathematically suppressing the kappa ratio. The second paradox occurs when asymmetrical marginal distributions between raters produce paradoxical increases in kappa relative to symmetric configurations.

These mathematical boundary conditions catalyzed an entire sub-discipline of psychometric refinement. Methodologists formulated several alternative metrics designed to mitigate the prevalence paradox, most notably Kilem L. Gwet’s AC1 statistic, which provides a more robust chance adjustment under skewed base rates. Similarly, Joseph L. Fleiss—a brilliant biostatistician who had studied closely with Cohen at NYU—extended Cohen’s nominal Kappa in 1971 to accommodate generalized multi-rater configurations involving more than two observers (Fleiss’ Kappa). Concurrently, communication theorist Klaus Krippendorff formulated Krippendorff’s Alpha ($\alpha$), which accommodates multi-rater designs, missing observational data, and multiple levels of measurement within a single unified formula.

Despite these subsequent mathematical extensions and competitive formulations, Jacob Cohen’s original 1960 and 1968 formulations retain an undisputed primacy across contemporary biomedical, clinical, and psychological research. Editorial guidelines for premier medical publications (such as the New England Journal of Medicine and The Lancet) and psychiatric guidelines globally continue to mandate Cohen’s Kappa as the primary benchmark for verifying diagnostic consensus, clinical trial inter-rater fidelity, and coding concordance in high-stakes human assessment.

8. Multiple Regression and Correlation as a Unified General Data-Analytic System

8.1 Demolishing the Rigid Dichotomy Between ANOVA and Multiple Regression

During the mid-twentieth century, behavioral researchers operated under an artificial, deeply entrenched methodological caste system. A rigid epistemic dichotomy divided quantitative psychology into two hostile, isolated camps: the experimentalists and the correlationalists. As Lee Cronbach highlighted in his famous 1957 APA Presidential Address, experimental psychologists viewed themselves as superior scientists who exclusively utilized Analysis of Variance (ANOVA) to analyze randomized, balanced laboratory interventions; observational, personality, and social psychologists were relegated to using correlation and multiple regression to evaluate observational, survey, and quasi-experimental data. ANOVA was viewed as the pristine tool of causal laboratory science, while regression was treated as a second-class metric for observational description.

In 1968, Jacob Cohen published one of the most intellectually disruptive papers in the history of behavioral statistics in Psychological Bulletin, bearing the bold title: “Multiple Regression as a General Data-Analytic System.” In this landmark work, Cohen demolished the false dichotomy between ANOVA and regression. He demonstrated mathematically that Analysis of Variance, Analysis of Covariance (ANCOVA), and multiple correlation are not distinct mathematical universes; rather, ANOVA and ANCOVA are merely narrow, mathematically constrained special cases of the General Linear Model (GLM), perfectly subsumed within multiple linear regression.

Cohen demonstrated that any ANOVA design—regardless of how complex the factorial layout, how unbalanced the cell sizes, or how extensive the covariate structure—can be translated into a multiple regression equation through the appropriate algebraic coding of categorical independent variables. By proving this equivalence, Cohen liberated behavioral researchers from the rigid mathematical handcuffs of classical ANOVA, which required balanced, orthogonal designs with equal cell sample sizes. Multiple regression, Cohen showed, was an inherently superior, vastly more flexible, and comprehensive analytical system capable of handling categorical predictors, continuous variables, continuous covariates, and complex non-linear trajectories simultaneously within a single mathematical framework.

Armed with this insight, Cohen mounted a ferocious, decades-long campaign against a pervasive, destructive practice in psychological research: the artificial dichotomization of continuous variables through median splits. Researchers routinely took rich, continuous individual difference variables (such as anxiety, age, or intelligence) and cut them in half at the sample median to shoehorn their data into a 2×2 factorial ANOVA. Cohen exposed the severe mathematical pathologies of this practice, proving that median splits sacrifice massive statistical power (equivalent to discarding up to one-third of the sample), artificially suppress genuine effect sizes, and can introduce catastrophic spurious interaction effects that do not exist in the underlying population.

8.2 The Collaborative Masterwork: Applied Multiple Regression/Correlation Analysis (1975, 1983)

Building upon the foundational insights of his 1968 paper, Cohen embarked upon an ambitious, decadelong scholarly collaboration with his spouse, the distinguished sociologist and developmental epidemiologist Patricia Cohen. This extraordinary intellectual partnership culminated in the 1975 publication of their monumental masterwork: Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences (with a substantially expanded, definitive second edition appearing in 1983). The Cohen and Cohen text fundamentally revolutionized the teaching and execution of advanced statistics throughout the social, educational, and behavioral sciences.

The Cohens’ text bridged the daunting chasm between high-level matrix algebra and real-world behavioral applications. Its defining contribution was the systematic, exhaustively detailed treatment of categorical variable coding within regression models. The Cohens provided working researchers with the definitive mathematical blueprints for transforming qualitative group structures into quantifiable vectors through three primary coding schemas:

• Dummy Coding: Assigning 0s and 1s to represent membership in specific treatment groups relative to a fixed, designated reference or control group, allowing regression intercepts to reflect baseline control means directly and slopes to reflect specific treatment differentials.
• Effects Coding: Utilizing -1, 0, and +1 vectors to constrain deviations to sum to zero, permitting the regression intercept to represent the grand mean of the entire sample and regression coefficients to reflect the main effects of traditional ANOVA.
• Contrast Coding: Developing orthogonal algebraic weight vectors that reflect targeted, a priori theoretical comparisons between specific clusters of treatment groups, directly mapping planned contrasts into single-degree-of-freedom regression coefficients.

Furthermore, the Cohens provided the first truly lucid, mathematically rigorous exposition of interaction terms, moderation effects, and polynomial non-linear curves in multiple regression accessible to social scientists. They demonstrated that an interaction between two continuous variables is simply modeled by the inclusion of their cross-product term in the regression equation, providing meticulous pedagogical guidelines for centering variables to eliminate artificial multicollinearity and outlining the exact protocols required to calculate and graphically plot simple slopes across varying levels of a moderating variable. This masterwork transformed regression from an obscure predictive tool into the preeminent analytic engine of empirical psychology.

8.3 Pedagogical Shift: From Experimental Orthogonality to Observational Realities

The conceptual revolution catalyzed by Cohen and Cohen went far beyond algebraic notation; it constituted an epistemological liberation for non-laboratory science. For generations, behavioral scientists had been indoctrinated in the experimental paradigm established by Fisher for agricultural plots, wherein experimental treatments could be engineered to be completely orthogonal (uncorrelated with one another) through strict random assignment and balanced sample designs. However, real-world observational researchers—developmental psychologists, sociologists, clinical epidemiologists, and organizational scholars—operate in a world where human traits, social backgrounds, and biological risks are inherently correlated, multi-determined, and messy.

Classical ANOVA was profoundly ill-equipped to handle observational data; it broke down completely in the presence of correlated independent variables and unbalanced, missing cell structures. Cohen’s multiple regression framework provided social scientists with a robust, mathematically sound system explicitly designed to navigate the non-orthogonal realities of observational research. The text provided comprehensive, pioneering protocols for diagnosing and managing the acute challenges of non-experimental data:

• Multicollinearity: Cohen and Cohen outlined systematic diagnostic metrics, such as the Variance Inflation Factor (VIF) and tolerance, illustrating how high correlations among predictors inflate standard errors and destabilize regression estimates, while providing strategic remedies including ridge regression, composite index construction, and hierarchical variable sets.
• Hierarchical and Stepwise Modeling: They vigorously condemned the atheoretical, automated practice of computerized stepwise regression, demonstrating that automated entry algorithms capitalize on idiosyncratic sample noise and produce wildly non-replicable models. In its place, they championed hierarchical regression, wherein independent variables are entered into the regression model in sequential, theory-driven blocks determined strictly by causal priority, temporal ordering, and theoretical logic.
• Missing Data Diagnostics: Decades before modern full-information maximum likelihood (FIML) and multiple imputation became computationally routine, the Cohens provided early, principled strategies for diagnosing whether data were missing randomly and established dummy-variable missing-data coding techniques to salvage observational cohorts.

Through this comprehensive pedagogy, Cohen liberated observational and non-experimental researchers from the lingering inferiority complex that had haunted non-laboratory science. He demonstrated that rigorous, causally nuanced, and empirically defensible insights could be extracted from observational field studies, provided investigators deployed the General Linear Model with theoretical discipline and quantitative sophistication. The Cohen and Cohen text became one of the most cited, revered, and foundational manuals in the history of empirical social science, forever altering how quantitative reasoning is taught across universities globally.

9. Critique of Null Hypothesis Significance Testing: ‘The Earth Is Round (p < .05)'

9.1 The Epistemological Fallacy of NHST and Inverse Probability

Throughout his academic career, Cohen’s specific methodological contributions—power analysis, standardized effect sizes, kappa, and regression modeling—were anchored by an overarching philosophical campaign against the uncritical, ritualized abuse of Null Hypothesis Significance Testing (NHST). In 1994, toward the culmination of his legendary career, Cohen synthesized his lifetime of methodological skepticism into a devastating, peerless polemical essay published in the premier journal of the discipline, the American Psychologist. The paper bore a title that would become legendary across the philosophy of science: “The Earth Is Round (p < .05).”

In this masterpiece of intellectual clarity and acerbic wit, Cohen laid bare the core epistemological fallacy underlying standard significance testing: the widespread, catastrophic misinterpretation known in formal logic and probability theory as the fallacy of the transposed conditional, or the illusion of inverse probability. Cohen demonstrated that researchers universally misinterpret the fundamental meaning of a p-value. A p-value is formally defined as the probability of observing data as extreme as, or more extreme than, the observed data, assuming that the null hypothesis ($H_0$) is strictly true:

p = P(Data | H₀)

However, as Cohen incisively documented, what researchers, journal editors, and the broader public desperately want to know—and what they erroneously believe the p-value tells them—is the inverse probability: namely, what is the probability that the null hypothesis is true, given that we have observed these empirical data?

P(H₀ | Data)

Cohen demonstrated that conflating these two conditional probabilities is a devastating logical blunder. Utilizing rigorous Bayesian logic, Cohen famously illustrated this fallacy with an unforgettable syllogism:

“If a person is an American, they are probably not a member of Congress.
This person is a member of Congress.
Therefore, this person is probably not an American.”

The formal logic of this syllogism is precisely identical to the backward reasoning utilized in standard NHST. In the syllogism, the conditional probability $P(\text{Congress} | \text{American})$ is minuscule, roughly 0.000002. From this rare conditional probability, we observe a data point (the individual is a member of Congress) and draw the absurd conclusion that the null hypothesis (the individual is an American) must be false. Cohen showed that researchers commit this exact logical violation every time they reject $H_0$ based on a low $P(\text{Data} | H_0)$, completely ignoring prior probabilities and generating a systemic, dangerous illusion of scientific certainty where none exists.

9.2 The Flaw of the Nil Hypothesis and the Scale of Rejection

The second devastating critique Cohen advanced in his 1994 masterwork targeted what he memorably termed the “nil hypothesis.” Cohen observed that in standard behavioral research, the null hypothesis tested is almost universally an assertion of absolute, exact nothingness: the hypothesis that the difference between two population means is precisely $0.000000…$, or that the correlation between two psychological constructs is absolute zero. Cohen argued that outside of pristine, idealized physical systems, the nil hypothesis is an ontological absurdity; in the complex, highly interconnected, multi-determined realm of human psychology and social behavior, two groups or two variables are never identical down to the infinite decimal place.

Citing the famous quantitative insights of Paul Meehl, Cohen pointed out the existence of the “crud factor”: in social reality, everything is correlated to some degree with everything else due to complex environmental, biological, and socio-historical interconnectivity. Therefore, asserting a true population difference of absolute zero is almost always false a priori, before a single data point is even gathered. The standard scientific exercise of rejecting a nil hypothesis is fundamentally trivial because the hypothesis being tested was never true in nature to begin with.

Furthermore, Cohen demonstrated the catastrophic interaction between the nil hypothesis and sample size. Because the standard error of any test statistic shrinks toward zero as sample size $N$ increases toward infinity, any nonzero population effect—no matter how microscopically trivial, meaningless, or clinically irrelevant—will inevitably achieve a statistically significant $p < .05$ threshold if the investigator simply gathers a large enough sample. Conversely, a transformative, massive effect will be labeled “non-significant” if the sample size is small. Cohen vigorously castigated journal editors and peer reviewers who deployed $p < .05$ as an unthinking, automated gatekeeping mechanism, treating statistical significance as an ontological switch that turns scientific phenomena on or off, rather than evaluating the substantive magnitude, theoretical coherence, and practical value of the empirical observations.

9.3 Proposed Remedial Methodologies: Confidence Intervals and Graphic Representation

Jacob Cohen was not merely an iconoclastic critic bent on tearing down traditional structures; he was, above all, a constructive pedagogue dedicated to offering actionable, mathematically superior solutions. In the concluding sections of “The Earth Is Round (p < .05)” and throughout his final writings, Cohen laid out a comprehensive blueprint for remedial methodological reform, urging behavioral scientists to permanently abandon their mechanical addiction to dichotomous hypothesis testing and embrace an alternative data-analytic philosophy.

At the center of Cohen’s remedial program was the routine, mandatory reporting of parameter estimates paired with Confidence Intervals (CIs). Cohen argued that confidence intervals possess immense cognitive and scientific superiority over raw p-values. While a p-value compresses all information down to an uninformative, binary decision metric, a confidence interval keeps the researcher rooted in the real-world units of the underlying scale or standardized effect size metric. A confidence interval simultaneously communicates two vital pieces of information: the point estimate of the effect magnitude and the precision of the measurement, starkly exposing the uncertainty of the estimate. If a 95% confidence interval for an effect spans from 0.02 to 0.85, it instantly signals to the researcher that their data are practically uninformative, even if the result happened to reach statistical significance ($p = .04$).

Cohen also passionately championed the primacy of graphical data exploration, visual displays, and cumulative replication. He urged researchers to spend time looking at their raw data, examining frequency distributions, plotting scattergrams, and visualizing overlapping group densities rather than immediately running mechanized inferential algorithms. Above all, Cohen called for a philosophical shift toward intellectual humility. He argued that true cumulative science is never established by a single, isolated, miraculous study that scrapes past the $p < .05$ barrier; rather, scientific truth emerges exclusively through the slow, iterative accumulation of replicated effect sizes synthesized across multiple independent laboratories. Cohen pleaded with the scientific community to replace mechanical rituals with critical thinking, transparency, and honest acknowledgments of empirical uncertainty.

10. Academic Pedagogy, Faculty Leadership, and Mentorship at New York University

10.1 Four Decades of Quantitative Pedagogy at NYU

While Jacob Cohen’s global reputation was forged through his written publications, his immediate academic environment was defined by four decades of transformative teaching, academic stewardship, and departmental leadership at New York University. Joining the faculty in 1959, Cohen served as the intellectual anchor of the NYU Department of Psychology until his retirement as Professor Emeritus in 1993. Over these thirty-four years of active faculty service, Cohen built NYU into an international powerhouse for training in quantitative methods, psychometrics, and advanced research design.

In the lecture hall, Cohen was a master pedagogue of legendary stature. He completely rejected the traditional mathematical pedagogical style that relied on terrifying graduate students with dense, opaque matrix proofs and endless mechanical derivations. Instead, Cohen prioritized conceptual intuition, graphical reasoning, and clear verbal translation. He believed that if a statistical concept could not be explained in plain English to an intelligent undergraduate, the instructor did not truly understand the concept. His classes were renowned for their warmth, their intellectual accessibility, and their vibrant wit. Cohen dismantled the pervasive “statistics anxiety” that paralyzed generations of clinical and behavioral graduate students, empowering them to engage with data not as passive technicians following a cookbook, but as creative, skeptical investigators.

Through this humanistic pedagogical philosophy, Cohen mentored a generation of doctoral students who went on to become leading figures in quantitative psychology, biostatistics, psychometrics, and behavioral science. His students populated psychology departments, medical schools, and research institutions across the world, carrying forward Cohen’s commitments to statistical power, measurement reliability, and inferential skepticism. Cohen demonstrated that quantitative psychology was not an isolated, sterile discipline of abstract numbers, but a profoundly human, interdisciplinary craft dedicated to uncovering truth amidst the noise of empirical reality.

10.2 Scholarly Partnership with Patricia Cohen

It is impossible to fully comprehend Jacob Cohen’s intellectual trajectory without examining his deep scholarly, intellectual, and personal partnership with his spouse, Dr. Patricia Cohen. A distinguished sociologist, quantitative methodologist, and developmental epidemiologist in her own right, Patricia Cohen brought a unique sociological and longitudinal perspective that beautifully complemented Jacob’s clinical and psychometric orientations. Their marriage was a true intellectual union, marked by decades of vibrant scholarly collaboration that permanently altered the landscape of social science methodology.

Patricia Cohen was the director of the groundbreaking Children in the Community study, one of the most comprehensive, scientifically rigorous longitudinal epidemiological investigations of child and adolescent mental health ever conducted. This massive, multi-wave project tracked a representative cohort of over seven hundred children from the early 1970s through adulthood across several decades. The Children in the Community study served as an invaluable empirical laboratory for the Cohens’ collaborative methodological innovations. It provided massive, complex, real-world data characterized by developmental trajectories, multi-wave longitudinal assessments, correlated observations, and inevitable participant attrition.

The integration of these rich developmental datasets directly into their joint textbooks and methodological treatises elevated their work far above standard statistical manuals. Together, Jacob and Patricia Cohen pioneered advanced pedagogical approaches to tracking growth curves, modeling non-linear developmental trends, disentangling age, cohort, and period effects, and handling complex multi-variable interactions over time. Their intellectual partnership was characterized by profound mutual respect and an egalitarian exchange of ideas, representing one of the most fruitful and transformative collaborative alliances in the history of modern social science.

10.3 Consultative Influence Across Medicine, Psychiatry, and Industry

Beyond his university duties, Jacob Cohen’s methodological expertise was continuously sought after by premier public health agencies, medical institutions, and the biomedical industry. Throughout his career, Cohen served as a high-level methodological consultant to the National Institute of Mental Health (NIMH), the Veterans Administration, the World Health Organization, and various prominent pharmaceutical and medical research organizations. In these consultative capacities, Cohen functioned as an indispensable arbiter of statistical validity and clinical trial integrity.

Cohen’s consultative impact was particularly profound in the domains of clinical psychiatry, psychopharmacology, and neurology. During the late twentieth century, as the psychiatric establishment shifted away from psychoanalysis toward biological psychiatry and somatic therapies, billions of dollars were pouring into clinical drug trials evaluating novel psychotropic compounds. Cohen routinely reviewed multi-center clinical trial designs, identifying catastrophic power deficiencies, advising on sample recruitment targets, and insisting on the inclusion of Weighted Kappa metrics to verify diagnostic and symptom rating concordance across multi-site clinical investigators.

Cohen’s consultative work was marked by an unyielding ethical commitment to scientific integrity. He consistently refused to endorse underpowered or methodologically flawed corporate research designs, warning industry executives and biomedical researchers that compromised methodologies inevitably produce false-positive therapeutic claims that mislead clinicians and endanger patient populations. His fierce independence and intellectual authority forced institutional and corporate entities across North America and Europe to dramatically elevate their methodological standards, further cementing his legacy as the uncompromising conscience of applied empirical research.

11. Awards, Professional Honors, and Academic Recognition

11.1 Major Honors from the American Psychological Association (APA)

Throughout his illustrious academic career, Jacob Cohen received the highest formal honors that the scientific community can bestow upon a behavioral scientist. His pioneering methodological contributions were celebrated not merely as technical innovations, but as transformative intellectual tools that permanently reshaped the trajectory of empirical psychology. In 1997, the American Psychological Association (APA) awarded Cohen its prestigious Distinguished Scientific Tool Award, an extraordinary honor reserved for individuals whose methodological, computational, or instrumental inventions have profoundly advanced the global landscape of psychological science.

The citation honoring Cohen for the Scientific Tool Award acknowledged his comprehensive, lifetime achievements across four distinct methodological frontiers: the development and popularization of statistical power analysis, the formulation of standardized effect sizes, the creation of Cohen’s Kappa for inter-rater agreement, and the mathematical unification of the General Linear Model via multiple regression. The APA noted that it was virtually impossible to open any major psychological journal anywhere in the world without encountering empirical investigations designed, analyzed, or interpreted through the methodological frameworks that Jacob Cohen had constructed.

In addition to this national recognition, Cohen received extensive lifetime contribution honors from Division 5 of the APA (Evaluation, Measurement, and Statistics, now known as Quantitative and Qualitative Methods). He served as President of Division 5, using his executive platform to advocate for sweeping curricula reforms and actively lobbying scientific journals to modernize their statistical reporting mandates. Cohen was elected a Fellow of the American Psychological Association, the American Association for the Advancement of Science (AAAS), the American Statistical Association (ASA), and the Association for Psychological Science (APS), standing as a revered elder statesman across the full spectrum of quantitative and behavioral disciplines.

11.2 Institutional Recognition and the 1998 Passing of Jacob Cohen

At New York University, Cohen was celebrated as one of the institution’s most distinguished faculty luminaries. Upon his formal academic retirement in 1993, he was awarded the status of Professor Emeritus of Psychology, continuing to maintain an active scholarly presence, consulting with students and colleagues, and continuing his relentless publication crusade for statistical reform. His retirement was marked by retrospective conferences and scholarly celebrations that brought together decades of former students, prominent biostatisticians, and international collaborators to reflect upon his transformative institutional footprint.

On June 18, 1998, Jacob Cohen passed away at the age of seventy-five in Allentown, Pennsylvania, leaving the international scientific community in profound mourning. His passing triggered an immediate, worldwide outpouring of academic tributes, memorial symposia, and reflective essays across major medical, psychiatric, and psychological journals. Leading scientific publications, including the American Psychologist, Psychological Science, and Educational and Psychological Measurement, published extensive retrospectives celebrating his extraordinary life and assessing the monumental magnitude of his intellectual legacy.

Colleagues, university administrators, and international scholars gathered at NYU and at international academic conventions to commemorate a man whose life work had permanently altered the epistemological architecture of behavioral research. The academic community recognized that with the passing of Jacob Cohen, science had lost not merely a brilliant mathematical innovator, but its foremost methodological conscience—a scholar who had dedicated his entire intellectual existence to elevating the truth-seeking capacity of empirical inquiry.

11.3 Tributes to Cohen’s Wit, Humanitarian Spirit, and Intellectual Clarity

What distinguished Jacob Cohen from virtually all other quantitative innovators of his generation was his extraordinary personal warmth, his boundless humanitarian spirit, and his peerless, self-deprecating wit. In a discipline often characterized by sterile academic posturing, impenetrable mathematical jargon, and dry exposition, Cohen wrote and lectured with the comedic timing and narrative flair of a classic New York raconteur. His writings were peppered with brilliant, memorable metaphors, satirical allegories, and poignant parables designed to puncture academic pretension.

Among his most famous pedagogical allegories was his satirical commentary on the absurdity of nil-hypothesis significance testing, exemplified by his celebrated “miracle cure” anecdote. Cohen would deadpan to his classes that if a psychiatric investigator discovered a treatment that cured 80% of institutionalized schizophrenics, but tested it on an experimental cohort of only four patients ($N = 4$), the result would be statistically non-significant ($p > .05$). Under standard journal review protocols, the researcher would be forced to conclude that the cure did not work and discard it, whereas an investigator who found an imperceptible, entirely useless 0.001 point reduction in symptom scores across a massive clinical sample of 100,000 patients would triumphantly proclaim a “highly statistically significant scientific breakthrough” ($p < .0001$). Through such vivid, humorous illustrations, Cohen permanently imprinted the critical distinction between statistical significance and real-world practical importance upon generations of scientists.

Testimonials penned following his death universally celebrated his immense personal generosity. Cohen was renowned for spending hours patiently answering unsolicited letters and telephone inquiries from confused, early-career researchers and graduate students across the world who were struggling with their experimental designs or power calculations. He treated every inquiry with total seriousness, kindness, and intellectual generosity. Cohen saw quantitative statistics not as an elite barrier designed to exclude outsiders, but as a universal, egalitarian instrument of human enlightenment that should be made accessible to every individual seeking empirical truth.

12. Enduring Impact on Modern Quantitative Psychology and the Open Science Movement

12.1 Anticipation of the Contemporary Replication Crisis

Viewed from the vantage point of twenty-first-century science, Jacob Cohen’s scholarship was not merely influential; it was extraordinarily prophetic. During the second decade of the twenty-first century, psychology, biomedicine, and the social sciences were struck by a profound epistemic earthquake universally known as the Replication Crisis (or Reproducibility Crisis). Flagship collaborative replication initiatives, such as the landmark 2015 study conducted by the Open Science Collaboration which attempted to replicate one hundred prominent psychological experiments, revealed that over 60% of published findings failed to successfully replicate when subjected to rigorous, independent empirical re-testing.

When the scientific community erupted in shock, despair, and self-flagellation over these catastrophic reproducibility failures, historians of science noted with grim clarity that Jacob Cohen had predicted this exact collapse more than fifty years earlier. The primary architectural culprits identified by modern open science reformers—publication bias, the systemic inflation of effect sizes, widespread p-hacking, flexible data-analytic stopping rules, and, above all, the routine execution of severely underpowered studies—were precisely the methodological pathologies that Cohen had meticulously documented in his 1962 audit of the Journal of Abnormal and Social Psychology.

Cohen had clearly warned the discipline that an academic culture which incentivizes the uncritical pursuit of $p < .05$ within small, underpowered samples is mathematically guaranteed to fill its journals with spurious, false-positive discoveries. The contemporary replication crisis was nothing less than the historical chickens of the twentieth century coming home to roost. The open science movement of today is, in its intellectual essence, the direct modern institutionalization of Jacob Cohen’s lifetime crusade, validating every mathematical and epistemological warning he issued over the course of four decades.

12.2 Institutionalization of Power Analysis in Modern Research Protocols

Today, the practical methodologies conceptualized and formalized by Jacob Cohen have been codified into the absolute, mandatory infrastructure of modern scientific research. Across the international scientific landscape, prospective statistical power analysis is no longer an optional recommendation; it is an ironclad, non-negotiable prerequisite enforced by major governmental funding bodies, institutional review boards, and premier academic journals. Grant review panels at the National Institutes of Health (NIH), the National Science Foundation (NSF), and the European Research Council (ERC) will instantly reject any empirical research proposal that fails to present a mathematically rigorous, prospective power analysis justifying its proposed sample size targets.

Furthermore, Cohen’s mathematical algorithms have been universally integrated into modern computational tools that are utilized daily by millions of scientists across the globe. Software packages such as G*Power—a world-standard computational tool developed by Franz Faul and colleagues—derive their underlying mathematical mechanics directly from the algorithmic models and effect size formalisms established in Cohen’s 1988 volume. Within the open-source statistical programming language R, foundational packages such as pwr (developed by Stephane Champely) and irr (for inter-rater reliability) are direct software implementations of Cohen’s classical power equations, effect size metrics, and Kappa formalisms.

Similarly, editorial guidelines across premier academic publishing organizations, spearheaded by the American Psychological Association’s Task Force on Statistical Inference (which was formed largely in response to Cohen’s 1994 paper), now universally mandate the reporting of standardized effect sizes, confidence intervals, and exact power parameters alongside, or entirely in lieu of, traditional null-hypothesis tests. The academic world has fundamentally restructured its reporting architecture along the precise blueprints that Jacob Cohen drafted.

12.3 Jacob Cohen’s Permanent Place in the History of Statistics

In the grand historical pantheon of quantitative science, Jacob Cohen occupies a unique, permanent, and revered position. He stands alongside towering figures such as Karl Pearson, Ronald Fisher, Jerzy Neyman, Egon Pearson, and John Tukey as one of the definitive architects of modern statistical reasoning. Yet, whereas Fisher and Neyman approached statistics through the prism of agricultural genetics and abstract mathematical proofs, Cohen approached statistics through the messy, vulnerable, and profoundly complex lens of applied human behavior and clinical suffering.

Cohen’s enduring genius was his synthesis of uncompromising mathematical precision with profound practical empathy for the working empirical researcher. He recognized that statistical methods are not sacred, divine tablets handed down from heaven; they are human-made intellectual tools designed to assist fallible human minds in the arduous labor of discovering truth. He was the great demystifier of quantitative science, sweeping away ritualized dogma, exposing statistical illusions, and replacing mechanical computation with transparent, rational, and intellectually humble inference.

Jacob Cohen was, in the truest and most noble sense, the conscience of behavioral science. His life’s work serves as an eternal reminder that the pursuit of scientific truth demands not merely the accumulation of data, but an unwavering, self-critical commitment to the integrity of the methods by which that data is evaluated. As modern science continues its halting, determined march toward greater transparency, reproducibility, and epistemological clarity, it walks along the intellectual trail blazed by Jacob Cohen—a trail constructed with deep mathematical insight, profound pedagogical generosity, and an enduring love for scientific truth.

Conclusion

The legacy of Jacob Cohen (1923–1998) is woven into the very fabric of how empirical inquiry is executed, evaluated, and conceptualized in modern behavioral and biomedical science. Across a luminous career that traversed five critical decades of twentieth-century academic development, Cohen fundamentally revolutionized the philosophy and practice of scientific inference. From his childhood in working-class Brooklyn and his wartime logistical training, to his pioneering clinical diagnostics within the Veterans Administration and his decades of faculty leadership at New York University, Cohen approached the challenge of quantitative measurement with an extraordinary combination of technical brilliance and deep humanistic commitment.

Through statistical power analysis, he provided the behavioral sciences with the mathematical machinery necessary to rationally plan research and avoid the catastrophic, waste-strewn landscape of false-negative experiments. Through standardized effect size metrics—Cohen’s d, f, $f^2$, q, and w—he unlocked the potential for scale-invariant measurement and paved the way for modern meta-analytic synthesis. Through Cohen’s Kappa and its weighted variants, he liberated clinical diagnostics from the illusions of chance-driven observer concordance. Through his unification of ANOVA within multiple regression, he demolished artificial barriers that had long divided laboratory experimentalists from field observers. And through his brilliant philosophical critiques, culminating in “The Earth Is Round (p < .05),” he struck a mortal blow against mechanical, thoughtless null-hypothesis testing, urging the global community toward confidence intervals, graphical clarity, and intellectual humility.

As the scientific community in the twenty-first century continues to navigate the challenges of the replication crisis and the open science transition, Jacob Cohen’s writings remain as vibrant, urgent, and indispensable as when they were first penned. He showed generations of researchers that true scientific rigor is not measured by the mechanical attainment of an arbitrary p-value, but by the magnitude, precision, and replicability of genuine empirical phenomena. Jacob Cohen stands immortalized in the history of ideas not merely as a quantitative master, but as a towering champion of scientific integrity, intellectual lucidity, and the humane pursuit of empirical truth.

References

• Cicchetti, D. V., & Feinstein, A. R. (1990). High agreement but low kappa: II. Resolving the paradoxes. Journal of Clinical Epidemiology, 43(6), 551–558. https://doi.org/10.1016/0895-4356(90)90159-M
• Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46. https://doi.org/10.1177/001316446002000104
• Cohen, J. (1962). The statistical power of abnormal-social psychological research: A review. The Journal of Abnormal and Social Psychology, 65(3), 145–153. https://doi.org/10.1037/h0045186
• Cohen, J. (1968a). Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychological Bulletin, 70(4), 213–220. https://doi.org/10.1037/h0026256
• Cohen, J. (1968b). Multiple regression as a general data-analytic system. Psychological Bulletin, 70(6, Pt.1), 426–443. https://doi.org/10.1037/h0026714
• Cohen, J. (1969). Statistical power analysis for the behavioral sciences (1st ed.). Academic Press.
• Cohen, J. (1977). Statistical power analysis for the behavioral sciences (Rev. ed.). Academic Press.
• Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates. https://doi.org/10.4324/9780203771587
• Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45(12), 1304–1312. https://doi.org/10.1037/0003-066X.45.12.1304
• Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49(12), 997–1003. https://doi.org/10.1037/0003-066X.49.12.997
• Cohen, J., & Cohen, P. (1975). Applied multiple regression/correlation analysis for the behavioral sciences (1st ed.). Lawrence Erlbaum Associates.
• Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.
• Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Lawrence Erlbaum Associates. https://doi.org/10.4324/9780203774441
• Cronbach, L. J. (1957). The two disciplines of scientific psychology. American Psychologist, 12(11), 671–684. https://doi.org/10.1037/h0043943
• Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. https://doi.org/10.3758/BF03193146
• Feinstein, A. R., & Cicchetti, D. V. (1990). High agreement but low kappa: I. The problems of two paradoxes. Journal of Clinical Epidemiology, 43(6), 543–549. https://doi.org/10.1016/0895-4356(90)90158-L
• Fisher, R. A. (1925). Statistical methods for research workers. Oliver and Boyd.
• Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5), 378–382. https://doi.org/10.1037/h0031619
• Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5(10), 3–8. https://doi.org/10.3102/0013189X005010003
• Gwet, K. L. (2008). Computing inter-rater reliability and its variance in the presence of high agreement. British Journal of Mathematical and Statistical Psychology, 61(1), 29–48. https://doi.org/10.1348/000711006X126600
• Hedges, L. V. (1981). Distribution theory for Glass’s estimator of effect size and related estimators. Journal of Educational Statistics, 6(2), 107–128. https://doi.org/10.3102/10769986006002107
• Krippendorff, K. (1970). Estimating the reliability, systematic error and random error of data. Educational and Psychological Measurement, 30(1), 61–70. https://doi.org/10.1177/001316447003000105
• Meehl, P. E. (1967). Theory-testing in psychology and in physics: A methodological paradox. Philosophy of Science, 34(2), 103–115. https://doi.org/10.1086/288135
• Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46(4), 806–834. https://doi.org/10.1037/0022-006X.46.4.806
• Neyman, J., & Pearson, E. S. (1933). On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, 231(694–706), 289–337. https://doi.org/10.1098/rsta.1933.0009
• Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. https://doi.org/10.1126/science.aac4716
• Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641. https://doi.org/10.1037/0033-2909.86.3.638
• Sedlmeier, P., & Gigerenzer, G. (1989). Do studies of statistical power have an effect on the power of studies? Psychological Bulletin, 105(2), 309–316. https://doi.org/10.1037/0033-2909.105.2.309
• Shrout, P. E. (1998). Jacob Cohen (1923–1998). American Psychologist, 54(6), 438. https://doi.org/10.1037/0003-066X.54.6.438
• Wilkinson, L., & the Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54(8), 594–604. https://doi.org/10.1037/0003-066X.54.8.594