Assessment of Situational Judgement (ASJ)

Abstract

The Assessment of Situational Judgement (ASJ) is an empirically validated psychometric instrument developed by Sarah J. Getz, Bonnie E. Levin, and James E. Galvin (2024) to evaluate individual susceptibility to financial fraud and consumer scams. Rooted in situational judgment testing methodologies and behavioral neuroeconomics, the ASJ addresses a critical public health and neuropsychological concern: the exploitation of financially vulnerable individuals, particularly older adults experiencing subtle age-related cognitive changes or preclinical neurodegenerative conditions such as mild cognitive impairment (MCI) and Alzheimer's disease. The ASJ consists of 17 ecologically valid vignette-based scenarios derived from real-world complaints registered with the Florida Division of Consumer Services, paired with benign, legitimate everyday transactions. The tool assesses an individual's capacity to discriminate between fraudulent solicitations and legitimate propositions across various modalities, including internet-based, telephonic, and in-person interactions. Respondents rate their behavioral likelihood of engaging with each presented scenario on a 7-point Likert scale ranging from 1 ("extremely unlikely") to 7 ("extremely likely"). Psychometric evaluations reveal a robust two-factor latent structure comprising a Scam Scenarios factor and a Legitimate Scenarios factor, explaining 50.6% and 10.6% of the variance, respectively. Internal consistency is exceptionally high for the Scam factor (α = 0.98) and adequate for the Legitimate factor (α = 0.73). Receiver operating characteristic (ROC) curve analyses demonstrate strong diagnostic discriminability, with an area under the curve (AUC) of 0.790 for the Scam factor, 0.730 for the Legitimate factor, and 0.767 for the composite Scam/Legitimate ratio. The instrument differentiates effectively based on digital literacy, sociodemographic factors, and history of scam victimization, establishing the ASJ as a vital assessment tool for clinical neuropsychology, geriatric medicine, and financial forensics.

Keywords

Assessment of Situational Judgement, ASJ, scam susceptibility, financial exploitation, situational judgment test, elder fraud, neuroeconomics, mild cognitive impairment, Alzheimer's disease, decision-making, consumer protection, psychometrics

Authors

The Assessment of Situational Judgement was conceived, developed, and validated by an interdisciplinary team of clinical neuropsychologists and behavioral neurologists:

• Sarah J. Getz, Ph.D. – Department of Neurology, Comprehensive Center for Brain Health, University of Miami Miller School of Medicine, Miami, FL, USA. Dr. Getz specializes in neuropsychological assessment, cognitive aging, and the ecological measurement of complex real-world decision-making capacities.
• Bonnie E. Levin, Ph.D. – Division of Neuropsychology, Department of Neurology, University of Miami Miller School of Medicine, Miami, FL, USA. Dr. Levin is an internationally recognized neuropsychologist focusing on cognitive decline, executive function, and behavioral biomarkers of neurodegenerative diseases.
• James E. Galvin, M.D., M.P.H. – Comprehensive Center for Brain Health, Department of Neurology, University of Miami Miller School of Medicine, Miami, FL, USA. Dr. Galvin is a leading behavioral neurologist renowned for developing screening tools for dementia, Lewy body disorders, and cognitive-functional resilience in late life.

Purpose

The primary purpose of the Assessment of Situational Judgement (ASJ) is to operationalize and quantify an individual's behavioral vulnerability to fraudulent schemes while concurrently examining their ability to maintain engagement in legitimate financial and social exchanges. Modern financial fraud has evolved from rudimentary unsolicited mail campaigns into sophisticated, multi-channel psychological manipulation operations, including advanced phishing, imposter schemes, deceptive home-repair solicitations, and artificial intelligence-augmented telecommunication scams. While consumer education campaigns often assume that financial knowledge alone confers protection, empirical evidence reveals that individuals with intact factual knowledge frequently fall victim due to situational, social, and emotional processing breakdowns.

In clinical and neuropsychological contexts, standard cognitive batteries (e.g., assessing abstract reasoning, episodic memory, or visuoperceptual processing) frequently fail to capture the nuanced lapses in social cognition, emotional regulation, and risk appraisal that precede overt functional disability. An individual can score within normal limits on the Mini-Mental State Examination (MMSE) or the Montreal Cognitive Assessment (MoCA) while exhibiting catastrophic deficits in ecological decision-making. The ASJ bridges this diagnostic gap by presenting realistic, emotionally evocative situational vignettes that require the integration of cognitive risk appraisal, skepticism, and contextual verification.

The clinical and research applications of the ASJ are extensive:

• Early Detection of Neurodegenerative Vulnerability: Serving as a behavioral proxy for dysfunction in neural circuits supporting social judgment, such as the ventromedial prefrontal cortex (vmPFC) and insular networks, which often deteriorate during preclinical stages of Alzheimer's disease and frontotemporal lobar degeneration.
• Forensic and Capacity Evaluations: Providing objective, standardized empirical data for forensic neuropsychologists, legal professionals, and adult protective services evaluating an older adult's testamentary capacity, financial competence, or need for fiduciary conservatorship.
• Intervention Design and Risk Stratification: Permitting clinicians and community advocates to profile an individual's specific risk signature. For instance, distinguishing between an individual who displays excessive credulity (elevated scam compliance) versus an individual demonstrating generalized paranoia or cognitive withdrawal (indiscriminate rejection of legitimate transactions).
• Financial Services and Consumer Protection: Offering a standardized baseline for financial institutions to formulate individualized safeguarding protocols, tailored fraud alerts, and supportive decision-making frameworks.

Psychological Construct

The ASJ measures Scam Susceptibility, conceptualized as a multi-dimensional psychological and behavioral construct characterized by the reduced capacity to detect deceptive intent, interrogate predatory heuristics, and withhold compliance when confronted with fraudulent propositions. Scam susceptibility exists at the confluence of cognitive processing, personality architecture, social cognition, and affective regulation. Crucially, the construct does not simply reflect generalized gullibility; rather, it reflects a failure of contextual discernment where deceptive markers are minimized and perceived credibility is inappropriately inflated.

To accurately capture this phenomenon without confounding scam vulnerability with generalized institutional distrust, the ASJ establishes a dual-construct measurement paradigm comprising two distinct latent dimensions:

1. Scam Susceptibility (Scam Scenarios Dimension)

This subscale evaluates the respondent's propensity to accept and act upon deceptive solicitations that mimic authentic economic, legal, or interpersonal propositions. The fraudulent vignettes operationalized in the ASJ mirror the tactical mechanics of contemporary fraud:

• Authority and Legal Coercion: Scenarios wherein perpetrators impersonate legal authorities, law enforcement agencies, or utility representatives demanding immediate compliance or financial remittance under threat of punitive sanctions.
• Phantom Riches and Reciprocity Exploitation: Situations involving sudden lottery notifications, unsolicited foreign investments, or grand sweepstakes that activate reward-seeking neurobehavioral pathways while requiring upfront "processing" or "administrative" fees.
• Urgency and Affective Hijacking: Scenarios engineering artificial time sensitivity (e.g., compromised computer networks, imperative bank account verifications, or urgent home maintenance emergencies) that force reflexive, intuitive decision-making by bypassing reflective deliberation.

High scores on the Scam Scenarios dimension indicate pronounced behavioral susceptibility, reflecting a breakdown in skepticism, failure to identify critical inconsistencies, and an unsafe willingness to yield personal identifying information, financial access, or monetary assets to unverified external agents.

2. Legitimate Transaction Discernment (Legitimate Scenarios Dimension)

An essential psychometric contribution of the ASJ is its explicit inclusion of authentic, benign scenarios that mirror daily civic and consumer activities. These items depict routine, authorized requests, such as standard municipal communications, verified banking notifications regarding suspicious transactions, or legitimate contractor estimates following requested appointments.

The inclusion of this dimension is vital for several theoretical and diagnostic reasons:

• Differential Diagnosis of Executive Processing: Older individuals who develop cognitive impairment may adopt a globalized defensive strategy characterized by complete behavioral inhibition and distrust, rejecting both fraudulent solicitations and essential life-management tasks (e.g., refusing to pay verified municipal utility bills or ignoring urgent legitimate banking alerts).
• Signal Detection Theory Integration: By pairing scam items (noise/target deception) with legitimate items (signals/authentic transactions), the ASJ permits the computation of discriminability indices (Signal Detection Theory). This separates true fraud discernment from extreme response biases (e.g., indiscriminate agreeableness or hyper-vigilant paranoia).

Theoretical Framework

The architectural foundation of the ASJ rests upon integrative neuroeconomic, cognitive, and social-psychological models of human decision-making, specifically the Dual-Process Theory of Cognition, the Somatic Marker Hypothesis, and the Scam Compliance Model.

Dual-Process Theory and Heuristic Hijacking

Under Dual-Process Theory, human cognitive processing operates via two interactive engines: System 1 (an intuitive, rapid, automatic, and emotionally charged system) and System 2 (a reflective, slow, deliberative, effortful, and rule-governed system). Fraud perpetrators intentionally craft solicitations to hijack System 1 heuristics. By embedding high emotional valence (e.g., fear of arrest, euphoria of an unexpected inheritance, or empathy for an imperiled grandchild), scammers induce an affective state that suppresses System 2 analytic verification.

In neurobiologically vulnerable individuals, such as aging adults showing volumetric reductions in the lateral prefrontal cortex, the ability of System 2 to override biased System 1 outputs becomes impaired. The ASJ captures this vulnerability by presenting contextual stimuli that trigger System 1 cognitive shortcuts (e.g., social proof, scarcity, perceived authority). The tool tests whether the respondent's supervisory attentional control can arrest the impulsive urge to comply and activate critical investigative behaviors.

The Somatic Marker Hypothesis and Prefrontal Networks

According to the Somatic Marker Hypothesis formulated by Antonio Damasio, bioreactive signals ("gut feelings") generated by the ventromedial prefrontal cortex (vmPFC), amygdala, and insula guide decision-making under conditions of risk and uncertainty. When healthy adults encounter an ambiguous or covertly exploitative offer, these somatic markers trigger a covert autonomic warning response (a sensation of doubt, distrust, or visceral uneasiness) before conscious intellectual reasoning can fully map the deceit.

Preclinical neurodegeneration, subclinical vascular white matter hyperintensities, and normative senescence often disrupt vmPFC-amygdala structural connectivity. This leads to what neuroscientists describe as an "insensitivity to future consequences." Affected individuals retain intact crystallized intellect and verbal fluency, yet their physiological and behavioral alarm systems fail to signal deception. The ASJ presents concrete, multi-modal situational contexts that simulate real-life interactions, effectively probing the functional integrity of these prefrontal warning circuits.

The Tripartite Model of Vulnerability

The ASJ also operationalizes the sociodemographic and cognitive vulnerability frameworks established by Lichtenberg and colleagues, which postulate that financial decision-making capacity relies on three interdependent pillars:

1. Intellectual Competence: Fundamental cognitive faculties, including memory, processing speed, and numeracy.
2. Contextual Judgment: Situational appraisal, recognition of interpersonal motives, and real-time behavioral self-regulation.
3. Social and Emotional Context: Factors such as loneliness, social isolation, technological estrangement, and trust dysregulation.

By capturing responses to situational dilemmas rather than testing decontextualized cognitive skills (such as serial subtractions or list learning), the ASJ functions as an ecologically valid operationalization of the contextual judgment pillar.

Validity

The psychometric validation of the ASJ, as reported by Getz, Levin, and Galvin (2024), provides extensive empirical evidence supporting its construct, criterion, and discriminative validity across diverse community-dwelling cohorts.

Construct and Discriminant Validity

Construct validity was demonstrated through clear dissociations between the tool's subscales and relevant sociodemographic and behavioral profiles. In initial validation studies conducted across adult cohorts recruited via Amazon Mechanical Turk (MTurk), the ASJ demonstrated significant sensitivity to factors known to influence financial risk:

• Technology and Digital Engagement: Participants who reported frequent, competent engagement with computers, email, and social media platforms scored significantly lower on the Scam Scenarios factor. This confirms that digital familiarity provides an experiential scaffold that facilitates the identification of deceptive online cues.
• Prior Scam Exposure: The ASJ effectively distinguished individuals with past histories of fraud victimization from non-victimized peers. Individuals with prior victimizations exhibited significantly higher self-reported willingness to engage with high-risk scenarios, confirming that the ASJ captures real-world vulnerability rather than hypothetical or socially desirable responses.
• Sociodemographic Divergence: Performance on the ASJ demonstrated predictable patterns across age cohorts, educational attainment levels, and cognitive engagement indices, corroborating its sensitivity to cognitive-developmental and experiential variations.

Criterion and Diagnostic Validity (ROC Analysis)

To establish the clinical and classification utility of the ASJ, the researchers conducted detailed Receiver Operating Characteristic (ROC) curve analyses to assess its accuracy in classifying individuals at elevated risk for scam victimization:

• Scam Factor Discriminability: The Scam Scenarios subscale achieved an Area Under the Curve (AUC) of 0.790 (95% CI [0.730, 0.850]), indicating good-to-excellent diagnostic accuracy in discriminating between individuals vulnerable to scamming and those with robust skepticism.
• Legitimate Factor Discriminability: The Legitimate Scenarios subscale produced an AUC of 0.730 (95% CI [0.660, 0.800]), proving its capability to identify individuals exhibiting maladaptive generalized distrust or impaired everyday functional engagement.
• Scam/Legitimate Ratio Metric: A composite metric calculated by evaluating the ratio of the Scam score to the Legitimate score generated an AUC of 0.767 (95% CI [0.702, 0.832]). This composite metric provides clinicians with a standardized index of relative discernment, balancing gullibility against excessive paranoia.

Reliability

The Assessment of Situational Judgement exhibits strong internal consistency, confirming that its items consistently measure their target constructs without excessive measurement noise.

Internal Consistency

In the primary psychometric validation study by Getz et al. (2024), reliability estimates across the instrument's constituent dimensions revealed clear internal coherence:

• Scam Scenarios Dimension: Demonstrated an internal consistency reliability of Cronbach's α = 0.98. This high alpha value confirms strong inter-item covariance among fraudulent vignettes, indicating that individuals who identify predatory markers in telephonic contexts consistently identify them across digital and face-to-face vectors.
• Legitimate Scenarios Dimension: Exhibited an internal consistency coefficient of Cronbach's α = 0.73. This value satisfies the standard psychometric threshold for clinical and research subscales (α ≥ 0.70). The lower alpha relative to the Scam factor reflects the intentional heterogeneity of the legitimate control items, which span diverse real-world activities such as municipal banking, retail transactions, and medical administration.

Standard Error of Measurement and Precision

The high internal consistency observed for the primary Scam subscale yields a minimal Standard Error of Measurement (SEM), maximizing the instrument's reliability across clinical cut-off boundaries. This reduces the risk of false-positive classifications when screening older adults for impaired financial capacity.

Factor Analysis

The latent structure of the 17-item ASJ was established using exploratory and confirmatory factor analytic frameworks, validating the theoretical distinction between fraudulent risk appraisal and authentic civic interaction.

Factor Extraction and Variance Explanation

Factor analysis with oblique rotation confirmed a clear, robust two-factor solution, explaining an aggregate of 61.2% of the total response variance:

• Factor 1: Scam Scenarios: This dominant factor accounted for the vast majority of the variance, explaining 50.6% of total scale variance (eigenvalue > 8.0). All fraudulent vignettes loaded onto this latent dimension with high factor loadings (predominantly > 0.70), demonstrating that scam compliance represents a unified, generalizable behavioral trait.
• Factor 2: Legitimate Scenarios: This factor accounted for 10.6% of the total variance (eigenvalue > 1.5). All authentic, non-deceptive vignettes loaded cleanly onto this dimension, confirming that an individual's willingness to engage in valid daily transactions represents a distinct cognitive and behavioral construct from scam susceptibility.

Structural Fit and Orthogonality

The correlation between Factor 1 and Factor 2 was modest and non-redundant, providing empirical justification for retaining both subscales independently. Goodness-of-fit parameters confirmed that forcing a unidimensional construct resulted in significant model degradation, whereas the two-factor paradigm accommodated both risk-seeking credulity and adaptive economic engagement.

Instrument / Measurement Tool

The ASJ is structured as an objective, vignette-based situational judgment test designed for rapid, standardized administration in clinical, home, or research environments.

Structural Attributes

• Instrument Type: Situational Judgment Test (SJT) / Vignette-based Behavioral Assessment.
• Total Item Count: 17 ecological scenarios.
  • Scam Scenarios: 11 to 13 realistic fraud vignettes representing telephonic, email, text message, postal mail, and door-to-door vectors.
  • Legitimate Scenarios: 4 to 6 authentic control vignettes representing ordinary, non-predatory daily interactions.
• Administration Format: Self-administered or clinician-administered via electronic platforms (Qualtrics, REDCap, tablet interfaces) or paper-and-pencil formats.
• Completion Duration: Approximately 8 to 12 minutes.
• Target Population: Community-dwelling older adults, individuals undergoing evaluations for subjective cognitive decline (SCD), Mild Cognitive Impairment (MCI), or dementia, as well as forensic evaluations of financial capacity.

Response Scale

Each vignette concludes with a concrete action statement prompting the respondent to rate their behavioral likelihood of completing the indicated action (e.g., sending funds, verifying sensitive account credentials, clicking a supplied link, or admitting a solicitor). Responses are scored on a standardized 7-point Likert scale:

• 1: Extremely unlikely
• 2: Moderately unlikely
• 3: Slightly unlikely
• 4: Neutral / Undecided
• 5: Slightly likely
• 6: Moderately likely
• 7: Extremely likely

Scoring Protocol and Algorithmic Interpretation

Scoring of the ASJ does not rely on a single, undifferentiated total score. Instead, it yields three complementary analytical metrics:

• Scam Score (Mean Likelihood): Computed by summing the ratings on all scam vignettes and dividing by the number of scam items (Scale: 1.00 to 7.00).
  
  Clinical Interpretation: Higher scores indicate elevated vulnerability to financial exploitation, impaired fraud discernment, and heightened behavioral risk.
• Legitimate Score (Mean Likelihood): Computed by summing the ratings on all legitimate vignettes and dividing by the number of legitimate items (Scale: 1.00 to 7.00).
  
  Clinical Interpretation: Lower scores indicate excessive generalized suspicion, functional withdrawal, or executive impairment regarding everyday economic demands.
• Scam/Legitimate Ratio Index: Computed as:
  
  Ratio = (Mean Scam Score) / (Mean Legitimate Score)
  
  Clinical Interpretation: A mathematically balanced metric that controls for individual response styles (e.g., acquiescence bias). Ratios approaching or exceeding 1.0 indicate an inability to differentiate safety from hazard, representing severe risk for financial exploitation.

Permissions & Fee and Test Year

The Assessment of Situational Judgement (ASJ) was formally published in 2024. The instrument and its foundational validation study appeared in the Journal of Alzheimer's Disease:

• Test Year: 2024
• Copyright: © 2024 by Sarah J. Getz, Bonnie E. Levin, James E. Galvin, and IOS Press.
• Accessibility and Licensing: The ASJ is an academic, non-commercial assessment instrument developed under institutional and federal research support. Qualified clinicians, university researchers, and health organizations may access and use the scale for research and non-profit clinical screening purposes, subject to proper academic citation and compliance with IOS Press permissions guidelines. Commercial deployment, inclusion within commercial diagnostic batteries, or integration into proprietary software requires formal licensing agreements with the copyright holders and the University of Miami Miller School of Medicine.

References

• Alves, R., & Wilson, S. J. (2008). The susceptibility of older adults to financial fraud: A review of the psychological literature. Journal of Financial Crime, 15(2), 161–174. https://doi.org/10.1108/13590790810866881
• Boyle, P. A., Yu, L., Mott, M., Dawe, R. J., & Bennett, D. A. (2020). Adverse financial outcomes in older adults with and without cognitive impairment. Annals of Internal Medicine, 172(5), 324–331. https://doi.org/10.7326/M19-2777
• Damasio, A. R. (1996). The somatic marker hypothesis and the possible functions of the prefrontal cortex. Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences, 351(1346), 1413–1420. https://doi.org/10.1098/rstb.1996.0125
• Getz, S. J., Levin, B. E., & Galvin, J. E. (2024). The Assessment of Situational Judgment questionnaire: A novel instrument to detect susceptibility to financial scamming. Journal of Alzheimer's Disease, 97(3), 1365–1379. https://doi.org/10.3233/JAD-231194
• Han, S. D., Boyle, P. A., James, B. D., Yu, L., & Bennett, D. A. (2016). Mild cognitive impairment is associated with poorer everyday investment decisions in older adults. Journal of the American Geriatrics Society, 64(10), 2097–2101. https://doi.org/10.1111/jgs.14309
• Kahneman, D. (2011). Thinking, fast and slow. Farrar, Straus and Giroux.
• Lichtenberg, P. A., Sugarman, M. A., Paulson, D., Ficker, L. J., & Rahman-Filipiak, A. (2016). Psychological and financial cognitive factors associated with financial exploitation of older adults. The Gerontologist, 56(5), 886–894. https://doi.org/10.1093/geront/gnv054
• Wood, S., Liu, P. J., Hanoch, Y., Xi, P. M., & Klapatch, L. (2018). Call to claim your prize: Perceived benefits influence elder susceptibility to lottery scams. Frontiers in Psychology, 9, Article 2633. https://doi.org/10.3389/fpsyg.2018.02633

Items of the Scale