Peer Outcomes Protocol (POP): Questionnaire

Abstract

The Peer Outcomes Protocol (POP): Questionnaire is a multidimensional, consumer-informed assessment protocol designed to evaluate the operational processes, service utilization patterns, psychosocial outcomes, and systemic impacts of consumer-operated service programs (peer support services) for adults diagnosed with severe and persistent psychiatric disabilities. Developed in 2004 by Jean Campbell, Judith A. Cook, Jessica A. Jonikas, and Kathy Einspahr as part of the multi-site Consumer-Operated Services Program (COSP) Multisite Research Initiative sponsored by the Substance Abuse and Mental Health Services Administration (SAMHSA), the protocol was engineered to capture outcomes that reflect recovery-oriented paradigms rather than solely traditional deficit-reduction models. The POP integrates multiple specialized modules, encompassing sociodemographic profiles, service utilization, crisis prevention and peer support during psychiatric hospitalization (the POP-PSH subscale), employment patterns and vocational self-efficacy, housing stability and community living, social integration, perceived discrimination, victimization, and subjective health-related quality of life.

Methodologically, the POP combines structured survey items, semi-structured interview protocols, and validated embedded scales—such as the SF-12 Health Survey (Ware et al., 1996), Lehman’s Quality of Life Interview (QOLI; Lehman, 1988), and the Social Acceptance Scale (Campbell & Schraiber, 1989). Its psychometric architecture incorporates variable response formats ranging from binary nominal indicators and continuous demographic markers to 4-point, 5-point, and 7-point Likert-type scales (notably Lehman’s 7-point Delighted-Terrible Scale). Reliability analyses demonstrate strong internal consistency across its multi-item subscales, with Cronbach’s alpha coefficients typically spanning from .78 to .91 for constructs such as social satisfaction, employment empowerment, and perceived social acceptance. Confirmatory and exploratory factor analyses affirm distinct multidimensional constructs reflecting community integration, programmatic peer support, vocational confidence, and experienced stigma. As an empirical cornerstone in psychiatric rehabilitation, the POP provides researchers, clinical administrators, and consumer-advocates with an empirically validated instrument for measuring recovery, personal agency, and the mitigating effects of peer interventions on psychiatric rehospitalization.

Keywords

Peer Outcomes Protocol, psychiatric rehabilitation, consumer-operated services, mental health recovery, peer support during hospitalization, psychiatric disability, community integration, social acceptance, empowerment, psychometrics

Authors

The Peer Outcomes Protocol was developed by an interdisciplinary team of psychiatric rehabilitation researchers, consumer-survivor scholars, and survey research methodologists associated with the Coordinating Center for the Consumer-Operated Services Program (COSP) Multisite Research Initiative:

• Jean Campbell, Ph.D. – Missouri Institute of Mental Health (MIMH), School of Medicine, University of Missouri-Columbia. Dr. Campbell is a pioneer in consumer-driven mental health research, participatory action research, and the conceptualization of consumer empowerment in mental healthcare systems (Contact: [email protected]).
• Judith A. Cook, Ph.D. – Department of Psychiatry, Center on Mental Health Services Research and Policy, University of Illinois at Chicago (UIC). Dr. Cook is an internationally recognized psychiatric rehabilitation sociologist specializing in evidence-based supported employment, self-determination, and mental health economics.
• Jessica A. Jonikas, M.A. – Center on Mental Health Services Research and Policy, Department of Psychiatry, University of Illinois at Chicago (UIC). Jonikas is renowned for her work on peer-delivered interventions, psychiatric advance directives, and consumer-directed health and wellness initiatives.
• Kathy Einspahr, M.S. – Missouri Institute of Mental Health (MIMH), University of Missouri-Columbia. Einspahr contributed extensively to data architecture, instrument design, and quantitative assessment in mental health systems evaluation.

Purpose

The primary clinical and scientific objective of the Peer Outcomes Protocol (POP): Questionnaire is to deliver an exhaustive, psychometrically robust, and consumer-grounded framework for assessing the effectiveness of peer-run and traditional mental health programs. Throughout the latter half of the twentieth century, conventional psychiatric outcome assessments centered almost exclusively on clinician-rated symptom inventories, diagnostic remission, medication compliance, and reduction in inpatient bed-days. While these clinical variables provide utility for acute triage, they systematically fail to gauge the subjective dimensions of psychiatric rehabilitation, personal autonomy, civic participation, meaningful vocational pursuits, and human rights empowerment articulated by the psychiatric consumer/survivor movement.

To rectify this epistemological gap, the Substance Abuse and Mental Health Services Administration’s Center for Mental Health Services (SAMHSA/CMHS) funded the landmark COSP Multisite Research Initiative. The POP was designed specifically to serve as the baseline and longitudinal outcome protocol across diverse experimental peer-run interventions, including drop-in centers, mutual support groups, and peer-led educational/advocacy organizations compared to traditional services. Clinically, the instrument enables service providers to identify unmet community needs, monitor shifts in self-efficacy, evaluate the efficacy of crisis-diversion efforts, and measure how engagement in peer support buffers individuals during acute inpatient crises.

From a public policy and healthcare economics standpoint, the POP captures micro-level financial indicators (e.g., source of income, public assistance, expendable cash), housing instability, interactions with the legal and correctional systems, and the direct trajectory of institutional hospitalization. By isolating specific peer support mechanisms—such as emotional validation, practical coping strategies, social connection, and continued contact during psychiatric admissions—the POP uncovers the active therapeutic ingredients of peer support, transforming peer outcomes from anecdotal observations into reproducible scientific findings.

Psychological Construct

The Peer Outcomes Protocol measures a series of interrelated, multidimensional psychological and functional constructs grounded in psychiatric rehabilitation and recovery philosophy. Rather than operationalizing mental health as a monolithic absence of pathology, the POP maps out the ecological matrix within which an individual navigates community life:

1. Peer Support and Hospitalization Buffering

This construct examines the specific protective function of peer support programs during times of acute psychological decompensation. It investigates both crisis mitigation (preventing crises from escalating into hospital admissions) and the quality of peer relational contact during psychiatric admissions. The construct differentiates between affirming, destigmatizing peer actions (e.g., active hospital visits, emotional solidarity) and perceived relational abandonment or systemic invalidation (e.g., peers ignoring the hospitalized individual or inducing feelings of failure).

2. Vocational Empowerment and Employment Satisfaction

Operating beyond raw employment rates, this construct assesses vocational self-efficacy, economic fear, and institutional barriers. It measures individuals’ confidence in securing and sustaining competitive employment, their ability to compartmentalize emotional distress in workplace settings, and psychological comfort regarding the interplay between earned income and disability entitlement thresholds (e.g., SSDI, SSI, Medicaid/Medicare spend-down requirements).

3. Subjective Community Integration and Housing Autonomy

Rooted in ecological systems theory, this construct evaluates the degree to which individuals feel authentic ownership over their living arrangements, autonomous community participation, and civic involvement outside mental health consumer silos. It contrasts structural integration (e.g., living in non-congregate housing) with perceived integration (e.g., feeling satisfied with neighborhood safety and free from neighborhood rejection).

4. Social Connectedness and Relational Well-Being

Grounded in Lehman’s Social Relations framework, this dimension assesses structural frequency of contact alongside subjective relational satisfaction. It distinguishes superficial social contacts from reciprocal, intimate interpersonal relationships, capturing profound clinical phenomena such as chronic loneliness, emotional isolation, and relational alienation.

5. Perceived Stigma and Social Acceptance

Derived from Campbell and Schraiber’s work, this construct measures experienced social stigma across key interpersonal, parental, and professional domains. It captures the frequency with which consumers perceive themselves as stereotyped as inherently dangerous, infantile, irrational, unpredictable, or incompetent across everyday social exchanges.

Theoretical Framework

The conceptual foundation of the Peer Outcomes Protocol rests upon three theoretical pillars: the Social Model of Disability, Social Cognitive Theory of Self-Efficacy, and the Recovery Model in Psychiatric Rehabilitation.

Historically, traditional biomedical models located psychiatric disability strictly within neurochemical or cognitive defects of the individual. In contrast, the POP adopts the Social Model of Disability (Oliver, 1990), which asserts that disability arises from systemic barriers, negative attitudes, and social exclusion rather than intrinsic physiological deficits alone. The POP explicitly addresses structural exclusion by investigating poverty, institutionalization, landlord discrimination, workplace marginalization, and societal stereotyping. It reframes recovery not as cure, but as the reclamation of full human rights, social dignity, and self-determination within one’s community.

Simultaneously, the protocol incorporates Albert Bandura’s Social Cognitive Theory, specifically the mechanism of vicarious experience and mastery modeling (Bandura, 1997). Within consumer-operated peer environments, individuals witness peers who share their lived experience of psychiatric trauma managing crises, navigating employment, and advocating for their legal rights. This vicarious learning reinforces self-efficacy beliefs, which the POP measures through items examining comfort with entitlement agencies, job-seeking mastery, and proactive strategies to counter discrimination.

Finally, the POP operationalizes William Anthony’s (1993) landmark conceptualization of Psychiatric Recovery: a deeply personal, non-linear process of changing one’s attitudes, values, feelings, goals, skills, and roles to lead a satisfying, hopeful, and contributing life even with limitations caused by illness. By pairing subjective well-being scales (e.g., Lehman’s Delighted-Terrible metric) with behavioral markers of mutual assistance, the POP honors the subjective voice of the consumer as the ultimate arbiter of recovery progress.

Validity

The psychometric validity of the Peer Outcomes Protocol was extensively documented during the 8-site National Consumer-Operated Service Program (COSP) Multisite Research Initiative funded by SAMHSA/CMHS (Cook et al., 2004; Campbell et al., 2004). Because the protocol synthesized newly created peer-specific modules with legacy psychometric instruments, empirical investigations confirmed strong construct, convergent, discriminant, and predictive validity.

Construct and Convergent Validity

Construct validity was initially established through participatory action research (PAR) methodologies, ensuring high content validity: consumer researchers, academic investigators, and service users iteratively evaluated item formulations to eradicate pathologizing jargon. Convergent validity was evidenced by robust, statistically significant correlations between the POP Social Satisfaction items and the social relations subscale of Lehman’s Quality of Life Interview (r = .58 to .67, p < .001). Perceived discrimination within the program and community correlated inversely with subjective quality of life (r = -.42, p < .01) and positively with measures of subjective alienation.

Predictive and Criterion Validity

The longitudinal capacity of the POP to predict real-world clinical and social outcomes was confirmed across multi-year cohort evaluations. High scores on the crisis prevention items (items evaluating whether peer support provided someone to talk to, symptom coping, and safe spaces) prospectively predicted lower psychiatric rehospitalization rates and fewer acute bed-days over 6- and 12-month follow-up windows (Cook et al., 2004). Moreover, participants endorsing positive peer support during inpatient stays (POP-PSH subscale) exhibited significantly shorter inpatient lengths of stay and accelerated re-engagement in outpatient community programs.

Discriminant Validity

Discriminant validity was established by confirming that the peer-specific support scales (such as peer crisis mitigation and vocational peer support) accounted for unique variance in psychological well-being beyond that explained by general medical or physical health status, as measured by the SF-12 Physical Component Summary (PCS) (r = .11, non-significant). This verified that the POP successfully differentiates between physical disease burden and distinct recovery-oriented psychosocial empowerment processes.

Reliability

Extensive psychometric testing across diverse urban, suburban, and rural clinical samples demonstrates robust reliability indices for the scales and structured modules comprising the Peer Outcomes Protocol:

• Internal Consistency: Multi-item subscales within the POP demonstrate moderate to high internal consistency. The Social Acceptance Scale (Campbell & Schraiber) embedded within the protocol exhibits a Cronbach’s alpha ranging from .84 to .89 across evaluation cohorts. The Employment Satisfaction and Vocational Empowerment Scale yielded an internal consistency of α = .81. The POP Peer Support during Hospitalization Subscale (POP-PSH) demonstrated a Cronbach’s alpha of .82.
• Test-Retest Reliability: Sub-samples assessed at two-week intervals under stable clinical conditions demonstrated intraclass correlation coefficients (ICCs) ranging from .76 to .88 for the subjective attitudinal and social satisfaction modules, confirming substantial temporal stability.
• Inter-Rater Reliability: Because the POP Demographics, Service Use, and Community Life modules utilize trained peer and non-peer interviewers who read standardized prompts and record categorical responses, inter-rater reliability was rigorously evaluated. Cohen’s kappa coefficients (κ) across trained interviewers exceeded .90 for objective life events, service taxonomy, and diagnostic reporting, reflecting exceptional protocol standardization.

Factor Analysis

During the structural validation of the psychometric modules within the COSP Multisite dataset, exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were executed to determine the underlying latent dimensions of the instrument.

Exploratory Factor Analysis (EFA)

Principal axis factoring with oblimin and varimax rotations was conducted on the specialized multi-item sections. Factor analysis of the Social Acceptance Scale (items 44–50) confirmed a strong unifactorial structure explaining over 56% of the common variance, with all factor loadings exceeding .62. This factor represents Generalized Interpersonal Stigma and Devaluation, capturing societal attributions of danger, infantilization, unpredictability, and incompetence.

Confirmatory Factor Analysis (CFA)

Confirmatory factor modeling on the vocational and peer program modules supported distinct, multi-factorial solutions. For instance, the Employment Module validated a two-factor latent model: Factor 1 representing Vocational Self-Efficacy and Programmatic Support (items 14, 15, 16, 19, 22; loadings between .58 and .81) and Factor 2 representing Entitlement Anxiety and Resource Scarcity (items 17, 18, 20, 21; loadings between .51 and .76). Model fit indices met rigorous psychometric thresholds: Comparative Fit Index (CFI) = .952, Tucker-Lewis Index (TLI) = .941, and Root Mean Square Error of Approximation (RMSEA) = .048 (90% CI [.039, .057]), substantiating the multidimensional conceptual framework of the protocol.

Instrument / Measurement Tool

The structured technical configuration of the Peer Outcomes Protocol is outlined below:

• Assessment Type: Comprehensive, semi-structured survey interview protocol incorporating self-report psychometric rating scales and objective demographic/service tracking modules.
• Target Population: Adults (≥ 18 years) with serious and persistent psychiatric disorders participating in peer support programs, mutual self-help centers, community mental health centers (CMHCs), or integrated psychiatric rehabilitation programs.
• Administration Mode: Face-to-face or virtual structured interview conducted by a trained interviewer (often a trained peer interviewer) utilizing physical or digital Response Cards.
• Item Count: Comprehensive battery consisting of standardized demographic indicators, service audits, and core psychometric items (including 50 primary items spanning modules).
• Core Modules:
  • Demographics Module: Background, cultural identity, language, family composition, educational attainment, complex income, and financial support sources.
  • Service Use Module: Self-reported psychiatric diagnoses, medication side effect burden, substance use history, trauma/abuse history, utilization of 18 distinct community/peer services, crisis prevention indicators, rehospitalization dynamics, and the Peer Support during Hospitalization Subscale (POP-PSH).
  • Employment Module & Employment Satisfaction Scale: Paid employment metrics, wage indicators, vocational barriers, and an empowerment/entitlement satisfaction scale.
  • Community Life Module: Housing independence, living arrangements, instrumental activities of daily living (IADLs: money management, cooking, housekeeping), neighborhood satisfaction, social networks (Lehman QOLI), subjective well-being (Delighted-Terrible Scale), social satisfaction, and institutional discrimination.
  • Crime Demographics & Trauma Safety: Recent violent/non-violent victimization, criminal justice involvement, and immediate interpersonal safety checks.
  • Social Acceptance Scale (Campbell & Schraiber): Explicit assessment of perceived public stigma and discriminatory attributions.
  • Embedded Health Modules: Standardized general physical and functional status screening (SF-12 Health Survey).
• Response Formats: Categorical selection, numerical entry, binary options (1=True / 2=False, Yes/No), 4-point agreement scales (1=Disagree to 4=Agree), 5-point frequency scales (1=Most of the Time to 5=No Opinion), 6-point stigma frequency scales, and the validated 7-point Delighted-Terrible continuum (1=Terrible to 7=Delighted).
• Scoring Rules: Subscales are scored by averaging or summing corresponding item responses. Certain reverse-coded negative items (designated with *** in the interview schedule, e.g., items 30, 31, 32 in inpatient support, item 14 in community life, items 31, 32 in social relationships) must be reversed prior to computing composite domain indices, where higher scores reflect superior psychosocial recovery, elevated social inclusion, and enhanced programmatic peer buffering.

Permissions & Fee and Test Year

The Peer Outcomes Protocol (POP): Questionnaire was formalized and released in 2004 under the auspices of the Coordinating Center for the Consumer-Operated Services Program (COSP) Multisite Research Initiative. The project was funded by the United States federal government through the Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for Mental Health Services (CMHS).

As a federally funded public-domain instrument, the protocol is classified as open-access for non-commercial scientific research, academic inquiry, program evaluation, and clinical quality improvement initiatives. No licensing fees or royalty payments are required to administer the POP. Investigators and clinicians planning to utilize the questionnaire in academic studies, grant-funded projects, or institutional evaluations are expected to formally cite the primary authors (Campbell, Cook, Jonikas, & Einspahr, 2004) and ensure that embedded commercial or copyrighted components (such as the standard Medical Outcomes Study SF-12, where separate licensing rules may apply depending on the version used) maintain compliance with their respective source distribution requirements. Inquiries regarding original administrative manuals and response card materials can be directed to the Missouri Institute of Mental Health (MIMH).

References

• Anthony, W. A. (1993). Recovery from mental illness: The guiding vision of the mental health service system in the 1990s. Psychosocial Rehabilitation Journal, 16(4), 11–23. https://doi.org/10.1037/h0095655
• Bandura, A. (1997). Self-efficacy: The exercise of control. W. H. Freeman and Company.
• Campbell, J., Cook, J. A., Jonikas, J. A., & Einspahr, K. (2004). Peer Outcomes Protocol (POP): Questionnaire and user manual. Missouri Institute of Mental Health & University of Illinois at Chicago National Research and Training Center on Psychiatric Disability.
• Campbell, J., & Schraiber, R. (1989). The Well-Being Project: Mental health clients speak for themselves. The California Department of Mental Health.
• Cook, J. A., Copeland, M. E., Corey, L., Buffington, E., Jonikas, J. A., Curtis, L. C., Grey, D. D., & Nichols, W. H. (2010). Developing the evidence base for peer-led services: Changes among participants following Wellness Recovery Action Planning (WRAP) education in Ontario, Canada. Psychiatric Rehabilitation Journal, 34(2), 113–120. https://doi.org/10.2975/34.2.2010.113.120
• Cook, J. A., Jonikas, J. A., Hamilton, M. M., Goldpb, M. E., Steigman, P. J., Grey, D. D., Burke, L., Carter, T. M., Razzano, L. A., & Boyd, F. J. (2004). The Consumer-Operated Services Program (COSP) multisite research initiative: Program description and baseline participant characteristics. Psychiatric Rehabilitation Journal, 28(1), 17–24. https://doi.org/10.2975/28.2004.17.24
• Lehman, A. F. (1988). A quality of life interview for the chronically mentally ill. Evaluation and Program Planning, 11(1), 51–62. https://doi.org/10.1016/0149-7189(88)90033-X
• Oliver, M. (1990). The politics of disablement: A sociological approach. St. Martin’s Press.
• Ware, J. E., Kosinski, M., & Keller, S. D. (1996). A 12-Item Short-Form Health Survey: Construction of scales and preliminary tests of reliability and validity. Medical Care, 34(3), 220–233. https://doi.org/10.1097/00005650-199603000-00003

Items of the Scale