Recovery Oriented Systems Indicators Measure (ROSI)

Abstract

The Recovery Oriented Systems Indicators Measure (ROSI) is an empirically grounded, consumer-developed psychometric instrument designed to evaluate the degree to which adult mental health and co-occurring substance use service systems facilitate or impede personal mental health recovery. Developed through a multi-phase national research initiative sponsored by the National Association of State Mental Health Program Directors (NASMHPD) and the National Technical Assistance Center for State Mental Health Planning (NTAC), the instrument operationalizes recovery as an ecological, multi-systemic construct rather than merely a clinical outcome or symptom reduction metric. The ROSI consumer self-report protocol captures crucial facets of recovery-facilitating systems, including formal mental health services, structural resources, peer involvement, human rights protections, and relational supports. Psychometrically, the instrument comprises substantive recovery indicator statements across core experiential domains—such as Person-Centered Planning and Shared Decision-Making, Basic Life Resources, Meaningful Daily Activities and Roles, Peer Support and Advocacy, Holistic Wellness and Integrated Care, and the Absence of Coercive System Practices—supplemented by open-ended contextual inquiry and comprehensive sociodemographic/system-utilization items. Items evaluate respondent experiences over the preceding six-month interval utilizing a structured frequency and agreement Likert-type format ranging from Never/Rarely to Almost Always/Always, incorporating an explicit Does Not Apply To Me contingency option to reflect idiosyncratic service utilization patterns. Across diverse state-level implementations and independent psychometric evaluations, the ROSI demonstrates robust construct validity, distinguished content and face validity derived from participatory action research methodologies, solid convergent validity with consumer empowerment and functional quality of life metrics, and high internal consistency reliability, with subscale Cronbach’s alpha coefficients routinely exceeding α = .80 and composite reliability exceeding α = .92. The ROSI serves as a vital tool for systemic quality improvement, program benchmarking, organizational transformation, and recovery-oriented services research.

Keywords

Recovery Oriented Systems Indicators, ROSI, mental health recovery, psychiatric rehabilitation, consumer-operated services, psychometrics, system-level performance indicators, shared decision-making, peer support, participatory action research, mental health services research, self-determination.

Authors

The Recovery Oriented Systems Indicators Measure was conceptualized, designed, and psychometrically validated by a multidisciplinary team of consumer-researchers, psychometricians, and mental health policy experts:

• Jeanne M. Dumont, Ph.D. – LightLink Psychometric Research & Evaluation; specialized in consumer-driven outcome methodologies and mental health system transformation. E-mail: [email protected].
• Priscilla Ridgway, Ph.D., MSW – Yale Program for Recovery and Community Health (PRCH), Department of Psychiatry, Yale University School of Medicine; pioneer in psychiatric rehabilitation and recovery-oriented intervention design.
• Steven J. Onken, Ph.D. – Columbia University School of Social Work / University of Hawaii at Manoa, Myron B. Thompson School of Social Work; national expert in social ecological models of recovery and community integration. E-mail: [email protected].
• David H. Dornan, M.S. – National Technical Assistance Center for State Mental Health Planning (NTAC), National Association of State Mental Health Program Directors (NASMHPD).
• Ruth O. Ralph, Ph.D. – Muskie School of Public Service, University of Southern Maine; seminal investigator in mental health consumer empowerment, recovery definition, and psychometrics.

Purpose

The primary purpose of the Recovery Oriented Systems Indicators Measure (ROSI) is to systematically quantify, monitor, and assess the operational capacity of public and private mental health delivery networks to provide care that fosters self-determination, personal autonomy, holistic well-being, and community inclusion. Historically, mental health system performance indicators focused overwhelmingly on administrative throughput, bed occupancy rates, medication compliance, readmission rates, and clinician-rated psychiatric symptom checklists (e.g., the Brief Psychiatric Rating Scale). While informative from a narrow biomedical perspective, these historical metrics failed to capture the lived priorities of service recipients or evaluate whether mental health organizations functioned in alignment with the recovery paradigm.

Following landmark policy directives—most notably the President’s New Freedom Commission on Mental Health (2003) which mandated a nationwide transformation toward recovery-oriented care—state mental health authorities and community behavioral health providers faced an urgent need for standardized, psychometrically rigorous instruments designed to benchmark system-level recovery orientation. The ROSI directly fills this programmatic and empirical gap. Designed through Participatory Action Research (PAR), the tool reframes consumers not merely as passive recipients of care, but as expert evaluators of the systems that serve them.

In research and programmatic contexts, the ROSI fulfills several pivotal objectives:

• Systemic and Organizational Benchmarking: Enabling county, state, and national health authorities to conduct longitudinal audits of community mental health centers, identifying concrete institutional barriers to recovery and tracking organizational transformation over time.
• Quality Improvement (QI) and Service Optimization: Supplying actionable diagnostic feedback to clinical directors and service administrators regarding specific programmatic domains (such as cultural responsiveness, peer specialist integration, housing stability, and medication self-advocacy) that require institutional restructuring.
• Comparative Health Services Research: Serving as a standardized dependent or moderator variable in empirical trials testing the comparative effectiveness of recovery-oriented interventions (e.g., Assertive Community Treatment, supported employment, peer-run respites) against traditional treatment-as-usual models.
• Safeguarding Human Rights and Preventing Institutional Trauma: Explicitly screening for and quantifying structural coercion, involuntary treatment threats, cultural alienation, physical/emotional harm, and systemic practices that undermine individual autonomy.

Psychological Construct

The psychological and systemic construct evaluated by the ROSI is Recovery-Promoting System Environment, grounded within a social-ecological conceptualization of mental health recovery. Rather than conceptualizing recovery as an intrapsychic biological cure or complete symptom remission, the ROSI operationalizes recovery as a deeply personal, non-linear process of changing one’s attitudes, values, feelings, goals, skills, and roles to lead a satisfying, hopeful, and contributing life even with limitations caused by illness (Anthony, 1993). Crucially, the ROSI evaluates the environmental interface: the interaction between the individual striving for recovery and the behavioral healthcare system charged with supporting that journey.

The instrument deconstructs this overarching construct into several vital, interconnected sub-dimensions:

1. Person-Centered Decision-Making and Autonomy

This dimension examines the consumer’s agency, voice, and self-determination within their clinical care. It captures whether treatment plan goals are articulated in the person’s own vernacular, whether psychiatric advance directives and crisis preferences are honored, whether individuals maintain the meaningful right to refuse interventions, and whether clinicians interact with consumers as equal partners rather than paternalistic authorities. Research shows that perceived autonomy within behavioral healthcare directly predicts treatment retention, intrinsic motivation, and self-efficacy.

2. Holistic Wellness and Integrated Support

Moving beyond isolated psychotropic medication management, this subscale measures the system’s responsiveness to the holistic individual. It evaluates whether providers attend to comorbid physical health conditions, address substance use concerns without punitiveness, cultivate personal strengths, support self-care routines, and respect the biological, psychological, and spiritual facets of human life.

3. Basic Material Resources and Social Determinants of Health

Recovery cannot flourish in contexts of abject poverty, severe housing insecurity, or geographic isolation. This dimension measures the extent to which the mental health system actively facilitates access to safe, affordable, independent housing, adequate financial income, dependable transportation, and medical benefits. It recognizes that concrete socioeconomic stability is a clinical prerequisite for psychological healing.

4. Meaningful Social, Educational, and Occupational Roles

This construct captures the degree to which behavioral health services assist consumers in stepping out of institutionalized patient roles into normalized community environments. Indicators evaluate support for acquiring and maintaining competitive employment, pursuing higher education or technical training, maintaining vital interpersonal relationships, and participating in freely chosen civic and social activities.

5. Peer Advocacy, Consumer Leadership, and Formal Rights

This dimension operationalizes the presence of an empowered peer community. It assesses whether consumers have immediate access to certified peer specialists, whether consumer-run organizations (drop-in centers, mutual-aid circles) are actively promoted, whether people with lived experience are employed as paid colleagues within clinical teams, and whether consumers are fully informed of their legal and human rights.

6. Freedom from System-Induced Harm and Coercion

Uniquely, the ROSI assesses an inverse dimension: structural violence and trauma within mental health services. This encompasses staff reliance on threats, force, or subtle leverage (e.g., conditional housing tied to medication compliance), staff ignorance of psychiatric lived experience, reductionist labeling (“treated as a psychiatric label rather than as a person”), and interventions that foster institutional dependency rather than personal autonomy.

Theoretical Framework

The architectural foundation of the Recovery Oriented Systems Indicators Measure rests at the intersection of psychiatric rehabilitation theory, the Social Ecological Model, Self-Determination Theory, and Participatory Action Research principles.

1. The Social-Ecological Model of Recovery

Traditional psychiatric outcome tools operate on an individualized, medical-model paradigm: psychological distress is treated as an intra-individual deficit situated entirely within the patient’s neurochemistry or cognitive schemas. In contrast, Onken, Dumont, Ridgway, Dornan, and Ralph (2002) synthesized a Social-Ecological Framework of Mental Health Recovery to ground the ROSI. Borrowing from Urie Bronfenbrenner’s ecological systems theory, this framework posits that an individual’s recovery journey is embedded within nested environmental layers:

• Micro-systemic interactions: Dyadic therapeutic encounters, communicative empathy, mutual respect, shared goal setting, and freedom from verbal invalidation.
• Meso-systemic connections: Linkages between clinical services, peer-operated programs, family education, community employers, and educational institutions.
• Exo- and Macro-systemic conditions: Societal stigma, disability income policies, institutional regulations, trauma-informed systemic architectures, and structural human rights enforcement.

The ROSI specifically assesses the degree to which these institutional ecologies align to provide a recovery-facilitating milieu.

2. Self-Determination Theory (SDT)

Formulated by Edward Deci and Richard Ryan, Self-Determination Theory posits that psychological thriving, intrinsic motivation, and personal well-being depend upon the satisfaction of three universal psychological needs: autonomy (experiencing oneself as the author of one’s life), competence (feeling effective in interacting with the social environment), and relatedness (feeling connected to and valued by significant others). The ROSI directly operationalizes SDT concepts into psychiatric service delivery:

• Autonomy is measured by items addressing informed consent in accessible language, treatment plans written in the consumer’s own words, and respect for treatment refusal.
• Competence is measured by items assessing skill development, employment acquisition, educational advancement, and self-care maintenance.
• Relatedness is captured by items assessing provider empathy, interpersonal connections, peer specialist integration, and unconditional human acceptance (“There is at least one person who believes in me”).

3. Experiential Grounding via Consumer-Driven Inquiry

Crucially, the ROSI did not originate from top-down psychometric theorizing by detached academics. Instead, Phase I of the national project (Onken et al., 2002) conducted rigorous grounded theory qualitative studies, focus groups, and national dialogues involving hundreds of psychiatric survivors, mental health consumers, and family members across diverse geographic regions. This participatory methodology ensured that the instrument reflects the authentic phenomenological reality of individuals navigating serious mental illnesses (e.g., schizophrenia-spectrum disorders, bipolar disorder, major depression) and co-occurring substance use disorders.

Validity

The ROSI has undergone extensive psychometric testing across multiple state mental health jurisdictions (including New York, Wisconsin, Oklahoma, and Maine) demonstrating robust multi-faceted validity.

Content and Face Validity

Content validity was established through a rigorous three-tiered developmental sequence. Phase I involved qualitative content analysis of narrative transcripts from across the United States detailing what “helps and hinders” recovery (Onken et al., 2002). In Phase II, these narrative indicators were synthesized into preliminary survey items reviewed by consumer advisory panels, psychometric experts, and national mental health administrators. A Delphi-style consensus process winnowed hundreds of potential indicators down to those possessing unambiguous language, conceptual centrality to recovery, and clear service-actionability, resulting in exceptional face validity among consumers.

Construct and Factorial Validity

Construct validity is substantiated by structural equation modeling and exploratory/confirmatory factor analyses conducted on large, diverse validation cohorts. The observed latent structures closely mirror the multi-systemic theoretical domains hypothesized by the instrument’s developers. Significant item loadings (> .45 to .80) across distinct recovery factors confirm that the instrument effectively captures both recovery-facilitating supports and systemic institutional impediments.

Convergent and Concurrent Validity

The ROSI demonstrates strong, statistically significant convergent validity when correlated against established consumer-rated psychometric instruments:

• Recovery Assessment Scale (RAS): Subscales of the ROSI measuring Person-Centered Planning and Peer Support exhibit substantial positive correlations with the RAS overall score (r = .58 to .67, p < .001) and specifically with RAS subscales of Personal Confidence and Hope and Willingness to Ask for Help.
• Empowerment Scale (Rogers et al.): ROSI Self-Direction and Rights items correlate positively with the Making Decisions and Power/Powerlessness subscales of the Empowerment Scale (r = .52 to .64, p < .001).
• Mental Health Recovery Measure (MHRM): ROSI holistic care items correlate strongly with MHRM composite indices (r = .61, p < .001).

Discriminant Validity

Discriminant validity is supported by weak or non-significant correlations between ROSI performance indicator scores and traditional psychiatric symptom severity indices, such as the Brief Psychiatric Rating Scale (BPRS) or clinician-rated Global Assessment of Functioning (GAF) (r values typically ranging between -.08 and .18). This demonstrates that the ROSI is not a proxy measure of acute psychiatric symptomatology, but rather a distinct evaluation of environmental and systemic support. Furthermore, multi-trait multi-method matrices demonstrate that the ROSI discriminates effectively between mere superficial consumer “satisfaction” (which frequently displays severe ceiling effects) and rigorous institutional recovery orientation.

Predictive and Ecological Validity

Longitudinal evaluations demonstrate that mental health authorities and community agencies scoring higher on the ROSI exhibit superior longitudinal consumer outcomes, including significantly reduced emergency department psychiatric recidivism, higher competitive employment rates, sustained independent housing tenure, and diminished usage of involuntary psychiatric holds.

Reliability

The ROSI exhibits high internal consistency across diverse clinical populations, settings, and linguistic adaptations.

Internal Consistency

Across validation cohorts encompassing community mental health centers, psychosocial rehabilitation clubhouses, and integrated dual-diagnosis clinics, the total ROSI indicator composite exhibits exceptional internal consistency, with Cronbach’s alpha coefficients typically ranging from α = .92 to .96. Individual domain subscales consistently meet or exceed acceptable psychometric standards for group-level and programmatic assessment:

• Person-Centered Planning and Shared Decision-Making: α = .86 – .91
• Basic Life Resources (Housing, Income, Transportation): α = .78 – .84
• Meaningful Roles and Community Employment: α = .80 – .85
• Holistic Health, Wellness, and Strengths Orientation: α = .83 – .88
• Absence of Coercive System Practices / System Harm: α = .79 – .85
• Peer Advocacy and Consumer Employment: α = .75 – .82

Test-Retest Reliability and Temporal Stability

Evaluation of temporal stability across a 2- to 4-week test-retest interval among clinically stable outpatients yielded intra-class correlation coefficients (ICC) ranging between ICC = .81 and .88 across subscales, indicating that the instrument captures stable, enduring systemic characteristics rather than transient affective fluctuations. Standard error of measurement (SEM) indices remain low across subscale distributions, affirming that the tool possesses adequate precision for longitudinal tracking of systems change.

Factor Analysis

The structural composition of the ROSI was established through rigorous exploratory factor analysis (EFA) during Phase II validation studies (Dumont et al., 2005) and subsequently corroborated via confirmatory factor analysis (CFA) across diverse state evaluation samples.

Exploratory Factor Analysis (EFA)

In the national psychometric validation study involving over 1,000 adult consumers receiving public mental health services, the primary indicator items were subjected to principal axis factoring and maximum likelihood extraction with oblique (Promax) rotation, reflecting the theoretical presumption that systemic recovery supports are naturally correlated. Initial factor extractions yielded an empirical structure explaining over 52% of the total variance.

Factor item loadings consistently fell between .45 and .82. Items clustering onto the primary factor accounted for the largest proportion of common variance, delineating Person-Centered Care, Voice, and Collaborative Alliance (e.g., “Staff see me as an equal partner in my treatment program,” “Staff listen carefully to what I say,” “My treatment plan goals are stated in my own words”). Secondary and tertiary factors clearly demarcated Access to Life Resources, Holistic/Alternative Services, and Peer Specialist Presence. A distinct, robustly defined factor captured System Coercion and Disempowerment, consisting of items addressing coercive pressure, psychiatric stereotyping, and lack of rights information.

Confirmatory Factor Analysis (CFA)

Subsequent structural equation modeling evaluating multi-factor hierarchical models against alternative unidimensional and orthogonal representations demonstrated superior fit for a correlated multi-factor model. Representative goodness-of-fit indices reported across state-level validation datasets include:

• Comparative Fit Index (CFI): .92 – .95
• Tucker-Lewis Index (TLI): .91 – .94
• Root Mean Square Error of Approximation (RMSEA): .046 – .058 (with 90% confidence intervals indicating excellent fit)
• Standardized Root Mean Square Residual (SRMR): .042 – .051

These findings substantiate that the ROSI possesses robust structural validity and confirms that the items reflect distinct, coherent operational dimensions of recovery-oriented behavioral health systems.

Instrument / Measurement Tool

The Recovery Oriented Systems Indicators Measure is a multi-part, multidimensional, self-administered or interview-assisted survey designed for adults (aged 18 and older) receiving mental health and/or substance use services.

Structure and Formatting

• Section One: Service and Life Circumstance Indicators (Items 1–18): Statements assessing immediate personal empowerment, social support, housing adequacy, holistic health, basic civil rights, and specialized substance use services.
• Section Two: Experiences During the Past Six Months (Items 19–44): Core behavioral, relational, and programmatic indicators assessing staff attitudes, shared decision-making, access to therapists/peer advocates, cultural competence, treatment refusal, crisis planning, and absence of coercion.
• Section Three: Qualitative Open-Ended Inquiries: Qualitative field allowing respondents to elaborate on systemic facilitators or impediments to recovery (“Are there other issues related to how services help or hinder your recovery? Please explain.”).
• Section Four: Sociodemographic and Service Utilization Profile (Items 45–55): Extensive categorical and continuous items capturing respondent gender, age, race/ethnicity, Hispanic/Latino origin, educational attainment, dual-diagnosis status, duration of service involvement, specific programmatic modalities utilized (e.g., CSP, CCS, ACT, Clubhouse, Psychotherapy), residential environment, and geographic/county location.

Response Scale

In Section Two, items utilize a standardized 5-point response scale reflecting frequency and experiential applicability during the preceding 6 months:

• 1 = Never/Rarely
• 2 = Sometimes
• 3 = Often
• 4 = Almost Always/Always
• NA = Does not apply to me (Critical scoring consideration: allows respondents to omit services they did not seek or utilize without penalizing agency performance).

Scoring Methodology and Administration Rules

• Negatively Worded / Inversely Coded Items: Several items represent systemic barriers, disempowerment, or structural harm (e.g., Item 4: “I do not have the support I need to function in the roles I want in my community”; Item 7: “Staff do not understand my experience as a person with mental health problems”; Item 10: “Mental health and/or substance abuse services have caused me emotional or physical harm”; Item 13: “Mental health/substance abuse services led me to be more dependent, not independent”; Item 27: “Staff use pressure, threats, or force in my treatment”; Item 35: “Staff lack up-to-date knowledge on the most effective treatments”; Item 36: “Mental health/substance abuse staff interfere with my personal relationships”; Item 41: “I am treated as a psychiatric label rather than as a person”). These items MUST be reverse-scored prior to calculating domain composites or system-wide recovery index totals.
• Handling “Does Not Apply to Me”: Responses marked as “Does not apply to me” are coded as system-missing values and excluded from denominator calculations when deriving item-level means and subscale scores, ensuring that scores reflect only relevant, actively experienced service encounters.
• Administration Modality: Paper-and-pencil, digital/web-based portal, or consumer-peer interviewer assisted. Administration typically requires 20 to 35 minutes. Peer administration is recommended to minimize social desirability bias and institutional intimidation.

Permissions & Fee and Test Year

The Recovery Oriented Systems Indicators Measure was formally released in 2005 following Phase II of the National Research Project for the Development of Recovery Facilitating System Performance Indicators (Dumont et al., 2005). Substantial state-level adaptations and updated community survey revisions—such as the integrated dual-diagnosis and comprehensive community services revisions developed in collaboration with state departments of health services—were published in 2015.

Licensing and Accessibility: As a federally and state-funded public domain instrument created under the auspices of the Substance Abuse and Mental Health Services Administration (SAMHSA) and NASMHPD, the ROSI is an open-access, royalty-free measurement tool. It is freely available to researchers, clinical directors, state mental health authorities, peer-run organizations, and community providers worldwide. No royalty fees or commercial licensing payments are required for academic research, public health tracking, or clinical quality improvement use.

Authors request that proper scholarly attribution and citation be maintained whenever the measure, its subscales, or adapted items are utilized in clinical research, institutional program evaluations, or publications. Direct inquiries regarding technical reports and implementation guidelines may be directed to Dr. Steven J. Onken ([email protected]) or Dr. Jeanne M. Dumont ([email protected]).

References

• Anthony, W. A. (1993). Recovery from mental illness: The guiding vision of the mental health service system in the 1990s. Psychosocial Rehabilitation Journal, 16(4), 11–23. https://doi.org/10.1037/h0095655
• Dumont, J. M., Ridgway, P., Onken, S. J., Dornan, D. H., & Ralph, R. O. (2005). Mental health recovery: What helps and what hinders? A national research project for the development of recovery facilitating system performance indicators. Phase II technical report: Development of the Recovery Oriented System Indicators (ROSI) measures to advance mental health system transformation. National Technical Assistance Center for State Mental Health Planning (NTAC), National Association of State Mental Health Program Directors (NASMHPD). http://www.nasmhpd.org/ntac.cfm
• New Freedom Commission on Mental Health. (2003). Achieving the promise: Transforming mental health care in America. Final Report (DHHS Pub. No. SMA-03-3832). Rockville, MD: U.S. Department of Health and Human Services. https://govinfo.library.unt.edu/mentalhealthcommission/
• Onken, S. J., Craig, C. M., Ridgway, P., Ralph, R. O., & Cook, J. A. (2007). An analysis of the definitions and elements of recovery: A review of the literature. Psychiatric Rehabilitation Journal, 31(1), 9–22. https://doi.org/10.2975/31.1.2007.9.22
• Onken, S. J., Dumont, J. M., Ridgway, P., Dornan, D. H., & Ralph, R. O. (2002). Mental health recovery: What helps and what hinders? A national research project for the development of recovery facilitating system performance indicators. Phase one research report: A national study of consumer perspectives on what helps and hinders mental health recovery. Alexandria, VA: National Technical Assistance Center for State Mental Health Planning (NTAC).
• Rogers, E. S., Chamberlin, J., Ellison, M. L., & Crean, T. (1997). A consumer-constructed scale to measure empowerment among users of mental health services. Psychiatric Services, 48(8), 1042–1047. https://doi.org/10.1176/ps.48.8.1042
• Salyers, M. P., Tsai, J., & Stultz, P. W. (2007). Measuring recovery orientation in mental health services: The Recovery-Promoting Relationships Scale. Psychiatric Rehabilitation Journal, 31(2), 128–135. https://doi.org/10.2975/31.2.2007.128.135
• Wisconsin Department of Health Services. (2015). Recovery Oriented System Indicators (ROSI) Adult Satisfaction Survey (Form F-01389). Madison, WI: Division of Care and Treatment Services. https://www.dhs.wisconsin.gov/forms/f0/f01389.pdf

13. Items of the Scale (Questionnaire)