Coaching Fidelity Scale (CFS)

1. Abstract

The Coaching Fidelity Scale (CFS) (Meadan et al., 2023) is an empirically validated observational rating checklist designed to assess the implementation fidelity of professional coaching practices within early intervention, special education, and behavioral healthcare settings. Structured around evidence-based, triadic coaching models utilized across naturalistic developmental behavioral interventions (NDBIs), the CFS operationalizes coaching into an 11-item checklist encompassing three sequential operational phases: (a) pre-observation, which establishes shared intentionality through strategy identification, contextual routine selection, and procedural task analysis review; (b) observation, which captures uninterrupted caregiver-child or practitioner-child behavioral interactions; and (c) post-observation, which facilitates reciprocal learning through coachee reflection, targeted supportive and constructive performance feedback, and collaborative problem-solving. Additionally, the CFS accommodates an optional instructional module evaluating video-recorded feedback sessions.

The scale was developed and psychometrically evaluated across multiple intervention cohorts comprising professionals—including board-certified behavior analysts (BCBAs), early childhood special educators, speech-language pathologists, and research interventionists—coaching adult coachees (e.g., caregivers of young children under five years old with autism spectrum disorder or related developmental disabilities, paraprofessionals, and registered behavior technicians). Psychometric evaluation confirms exceptional internal consistency (Cronbach's alpha = .91, 95% CI [.89, .93]; average inter-item correlation = .47) and outstanding interrater reliability (intraclass correlation coefficient [ICC] > .96). Exploratory factor analysis (EFA) substantiated an unequivocal unidimensional latent structure accounting for 77.84% of the total variance, accompanied by robust model fit indices (CFI = .981, TLI = .975, RMSEA = .065, WRMR = .0449). Known-groups validity demonstrated high test sensitivity, capturing statistically significant, large-magnitude advancements in coaching fidelity from pre-intervention baseline to post-intervention assessment (Mann–Whitney W = 327.00, p < .001, rank-biserial correlation = .84). The CFS addresses an acute methodological gap in implementation science, providing researchers, administrators, and clinical supervisors with a psychometrically sound, low-inference observational metric to evaluate treatment adherence and optimize coach training.

2. Keywords

Coaching Fidelity Scale, implementation fidelity, observational assessment, adult coaching practices, autism spectrum disorder, developmental disabilities, early childhood intervention, parent-mediated interventions, naturalistic developmental behavioral interventions, psychometrics.

3. Authors

The Coaching Fidelity Scale was conceptualized, developed, and psychometrically validated by an interdisciplinary team of researchers specializing in early childhood special education, behavioral consultation, and family-centered intervention methodologies:

• Hedda Meadan, Ph.D., BCBA-D — Department of Special Education, College of Education, University of Illinois Urbana-Champaign, Champaign, Illinois, United States. Correspondence Email: [email protected]. Institutional Address: 1310 South Sixth Street, Champaign, IL 61820.
• James D. Lee, Ph.D. — Department of Special Education, University of Illinois Urbana-Champaign, Champaign, Illinois, United States.
• Michelle M. Sands, Ph.D. — Department of Special Education, University of Illinois Urbana-Champaign, Champaign, Illinois, United States.
• Moon Y. Chung, Ph.D. — Department of Special Education, University of Illinois Urbana-Champaign, Champaign, Illinois, United States.
• Pau García-Grau, Ph.D. — Department of Teaching and Learning, University of Valencia, Valencia, Spain / Department of Special Education, University of Illinois Urbana-Champaign, United States.

4. Purpose

The primary purpose of the Coaching Fidelity Scale (CFS) is to provide an objective, low-inference, criterion-referenced observational instrument that assesses the fidelity with which professionals deliver collaborative coaching interventions to adult learners. In contemporary pediatric and educational intervention paradigms—most notably within family-centered early intervention for young children with autism spectrum disorder (ASD) and developmental delays—service delivery has fundamentally transitioned from direct clinician-to-child intervention toward a triadic consultation model. In this triadic framework, the professional serves as a coach who guides, scaffolds, and empowers primary caregivers or frontline educational staff (the coachees), who subsequently deliver naturalistic, evidence-based instructional strategies directly to the child within everyday family routines.

Despite the ubiquitous empirical endorsement of caregiver coaching as an active mechanism for developmental acceleration, implementation science literature has persistently highlighted a "black box" phenomenon regarding coaching delivery. Historically, researchers and clinical supervisors lacked standardized, validated measurement tools to verify whether interventionists adhered to the specified active ingredients of coaching or whether the observed child outcomes were attenuated by fidelity slippage. Without direct measurement of coaching fidelity, attributing child progress or stagnation to a given intervention model remains compromised by Type III error (evaluating an intervention that was not implemented as planned). The CFS systematically solves this methodological vulnerability by operationalizing the multi-step coaching cycle into clear behavioral descriptors that can be reliably evaluated through live observation or recorded video analysis.

In research contexts, the CFS serves as an indispensable integrity check across randomized controlled trials (RCTs), single-case experimental designs (SCEDs), and multisite translational studies. It allows principal investigators to quantify treatment differentiation, monitor fidelity drift over time, compute dosage-fidelity interaction matrices, and ensure that multi-tiered coaching protocols remain uniform across diverse interventionists and service delivery modalities (e.g., in-person home visits versus synchronous telepractice). Clinically, the CFS functions as a formative and summative supervisory assessment tool. Clinical directors, BCBA supervisors, and early intervention administrators utilize the scale to conduct objective baseline audits of practitioner skills, guide individualized supervision plans, track professional competency acquisition during onboarding or practicum training, and identify specific practice deficits—such as superficial problem-solving or the omission of systematic reflection—that require targeted scaffolding.

5. Psychological Construct

The psychological construct underlying the CFS is implementation fidelity within professional coaching frameworks, defined as the degree to which an interventionist delivers an evidence-based coaching methodology in complete alignment with its intended procedural architecture, theoretical mechanisms, and operational principles. In the CFS framework, fidelity is conceptualized as a multi-stage, interactive process rather than a static clinician characteristic. The instrument decomposes coaching into three interconnected behavioral phases, augmented by an optional modality-specific feature:

Pre-Observation Phase

The pre-observation phase captures the coach's capacity to establish instructional alignment and mutual intentionality prior to active intervention delivery. In this preparatory phase, the coach refrains from unilateral didactic instruction and instead initiates a shared dialogue with the coachee to anchor the upcoming learning episode. The specific behavioral elements within this dimension include:

• Collaborative Target Identification: Explicitly determining the specific evidence-based strategy, developmental skill, or behavioral target that the coachee will implement with the child during the session (e.g., naturalistic environmental arrangement, mand training, or expectant waiting).
• Contextual Routine Selection: Establishing the natural everyday family or classroom routine within which the strategy will be situated (e.g., snack time, joint book reading, sensory play, or transitional routines), ensuring contextual relevance and ecological validity.
• Procedural Task Analysis Review: Reviewing the explicit behavioral steps, scaffolding prompts, and contingent reinforcement parameters necessary to execute the targeted intervention strategy correctly, thereby establishing a shared mental model before approaching the child.

Observation Phase

The observation phase assesses the coach's adherence to structured observational discipline. During this phase, the coach observes the uninterrupted natural interaction between the coachee and the child. The critical psychological parameter evaluated here is the coach's restraint from unnecessary verbal intrusion, physical overshadowing, or co-opting the caregiver's parental role. By maintaining an unobtrusive presence, the coach affords the coachee genuine psychological agency, authentic practice opportunities, and space to experience naturalistic contingencies. Concurrently, the coach systematically collects observational data, notes procedural adherence, and identifies salient interaction moments to inform the subsequent feedback phase.

Post-Observation Phase

The post-observation phase represents the communicative core of adult coaching, where experiential learning is transformed into lasting behavioral competence through structured debriefing. This phase incorporates three interdependent communicative interactions:

• Coachee Reflection: Eliciting the coachee's internal appraisal and cognitive self-assessment regarding the completed interaction. Rather than delivering immediate evaluation, the coach invites the coachee to articulate what went well, what challenges arose, and how the child responded, cultivating long-term self-monitoring and metacognitive awareness.
• Targeted Performance Feedback: Delivering balanced, explicit, and objective feedback. This encompasses supportive feedback that reinforces correct strategy implementation through descriptive praise linked to child behavioral outcomes, alongside constructive feedback that non-judgmentally pinpoints procedural deviations and opportunities for behavioral refinement.
• Collaborative Joint Problem-Solving: Engaging in bidirectional brainstorming and systematic problem-solving to overcome identified barriers. Expanded during scale iteration to include specific items evaluating the systematic generation, evaluation, and selection of adaptations, this dimension ensures that future practice attempts are rooted in data-driven behavioral solutions rather than passive compliance.

Optional Video Feedback Phase

To support contemporary technological adaptations, the CFS integrates an optional evaluative unit governing video feedback. In this component, the coach and coachee jointly review recorded video footage of previous or immediate interactions. The coach pauses video segments at critical decision points to highlight nuanced nonverbal cues, analyze child communicative initiations, review micro-behaviors that are challenging to discern in real time, and prompt deeper coachee reflection through objective visual playback.

6. Theoretical Framework

The Coaching Fidelity Scale is grounded at the intersection of three theoretical paradigms: Andragogy (Adult Learning Theory), Social Cognitive Theory, and the Behavioral Consultation and Coaching Model. Together, these frameworks establish the foundational premise that adult skill acquisition requires collaborative, non-hierarchical, practice-based instructional architectures.

The foundational paradigm is Malcolm Knowles' Adult Learning Theory (Andragogy) (Knowles, 1984). Knowles posited that adult learners are fundamentally distinct from pedagogical subjects: they possess self-directed self-concepts, accumulate rich experiential reservoirs that serve as learning resources, exhibit learning readiness tightly coupled to social roles, and require immediate, problem-centered applicability of knowledge. The CFS directly reflects these principles. By organizing coaching into collaborative pre-observation planning, open-ended reflective questioning, and reciprocal problem-solving, the scale penalizes top-down, authoritarian lecturing. It operationalizes a coaching dynamic where the professional serves as a reflective facilitator who honors the caregiver's expertise regarding their child while integrating evidence-based behavioral strategies into established family routines.

Concurrently, the CFS draws extensively from Albert Bandura's Social Cognitive Theory (Bandura, 1986), particularly the mechanisms of vicarious modeling, enactive mastery experiences, and the development of perceived self-efficacy. Self-efficacy—the personal belief in one's capability to execute behaviors necessary to produce specific performance attainments—is the single strongest predictor of sustained parental intervention implementation. Within the CFS architecture, enactive mastery is deliberately cultivated during the observation phase, where caregivers actively execute intervention strategies in real time. The subsequent post-observation feedback and collaborative problem-solving phases provide psychological safety, positive verbal persuasion, and attributional framing, transforming behavioral practice into robust, enduring parental self-efficacy.

Finally, the CFS operationalizes the empirical Triadic Behavioral Consultation and Coaching Model formulated within applied behavior analysis and early childhood intervention literature (e.g., Rush & Shelden, 2011; Snyder et al., 2015). This framework conceptualizes the coach-coachee-child relationship as a systemic triad wherein the ultimate beneficiary (the child with developmental disabilities) is reached through the capacity-building of the primary agent (the coachee). The triadic model insists that coaching cannot be haphazard or conversational; it requires a structured, empirical learning cycle. The CFS provides the operational blueprint for this cycle, establishing clear empirical criteria for each phase to ensure that coaching practices retain their programmatic integrity across interventions.

7. Validity

The construct validity, test sensitivity, and criterion-related integrity of the Coaching Fidelity Scale have been rigorously evaluated across empirical intervention trials (Meadan et al., 2023), drawing observational data from three distinct multi-baseline and group-design studies focusing on naturalistic caregiver coaching (Chung et al., 2020, 2021; Meadan et al., 2020).

Test Sensitivity and Known-Groups Validity

To ascertain whether the CFS is sensitive to meaningful differences in coaching competence, researchers conducted known-groups comparisons contrasting untrained baseline observations against post-training implementation sessions. Independent sample comparisons were performed between a baseline cohort of pre-intervention observations ($n = 50$) and a post-intervention cohort ($n = 81$). Given the non-normal distributional properties typical of fidelity rating data, a non-parametric Mann–Whitney U test was executed.

The statistical analyses yielded a profound, statistically significant differentiation between the two conditions. The pre-intervention baseline observations exhibited minimal coaching fidelity, characterized by a mean score of $1.00$ ($SD = 1.20$). In contrast, post-intervention fidelity ratings climbed to a mean of $5.91$ ($SD = 2.52$). The Mann–Whitney test confirmed this difference was highly significant: $$W = 327.00, \quad p < .001$$ The associated effect size, quantified via the rank-biserial correlation ($r_{rb}$), was calculated at $.84$. This extraordinarily large effect size indicates that the CFS possesses exceptional test sensitivity, demonstrating that the checklist accurately captures systematic shifts in professional coaching behavior following empirical training programs and is free of floor or ceiling artifact constraints.

Content and Social Validity

Content validity was established through an iterative developmental process involving expert panels in speech-language pathology, special education, and applied behavior analysis. Initial implementations of the original 9-item prototype revealed that practitioners frequently struggled to engage caregivers in rigorous, ongoing adaptation of strategies when initial efforts stalled. Based on qualitative feedback and error analysis across two empirical trials (Chung et al., 2020; Meadan et al., 2020), the author team added two granular items targeting iterative problem-solving (specifically labeled Problem-Solving items 2 and 4). This refinement expanded the tool to 11 core items, ensuring comprehensive operational coverage of the collaborative problem-solving construct and establishing robust face and content validity.

8. Reliability

The reliability of the Coaching Fidelity Scale has been extensively documented through evaluations of both internal consistency and inter-observer agreement across live and video-recorded observations.

Internal Consistency

The scale's internal consistency was evaluated by computing Cronbach's alpha across the full item matrix. The CFS demonstrated outstanding internal consistency: $$\alpha = .91 \quad (95% \text{ CI } [.89, .93])$$ This coefficient exceeds the standard threshold of $.80$ required for basic research instruments and satisfies the stringent $.90$ threshold demanded for high-stakes individual diagnostic and professional supervisory assessments. Furthermore, the average inter-item correlation was computed at: $$r = .47 \quad (95% \text{ CI } [.41, .53])$$ In psychometric scale development, an average inter-item correlation between $.40$ and $.50$ represents the optimal psychometric balance; it confirms that the items are sufficiently homogeneous to measure a coherent underlying construct without suffering from item redundancy or overly narrow operationalization.

Interrater Reliability (IRR)

Because the CFS is an observational measurement system completed by third-party evaluators, demonstrating high interrater reliability is essential to rule out observer drift and subjective scoring bias. Paired independent raters were trained to criterion using video benchmarks prior to scoring study sessions. The degree of agreement between independent raters for the overall CFS score was evaluated using the Intraclass Correlation Coefficient (ICC):

An ICC value surpassing $.96$ indicates nearly perfect agreement between independent raters, confirming that the operational definitions within the CFS checklist provide sufficiently low-inference behavioral anchors for observers to evaluate coaching fidelity with negligible measurement error.

9. Factor Analysis

To investigate the latent dimensionality of the Coaching Fidelity Scale and verify whether its component behaviors reflect a single global fidelity dimension or distinct subcomponents, an exploratory factor analysis (EFA) was conducted on the core item matrix (Meadan et al., 2023).

Factor Extraction and Variance Explained

The exploratory factor analysis extracted a clear, uncompromised unidimensional solution. A single general factor accounted for 77.84% of the total variance across the scale items. In psychometric research, a single dominant factor accounting for over 50% (and particularly over 70%) of total variance provides decisive empirical justification for treating a scale as strictly unidimensional, confirming that the three sequential phases (pre-observation, observation, post-observation) function cohesively to reflect a solitary latent construct: overall coaching fidelity.

Factor Loadings and Communalities

All evaluated items demonstrated high factor saturations on the single extracted factor. Every item exhibited a primary factor loading exceeding .76 (with specific item loadings reaching up into the high .80s, exemplified by benchmark item JP1). Commensurately, item communalities ($h^2$) were uniformly high, with all values exceeding .609. These elevated communalities indicate that the latent coaching fidelity factor explains more than 60% of the variance in each individual item, leaving minimal residual uniqueness or measurement error. Crucially, the analysis revealed no significant cross-loadings or anomalous secondary factor bifurcations.

Structural Fit Indices

The structural fit of the unidimensional measurement model was comprehensively assessed against conventional fit indices, with the empirical data demonstrating good-to-excellent alignment with the theoretical model across all evaluated parameters:

• Chi-Square Goodness of Fit: $\chi^2(27) = 9.423, ; p = .99$ (a non-significant chi-square relative to degrees of freedom indicates exceptional model-data fit with zero evidence of misspecification).
• Comparative Fit Index (CFI): $.981$ (well above the conservative $ge .95$ benchmark).
• Goodness-of-Fit Index (GFI): $.975$.
• Tucker–Lewis Index (TLI): $.975$ (exceeding the standard $.95$ cut-off).
• Root Mean Square Error of Approximation (RMSEA): $.065$ (falling within the accepted $le .08$ acceptable fit interval, demonstrating adequate control for parsimony).
• Root Mean Square Residual (RMSR): $.0860$.
• Weighted Root Mean Square Residual (WRMR): $.0449$ (well below the recommended $1.00$ threshold for categorical/dichotomous observational indicators).

These empirical findings confirm that calculating a single, composite total fidelity score by summing the adhered items is psychometrically justified and theoretically robust.

10. Instrument / Measurement Tool

The Coaching Fidelity Scale (CFS) is structured as a low-inference, criterion-referenced observational checklist. The key administrative and structural specifications of the instrument include:

• Test Type: Direct observational rating scale / Implementation fidelity checklist.
• Administration Format: Completed by an external trained observer (e.g., researcher, peer reviewer, or clinical supervisor) via live, real-time observation or synchronous/asynchronous video review.
• Item Count: 11 core items structured across three sequential phases, supplemented by 1 optional video feedback instructional item (totaling 12 operational entries when video debriefing is deployed).
• Response Scale: Dichotomous categorical scoring:
• Yes (1): The coach clearly and unambiguously demonstrated the target coaching behavior during the designated session phase in accordance with operational criteria.
• No (0): The target coaching behavior was omitted, incomplete, or executed contrary to operational guidelines.
• Not Applicable (N/A): Permitted solely for designated contextual items (e.g., the optional video review module when video analysis is not utilized).
• Subscale Structural Breakdown:
  • Phase 1: Pre-Observation (Operational items focusing on goal consensus, routine identification, and procedural review).
  • Phase 2: Observation (Operational item evaluating coach non-intrusiveness and systematic observation of coachee-child interactions).
  • Phase 3: Post-Observation (Operational items evaluating coachee self-reflection, supportive praise, constructive error-correction feedback, and iterative joint problem-solving).
  • Optional Instructional Module (Evaluates the fidelity of joint video replay and targeted video-based debriefing).
• Scoring Rules: A total composite fidelity score is generated by calculating the sum of all observed behaviors ($1$ point per fulfilled item) divided by the total number of applicable items, multiplied by $100$ to produce a Percentage of Fidelity Achieved: $$\text{Fidelity Percentage} = \left( \frac{\sum \text{Observed Items (Yes)}}{\text{Total Applicable Items}} \right) \times 100$$ In standard implementation intervention trials, an implementation fidelity threshold of $ge 80%$ is generally adopted as the benchmark for acceptable delivery.
• Time Required: Varies according to coaching session duration; typically requires 15 to 45 minutes of observation with an immediate 5-minute post-session scoring finalization.
• Target Population: Adult professionals delivering coaching services (e.g., special educators, early interventionists, speech-language pathologists, Board Certified Behavior Analysts, paraprofessionals, and behavior technicians) working with coachees (parents, primary caregivers, or classroom instructional aides) of children aged birth through five years old presenting with autism spectrum disorder or developmental disabilities.

11. Permissions & Fee and Test Year

The Coaching Fidelity Scale was formally published in 2023 in the peer-reviewed journal Infants & Young Children (Meadan et al., 2023).

• Permissions: The scale is protected under academic copyright held by Wolters Kluwer Health, Inc. and the study authors. Researchers and clinical practitioners wishing to utilize, reproduce, adapt, or digitize the CFS for academic research, thesis projects, or clinical supervision are required to contact the corresponding author, Dr. Hedda Meadan, for written permission and operational scoring guidelines.
• Correspondence Contact: Dr. Hedda Meadan, Department of Special Education, University of Illinois Urbana-Champaign, 1310 South Sixth Street, Champaign, IL 61820. Email: [email protected].
• Commercial Use: Commercial exploitation, inclusion in paid software suites, or unauthorized resale without explicit legal licensing is strictly prohibited.
• Fee: There are no licensing fees associated with standard non-commercial academic research and clinical training use, provided formal permission is obtained and appropriate academic citation is maintained.

12. References

The following peer-reviewed publications provide the theoretical foundation, methodological development, and empirical psychometric validation of the Coaching Fidelity Scale:

• Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Prentice-Hall.
• Chung, M. Y., Meadan, H., & Lee, J. D. (2020). Parent-implemented communication strategies: A coaching manual for practitioners. Journal of Special Education Technology, 35(4), 241–253. https://doi.org/10.1177/0022466920950640
• Chung, M. Y., Meadan, H., Sands, M. M., & Lee, J. D. (2021). Preparing early interventionists to coach caregivers via telepractice. Teacher Education and Special Education, 44(3), 218–234. https://doi.org/10.1177/08884064211001456
• Knowles, M. S. (1984). Andragogy in action: Applying modern principles of adult learning. Jossey-Bass.
• Meadan, H., Lee, J. D., Sands, M. M., Chung, M. Y., & García-Grau, P. (2023). The Coaching Fidelity Scale (CFS): Development and evaluation of an observational measure of coaching fidelity. Infants & Young Children, 36(1), 37–52. https://doi.org/10.1097/IYC.0000000000000231
• Meadan, H., Sands, M. M., Chung, M. Y., & Lee, J. D. (2020). Internet-based caregiver-mediated intervention: A systematic review. Topics in Early Childhood Special Education, 40(1), 4–18. https://doi.org/10.1177/0271121419890008
• Rush, D. D., & Shelden, M. L. (2011). The early childhood coaching handbook. Paul H. Brookes Publishing.
• Snyder, P. A., Hemmeter, M. L., & Fox, L. (2015). Supporting early childhood educators’ use of evidence-based practices through practice-based coaching. Handbook of Early Childhood Special Education, 301–320. https://doi.org/10.1007/978-3-319-28492-7_16

13. Items of the Scale