1. Abstract
The Assistive Technology Usability Questionnaire for people with Neurological diseases Questionnaire (NATU Quest) is an innovative, specialized psychometric instrument engineered to evaluate the perceived usability, operational efficacy, and subjective user satisfaction associated with assistive technology (AT) solutions among individuals diagnosed with acute and chronic neurological disorders. Developed and psychometrically validated by Masbernat-Almenara and colleagues in 2023, the scale addresses a profound methodological gap in neurorehabilitation and human-computer interaction: the lack of accessible, psychometrically sound, patient-centered usability measures tailored specifically to individuals facing sensory, cognitive, communication, and motor impairments. The NATU Quest comprises 10 unidimensional items administered through an electronic interface featuring cognitive accommodations, specifically a 6-point Likert-type response format (anchored from 0 = Strongly Disagree to 5 = Strongly Agree), coupled with an intuitive chromatic traffic light visual schema and calibrated facial expression icons (emojis) to facilitate unambiguous comprehension and reduce cognitive response load.
The scale development was grounded in a multi-stage empirical paradigm beginning with a rigorous Delphi consensus process involving multidisciplinary clinical experts and end-users with neurological conditions. Psychometric evaluation conducted with a Spanish cohort of AT users and domain experts established exceptional psychometric properties. The instrument demonstrated robust internal consistency reliability (Cronbach’s alpha, α = 0.895) and high longitudinal temporal stability (Spearman’s rank correlation coefficient ρ = 0.818, p < 0.001; Intraclass Correlation Coefficient, ICC = 0.869, 95% CI [0.781, 0.923]). Content validity was confirmed through quantitative Delphi thresholds exceeding 0.58. Concurrent criterion-related validity was confirmed by strong positive correlations with established benchmark usability instruments (ρ = 0.756, p < 0.001). Furthermore, advanced latent trait modeling through modern Item Response Theory (IRT) demonstrated that all 10 items exhibit an excellent fit to the unidimensional Rasch measurement model, ensuring invariant interval-level measurement across the latent continuum of assistive technology usability.
2. Keywords
Assistive Technology, Usability Assessment, Neurological Diseases, NATU Quest, Neurorehabilitation, Psychometrics, Rasch Measurement Model, Human-Computer Interaction, Patient-Reported Outcome Measures, User Satisfaction
3. Authors
The NATU Quest was conceived, constructed, and psychometrically validated by a collaborative multidisciplinary consortium of researchers, clinical neurorehabilitation specialists, physiotherapists, and biomedical engineers based in Catalonia, Spain:
- Maria Masbernat-Almenara, PhD, MSc, PT: Department of Nursing and Physiotherapy, Faculty of Nursing and Physiotherapy, University of Lleida, Lleida, Spain. ORCID: 0000-0002-5822-239X.
- Francesc Rubí-Carnacea, PhD, MSc, PT: Department of Nursing and Physiotherapy, Faculty of Nursing and Physiotherapy, University of Lleida, Lleida, Spain. Email: [email protected] (Corresponding Author).
- Eloy Opisso, PhD: Institut Guttmann, Institut Universitari de Neurorehabilitació adscrit a la UAB, Badalona, Barcelona, Spain; Fundació Institut d’Investigació en Ciències de la Salut Germans Trias i Pujol, Badalona, Spain. ORCID: 0000-0002-6868-6737.
- Esther Duarte-Oller, MD, PhD: Physical Medicine and Rehabilitation Department, Parc de Salut Mar (Hospital del Mar, Hospital de l’Esperança), Rehabilitation Research Group, Hospital del Mar Medical Research Institute (IMIM), Universitat Autònoma de Barcelona, Barcelona, Spain.
- Josep Medina-Casanovas, PhD, PT: Institut Guttmann, Institut Universitari de Neurorehabilitació adscrit a la UAB, Badalona, Barcelona, Spain.
- Fran Valenzuela-Pascual, PhD, MSc, PT: Department of Nursing and Physiotherapy, Faculty of Nursing and Physiotherapy, University of Lleida, Lleida, Spain. ORCID: 0000-0002-1945-629X.
4. Purpose
The primary clinical and psychometric objective of the NATU Quest is to provide a standardized, psychometrically rigorous, and accessible instrument designed specifically to measure the perceived usability, operational utility, safety, learnability, and overall satisfaction of assistive technology (AT) devices among individuals living with acquired or progressive neurological conditions. Neurological conditions—including stroke, traumatic brain injury, spinal cord injury, multiple sclerosis, Parkinson’s disease, and cerebral palsy—characteristically generate heterogeneous functional deficits across physical, cognitive, sensory, and affective operational domains. In this population, assistive technologies serve as crucial bridges toward restoring independence, functional communication, mobility, and community reintegration.
Historically, evaluating the usability of such devices relied heavily on generic usability instruments derived from standard human factors engineering and commercial software development, such as the System Usability Scale (SUS) or the Post-Study System Usability Questionnaire (PSSUQ). Although methodologically robust in neurotypical populations, these generic legacy instruments possess notable design limitations when deployed in clinical neurorehabilitation environments. First, their textual syntax often includes complex, alternating negatively keyed statements (such as SUS Item 2: “I found the system unnecessarily complex” versus Item 3: “I thought the system was easy to use”). For patients presenting with post-stroke aphasia, executive dysfunction, cognitive slowing, or working memory constraints, alternating item valence frequently triggers cognitive fatigue, acquiescence bias, or systematic misinterpretation. Second, generic tools lack items that capture functional autonomy, physical safety, ergonomic comfort, and clinical support, which are central to healthcare-related assistive devices.
The NATU Quest was engineered to overcome these barriers through universal design principles and targeted psychometric construction. By utilizing exclusively positively keyed items, eliminating linguistic ambiguity, and pairing each response anchor with an intuitive chromatic traffic light spectrum and facial affect icons, the NATU Quest minimizes construct-irrelevant cognitive load. In clinical contexts, occupational therapists, physiotherapists, physiatrists, and rehabilitation engineers can utilize the instrument to track longitudinal user adaptation, systematically benchmark competing commercial devices, detect early risk indicators of device abandonment, and guide personalized equipment customization. In research and industrial design contexts, the scale provides a validated outcome measure for clinical trials and user-centered design iterations, ensuring assistive products genuinely align with the needs of end-users.
5. Psychological Construct
The underlying construct operationalized by the NATU Quest is Assistive Technology Usability within Neurorehabilitation. Grounded in the ISO 9241-11 standard for ergonomics of human-system interaction, usability is traditionally defined as the extent to which a system, product, or service can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use. Within the context of neurorehabilitation, however, usability expands beyond simple cognitive efficiency to embrace deeply interrelated physical, ergonomic, psychological, and environmental dimensions.
Through item development and modern psychometric validation, the NATU Quest operationalizes usability as an integrated, unidimensional latent continuum consisting of five primary operational facets:
- Operational Effectiveness and Functional Utility: This dimension assesses the user’s perception of whether the assistive device reliably facilitates the successful completion of target daily activities and compensatory tasks (e.g., Item 2: “The product allows me to perform activities effectively”). Effectiveness reflects functional performance: the degree to which motor, communication, or cognitive execution is restored or compensated for without technological disruption.
- Ergonomic Comfort, Perceived Safety, and Physical Risk: Unlike software interfaces, physical and biomechanical assistive technologies interface directly with impaired sensory and neuromuscular anatomy. Consequently, physical comfort and psychological safety are primary determinants of usability (e.g., Item 3: “The product is comfortable and safe to use”). If an assistive device induces muscular strain, skin shear, or anxiety regarding falling or malfunctioning, perceived usability drops substantially regardless of functional performance.
- Learnability and Cognitive Accessibility: Learnability reflects the mental effort and temporal investment demanded of the user to achieve initial operational proficiency (e.g., Item 1: “The product is easy to use”; Item 4: “It took me a short time to learn how to use the product”). Given that neurological pathology frequently compromises processing speed, divided attention, and procedural memory acquisition, an assistive product must present intuitive affordances that minimize cognitive strain.
- Psychosocial Empowerment, Identity, and Autonomy: The adoption of assistive technology profoundly influences the user’s self-concept, sense of self-determination, and personal dignity. The NATU Quest explicitly captures psychological empowerment and daily independence (e.g., Item 6: “I feel autonomous and independent when using the product”). Furthermore, it gauges aesthetic acceptability and social stigma reduction (e.g., Item 7: “The product design is appealing and appropriate for daily use”), recognizing that non-stigmatizing, aesthetically pleasing devices support long-term psychosocial integration.
- System Support, Expectation Fulfillment, and Global User Satisfaction: The final facet incorporates the ecological support surrounding the technology, including instructional guidance and professional support (e.g., Item 8: “The instructions or support provided were clear and helpful”), expectation congruence (e.g., Item 5: “The product meets my expectations and needs”), advocacy/loyalty intentions (e.g., Item 9: “I would recommend this product to other people in a similar situation”), and holistic affective satisfaction (e.g., Item 10: “Overall, I am satisfied with the product”).
Through Rasch modeling, these diverse operational manifestations have been empirically proven to map onto a single, cohesive latent trait continuum (θ), demonstrating that in neurological users, usability operates as a unified construct spanning cognitive, physical, and emotional adaptation.
6. Theoretical Framework
The construction and validation of the NATU Quest are rooted in three foundational theoretical frameworks: the Matching Person and Technology (MPT) Model, the Technology Acceptance Model (TAM), and the International Classification of Functioning, Disability and Health (ICF).
The Matching Person and Technology (MPT) Model
Formulated by Marcia Scherer, the Matching Person and Technology model posits that successful assistive technology integration depends upon the reciprocal alignment between three interactive domains: the personal characteristics of the user (personality, expectations, coping style, cognitive-motor capabilities), the physical and social milieu (environmental demands, professional support, structural barriers), and the specific attributes of the technology (complexity, comfort, appearance, reliability). Scherer’s empirical investigations indicated that technological abandonment rates in neurorehabilitation historically exceed 30% to 50%, primarily because clinicians evaluate physical impairment while overlooking user expectations, personal preferences, and psychosocial comfort. The NATU Quest operationalizes MPT tenets by explicitly balancing functional capability against personal satisfaction, emotional autonomy, aesthetic acceptability, and the adequacy of environmental training support.
The Technology Acceptance Model (TAM) and Cognitive Load Theory
Originating from human factors psychology, Fred Davis’s Technology Acceptance Model asserts that user behavioral intention and ultimate system adoption are determined by two core cognitive appraisals: Perceived Usefulness (PU) and Perceived Ease of Use (PEOU). Within the NATU Quest, items assessing operational effectiveness (Item 2) and expectation fulfillment (Item 5) reflect Perceived Usefulness, while items assessing ease of use (Item 1) and learnability (Item 4) capture Perceived Ease of Use. Moreover, John Sweller’s Cognitive Load Theory guided the design of the instrument itself. Recognizing that neurological injury reduces available working memory capacity, the scale incorporates dual-coding visual stimuli—chromatic gradients and stylized facial affect representations—which allow patients to process response anchors through parallel visual and emotional channels, lowering extraneous cognitive load during self-report administration.
The World Health Organization ICF Framework
Under the World Health Organization’s International Classification of Functioning, Disability and Health (ICF), disability is conceptualized not as an isolated biological deficit, but as a dynamic interaction between health conditions and contextual (environmental and personal) factors. Assistive technologies serve as environmental facilitators designed to bridge the gap between biological impairment (body functions/structures) and societal participation. The NATU Quest was engineered in harmony with this biopsychosocial architecture: its items do not gauge neuromuscular deficits, but rather evaluate how effectively the environmental tool modifies functional activity performance, fosters autonomy, and fits within the user’s lived context.
7. Validity
The psychometric validation of the NATU Quest followed a multi-phase experimental methodology adhering to the COSMIN (Consensus-based Standards for the selection of health Measurement Instruments) recommendations, establishing robust content, concurrent, and construct validity across targeted neurorehabilitation cohorts.
Content Validity and the Delphi Consensus Method
Content validity was established through a formal, iterative Delphi consensus study. A panel composed of both multidisciplinary clinical and academic experts (neurologists, physiotherapists, occupational therapists, biomedical engineers) and experienced adult assistive technology end-users diagnosed with diverse neurological conditions evaluated candidate items for clarity, semantic relevance, clinical appropriateness, and universal applicability. Quantitative content validity criteria were applied, requiring items to surpass an established content validity index threshold of 0.58. Nine initial core items met this strict statistical threshold; a tenth synthesizing item capturing overall holistic satisfaction was integrated based on expert consensus, finalizing the 10-item instrument. This collaborative process ensured comprehensive content coverage of the usability domain while preventing extraneous or confusing items from entering the scale.
Concurrent Criterion Validity
To establish concurrent criterion validity, the NATU Quest was administered alongside established, internationally recognized benchmark usability questionnaires within a representative clinical sample of individuals with neurological diseases in Spain. Non-parametric correlation analyses revealed a strong, statistically significant positive association between total NATU Quest scores and the benchmark usability scores, demonstrated by a Spearman’s rank correlation coefficient of:
ρ = 0.756 (p < 0.001)
This high correlation demonstrates that the NATU Quest measures the target construct of technological usability without operational divergence, while providing a format tailored specifically to the cognitive and sensory needs of neurological patients.
Construct Validity via Modern Item Response Theory (Rasch Analysis)
Construct validity was evaluated using modern Item Response Theory (IRT), specifically through the application of the unidimensional Rasch measurement model. In psychometrics, Rasch modeling evaluates whether individual items calibrate along a continuous latent trait continuum (θ) such that the probability of endorsing a higher rating is governed solely by the person’s underlying trait level and the item’s inherent difficulty or endorsement threshold. All 10 items of the NATU Quest demonstrated a satisfactory mathematical fit to the Rasch model expectations (with mean-square infit and outfit statistics conforming to recommended psychometric tolerances). This confirmed that:
- The 10 items function cooperatively along a single, coherent latent continuum of usability;
- The visual 6-point response scale progresses monotonically across all functional ability levels; and
- Raw total ordinal sum scores can be mapped onto an interval-level linear measurement scale, facilitating robust parametric statistical analyses in clinical trials.
8. Reliability
The reliability of the NATU Quest was demonstrated through internal consistency analyses and temporal test-retest reproducibility evaluations.
Internal Consistency Reliability
Internal consistency reflects the extent to which all items within an instrument assess facets of the same underlying construct. Evaluated on the validation cohort, the NATU Quest exhibited high internal consistency, yielding a Cronbach’s alpha coefficient of:
α = 0.895
In psychometrics, an alpha coefficient between 0.85 and 0.90 represents optimal internal consistency; it confirms that the items are closely interrelated and free of substantial random measurement error, while remaining sufficiently distinct to avoid excessive item redundancy (which is typically observed when alpha exceeds 0.95).
Test-Retest Stability and Reproducibility
To assess temporal stability, the questionnaire was readministered to stable clinical respondents across a standardized time interval during which their assistive technologies and neurological status remained unchanged. Item-level agreement was evaluated using the weighted Cohen’s Kappa coefficient, showing moderate to considerable concordance across all individual scale items.
At the aggregate scale level, non-parametric rank stability assessed via Spearman’s rank correlation coefficient demonstrated a strong positive association between repeated administrations:
ρ = 0.818 (p < 0.001)
Furthermore, the scale’s relative reliability was corroborated using the Intraclass Correlation Coefficient (ICC), calculated across test-retest administrations as:
ICC = 0.869 (95% Confidence Interval: [0.781, 0.923])
An ICC value of 0.869 surpasses the recognized benchmark of 0.75 for clinical assessment tools, confirming that the NATU Quest is resistant to transient temporal fluctuations and suitable for tracking long-term neurorehabilitation outcomes.
9. Factor Analysis
In classical psychometric scale validation, researchers frequently utilize Exploratory Factor Analysis (EFA) or Confirmatory Factor Analysis (CFA) based on linear Pearson covariance matrices. However, in the psychometric validation of the NATU Quest, the investigators employed modern Item Response Theory (IRT)—specifically the Rasch Measurement Model—as the primary analytical framework for examining structural dimensionality and construct coherence.
Rasch Latent Trait Modeling vs. Classical Factor Analysis
Traditional factor analysis assumes continuous, normally distributed interval data; applying it to ordinal Likert scales can lead to spurious difficulty factors and distorted variance estimates. In contrast, the Rasch model explicitly transforms ordinal response data into an invariant, linear log-odds (logit) scale. In the NATU Quest validation, the Rasch analysis confirmed that all 10 items conform to the assumption of unidimensionality. The items exhibited satisfactory infit and outfit statistics within standard target ranges (0.6 to 1.4), verifying that unexplained residual variance was minimal and random, without secondary multidimensional clusters.
Item Hierarchy and Targeting
The Rasch calibrated item difficulty hierarchy demonstrated that the 10 NATU Quest items span the latent usability spectrum (θ) in an orderly manner. Easy-to-endorse items capture baseline operational usability and safety, whereas higher-threshold items require higher degrees of device integration (such as feeling completely autonomous or recommending the device to peers). This distribution ensures that the instrument can distinguish among various levels of assistive technology performance across clinical populations.
10. Instrument / Measurement Tool
- Instrument Name: Assistive Technology Usability Questionnaire for people with Neurological diseases Questionnaire (NATU Quest)
- Original Acronym: NATU Quest / NATU Questionnaire
- Test Type: Original Patient-Reported Outcome Measure (PROM) / Clinical Usability Inventory
- Administration Format: Electronic digital administration (compatible with touchscreens, tablets, modified mice, and assistive input peripherals)
- Target Population: Adults (18 years and older) diagnosed with acute or chronic neurological conditions (e.g., stroke, spinal cord injury, traumatic brain injury, multiple sclerosis, cerebral palsy) who use assistive technologies; also validated for use by neurorehabilitation clinical experts evaluating assistive technology products.
- Item Count: 10 discrete declarative items
- Response Scale: 6-point Likert-type scale, ranging from 0 to 5:
- 0 = Strongly Disagree
- 1 = Disagree
- 2 = Somewhat Disagree
- 3 = Somewhat Agree
- 4 = Agree
- 5 = Strongly Agree
- Visual Accommodations: Each numerical response point is paired with a chromatic traffic light schema (e.g., green tones for agreement, red tones for disagreement) and calibrated facial affect icons (emojis) ranging from distressed/frowning to smiling expressions.
- Scoring Paradigm: All 10 items are positively keyed. Numerical responses are summed to yield a total raw usability score ranging from 0 to 50, where higher scores reflect superior perceived usability, operational efficacy, and user satisfaction. Rasch logit conversion tables can be applied for interval-level measurement in research trials.
11. Permissions & Fee and Test Year
- Publication Year: 2023
- Copyright & Licensing: The NATU Quest is distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0).
- Usage Permissions: Under the CC BY 4.0 open-access framework, clinicians, researchers, and healthcare institutions are permitted to share, copy, redistribute, adapt, transform, and build upon the material for any purpose, including commercial and non-commercial applications, provided appropriate credit is given to the original authors and the source publication is cited.
- Fee: Free of charge / Open-Access (No license or royalty fees).
- Correspondence Contact: Francesc Rubí-Carnacea, Department of Nursing and Physiotherapy, University of Lleida, Lleida, Spain (Email: [email protected]).
12. References
- Brooke, J. (1996). SUS: A ‘quick and dirty’ usability scale. In P. W. Jordan, B. Thomas, I. L. McClelland, & B. Weerdmeester (Eds.), Usability Evaluation in Industry (pp. 189–194). Taylor & Francis.
- Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340. https://doi.org/10.2307/249008
- International Organization for Standardization. (2018). Ergonomics of human-system interaction — Part 11: Usability: Definitions and concepts (ISO Standard No. 9241-11:2018). https://www.iso.org/standard/63500.html
- Masbernat-Almenara, M., Rubi-Carnacea, F., Opisso, E., Duarte-Oller, E., Medina-Casanovas, J., & Valenzuela-Pascual, F. (2023). Developing an assistive technology usability questionnaire for people with neurological diseases. PLoS ONE, 18(1), Article e0281197. https://doi.org/10.1371/journal.pone.0281197
- Mokkink, L. B., Terwee, C. B., Patrick, D. L., Alonso, J., Stratford, P. W., Knol, D. L., Bouter, L. M., & de Vet, H. C. (2010). The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. Journal of Clinical Epidemiology, 63(7), 737–745. https://doi.org/10.1016/j.jclinepi.2010.02.006
- Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Danmarks Paedagogiske Institut.
- Scherer, M. J. (2002). Assistive technology: Matching device and consumer for successful rehabilitation. American Psychological Association. https://doi.org/10.1037/10420-000
- World Health Organization. (2001). International Classification of Functioning, Disability and Health: ICF. World Health Organization. https://apps.who.int/iris/handle/10665/42407