The University of Toledo The University of Toledo Digital Repository Theses and Dissertations 2013 Assessment of the measurement properties of the NHCAHPS family survey : a Rasch scaling approach Matthew S. O'Connor The University of Toledo Follow this and additional works at: http://utdr.edu/theses-dissertations Recommended Citation O'Connor, Matthew S., "Assessment of the measurement properties of the NHCAHPS family survey : a Rasch scaling approach" (2013). Theses and Dissertations. This Dissertation is brought to you for free and open access by The University of Toledo Digital Repository.
It has been accepted for inclusion in Theses and Dissertations by an authorized administrator of The University of Toledo Digital Repository. For more information, please see the repository's About page. A Dissertation entitled Assessment of the Measurement Properties of the NHCAHPS Family Survey: A Rasch Scaling Approach by Matthew S. O’Connor Submitted to the Graduate Faculty as partial fulfillment of the requirements for the Doctor of Philosophy Degree in Foundations of Education _________________________________________ Christine M.
Fox, PhD, Committee Chair _________________________________________ Gregory E. Stone, PhD, Committee Member _________________________________________ Noela Haughton, PhD, Committee Member _________________________________________ John Gallick, PhD, Committee Member _________________________________________ Patricia R. Komuniecki, PhD, Dean College of Graduate Studies The University of Toledo May 2013 Copyright 2013, Matthew Stephen O’Connor This document is copyrighted material. Under copyright law, no parts of this document may be reproduced without the expressed permission of the author.
An Abstract of Assessment of the Measurement Properties of the NHCAHPS Family Survey: A Rasch Scaling Approach by Matthew S. O’Connor Submitted to the Graduate Faculty as partial fulfillment of the requirements for the Doctor of Philosophy Degree in Foundations of Education The University of Toledo May 2013 The introduction of the Consumer Assessment of Healthcare Providers and Systems (CAHPS), a family of survey instruments designed to capture and report people’s experiences obtaining health care could soon add satisfaction as a consistent dimension of quality that skilled nursing facilities (SNFs) are required to assess and report. The SNF setting has not yet been mandated to implement CAHPS for Nursing Homes (NHCAHPS). Given the critical implications (e., comparisons of SNFs on NHCAHPS scores) and decisions resulting from performance on the NHCAHPS survey (i., Medicare reimbursement) it was imperative to construct a measure of family satisfaction.
The data were analyzed with the Rasch rating scale model. Rasch analyses demonstrate that the NHCAHPS Family Survey has adequate reliability, separation, fit, rating scale functioning, and dimensionality. Particular attention was given to the Overall Rating of Care item. According to the Rasch diagnostic indices, the NHCAHPS Family Survey composite domains did not function well.
Results are discussed in terms of their application to usefulness by SNF management teams for process improvement and to include in the Five-Star Quality Rating System. iii This dissertation is dedicated to my wife, Karen, and my children: Jason and Drew. Your tremendous patience and sacrifice as I pursued a lifelong dream will always be something I remember. I hope that one day Jason and Drew pick up this document and understand that when they have a dream they will need to work hard to accomplish it.
Working hard will mean giving up immediate gratification to achieve something more fulfilling. When I started on this doctoral journey, Jason was 5 and Karen was pregnant with Drew. This meant I had to leave Karen many evenings with 2 young boys as I attended class. She was able to make it through some tough times and will be happy this chapter of my life is now over.
I know it was not easy and I will always be thankful. There are many other people who were very helpful and supportive during this journey. My parents, Bill and Mary Pat, were always supportive of my dream. They helped me tremendously when this dream was derailed at one point in my life but knew it would be something I accomplished later.
My in-laws, Ron and Helen Payeff, took the boys many evenings when I was gone and helped to Karen out more than I will ever know during this time. Thank you for your support. iv Acknowledgements First and foremost, I would like to thank my dissertation committee collectively for generously giving their time and feedback to me throughout this process. I would especially like to thank my chair, Christine Fox.
Christine pushed me to develop my writing and research skills. I will always appreciate the amount of time she put in reviewing my drafts. Every time a draft was returned, I knew it would be filled with numerous “track changes” comments and grammatical changes that made my document better each time. Next, I need to thank HCR ManorCare for their financial support to complete this degree.
I will always remember Stephen Guillard as he made a significant company policy exception for me to achieve my dream. Also, I need to recognize John Huber as he provided me flexibility in my work schedule to attend courses that often began before the work day ended. Finally, John Gallick needs recognition for being a member of my committee and a support when I needed to discuss the project with someone. It is my desire to use what I have learned to continue to make HCR ManorCare the preeminent provider of long-term care.
I also need to thank Jamie Trabbic and Jennifer Orcelletto for putting up with me during the past year as I worked on this project. They listened to me vent about the process but were supportive of the goal often by picking up some extra work. v Table of Contents Abstract iii Acknowledgements v Table of Contents vi List of Tables x List of Figures xii List of Abbreviations xiii List of Symbols xiv I. Five-Star Quality Rating System 2 B.
Consumer Assessment of Healthcare Providers and Systems (CAHPS) 4 a. NHCAHPS Family Survey Scoring and Reporting 5 C. Impact of Satisfaction Measurement 7 a. Statement of the Problem 12 F.
Research Questions 15 II. The Current Methods for Collecting and Applying Quality Data in Skilled Skilled Nursing Facilities 16 A. Groups Advancing SNF Quality 17 B. Current View of Quality in Skilled Nursing Facilities 19 vi a.
Annual Survey Inspection 19 b. Complaint Survey Investigations 20 c. Nursing Home Compare 22 d. Health Inspection Domain 22 e.
Quality Measure Domain 29 g. Overall SNF rating (Composite Measure) 32 C. Issues with Current Quality Indicators 33 a. Consumer-Determined Quality 36 D.
The Rasch Model 39 E. Outcomes of Quality Indicators 40 a. Pay for Performance 41 b. Accountable Care Organizations 43 III.
Methods and Procedures 45 A. Data Management and Collection 47 a. Phase 1: Facility Notification of the Survey 48 b. Phase 2: Assignment of survey recipient 48 c.
Phase 3: Survey Notification and Distribution 48 D. Theoretical Framework for Measure Construction 49 vii a. Rating Scale Rasch Model 50 E. Scale Category Analyses 58 i.
Rating scale type 1 58 ii. Rating scale type 2 60 iii. Rating scale type 3 62 iv. Rating scale type 4 62 v.
Rating scale type 5 64 B. Improving the Rating Scales 65 a. Collapsing Recommendation to Others Scale 66 b. Overall Rating of Care 69 C.
Diagnostic Analysis with Random Samples 71 a. Scale Category Analysis 72 D. Item Fit Analyses 73 E. Principle Contrasts Analysis 83 G.
Composite Domain Analysis 84 a. Meeting Basic Needs Composite Domain 85 viii b. Nurses and Aides Kindness and Respect Towards Resident 85 c. Nursing Home Provides Information and Encourages Respondent Involvement 87 d.
Nursing Home Staffing, Care of Belongings and Cleanliness 88 V. Summary of Rasch Findings 90 B. Application of Findings 92 C. Overall Rating of Care 97 D.
Limitations 98 References 99 Appendices A. NHCAHPS Family Survey 104 B. Composite Measure Proportional Scoring Method Description and Example 111 C. Composite Measures for the NHCAHPS Family Member Survey 113 D.
Detailed Description of Scale Collapsing for Overall Quality of Care Item 115 ix List of Tables Table 1 Five-Star Ratings Definitions. 4 Table 2 Types of NHCAHPS Family Survey Scores. 8 Table 3 Health Inspection Score: Weights for Different Types of Deficiencies. 24 Table 4 Weighting Factor for Annual Survey Inspections and Complaint Survey Investigations.
25 Table 5 Weights for Repeat Revisits. 26 Table 6 National Average Hours per Patient Day Used in Calculation of Adjusted Staffing. 27 Table 7 National Star Cut points for Staffing Measures. 28 Table 8 Staffing Points and Five-Star Rating.
28 Table 9 MDS-Based Quality Measures. 30 Table 10 Five Star Cut Points for MDS Quality Measure Domain Score. 31 Table 11 Five-Step Process for Assigning Overall Five-Star Rating. 33 Table 12 Rating Scale Type 1 Diagnostics.
58 Table 13 Rating Scale Type 2 Diagnostics. 60 Table 14 Rating Scale Type 4 Diagnostics. 62 Table 15 Rating Scale Type 5 Diagnostics. 65 Table 16 Recommendation to Others 3-Point Rating Scale Diagnostics.
67 Table 17 Description of the Overall Rating of Care Collapsing Options. 70 Table 18 Summary of Changes in Person and Item Separation, Reliability, Variance Explained, and Fit Indices for Overall Care Rating Scale Collapsing Options,. 71 x Table 19 Separation, Reliability and Variance Explained for Random Samples. 72 Table 20 Range of the Item Fit Statistics for the Five Random Samples.
74 Table 21 Range of Item Measures and Standard Errors. 77 Table 22 Standardized Residual Variance in Eigenvalue Units. 84 xi List of Figures Figure 1 Sample webpage from the Nursing Home Compare website. 23 Figure 2 Rating Scale Type 1 Probability Curves.
59 Figure 3 Rating Scale Type 2 Probability Curves. 61 Figure 4 Rating Scale Type 4 Probability Curves. 63 Figure 5 Rating Scale Type 5 Probability Curves. 66 Figure 6 Recommendation to Others 3-Point Rating Scale Probability Curves.
68 Figure 7 Person Map of Items. 80 Figure 8 General Keyform Map. 82 Figure 9 An Individual Survey Response for Service Recovery Intervention. 94 xii List of Abbreviations ACA.
Affordable Care Act ACO. Accountable Care Organization ADL. Activities of Daily Living AHCA. American Health Care Association AHRQ.
Agency for Healthcare Research and Quality AIR. American Institute of Research CASPER. Certification and Survey Provider Enhanced Reports CAHPS. Consumer Assessment of Healthcare Providers and Systems CMS.
Centers for Medicare and Medicaid Services CSSR. Customer Service Survey Recipient HCAHPS. Hospital Consumer Assessment of Healthcare Providers and Systems HHCAHPS. Home Health Consumer Assessment of Healthcare Providers and Systems IOM.
Institute of Medicine LPN. Licensed Practical Nurse MDS. Minimum Data Set MedPAC. Medicare Payment Advisory Commission MIV.
My InnerView NHCAHPS. Nursing Home Consumer Assessment of Healthcare Providers and Systems NRC. National research Corporation NQF. National Quality Forum PCC.
Point Click care P4P. Pay for Performance QI. Quality Improvement RN. Registered Nurse RPCA.
Rasch Principle Contrasts Analysis RUG. Resource Utilization Group SFF. Special Focus Facility SNF. Skilled Nursing Facility VBP.
Value Based Purchasing xiii List of Symbols Bp Rasch Person Ability Di. Rasch Item Difficulty G. Rasch Person and Item Separation Statistics S. Observed Item Score P.
Proportion of Correct Items xiv Chapter One Introduction More than 3 million elderly and disabled individuals will rely on services provided by a skilled nursing facility (SNF) at some point during the year, and among them 1.5 million will stay long enough to consider the SNF their main residence (Doshi, Shaffer, and Briesacher, 2005). These individuals, their families, and their friends count on SNFs to provide care that is of high quality. Enduring issues surrounding SNFs have been quality related. The often-poor quality of SNFs has been a consistent issue of concern for consumers, governments and researchers.
Numerous definitions of quality exist. The Institute of Medicine (IOM) (1996) provides one of the most widely cited definitions: “Quality is the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge”.