| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
|
|
|
|||||||
|
|||||||||
Correspondence: Address correspondence to Nicholas G. Castle, PhD, Health Policy and Management, University of Pittsburgh, A649 Crabtree Hall, 130 DeSoto Street, Pittsburgh, PA 15261. E-mail: castlen{at}pitt.edu
 |
Abstract |
|---|
 TOP Abstract Nursing Home CompareMethodsResultsDiscussionReferences |
|---|
Key Words: Report cards • Nursing Home Compare • Quality
Report cards derive their popularity from their utility in several areas. First, they can assist health care purchasers in evaluating the value they are receiving from their expenditures (Scanlon et al., 1998). As Mukamel and Mushlin described, "public dissemination of accurate information about quality restores the imbalance between quality and costs in the ‘value’ equation" (2001, p. 21). Second, the collecting and reporting of information regarding quality may encourage providers to improve their performance. As Davies, Washington, and Bindman reported, "very public judgments about the quality of services affect the reputation of organizations, and organizations so affected might be expected to respond" (2002, p. 382). Third, report cards may educate health care purchasers about the care provided (Kroll Letwat, 1999), by providing information on quality indicators, for example.
Despite these potential benefits of report cards, they also have their critics. The public may not actually use the information they contain (Marshall, 2001). Robinson and Brodie (1997), for example, found that only 34% of consumers with report-card information actually used it. The data used by report cards may not be timely, and may not report quality measures that providers can improve in any significant way (Thomas, 1998). In addition, if report cards are successful, Mukamel and Mushlin (2001) noted that high-quality providers may not be able to respond to any resulting increase in demand for their services.
With these advocates and detractors as a backdrop, in November of 2002 the CMS publicly reported on the quality of nursing homes on a national basis on the Nursing Home Compare (NHC) Web site (www.Medicare.gov/NHCompare/home.asp). This was significant. Although report cards have been used for a considerable period of time (as cited in Marshall, 2001, Florence Nightingale is credited as producing one of the earliest report cards in 1863, describing mortality rates in London hospitals), large-scale report-card initiatives are relatively recent developments and generally have not diffused to long-term-care settings. Large-scale report-card initiatives had generally been used for hospitals and managed care providers. The NHC report card now "expanded the availability of public information on nursing homes and the quality of care provided" (General Accounting Office [GAO], 2002, p. 1). However, as with most report cards preceding NHC, questions remain regarding the use and usefulness of the information collected and reported.
In this investigation, I used a questionnaire to examine administrators' opinions of the NHC initiative and to determine whether the NHC Web site has fostered or helped with quality-improvement initiatives. Administrators are in contact almost daily with consumers; their opinions are important, as they can give some indication of the use and usefulness of the NHC website from the perspectives of both the facility and the consumer. Whether NHC information has fostered or helped with quality improvement is a particularly important question. For the NHC Web site to influence quality, nursing home quality-of-care initiatives are essential.
A prior literature review of report cards identified quality improvement by providers to be a likely outcome of disclosure of report-card information (Marshall, Shekelle, Leatherman, & Brook, 2000). Indeed, the New York Cardiac Surgery Reporting System influenced provider behavior (Hannan et al., 1997), as did the obstetrics consumer report developed by the Missouri Department of Health (Longo et al., 1997).
The NHC Web site is intended to provide assistance to consumers in choosing nursing homes and to improve the quality of care of nursing homes (GAO, 2002). Because this information could affect each nursing home's market share of incoming residents, one would expect most administrators to examine this report card. Administrators also may be sensitive to the public image of their facility (Marshall et al., 2000) or to peers' opinions (Longo et al., 1997). Professional norms also can create pressure for changes in quality. A recent study by Frayne and Geringer (2000) supports the notion that professional norms can influence outcomes. In turn, administrators are likely to have an opinion on the validity of the information presented, and some may have developed quality-of-care initiatives as a result of the information contained in the NHC Web site.
Thus, on the basis of (a) prior literature showing quality improvement by providers to be a likely outcome of disclosure of report-card information; (b) the influence of market forces; (c) public image; and (d) professional norms, I hypothesize that public dissemination of nursing home quality through the NHC report card has motivated administrators to improve quality of care.
|
Nursing Home Compare |
|---|
|
TOPAbstractNursing Home CompareMethodsResultsDiscussionReferences |
|---|
It is worth noting that the quality measures used in the NHC report card are not all the more familiar quality indicators used by surveyors during the survey and certification process. These quality indicators were developed by the Center for Health Systems Research and Analysis (CHSRA) at the University of Wisconsin (Zimmerman, 2003); they include, for example, use of nine or more different medications, prevalence of bedfast residents, and incidence of new fractures. They were developed for use by surveyors as indicators of quality problems, and not for public reporting. The CMS wanted to use quality measures in the NHC report card that were reliably associated with quality, and of course could be publicly reported. In short, some quality measures may be quality indicators, but not all quality indicators are quality measures (Zimmerman). Therefore, under a contract from the CMS, Abt examined the quality indicators and other quality measures specifically for use in the NHC report card. This development is described in a GAO report (GAO, 2002). The Abt research was intended to identify quality measures most reflective of nursing home quality. This included risk adjustment, to take into account resident and facility characteristics.
The CMS selected 10 measures for use in the NHC report card—these are called the core quality measures. Loss of ability in basic daily tasks, pressure ulcers, physical restraint use, and infections were used for long-stay residents with no risk adjustment, along with pain with resident-level risk adjustment, and pressure ulcers with facility-level risk adjustment. Delirium and pain were used for short-stay residents with no risk adjustment, along with delirium with resident-level risk adjustment, and walking with facility-level risk adjustment. The walking quality measure indicates the percentage of residents whose independence in walking has been maintained or improved. The assessments used to calculate this measure are those conducted between Days 5 and 14 for short-stay residents.
Resident-level risk adjustment is not unusual in the creation of report-card information. This controls for confounding effects that might be attributed to differences in resident populations (Mor et al., 2003). Facility-level risk adjustment is more uncommon than resident-level risk adjustment. Rather than account for individual resident characteristics alone, this approach accounts for the fact that, in the aggregate, some facilities may admit more impaired residents than other facilities. This facility admission profile (FAP) was used to risk-adjust quality measures that are considered more difficult for a facility to change, or in which one could reasonably anticipate some reoccurrence of the condition (e.g., pressure ulcers). Mor and associates recently outlined the rationale behind this approach to risk adjustment.
The CMS initiative in early 2002 was intended as a pilot program, and NHC information was included only for six states: Colorado, Florida, Maryland, Ohio, Rhode Island, and Washington. These pilot activities were expanded into a national initiative in November 2002, including all 50 states. In addition, as part of this initiative, Medicare Quality Improvement Organizations (QIOs) also were supposed to work with nursing homes to improve their quality and help consumers understand and use the NHC report card. Kissam and colleagues (2003) provide some information on how QIOs are responding to this initiative.
Finally, in January 2004, the CMS further changed the core quality measures to include a total of 14 indicators. For long-stay residents this included the percentage with an increased need for help with daily activities; the number of high-risk and low-risk residents with pressure ulcers; physical restraint use; the number of low-risk residents with loss of bladder or bowel control; the number of residents that had a catheter inserted and left in the bladder, spent most time in bed or in a chair, and had a urinary tract infection, with no risk adjustment; and percentage with moderate to severe pain, who were more depressed or anxious, and whose ability to move in or around the room got worse, with resident-level risk adjustment. For short-stay residents this included the percentage with delirium, with moderate to severe pain, and with pressure ulcers, all with resident level risk-adjustment. Thus, the quality measures were expanded, and the FAP, which was a little controversial (GAO, 2002), was no longer used.
|
Methods |
|---|
|
TOPAbstractNursing Home CompareMethodsResultsDiscussionReferences |
|---|
The sample was divided in this way because I was interested in examining whether prior experience with Web site report cards influenced administrators' responses to the NHC report card. Previous research has shown that prior exposure to report cards may influence how they are used and their usefulness; however, the literature has mixed results as to whether prior exposure increases users' receptivity toward report cards, or decreases receptivity. Results from the study by Mukamel and Mushlin (1998) show that facilities are likely to react to the first report card, whereas results from other analyses show that one report card may be a catalyst to the use of the next (Longo et al., 1997).
For the questionnaire mailing to administrators, I stratified facilities by state and then chose a random sample of approximately 30% of facilities from each state's pool of eligible facilities. I defined eligible facilities simply as those facilities participating in Medicare or Medicaid certification. I used this eligibility definition because these are the facilities included in the NHC report card. Eligible facilities included 252 nursing homes from CT, 749 from PA, 246 from MD, and 339 from TN. I sent the questionnaires to administrators in January of 2003.
I used some facility characteristics in the analyses (e.g., bed size, ownership, chain membership, Medicaid occupancy, and overall resident census), which I obtained from the 2003 Online Survey, Certification, and Reporting (OSCAR) data. These data come from the annual survey inspection in which all Medicare- or Medicaid-certified nursing homes must partake. The facility characteristics are considered reliable, and the data have been extensively described by others (e.g., Hughes, Lapane, & Mor, 2000).
Questionnaire
To my knowledge, no previous questionnaire has examined administrators' experiences with the NHC or any other nursing home report card. However, numerous publications have examined health-plan report cards. I examined this literature for guidance in developing my questionnaire. From this literature search, I found that Harris-Kojetin, McCormack, Jael, Sangl, and Garfinkel (2001) determined from interviews of consumers that report-card issues often fall into four main areas: content, comprehension, navigation, and decision processes. I used these issues in developing the questionnaire. Content refers to whether the report-card materials are relevant and complete. Comprehension questions examine whether the information contained in report cards is understood. Navigation examines how easily the report-card information is found. Decision process questions examine whether the report-card materials are helpful in making health care choices. In general, my questions first sought the administrators' opinions, and then asked their opinions from the perspective of consumers. Most questions used a 1-to-10 visual-analog rating scale, with 1 anchored by the least positive rating (e.g., not at all useful) and 10 anchored by the most positive rating (e.g., extremely useful; see Castle & Engberg, 2004).
I pilot tested the 45 items in the questionnaire with 10 administrators. This included mailing the questionnaire to the administrators and following up with phone calls. These calls lasted between 15 and 35 minutes. This resulted in minor changes to the questions and minor wording revisions to the instructions given at the beginning of the questionnaire.
Analyses
Descriptive analyses are presented consisting of the percent or mean for each of the questionnaire items. I calculated the average values for each of the two groups of states, and I used t tests to compare the significance of the difference in values between the groups. The total sample percent or mean also is presented, along with bar charts showing the distribution of scores. The score distributions show the percentage of responses for each response category, which is useful in determining whether the means are centered on a normal distribution or whether the data were skewed in some way. In addition, I used bivariate comparisons for respondent and nonrespondent facilities by using the OSCAR data.
For those nursing homes using the NHC information, I included on the questionnaire two open-ended questions that asked (a) how the information was used by the facility and (b) for any additional comments. I also examined these qualitative data and areas where NHC information was used are presented. In addition, areas where other comments were made are presented.
|
Results |
|---|
|
TOPAbstractNursing Home CompareMethodsResultsDiscussionReferences |
|---|
Most (81%) of the questionnaires were returned by mail within 1 month. In addition, because I was able to link facilities with OSCAR data, I determined that no significant differences on facility characteristics (i.e., bed size, ownership, chain membership, Medicaid census, staffing levels, and deficiencies) existed for respondents compared with nonrespondents.
Table 1 presents descriptive statistics of administrators and their use of the NHC report card, along with facility characteristics. Of most significance, I found a majority (90%) of administrators to have examined the NHC Web site. In general, few differences among the state samples were observed, but the gender and education level of administrators were different (p <.05). This may reflect employment conditions in these states.
|
|
|
|
Information was provided by 73 administrators (22%) to the open-ended question asking whether quality improvements were made as a result of the NHC information. Most commonly, administrators gave general or nonspecific comments (42% of all comments) indicating that the information was available, used, or passed on to the correct person for quality improvement. A number of administrators (17% of all comments) indicated that the information was used in specific ongoing quality initiatives (e.g., restraint reduction). Other responses included using the information to initiate restraint reduction, pressure ulcer prevention, and pain reduction (7%, 9%, and 12% of all comments, respectively).
|
Discussion |
|---|
|
TOPAbstractNursing Home CompareMethodsResultsDiscussionReferences |
|---|
In general, the results would seem to support my hypothesis. I found 33% of facilities to be using NHC information, and a further 51% state that they would be using the information for quality improvement in the future. Given that the NHC information was released only 3 months prior to data collection, this would seem to be a large number of facilities. However, clearly interpreting my findings as positive and supporting my hypothesis depends somewhat on whether one considers 33% of facilities to be using NHC information a high percentage or not. I found this number to be surprisingly large given the short time frame since the release of the NHC information, and the fact that administrators also reported that they believed few consumers were using the NHC Web site. However, that is not to say that administrators were not critical of the NHC information.
Administrators were critical of, if not a little perplexed by, the risk adjustment used in the NHC information. Most issues that administrators had with risk adjustment were simply related to understanding the techniques used. This understanding and potential misunderstanding of risk adjustment may be important. Previous research has shown that clarity is important in report cards. As Tumilson and associates (1997) found, clarity may facilitate getting consumers to use report cards. The same also may be true for nursing home administrators. It should be noted, however, that a gold standard to risk-adjust nursing home outcomes does not exist (Mor et al., 2003).
Administrators also were critical about using deficiency-citation information. They did not believe that deficiency citations well represented the quality of their own facility, and in general they believed that other facilities may be able to "game" the inspection process. This information came from an open-ended question and may not be representative of all administrators. Still, this belief is in line with recent GAO (1999) conclusions regarding the survey and certification system.
Regarding the specific MDS quality measures, administrators were asked which measures they believed were most and least credible. The pressure ulcer measure was identified as the least credible measure. This is congruent with the belief that pressure ulcers can develop during relatively short stays in other facilities. Nursing homes that hospitalize residents, for example, may experience higher rates of pressure ulcers and be unfairly classified as providing poor-quality care. The high degree of concern with pressure ulcers as a quality measure is important. As Mennemeyer, Morrisey, and Howard described, "either the providers will lose credibility, or the quality measures will" (1997, p. 126). It certainly is the case that, from administrators' perspective, the number of residents with pressure ulcers is not a credible measure.
It is interesting that administrators cite hot-button issues such as certification and pressure ulcers as important, yet, overall, they rate the NHC information as useful. It also is interesting that administrators appeared to rate the NHC information as more useful for themselves than for elders. In one sense this is rational, as some elders and family members tend to have low levels of education, limited reading skills, limited access to the Internet, and difficulty seeing (Goldstein & Fyock, 2001). In another sense this in not rational, because this is certainly not the case for all elders or family members and it is a little surprising that administrators would have such a dim view of the capabilities of consumers.
This view that the information is more useful for themselves than elders also is contrary to the belief that the NHC information will help consumers. Therefore, I speculate that administrators appear to be responding to the NHC information more because of a professional responsibility than because they fear potential consumers will use the information for admission decisions.
Facility characteristics (listed in Table 1) and administrator characteristics, and items from the content, comprehension, navigation, and decision processes sections of the questionnaire, were also used in multivariate analyses (not shown) examining the association of administrators' opinions of the usefulness of NHC information. The results were highly similar to those presented in the descriptive analyses. That is, relevance, completeness, ease of understanding, amount understood, ease of exploration, information interpretation, and confidence in choosing facility all had a significant association with whether administrators thought the NHC Web site was useful. Some of the results for the facility characteristics also warrant mentioning. Administrators in facilities with larger bed size, for-profit ownership, chain membership, and high Medicaid occupancy were associated with a lower likelihood of finding the NHC Web site extremely useful.
Limitations
I used MD and PA as states using report cards prior to the NHC report card. One limitation of this approach is that the MD and PA report cards are quite different. For example, the MD report card provides more information than the PA report card. This could influence administrators' opinions of the usefulness of report cards. We found that administrators in MD rated the usefulness of the NHC report card lower than did administrators from other states (analyses not shown). Alternatively, the information found on the state report cards may be seen as better (or worse) by administrators than that found in the NHC report card. The length of time the state report cards have been available also could influence administrators' opinions. For example, in states where report cards have been available for several years and consumers have (or have not) been influenced by the information, administrators' opinions of report cards in general could be influenced. In retrospect, a series of questions asking about administrators' prior experience and use of report cards may be a better approach to examining whether prior experience with state website report cards influenced administrators' response to the NHC report card in any way.
My results represent the opinions of administrators. These administrators may not be good proxies for opinions of residents and family members. In addition, the NHC web site is intended to be used primarily by consumers. As Harrington, O'Meara, Kitchener, Simon, and Schnelle described, "ultimately, report cards cannot serve the interests of all users equally" (2003, p. 48).
A further limitation of using administrators as respondents is the disparity between what administrators report on our questionnaire regarding report cards and whether they actually use the information. An observational examination of whether and how nursing homes use NHC information is still needed as a follow-up to this study. A related issue is that I did not define what I meant by quality initiatives in the questionnaire. This was left to the discretion of the administrator. Thus, the results I present for use of NHC information in quality initiatives are likely to vary from widespread to minimal quality-improvement activities.
The visual-analog rating scale used on the questionnaire was well received by administrators during pilot testing. Nevertheless, only two points on these scales were labeled at the highest and lowest values. Thus, some caution is required in interpreting scores between the poles. A score of 8 is more positive than a score of 7, for example, but further interpretation of either of these scores as representing a "moderate" rating cannot truly be made.
The questionnaire was administered during the first 3 months of the release of the NHC information. Administrators may have visited the website more frequently because of its novelty or newness, so the information reported for the number of visits to the NHC Web site may be biased. Of course, the alternative also is possible. That is, over time, the NHC report card will influence a greater number of administrators, and we will see more use of this Web site. The questionnaire also may have prompted administrators to visit the NHC Web site. I have no way of knowing whether this was the case. An examination of administrators' opinions of the NHC Web site now that it has been in operation for more than 1 year would probably be a useful extension to this work, and it should include an evaluation of whether and how NHC information has become institutionalized in the daily operations of the facility.
Conclusions
We know very little about nursing home report cards and the impact they are having on consumers and providers. With the exception of a recent GAO report (2002) and four recent articles (Castle & Lowe, 2005; Harrington, Collier, & O'Meara, 2003; Harrington, O'Meara, et al., 2003; Mukamel & Spector, 2003), I could not identify any research on this topic. Furthermore, these prior works are descriptive; I could not find any empirical research on this topic. So, despite the unquestionable need for report cards on nursing home care, some research in this area is clearly warranted. In this research, I focus on the impact that the NHC report card has had on administrators. My results show that administrators believe the NHC information is more useful for themselves than for potential consumers, and it is equivocal as to whether the information has stimulated much in the way of quality-improvement initiatives. Many other research topics readily come to mind. These include whether report-card information influences consumer cost–quality trade-offs, if the information is used by family members or elders, if market conditions influence consumer and provider reactions to NHC information, and if some groups of consumers find the NHC information more (or less) useful. In my research I have identified nursing homes to be using NHC information; the most significant future area of research I can think of is whether this improves quality of care—a badly needed outcome for our nation's nursing homes.
|
Footnotes |
|---|
Decision Editor: Linda S. Noelker, PhD
Received for publication January 29, 2004. Accepted for publication November 2, 2004.
|
References |
|---|
|
TOPAbstractNursing Home CompareMethodsResultsDiscussionReferences |
|---|
This article has been cited by other articles:
|
|
 |
|
  N. G. Castle Nursing Home Caregiver Staffing Levels and Quality of Care: A Literature Review Journal of Applied Gerontology, August 1, 2008; 27(4): 375 - 405. [Abstract] [PDF]
|
|
|
|
 |
|
 N. G. Castle, J. Engberg, and Aiju Men Nurse Aide Agency Staffing and Quality of Care in Nursing Homes Med Care Res Rev, April 1, 2008; 65(2): 232 - 252. [Abstract] [PDF]
|
|
|
|
 |
|
 D. B. Mukamel, W. D. Spector, J. S. Zinn, L. Huang, D. L. Weimer, and A. Dozier Nursing Homes' Response to the Nursing Home Compare Report Card J. Gerontol. B. Psychol. Sci. Soc. Sci., July 1, 2007; 62(4): S218 - S225. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 N. G Castle, J. Engberg, and D. Liu Have Nursing Home Compare quality measure scores changed over time in response to competition? Qual. Saf. Health Care, June 1, 2007; 16(3): 185 - 191. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. K. Inouye and L. Ferrucci Introduction: Elucidating the Pathophysiology of Delirium and the Interrelationship of Delirium and Dementia J. Gerontol. A Biol. Sci. Med. Sci., December 1, 2006; 61(12): 1277 - 1280. [Full Text] [PDF]
|
|
|
|
 |
|
 D. G. Stevenson Is a public reporting approach appropriate for nursing home care? Journal of Health Politics Policy and Law, August 1, 2006; 31(4): 773 - 810. [Abstract] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
| All GSA journals | Journals of Gerontology Series A: Biological Sciences and Medical Sciences | Journals of Gerontology Series B: Psychological Sciences and Social Sciences |
