| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
|
|
|
|||||||
|
|||||||||
Correspondence: Address correspondence to Charles Phillips, PhD, Texas A&M University Health Science Center, School of Rural Public Health, Department of Health Policy and Management, 1266 TAMU, College Station, TX 77843-1266. E-mail. Phillipsed@srph.tamhsc.edu
 |
Abstract |
|---|
 TOP Abstract MethodsResultsDiscussionReferences |
|---|
Key Words: Long-term care • Nursing homes • Pay for performance • Performance measurement • Quality indicators, Quality of care
Some of the most commonly used quality indicators in nursing home research focus on change in residents' ability to perform tasks concerning activities of daily living (ADL; Castle & Lowe, 2005; Harrington et al., 2003). Such a focus is not surprising, given that, when one is thinking about the needs of older individuals, functionality often plays a central role (Fillenbaum, 2006). In essence, it is the field of gerontology's approach to summarizing the "burden" of the multiple conditions and ailments so common among elders. The research using ADLs is voluminous, and change in ADL function is one of the quality indicators used in the Centers for Medicare and Medicaid Services Nursing Home Compare Web site, which rates the performance of all Medicare- or Medicaid-certified homes on a variety of quality indicators (Morris et al., 2002; Zimmerman et al., 1995).
Unfortunately, some recent research implies that where individuals receive nursing home care may have little effect on whether their ADL status in their first months of care declines or improves (Phillips, Chen, Shen, & Sherman, 2007). These results indicate that facility identity alone (represented as a set of over 1,300 dummy variables) explains only 8% of the variance in ADL change in residents' first 3 months in a nursing home. This figure increases to 10% when one looks at ADL change at the end of residents' first 6 months in a nursing home. This figure again increases to 14% when one focuses one's analysis on only those residents who declined or remained stable during their first 3 months (Phillips, Chen, et al., 2007).
Such results raise the question of how one can develop a meaningful performance-measurement indicator based on ADL function for nursing homes when so much of the variation in residents' functional status is driven by factors other than facility performance. Although we lack a gold standard for how much variance the care site should explain in one's outcomes, few of us, we think, would feel sanguine about basing a performance measure on an indicator in which 85% to 90% of the variance is beyond the home's or provider's control.
Some researchers may, however, be comfortable with such indicators. For example, a number of researchers seem relatively comfortable with quality indicators in which the overwhelming majority of the variance (80% to 90%) is explained either by individual characteristics or measurement error (Degenholtz, Kane, Kane, Bershadshy, & Kling, 2006; Kane et al., 2003; Morris et al., 2002). This position becomes troublesome to the degree that one accepts the idea that the best quality indicators are those heavily affected by provider performance.
However, the idea that care site, home, or provider makes no important contribution to functional outcomes and that over 90% of the variance in ADL change is beyond their control is troublesome. This finding seems contrary to our personal experience and, we suspect, that of our readers as well. Many of us have seen homes where residents walk in alertly, only to be glassy-eyed, slumped to one side, and pushed about in a wheelchair after a few short months. Many of us have also seen residents who were wheeled into homes and walk away unaided after the same few short months. Most of us know intuitively and anecdotally that the identity of the provider from whom a resident receives care must matter.
In this research, we investigate the validity of nursing home quality indicators or performance measures across all providers and all consumers. We seek to determine the circumstances when one can say that a provider's or home's performance does indeed make a difference in these indicators. To do this, we evaluate the impact of provider performance for residents in nursing homes at increasing extremes in terms of their unique positive or negative effect on ADL change. Our basic perspective or hypothesis is as follows: Only when one restricts analyses to homes at extreme ends of the quality continuum will the amount of variance in quality attributable to facility performance be substantial.
We also investigate which individual characteristics may affect the degree to which provider performance matters. Previous research has shown that the amount of variance in the direct-care time explained by site of care is positively correlated with a consumer's level of cognitive impairment (Phillips & Hawes, 1992, 2005). For example, in supportive housing, provider identity explained almost 60% of the variance in the frequency with which a resident received "cueing," a care strategy used with cognitively impaired residents, whereas individual characteristics explained slightly less than 20% of the variance in that measure (Phillips & Hawes, 2005). The same may be true of variance in ADL change.
Specifically, the research team investigated the impact of cognitive impairment, gender, and race on the importance of provider performance. As Mor and his colleagues have demonstrated (Mor, Zinn, Angelelli, Teno, & Miller, 2004), African Americans, whose occupancy of nursing home beds is increasing (Sahyoun, Pratt, Lentzner, Dey, & Robinson, 2001), often reside in poorer quality homes. In addition, men in nursing homes constitute a definite minority, and they may cluster in specific types of homes as well. Whether the homes in which they cluster might provide poorer or better quality is unclear at this point. Our basic hypothesis here is as follows: When analyses are restricted to minority groups or more homogeneous groups of residents, the amount of variance explained by facility performance will increase.
|
Methods |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
We used an admission cohort because it allowed the research team to develop measures of consumer baseline status that were free of any provider influence (Phillips, Hawes, et al. 2007; Phillips, Chen, et al., 2007). We looked at those individuals who were admitted to a nursing home and remained in that nursing home 3 months later and received an MDS quarterly assessment. We did not exclude individuals with an explicit terminal prognosis from our analyses. The MDS item that supposedly captures that information is of questionable usefulness (Finne-Soveri & Tilvis, 1998). This approach purposely excludes short-stay residents who were discharged or died prior to the scheduled quarterly assessment. The focus of this research is on outcomes of those residents likely to be long-stay residents.
Measurement
Items on the MDS have been tested for reliability (Hawes et al., 1995; Morris et al., 1990). Although it is often assumed that research data are far superior to administrative or clinical databases, this is not always the case. Evidence has shown that MDS ADL data developed in research studies have reliability similar to that found in administrative or clinical databases (Phillips & Morris, 1997). The greatest concern with these data is that, at times, MDS data do not agree with observational data gathered during specific time periods in nursing homes. However, this same research also indicates that MDS data can be used to differentiate among nursing homes providing either low- or high-quality care (Bates-Jensen et al., 2004; Codogan, Schnelle, Yamamoto-Mitani, Cabrere, & Simmon, 2004; Schnelle et al., 2004).
In addition, it is unclear how much agreement between observational and clinical data may be expected. There is always the concern that observations may be "reactive" and affect the behavior of those being observed. Some evidence indicates that care observations in nursing homes may not be reactive (Schnelle, Osterweil, & Simmons, 2005). However, there is also evidence that care observations for regulatory purposes may be reactive in residential care settings (Reid, Parsons, Green, & Shepsis, 1991).
Dependent Variable
Change in ADL function is the dependent variable in this research. This variable is the sum of scores for seven ADL indicators: bed mobility, transfer, locomotion, dressing, eating, toilet use, and personal hygiene. Each ADL had five potential response levels (0–4). This additive ADL scale, which ranges from 0 to 28, had a Cronbach's alpha of 0.91. Change in ADL function was the difference between the ADL scale score for each resident at admission and at his or her quarterly assessment. Decline is denoted by positive values, and improvement is indicated by negative values.
Independent Variables
We derived the individual-level covariates included in our analyses from each resident's intake MDS assessment. They included the ADL scale score just discussed, the MDS Cognitive Performance Scale (Morris et al., 1994; Hartmaier et al., 1995), a modified version of the MDS Changes in Health, End-Stage Disease, and Signs and Symptoms Scale (Hirdes, Fritjers, & Teare, 2003), the MDS Mortality Risk Index score (Flacker & Kiely, 2003), and the resident's age, gender, and living arrangement prior to entering the nursing home.
We measured provider performance by using a series of dummy variables representing nursing home identity. After we controlled for individual characteristics, these variables capture differences in resident outcomes related to the home in which each resident resided. It is important to note that this strategy does not tell one what aspect of a home's characteristics or behavior resulted in better or worse performance. Instead, our approach simply identifies homes in which the residents fared better or worse in their ADL function.
Analytic Strategy
We carried out all analyses at the individual level. For our different groups of residents, we estimated three models by means of ordinary least squares (OLS) methodology. We used individual characteristics at admission alone in the first model. A series of dummy variables representing home identity comprised the second model. We included both individual characteristics and the variables representing home identity in the third model (the combined model). In the combined model, the home variables represented the provider's impact on the outcome (performance), over and above the impact of individual characteristics. We compared the strength of these models on the basis of the explanatory power of each model (R2). In all analyses, the contribution of facility performance to explaining variance over and above individual characteristics is the difference between the R2 for the individual model and the combined model. We did not adjust the results for possible intrahome correlation or dependence because our focus is on the overall power of different models, not individual parameters within each model.
We estimated these models by using all homes and homes that our results indicated were in the top 20%, 10%, or 5% of homes versus the bottom 20%, 10%, or 5% of homes. We identified these homes at differing ends of the quality distribution by evaluating the relative size and direction of their individual parameters in our combined model. Facility parameters in the combined model indicated how well the home performed on the quality indicator after we adjusted for all the individual resident characteristics in the model.
In other analyses, we used only those homes in the top or bottom 5%, and we examined the degree to which resident characteristics affected the importance of homes' performance. As the analyses move to more and more heterogeneous resident populations (e.g., from all residents to only those in bottom and top 5% of homes), the R2 of the models will automatically increase. That progression is inherent in the way the bottom and top homes were chosen.
When we compare homes at the "extremes," we purposely choose groups of homes in which the variance in outcomes has a larger between-home variance component than the between-home variance component for outcomes among the entire population of homes. What is not inherent in this process is the levels reached by the R2 value. To investigate this issue, we performed a simulation with randomly generated data that mimicked the structure of our database. The value for the model using the top and bottom 5% of homes in this simulation was R2 =.12.
|
Results |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
|
|
Although the results just presented dealt with the impact of home performance on ADL change for the average resident, we also expected home performance to affect the outcomes of different types of residents differently. In Figure 2, we present results of our decomposition of the variance in ADL change for residents with differing levels of cognitive impairment. We ran our three basic models separately for residents' scoring at each of the seven levels of the MDS Cognitive Performance Scale.
|
Figure 3 replicates the information presented in Figure 2, but Figure 3 includes only those residents in the 5% worst or 5% best performing providers. The results were quite similar in form to our earlier results, but the levels of explained variance were much higher. Among the poorest and best performing homes, for the most cognitively impaired residents, the full model explained over 80% of the variance in outcomes. For these same residents, facility performance explained, in isolation, almost 70% of the variance in change in ADLs.
|
|
|
Discussion |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
Our research has a number of limitations. Our analyses deal with only one measure of change in ADL function. These results may not generalize to other nursing home quality indicators, including the specific indicators used on the Center for Medicare and Medicaid Services Nursing Home Compare Web site. In addition, we used data from only the first 3 months of the residence in a nursing home. Any cumulative effect of facility performance over a longer time period than 3 months was not captured in our analyses. In addition, our approach gives us no insight into what aspects of a home's operations or characteristics may have led to its position as a poor or excellent performer. Finally, our model may be poorly specified and some of the error variance may be attributable to unobserved factors that might affect our results.
Nonetheless, as we noted in our introduction, a good nursing home quality indicator largely varies only with differences in the quality of care provided by homes. However, the degree to which quality indicators are driven by variance in home performance or identity has not been a major research issue or a major concern. Only a few researchers in long-term care have given this issue the attention it deserves. One potential result of this attitude is that we may unknowingly have developed indicators for nursing homes in which the provider performance plays little role.
Our results concerning the importance of home performance imply that when one looks at homes that score the poorest or the best on our chosen outcome, then one sees a relatively strong effect of home performance. Beyond that, home performance has a rather minimal effect. This makes a provider's score on this indicator relatively meaningless to those trying to identify anything other than the best and worst homes, in terms of maintaining ADL function.
As we noted earlier, the results in Figures 2 and 4, indicating that the importance of facility performance varied by the resident's degree of cognitive impairment, were not wholly unexpected. Previous research on residential long-term care indicates that as individuals become more cognitively impaired, they became more dependent for the content of their care on provider policies (Phillips & Hawes, 1992, 2005). When a resident is too cognitively impaired to clearly express her or his needs, then general institutional procedures and practices, rather than individual needs, may design and dominate the care plan.
The implications of the findings concerning the impact of race and gender are less clear. The only clear conclusion is that facility performance is more important for the care of African Americans than it is for the care of other residents. This seems consistent with the finding by Mor and his colleagues (2004) that minority residents are most often found in "second-tier" homes with poorer staffing and outcomes. The more general implication of these results may be that if we allow the segregation of specific groups of residents into different types of homes, then the identity of the home and its performance may become more important determinants of these residents' outcomes.
Though this research focused only on ADL change, this research may indicate that researchers need to move toward analyses of outcomes for more homogeneous populations or look at only the best or worst providers, or both. In these instances, provider performance may explain enough variance in an outcome for one to consider using the indicator to help consumers choose among providers or for choosing which providers to reward in pay-for-performance reimbursement model.
As we noted herein, this research has significant limitations. Nevertheless, it may serve as a cautionary tale for those committed to the development of a rational system for rewarding provider performance and offering consumers meaningful information about providers. Depending on the results of future research, how much variance in a quality measure is attributable to provider performance, and how that level of determination varies across different types of consumers, may be matters of primary importance to those interested in the development of performance-measurement systems for nursing homes.
|
Footnotes |
|---|
2 Department of Statistics, Texas A&M University, College Station.
Decision Editor: William J. McAuley, PhD
Received for publication June 25, 2007. Accepted for publication January 15, 2008.
|
References |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
| All GSA journals | Journals of Gerontology Series A: Biological Sciences and Medical Sciences | Journals of Gerontology Series B: Psychological Sciences and Social Sciences |
