| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
|
|
|
|||||||
|
|||||||||
Correspondence: Address correspondence to Jennifer Craft Morgan, CB#1030, 720 MLK Jr. Boulevard, Chapel Hill, NC 27599-1030. E-mail: craft{at}email.unc.edu
 |
Abstract |
|---|
 TOP Abstract Background: The Direct Care...The WIN A STEP...MethodsResultsDiscussionReferences |
|---|
Key Words: Direct care workers • Turnover • Nursing homes • Workforce development • Supervision • Retention
|
Background: The Direct Care Workforce in Crisis |
|---|
|
TOPAbstractBackground: The Direct Care...The WIN A STEP...MethodsResultsDiscussionReferences |
|---|
The problems of a vulnerable workforce, low job quality for DCWs, and instability in care for residents of long-term care are pervasive. Potter, Churilla, and Smith (2006) showed that this workforce is particularly vulnerable. For example, 29% of all female workers in the United States are non-White, whereas 53% of the female direct care workforce in the United States is non-White. DCWs are less likely to be married than the average worker. Only 33% of the direct care workforce has had some college education, in contrast to the 62% of all workers who have at least some college education. Potter and colleagues also spoke to job quality and vulnerability. Fewer DCWs (41%) than other workers (53%) receive employer-based health insurance, and one third of women in the direct care workforce live in families whose income is at or below 150% of the poverty level (Potter et al., 2006). Instability in this workforce is also pervasive. According to a report prepared by Harmuth and Dyson (2004), more than 75% of states surveyed indicated that DCW recruitment and retention was a major workforce issue in their state. The discontinuity in staffing and low job quality is likely to have an impact on quality of care (Castle & Engberg, 2005; Noelker & Harel, 2000; Stone & Weiner, 2001).
Low pay, few benefits, and heavy workloads characterize the job of the DCW in North Carolina, the state where we conducted the present evaluation (Konrad, 2002). In 2004, active North Carolina Nursing Assistant I registrants had a median annual wage of only $14,912 and held an average of two jobs either consecutively or concurrently (Konrad, Morgan, & Ribas, 2006). In addition, 32% of NAs surveyed in North Carolina lack health insurance from any source, and 30% report often or routinely being physically exhausted at the end of a shift (Morgan, 2005). This job quality problem is compounded by disturbing rates of turnover, particularly in residential care settings where it is consistently has been over 100% per year for the past 3 years (Konrad, Morgan, & Dill, 2006). The socioeconomic vulnerability of the workforce adds to the complexity of the situation of the direct care workforce. In North Carolina, about half of DCWs are African American, 13% have less than a high school diploma, and, on average, these workers have 12.5 years of school (Morgan, 2005).
This article presents the results of an evaluation of an intervention designed to address some of the pervasive problems of the direct care workforce in long-term care, including education, compensation, commitment, and ultimately turnover and quality of care. The intervention, called WIN A STEP UP (Workforce Improvement for Nursing Assistants: Supporting Training Education and Payment for Upgrading Performance), focuses on NAs in skilled nursing facilities in North Carolina.
|
The WIN A STEP UP Program |
|---|
|
TOPAbstractBackground: The Direct Care...The WIN A STEP...MethodsResultsDiscussionReferences |
|---|
WIN A STEP UP began its operational phase in North Carolina's nursing homes with funding directed through the North Carolina Department of Health and Human Services from civil monetary penalties. Currently in its sixth year of operation, the program includes a 33-hr curriculum that focuses on clinical and interpersonal topics such as infection control, being part of a team, and dementia care. A core feature of the program is that it requires commitments from each NA, the nursing home management, and the WIN A STEP UP program staff.
Approximately 10 NAs per facility agree to attend the classes and remain employed at the facility for an agreed-upon amount of time. The facility agrees to commit staff time to completing the program and distribute a retention bonus or wage increase to NAs who complete the program. The program provides the curriculum, educational incentives to NAs ($70 per class), and a $75 retention bonus to NA participants completing the program.
Focus groups conducted during the pilot phase revealed pervasive problems regarding communication and perceived lack of teamwork between frontline nurses and NAs. This is consistent with the literature (Brannon, Zinn, Mor, & Davis, 2002; Cohen-Mansfield, 1995; Eaton, 2001; Feldman, 1994; Garland, Oyabu, & Gipson, 1988; Noelker, Ejaz, Menne, & Jones, 2006). In order to improve this situation for both nurses and NAs, the implementation team incorporated a program developed to improve the management and communication skills of frontline nurses. The Paraprofessional Healthcare Institute developed this program, called Coaching Supervision, to teach supervisors how to encourage and enable problem solving among their staff (Paraprofessional Healthcare Institute, 2005). It uses a curriculum that consists of seven modules designed to improve active listening, self-management, self-awareness, and self-presentation. The curriculum was incorporated into WIN A STEP UP by certifying local WIN A STEP UP team members as trainers through Paraprofessional Healthcare Institute and asking participating facilities to enroll 6 to 10 frontline supervisors to participate in the 2-day training onsite or nearby prior to NA participation.
|
Methods |
|---|
|
TOPAbstractBackground: The Direct Care...The WIN A STEP...MethodsResultsDiscussionReferences |
|---|
Data Collection
A mixed-method matched control evaluation design was built on the inclusion of eight participating sites that were recruited using the WIN A STEP UP waiting list and study team assessments of sites' ability to adhere to the tight timeframe required by the evaluation design. The study team attempted to recruit two comparison sites per participating site to guard against comparison site drop out. The study team hand-matched comparison sites to these eight sites based on organizational size, labor market characteristics, and management style. In all, 8 sites successfully completed the intervention and 10 sites completed the comparison site obligations. This process yielded six groups of sites that were roughly comparable based on size, region, and labor market characteristics (see Table 1) and that we used as the basis of the individual NA-level participant–control matching described in "Analytic Strategy." One of eight participating facilities and 2 of 10 comparison sites were nonprofit facilities.
|
After the completion of the program, informants at participating nursing homes received a brief follow-up paper-and-pencil survey. Facility staff completed all eight surveys during the 3-month follow-up site visit. Two paper-and-pencil surveys were given to NAs at program and comparison facilities at baseline (prior to intervention) and at 3-month follow-up (after completion of WIN A STEP UP training). Participating sites had a mean response rate of 97% (n = 238) at baseline and 81% at follow-up (n = 192). Comparison sites had a mean response rate of 93% (n = 224) at baseline and 75% at follow-up (n = 198). Supervisors of NAs completed the NA performance measures survey, in which they individually evaluated NAs on clinical and interpersonal skills at baseline and at 3-month follow-up. At the eight program sites, baseline and follow-up performance measures were available on all participating NAs completing the program (N = 77) from 68 distinct supervisors. At each of the 10 comparison sites, performance measures were requested from supervisors on a randomly drawn sample of 10 NAs, yielding baseline and follow-up data for 85% (n = 85) of individuals targeted. Finally, Coaching Supervision participants were anonymously surveyed (no survey identifier to link to respondent name) after completing training in classes with 4 to 10 participants. Response rates at facilities ranged from 50% to 100%, with a mean of 74% (N = 33). For the two sites with less than 75% response rates (3/6 and 3/5), we were unable to follow up with respondents because of the anonymity of the survey.
Analytic Strategy
Matched Design for Self-Assessed Outcomes and Turnover
In order to assess the impact of the program on participants, we compared program participants to controls. In the case of the two NA surveys, we did this by matching control NAs to each participant. Matching is important here because NA participants in the participating facilities were self-selected or manager selected (and therefore not randomly assigned). Matching limited the comparison group to NAs that were similar to the selected group in their own facility and in comparison facilities. Hence, we matched two types of individual controls to participating NAs: (a) control NAs who worked within the same facility as the participating NA (i.e., participating site controls) and (b) controls who worked in a comparison site within the matched group of sites displayed in Table 1 (i.e., comparison site controls). We selected both types of controls using propensity score matching. A propensity score is a device used to construct matched pairs that balance observed covariates. This score is simply the conditional probability of selection into a treatment group given these observed covariates (Joffe & Rosenbaum, 1999). Removing unmatched cases from the potential sample is likely beneficial even with the resultant statistical inefficiency. By using matched cases, we limited the sample to the NAs who were similar to those in the treatment group; this was thus a conservative way to estimate whether there was indeed an intervention effect. In Smith's (1997) words, "Restricting estimation to samples in which, for fixed x, there are both treatments and controls has the salutary effect of delimiting the range of our inferences, and specifying the conditions under which manipulation of the treatment will plausibly result in the estimated effect" (p. 334).
Individual NA-level independent variables included in the development of the propensity scores included work-related variables (tenure in the organization, tenure as a caregiver, shift worked, daily tasks, prior work experience in health care, current hourly wage, benefits, hours worked per week, concurrent work in a second paid job) and demographic characteristics (education, family income, race/ethnicity, gender, marital status, and parenthood). We chose individuals from the same facility and those from the same facility group as controls if they had a propensity score within 0.5 standard deviation of the participant. Through this process, we identified 81 participating site controls and 135 comparison site controls and included them in the analysis data set.
Supervisor Ratings of NAs
Resource constraints prevented the collection of pre- and postintervention supervisory ratings on all potential control NAs at the study organizations. Instead, we randomly selected all participants and 10 comparison NAs from each participating and each comparison organization for pre- and postintervention ratings. In the case of performance measures, we compared supervisory assessments of participants directly to those of nonmatched randomly selected controls.
Dealing With Missing Data
When item values were missing, we used a multiple imputation method to take advantage of the full information provided by the data set. We used information from each respondent to predict his or her missing values with random error added to ensure the appropriate amount of variability in the imputed data (Schafer & Graham, 2002). This strategy, which assumes missing data are missing at random, is robust to the use of ordered categorical indicators rounded to the nearest integer values after imputation (assuming an underlying continuous multivariate normal latent construct; Rubin, 1987; Schafer & Graham, 2002). We assessed changes between pre and post self-assessed measures and performance measures across participants and both groups of controls by using zero-order bivariate regression models using the MIANALYZE procedure in SAS 9.1. Rubin's multiple imputation strategy replaces each missing value with a set of plausible values that represent the uncertainty about the right value to impute. The imputed data sets are created and then analyzed using MIANALYZE, which, in this case, conducted bivariate ordinary least squares regression and combined the results of analyses across the imputed data sets. In this way, the procedure results in a t statistic that can be interpreted similarly to a t test but allows the researcher to utilize a full-information data set. This procedure results in valid statistical inferences that properly reflect the uncertainty due to missing values (Yuan, 2000). Given the small number of cases involved and the conservative statistical assumptions, we used a p value of.10 as the threshold for statistical significance.
Using Qualitative Data
Three coders coded interview transcripts with the aid of a qualitative software package (NVivo 4.0) after a codebook had been collaboratively developed during a first-pass reading. The goal of the content analysis of this study was to understand (a) how individual sites varied in their implementation of the program, (b) what events happened between baseline and follow-up that may have influenced the intervention and were not captured by survey data, and (c) what was the perceived impact of the intervention on the facility and staff from the perspective of managerial informants (at program sites). For this final goal, the three coders identified and coded broad themes in responses.
Measurement and Hypotheses
In this section we describe the measurement of the principal outcome variables used in the study, provide psychometric characteristics of scales as appropriate, and state our hypotheses as related to each of outcome measure.
Self-Assessed Outcomes
We developed or acquired outcome measures both for perceived quality of care and for job quality. Four subscales were developed in the domain of self-assessed quality of care. These items were developed using NA focus groups in which participants were asked to talk about and define good care. Exploratory factor analysis was employed to develop scales using data from the entire sample of NAs who completed the tool (N = 529). The four subscales included team care, workload, interpersonal care, and quality of coworkers. Table 2 provides sample items and alphas.
|
Turnover
For purposes of comparison, we assessed turnover as the percentage of NAs in each group (participant or control) who left between data collection points. If NAs did not participate in a 3-month follow-up data collection, we consulted human resource informants at the site to determine whether those individuals who had participated in data collection at baseline were still employed. At 6 months, we ascertained employment status only from organizational informants such as trainers or human resource/payroll informants. We matched data collection timing within the six facility groups to minimize potential confounding from seasonal differences in turnover and regional changes in unemployment rates.
Job Performance
The study team developed a rigorous measure of NA job performance using supervisor ratings of NA performance on items describing activities that reflected the specific learning objectives and content of the WIN A STEP UP curriculum. Prior to the evaluation study presented here, a scale was developed using a content expert panel review and an exploratory factor analysis approach on an independent sample of 181 supervisors rating their own NA subordinates. Exploratory factor analysis applied to job performance data yielded four subscales: nursing care, supportive leadership, communication, and resident-focused care. Table 2 provides sample items and alphas on these scales, the latter of which range from.90 to.97.
Hypotheses
We proposed four hypotheses to be tested.
Hypothesis 1: Program participants will show significant improvement on self-assessed quality-of-care subscales such as team care and quality of coworkers. Through engagement in the interpersonal dimensions of the WIN A STEP UP curriculum and the enhancement of communication at the facility through Coaching Supervision training, program participants should compare favorably over the intervention period to comparison site controls.
Hypothesis 2: Program participants will not be significantly different from controls in terms of workload or interpersonal care subscales. As these outcomes were not directly targeted by the educational interventions, these measures will not likely be influenced by the WIN A STEP UP intervention.
Hypothesis 3: Program participants will terminate employment at a lower rate than participating site and comparison site controls. The combination of compensation, retention commitments, and continuing education should improve retention at intervention sites, particularly for program participants.
Hypothesis 4: Program participants will show improvement in performance in nursing care, supportive leadership, communication, and resident-focused care. These topics are covered across modules in the WIN A STEP UP curriculum, and this learning should be supported by interactions at work with supervisors engaging in Coaching Supervision training.
|
Results |
|---|
|
TOPAbstractBackground: The Direct Care...The WIN A STEP...MethodsResultsDiscussionReferences |
|---|
Program Outcomes
We group the results of this evaluation into six areas: (a) qualitative assessments of programmatic success, (b) impact on turnover, (c) impact on perceived quality of care and job quality, (d) impact on job performance of program participants, (e) impact of Coaching Supervision, and (f) the extent to which positive program impact diffused beyond program participants to other staff at participating sites (see Table 3). This section is followed by a summary of the results of tests of the four evaluation hypotheses.
|
Impact on Turnover
Turnover of managers, as well NAs, continues to be the greatest challenge affecting both program implementation and program evaluation. Most sites that pulled out of the program after making an initial commitment to participate did so because a key management person turned over. When we compared participating NAs (N = 77) to NA controls in the same facilities (N = 81) and to comparison site NA controls (N = 135) separately, we observed no statistically significant difference between the two groups in the number of NAs who remained employed from baseline to 3 months after the intervention. However, when we examined the qualitative data, the adverse circumstances of two participating sites stood out. The first site experienced heavy turnover after one participant displayed animosity toward the administrator and convinced about half of her fellow participants to quit their jobs. The second participating site experienced turnover of the WIN A STEP UP trainer not once, but twice. Two major disruptions in program implementation made this second participating facility unique.
When we held these two anomalous participating sites and their comparison sites from the analysis as outliers, we found a modest yet significant (p <.10) difference in separation rate between the percentage of participants (5%) and the percentage of comparison site controls (13%) leaving within 3 months of the completion of the program. In these analyses, we detected no significant difference in turnover between NA participants and participating site controls. Furthermore, we detected no significant difference in the percentage of leavers between participants and comparison site controls at 6 months after program completion.
Impact on Perceived Quality of Care and Job Quality
We compared changes between baseline and follow-up on all four subscales of perceived quality of care between participants and both sets of controls using multiple imputation methods and ordinary least squares regression. Team care improved more for participants (N = 77) than for comparison site controls (N = 135) between baseline and 3-month follow-up. We found no significant difference between participants and controls on the other three subscales (i.e., workload, interpersonal care, and quality of coworkers; see Table 4).
|
Impact on Job Performance of Program Participants
At baseline, we detected no significant differences between participants and controls on any of the four performance rating subscales, supporting the notion of comparability of NA skill sets and work styles at participating and control sites as well as suggesting that the design had minimized program participant selection bias. However, between baseline and the 3-month follow-up, participants showed significant improvement in both nursing care and supportive leadership scores compared to controls. We found no significant changes in communication or resident-focused care over the same time period (see Table 5).
|
I do think they give more of a thought before they jump when they need to deal with conflict or disciplinary action ... From the supervisory level, I think it has created the ability for them to grow into what a lot of us have learned from many years of practice, and that is, don't leap into things. Let's look at the whole picture. Let's get our facts together. Let's really evaluate how can we best respond and address an individual and get positive results versus negative.
Managers also reported that their staff worked more as a team because of the training, as indicated in following quotes from our debriefing with two particular nursing home administrators:
They just kind of pulled together; the nurses and the [certified NAs] pulled together. And, so that relationship was smoothed and a respect for each other made there.With the supervisors and the charge nurses, I think they're listening; they're trying to understand. There were always those [nurses] that helped out, those types of things. I think it certainly put more people in tune with things.
Nurse supervisor participants (N = 33) echoed similar positive sentiments when they were surveyed about 3 months after their participation in Coaching Supervision training. All participants reported that they had engaged in informal discussions about the principles of Coaching Supervision with other NA supervisors. Virtually all (i.e., more than 95%) reported that their training had been instrumental in altering their personal style of supervision, positively influenced they way they supervise NAs, and had a positive impact on other supervisors' style of supervision. Nearly 76% reported that communication between NA managers and NAs had improved as a result of the training (or informal discussions about it), whereas almost 70% believed that the quality of care had improved as a result of Coaching Supervision training (or informal discussions about it).
Diffusion of Program Outcomes
WIN A STEP UP seemed to have its greatest impact on program participants, although there was some evidence of diffusion to those NAs working alongside program participants. Thus, both participants and controls at participating nursing homes perceived greater career rewards after the program was implemented than did NAs at control nursing homes, suggesting that some program benefits may have spilled over to others. Furthermore, overall turnover rates were lower on average at participating sites than were the rates for controls in the 3-month window following program completion. During the 3 months after program completion, turnover for all NAs at comparison sites increased somewhat (+10%), whereas it declined slightly (–2%) at participating sites. However, this difference washed out over the next 3 months.
Results of Evaluation Hypotheses
Hypothesis 1 was partially supported
Program participants showed significant improvement on one of two self-assessed quality-of-care subscales: team care (where change was expected) and quality of coworkers.
Hypothesis 2 was supported
Program participants were not significantly different from controls in terms of workload or interpersonal care subscales.
Hypothesis 3 was partially supported
Although program participants did not terminate employment at a lower rate than participating site controls, program participants were slightly less likely than comparison site controls to leave in the 3 months following completion of the program.
Hypothesis 4 was partially supported
We found improved performance in two of the four dimensions of care: nursing care and supportive leadership. Performance was not improved in the other dimensions: communication and resident-focused care.
|
Discussion |
|---|
|
TOPAbstractBackground: The Direct Care...The WIN A STEP...MethodsResultsDiscussionReferences |
|---|
Given the enormity of the challenge of improving DCW retention, a program such as WIN A STEP UP may offer a step in the right direction with regard to DCW turnover. The logical next steps for addressing DCW retention challenges include building programs like WIN A STEP UP into continuing education and compensation systems for entire facilities, articulating the program with formal educational institutions (e.g., community colleges) to build meaningful credentials and effective career ladders for DCWs, and integrating similar programs into larger "culture change" and public policy initiatives.
Coaching Supervision is the first programmatic element to reach beyond NAs and solidify teamwork between nurses and NAs as they provide care to residents. Part of the improvement in perceived quality of care reported by supervisors—team care—may have resulted from the introduction of the Coaching Supervision component. Because all program facilities received both Coaching Supervision and the NA curriculum, disentangling the impact of the Coaching Supervision component from the NA educational program was impossible. However, what may be a programmatic strength presents a scientific challenge: Because the effects of the Coaching Supervision component probably diffuse beyond direct program participants, it is likely that the contrast between program participants and their coworker DCWs not receiving the instructional component of the intervention is attenuated.
The high cost of the WIN A STEP program is likely to be a barrier to its wide-scale implementation unless major supportive changes in funding and policy environments occur. Subsequent to this evaluation period, WIN A STEP UP cut NA stipend payments in half in an effort to reduce costs, yet the implementation team observed no evident changes in sign-on or dropout rates for facilities. Currently, employers bear an estimated one third to one half of the total program cost, largely by contributing instructor time and paying for participants' raises and bonuses. Subsequent cycles of evaluation of this program would be enhanced by a better understanding of how varied amounts of retention bonuses may affect various program outcomes. In addition, focused return-on-investment studies that show financial costs and benefits of such programs will be needed. Such studies will contribute to building a stronger business case for nursing homes to take on the additional costs of workforce improvement programs that will be required to stabilize NA staff, improve the workplace, and enhance and sustain quality long-term care.
Finally, two major limitations to this study are identified. Although managers, supervisors, and NAs at participating nursing homes all perceived improvements in quality, the fact that our evaluation did not directly measure resident outcomes constitutes an important limitation. Programs aimed at improving care by improving jobs will be strengthened by evidence that they affect those resident care outcomes that are most meaningful to consumers, families, and purchasers of care. Experimental studies involving such outcomes are needed. Secondly, because we were unable to randomize participants to employment settings or randomize nursing homes to the intervention group, we used propensity scores to control for selection bias.
|
Footnotes |
|---|
We would like to acknowledge the WIN A STEP UP implementation team and the evaluation study team, particularly Sara Haviland, Heather Kane, Ashley Stout, Cheryl Thompson, Kathryn Wessel, and Ally Woodside, who participated extensively in the data collection and/or compilation of data used in this article. Special thanks to Anne Jackman, who helped manage the project. We would also like to thank the participating and comparison nursing home managers, frontline supervisors, and the many direct care workers who took time out of their busy lives to participate in our data collection activities.
1 University of North Carolina Institute on Aging, Chapel Hill.
2 Cecil G. Sheps Center for Health Services Research, University of North Carolina Institute on Aging, Chapel Hill.
|
References |
|---|
|
TOPAbstractBackground: The Direct Care...The WIN A STEP...MethodsResultsDiscussionReferences |
|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
| All GSA journals | Journals of Gerontology Series A: Biological Sciences and Medical Sciences | Journals of Gerontology Series B: Psychological Sciences and Social Sciences |
