| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
|
|
|
|||||||
|
|||||||||
Correspondence: Address correspondence to Dr. Louise Lemieux-Charles, Department of Health Policy, Management and Evaluation, University of Toronto, McMurrich Building, 12 Queen's Park Crescent West, Toronto, ON M5S 1A8, Canada. E-mail: l.lemieux.charles{at}utoronto.ca
 |
Abstract |
|---|
 TOP Abstract MethodsResultsDiscussionReferences |
|---|
Key Words: Organizational performance • Care recipients • Caregivers • Networks
Purpose of This Study
Establishing networks of care-provider agencies is often proposed or recommended to reduce the fragmentation of care. Currently, however, there is a lack of empirical evidence revealing why some networks are effective whereas others are not, especially in cases in which effectiveness is linked to functional and clinical outcomes (Provan & Milward, 2001; Salancik, 1995).
We undertook the Dementia Care Networks' Study to explore how best to systematically evaluate the effectiveness of four community-based, not-for-profit dementia networks. Because there is a dearth of research on evaluating network effectiveness, this was an exploratory study. The formally established networks we evaluated were located in Ontario, Canada, and consisted of governmental and nongovernmental organizations involved in the care of people with dementia and their caregivers. The principal aim of these dementia care networks was to decrease the fragmentation of community-based services.
Our research examined the networks' structure, the nature of the exchanges within the networks, and the potential relationship of those exchanges to perceived network effectiveness. The type of administrative and service-delivery exchanges we selected for our study has been tied to service integration and coordination (Provan & Milward, 1995; Morrissey et al., 2002). To assess effectiveness, we adapted the framework of Provan and Milward (2001) for assessing the effectiveness of community-based networks of publicly funded health and human service organizations. In this article we address care providers' perceptions of their networks' effectiveness; we focus on the system required to manage and care for individuals with complex needs. We also examined care recipients' and their caregivers' perceptions of effectiveness of the care received within the network; those findings are reported elsewhere (Cockerill et al., in press).
Theoretical Perspectives
Several studies have examined different models of care for people with dementia (Johri, Beland, & Bergman, 2003; Kodner, 2002). In the 1980s, the U.S. National Long-Term Care Demonstration (known as Channeling) showed how case management could channel patients to the correct level of care. Aggregated findings also revealed the difficulty that case managers have in targeting community services to a group that would likely enter nursing homes (Kane & Kane, 1987). In the 1990s, the U.S. Medicare Alzheimer Disease Demonstration (MADD) targeted a group of Medicare recipients deemed to have irreversible dementia. "Conventional care" was compared with case management that linked caregivers with community services. Researchers found that informal care networks that provided care to demented beneficiaries were generally able to function effectively regardless of whether a case manager was involved.
Neither Channeling nor MADD, however, fully explored integration mechanisms. Johri and colleagues (2003), in a recent systematic review of seven post-Channeling demonstration projects that tested innovative models of care for elderly individuals in selected OECD (Organization for Economic Cooperation and Development) countries—the United Kingdom, Italy, the United States, and Canada—concluded that "to date the only reform initiatives that have been successfully implemented on a large scale are single-entry point systems with geriatric assessment and case management in publicly funded systems" (p. 234). The authors also noted that, despite positive results, none of the experimental models had been successfully generalized on a large scale. In these studies, the nature of the relationships and processes that exist at the administrative and service levels between the networked organizations is a black box in terms of where emphasis is placed on study evaluation.
In the health care system in general, results of empirical studies of service integration have also been inconclusive (Conrad & Shortell, 1996; Glisson & Hemmelgarn, 1998; Leatt, Pink, & Guerriere, 2000). Similar findings are noted in research in community mental health systems, where there is a limited evidence base for systems integration (Goldman, Morrissey, & Ridgely, 1994; Rosenheck et al., 1998). The systems are difficult to evaluate partly because the form that integration takes can vary (Hoge & Howenstine, 1997; Provan & Milward, 1995), as can the scope of the system being evaluated. Of particular note are results of the ACCESS Demonstration Program, a vast experimental program implemented in the United States between 1993 and 1997 to improve services for homeless people, which found that system-level integration was not achieved, although project-level integration was achieved at the local level (Morrissey et al. 2002).
Over the past 15 years, new analytic tools have been developed that allow one to measure processes such as coordination and continuity of care within community-based health care networks (Fried, Johnsen, Starrett, Calloway, & Morrissey, 1998), as well as the impact of organizational characteristics and agency type on integration and cooperation (Banaszak-Holl, Allen, Mor, & Schott, 1998). These tools have also been applied to the evaluation of the delivery of health and human services (Provan & Milward, 1995; Van de Ven & Ferry, 1980; Morrissey, 1992). Morrissey and colleagues (2002) noted that the network analysis of such services assumes that service integration and coordination can be inferred from interorganizational links and resource-exchange patterns (e.g., of referrals and information).
Network development is based "on the recognition that there is strong interdependence among the actors and organizations in a particular sphere of intervention. It depends on reciprocal exchange; complementary activities and functions among organizations and actors; the expertise, reputation, and trust that have been developed between partners; and information sharing aimed at meeting needs efficiently" (Fleury & Mercier, 2002, p. 60). In health care, service delivery is seen as an interorganizational process by which needed assistance is secured through a network of local organizations that offer clinical and supportive services (Morrissey et al., 2002).
Network analysis coupled with the theory of resource exchange provided the framework for our consideration of the relationships within the four dementia care networks. Network analysis captures the embedded nature of a network's organizational actors and structural elements. It focuses on patterns of communication and information flows without placing value on the nature of the exchanges. As a result, however, network analysis is sometimes criticized for being a nontheoretical approach to examining interorganizational ties. The theory of resource exchange, meanwhile, assumes that the ties between individuals or organizations consist of exchange relations of valued items and that what matters is the exchange value of the items (Oliver & Montgomery, 1996). When combined, network analysis and resource-exchange theory permit researchers to understand more fully the relationships that exist and the nature of these links.
Because networks are seen as a valuable way to promote integration but are much more complex than single organizations, linking their structures and processes to their "effectiveness" poses many challenges.
As Provan and Milward (2001) observed, "there is little agreement among organizational and public-policy scholars ... about how community-based networks of health and human-service organizations should be evaluated" (p. 415). They proposed a framework for evaluating network effectiveness criteria at three levels of analysis: community, network, and organization or participant. Provan and Milward further noted that "networks must be evaluated as service delivery vehicles that provide value to local communities in ways that could not have been achieved through the uncoordinated provision of services by fragmented and autonomous agencies" (p. 415). For our study, we adapted the framework of Provan and Milward to focus on (a) the networks themselves and (b) the networks' organizational participants, including care-provider agencies.
|
Methods |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
Network Descriptions
At the time of our study, there were the only four dementia care networks in Ontario that had formalized relationships through membership. These were public-sector networks in which the majority of members were not-for-profit agencies. Network membership ranged in size from 17 to 24 organizations. The majority of these organizations were agencies involved in providing services, although not every agency that provided services to people with dementia had chosen to participate in the networks we studied. All four dementia networks had evolved from either preexisting networks (e.g., Health Service Network for Older Adults in Hamilton) or working groups for older adults. They all wanted to identify and take action on gaps in service delivery, improve coordination of services, and share information by realigning some aspects of their services and contributing resources to joint initiatives. Toronto and Niagara regarded advocacy as a prime activity. Some specific activities included the development of a dementia registry (Toronto) and the creation of a resource guide and database of services (Hamilton). Ottawa had formalized some of its activities through the formation of subcommittees that focused on service delivery, education, and research.
Table 1 describes the type and number of individuals and organizations that belonged to the networks. The number of care-provider agencies in each network ranged from 13 to 17: Niagara and Ottawa each had 13, Toronto had 16, and Hamilton had 17. Local Community Care Access Centres (CCACs) were the central agencies in all four networks. In Ontario, a CCAC is a not-for-profit agency that provides case management in home health and social support services, as well as information, referral, and coordination services. CCACs also arrange for assessment and treatment, government-paid placement into facilities, personal support and homemaking, and daycare services. A multiplicity of other providers, most paid by nongovernmental sources (e.g., local chapters of the Alzheimer's Society), offer other health care and social support to clients and caregivers. Ottawa had a greater number of organizations that were not directly involved in service provision but had an interest in the planning of services for this client group. They also provided more specialized services including geriatric assessment and the allocation of specialized home-care case-management services. There was no single entry point into the network.
|
Resources allocated by each network ranged from minimal secretarial support to project-based funding for the study of elderly individuals. At the time our study ended, the networks continued to be voluntary without contractual obligations between network members, as contrasted with some publicly funded networks that are led, coordinated, and governed by a centralized local administrative authority (Provan & Milward, 2001). A more in-depth description of each network can be found in the technical report (available at http://www.utoronto.ca/hpme/faculty/lemieux-charles.htm).
Network Structure
Network analysis provided a framework for describing the system of relations among agencies related to nine particular types of administrative and service-delivery links and for locating patterns in these data (Morrissey, 1992; Wasserman & Faust, 1994). The strength of a network can be measured through the commitment of network members to one another as reflected in their engagement in multiple types of links and exchanges (Provan & Milward, 2001). With the use of the Measuring Inter-Organizational Relationships questionnaire developed by Provan, Sebastian, and Milward (1996), key informants (one clinician, one administrative person) from each agency were asked to indicate how their organizations were linked at the administrative and service-delivery levels with other groups in their respective networks.
We examined the extent to which agencies were linked within each network (density) and the degree to which some agencies were more central within their network than others (centrality). We considered a link to exist between two agencies within a network if they both reported that a relationship existed between them. Administrative links included resource transactions such as funding, shared staff or facilities, joint training programs, and shared administrative information (Banaszak-Holl et al., 1998; Bolland & Wilson, 1994). Clinical service links included referrals sent, referrals received, case coordination, joint clinical programs, service contracts, and shared client-related information (Milward, Provan, & Smith, 1994). Agencies were asked whether they engaged in each of these activities with other agencies in their network and, in the case of information sharing, referrals, and case coordination, they were asked the frequency of their engagement. We deemed a link to exist if referrals sent and received occurred on a monthly or weekly basis. We included other links (e.g., case coordination and sharing of client information) if they took place once a year or more.
Provan and Sebastian (1998) argued that network effectiveness may owe far less to integration across a network (density) than to ties among a few organizations that provide the bulk of relationships and services to clients. These ties are measured through cliques, which are groups of tightly connected members that form the microstructure of a given network boundary (Wasserman & Faust, 1994). Clique overlap, meanwhile, represents the degree to which network members share multiple ties with members of one or more cliques. Following Provan and Sebastian, we calculated clique overlap as the number of agencies in at least half the cliques of a particular administrative or service link expressed as a percentage of the total number of agencies in that clique type. Recently, Provan and Sebastian suggested that agencies that overlap according to service-relationship type (e.g., same agencies involved in cliques related to sharing information and referrals) represent stronger clique overlap relationships and these might be tied to outcomes. This has been labeled as multiplexity. Provan and Sebastian posited that, to be effective, clique integration "must be intensive, involving multiple and overlapping links both within and across the organizations that compose the core of the network" (p. 460).
Network Effectiveness
As noted earlier, we measured network effectiveness at two levels: (a) a network itself and (b) perspectives of a network's organizational participants (i.e., service-delivery agencies). Figure 1, an adaptation of the Provan–Milward (2001) framework, summarizes the perspectives and measurement criteria operating at each level.
|
Network members involved in direct service delivery were asked how the network to which they belonged facilitated the administrative and service-delivery relationships and activities that were assumed to increase care coordination and integration. Items were worded (see Figure 1) to reflect benefits from a network perspective (e.g., "facilitates case coordination"). We used a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree) to score responses. All service-delivery agencies completed the survey. There was a 100% response rate.
Members of network steering committees, which included service-delivery and non-service-delivery agencies, also were asked, on a 3-point scale (agree, neutral, disagree), to evaluate the extent to which their networks acted as overarching organizations to support network goals. Seven items (see Figure 2) measured the extent to which each network was perceived to be effective. Three of the four networks (i.e., Toronto, Ottawa, and Hamilton) responded to the survey with an overall response rate of 80%. Niagara's steering committee had not met in more than a year because of other regional priorities; the network was therefore reluctant to participate in the survey.
|
|
Results |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
Network Structure: Links Among Member Organizations
Table 2 summarizes the degree to which agencies in each network were connected (density) in relation to particular types of links and the degree to which the networks were dominated by one or two agencies (centrality). Of the nine possible types of administrative and service-delivery links, agencies in all four networks were most frequently involved in five types of exchanges: (a) shared administrative information, (b) referrals sent, (c) referrals received, (d) case coordination, and (e) shared client information. In relation to the number of links between agencies (density), the networks were not highly integrated. Where there were links, the greatest number of agencies were involved in "shared client information" and, in this case, the Ottawa network had the highest density scores. This meant that Ottawa had more agencies involved in sharing information about clients than the other networks. In examining the centrality scores, we noted that the networks were more likely to be dominated by a few agencies. Hamilton and Niagara were highly centralized in relation to their referrals and shared administrative information. Shared administrative information entailed sharing information about policies and procedures and funding opportunities. Ottawa was also highly centralized in relation to shared client information, which included sharing client history and case consultation.
|
|
|
|
|
Discussion |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
We found that perceptions of effectiveness varied across the four networks. Ottawa agencies believed their network to be effective in terms of both administration and service delivery. Agencies in the Niagara network, meanwhile, claimed that network membership enhanced service-delivery effectiveness. Finally, although Toronto and Hamilton agencies perceived benefits from membership in their respective networks, the advantages were not determined to be as great as they were in the other two networks.
What might have accounted for these differences among the four networks? We know that information was shared broadly in the Ottawa network and that it had the highest percentage of agencies that belonged to both shared administrative information and shared client information cliques. Ottawa also was highly formalized; that is, it had developed subcommittees that addressed education, research, and clinical issues pertaining to service delivery across the network. In addition, Ottawa was involved in several high-profile activities, such as liaising with specialty services and standardizing driving-assessment tools for use by family physicians. It had also connected institutional and community care in a manner that had not been accomplished in the other three networks. A recent literature review on mental health system integration found that the size and scope of the service-delivery network might play important roles in relation to network success (Durbin, Rogers, Macfarlane, Baranek & Goering, 2001). Ottawa's membership spanned clinical service agencies, planning, and policy-making agencies. Although the benefits of network membership might not accrue to specific member agencies, these findings suggest that agency participation in an effective service-delivery network can lead to improved awareness and coordination of services through improved information sharing.
From an agency perspective, Niagara care providers saw a benefit from being a member of their network (as evidenced by their perception of service-delivery effectiveness). Because the Niagara network had a highly centralized administrative information-sharing system that promoted better ways to carry out work, policies, and procedures, member agencies might have become more aware of enhanced service-delivery processes. Niagara is a largely rural community with relatively fewer agencies; therefore, one might also expect there to have been fewer service providers with which agencies could engage and communicate as well as more extensive informal collaboration outside the formal network structure.
Across all four networks, there were more reports of service-delivery than administrative links. This finding suggests that networks and agencies might miss many opportunities to engage in administrative activities in order to overcome service-delivery fragmentation. Our finding is similar to that of Banaszak-Holl and colleagues (1998), whose study of agencies within community care networks suggested that agencies were less likely to engage in lower-level administrative activities than patient-level ones. It is widely acknowledged that bridging the gap between administration and clinical service is a challenge in most health care settings because client care is most often given priority. Future research that examines how networks share and allocate resources would likely contribute to our understanding of administrative effectiveness.
We believe strong perceptions of service-delivery effectiveness among care providers might be explained by each network's unique structure. Among the three aspects of network structure examined in our study, centrality and multiplexity seem to play the most significant roles in perceptions of network effectiveness. Our findings support those of Provan and Sebastian (1998), who contend that the relationship between groups of agencies (i.e., cliques) in a network might be more critical than the relationship that exists between all the agencies in a network. As noted earlier, systemwide integration may not be the level at which integration is observed (Morrissey et al., 2002).
It is also important to consider that network effectiveness could be due to many health service-delivery and community "system" factors (Rothman & Wagner, 2004). First, optimal network operation requires planning, network member meetings, data reviews, and arranging of links and resources—the sort of work that is first to be postponed in busy agencies. In our research, more resources, both financial and human, were allocated to the Ottawa network than to the Toronto network. This difference might explain, at least in part, the lower scores among Toronto agencies.
Networks will have a communitywide impact only if the care experience of the average care recipient changes. All four networks reported inconsistent agency attendance at network meetings, a factor that reflects uneven support among the various agencies for achieving broader network goals. Network effectiveness will be improved only when the majority of care recipients experience an integrated system designed to meet their needs.
Financial disincentives also can be a barrier to network effectiveness. We learned that minimal family physician involvement in network operations arose from the fee-for-service payment system that penalizes family physicians when they leave their practices to attend network meetings or attend case conferences for care recipients with complex care issues. Involvement of family physicians is key to the diagnosis, referral, and follow-up of individuals with dementia, yet their formal involvement in integration activities is underrepresented. As noted by Goldman and colleagues (2000) "primary care settings are where the patients present for care and treatment—and so public mental health policies must focus attention on improving diagnosis and treatment of mental disorders in primary care settings" (p. 74).
Finally, leadership in agencies is very important. Motivated clinical and administrative staff can design and test changes arising from network initiatives, but they need support and access to resources that only senior leaders in their head offices and government departments can supply.
Limitations
The particular selection of care-provider respondents to represent each of the agencies might have resulted in perceptions of network effectiveness that might have been different had other agency representatives been selected. Although this limitation qualified the generalizability of these groups' responses, the fact that we approached all the agency members within each of the four networks and the resulting high response rates increase our study's generalizability.
Although we examined the evolution of the networks over time, we surveyed the care-provider agencies at only one point in time. Our measurement approaches therefore did not attempt to assess the dynamic nature of community processes across time.
Implications
Our work is important because the findings demonstrate to health care planners and providers the relative contribution of dementia care networks to service delivery. One of its main contributions was further collaboration on developing a conceptual framework and its operationalization in measuring network effectiveness. The more deeply we progressed with our research, the more we realized the importance of understanding the different types of relationships that existed among agencies. The four networks were established to identify and take action on gaps in service delivery, to improve coordination of services, and to share information. The assumption was that, to accomplish these goals, integration of services, in which there are links among all network members, was necessary. We found that the relationships were more complex than previously imagined, and we suggest that network members need to examine more closely the nature of the relationships in which they are engaged and the self-reinforcing dynamic of overlapping groups. It might well be that, in delivering care to individuals with complex needs that change over time, constellations of agencies within networks need to work more closely together. It was critical that we examine integration from the network perspective as well as the agency perspective. We would propose that some of the activities in which members engaged as a network would not have been possible if they had not coordinated their efforts. Traditionally, it has been assumed that integration of all services within networks is required; however, as Provan and Sebastian (1998, p. 462) noted, "when broad, poorly specified structural issues like integration" are of interest, a more finely grained analysis is required.
|
Footnotes |
|---|
1 Department of Health Policy, Management and Evaluation, University of Toronto, ON, Canada.
2 Elisabeth Bruyère Research Institute, Ottawa, ON, Canada.
3 Department of Physical Therapy, University of Toronto, ON, Canada.
4 Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada.
5 Department of Psychiatry, University of Toronto, ON, Canada.
6 Division of Geriatric Psychiatry, Queen's University, Kingston, ON, Canada.
7 Division of Geriatric Medicine, University of Ottawa, ON, Canada.
8 SCO Health Service, Ottawa, ON, Canada.
Decision Editor: Linda S. Noelker, PhD
Received for publication February 26, 2004. Accepted for publication January 10, 2005.
|
References |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
This article has been cited by other articles:
|
|
 |
|
  K. G. Provan and P. Kenis Modes of Network Governance: Structure, Management, and Effectiveness J. Public Adm. Res. Theory., April 1, 2008; 18(2): 229 - 252. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. Jaglal, R. Cockerill, L. Lemieux-Charles, L. W. Chambers, K. Brazil, and C. Cohen Perceptions of the Process of Care Among Caregivers and Care Recipients in Dementia Care Networks American Journal of Alzheimer's Disease and Other Dementias, April 1, 2007; 22(2): 103 - 111. [Abstract] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
| All GSA journals | Journals of Gerontology Series A: Biological Sciences and Medical Sciences | Journals of Gerontology Series B: Psychological Sciences and Social Sciences |
