Large-Scale Improvement Initiatives in Health Care:
A Review of the Literature Rocco J. Perla,1 Elizabeth Bradbury,2 Christina Gunther-Murphy3
This paper was commissioned by the conference chairs for delegates of the inaugural Conference to Advance the State of the Science and Practice on Scale-up and Spread of Effective Health Programs, Washington, DC, July
6-8. Correspondence to R. Perla, Rocco.Perla2@umassmemorial.org Funding for this conference was made possible in part by grant 1R13HS019422-01 from the Agency for Healthcare Research and Quality (AHRQ). The views expressed in written conference materials or publications and by speakers and moderators do not necessarily reflect the official policies of the Department of Health and Human
Services; nor does mention of trade names, commercial practices, or organizations imply endorsement by the U.S. Government. The Commonwealth Fund, The Veteran’s Health Administration, The Donaghue Foundation and The John A. Hartford Foundation also provided meeting support.
Background: The movement to improve the quality of health care does not lack established interventions, powerful ideas, and examples of success and breakthrough results. Uptake of these advances, however, is limited, uneven, and slow. As a result, many patients receive less than basic care, thereby increasing the risk of negative outcomes. A major challenge for health systems in the United States and globally is to spread these advances broadly and rapidly, adapting them for different care settings.
Aim: The goal of this paper is to provide a succinct review of the literature as it relates to the current thinking, practice, and knowledge base that informs large-scale improvement initiatives in health care as we seek to close the gap between best practice and common practice.
Method: We employed a largely non-systematic review of the literature followed by a modified Delphi technique, with three reviewers to organize themes that emerged during the review process. A standard review form was developed and used. The review was limited to large-scale spread efforts in hospitals and health care systems; the search method keyed on the following terms: large-scale, spread, scale-up, change, hospitals, health care, health systems, human factors, innovation, collaboration, quality improvement, learning networks, diffusion, improvement capability, capacity, re-engineering, outcomes measurement, evaluation, and social movements. Although there were no geographic limitations to the search, we excluded any works associated with large-scale spread in the public health sector or developing countries. Each of the main themes (primary drivers or factors) that emerged during the review were linked to secondary and tertiary factors and organized using a driver diagram.
Results: The four primary drivers that emerged during our review as important to success include: Planning and Infrastructure; Individual, Group, Organizational, and System Factors;The Process of Change; and Performance Measures and Evaluation. The Planning and Infrastructure Driver was defined by a focus on clear aims and a compelling vision, explicit guidance on the intervention (the “how”), a strong and committed management presence, and sufficient resources and infrastructure (materials, contact information, meeting rooms, standard resources) needed to execute the initiative. The Individual, Group, Organizational, and System Driver centered on understanding the social-cognitive dimension of spread—specifically, how people individually and in groups think about and interact with the innovation, the implications its adoption has for them personally and for their organization, and how organizational culture and capability can influence widespread adoption. Further, many reports of successful large-scale change had visible leaders and project champions who were close to the front lines. The Process of ChangeDriver was defined by at least three dimensions, including the extent to which the effort is actively pushed to participants, the underlying change theory that drives the work (e.g., social movement theory or the Model for Improvement), and the mechanism used to spread the intervention (e.g., campaign, collaborative, or extension agent model). Almost every report and study we reviewed recognized the importance of the Performance Measures and Evaluation Driver.This driver was characterized in the literature as a tension between the call for more controlled and rigorous approaches to assess, measure, and evaluate the factors associated with successful outcomes of large-scale improvement efforts and the inherent complexity of doing just that. As some authors pointed out, it was difficult to know with a high degree of confidence about the fidelity of implementation of the program in relationship to its specifications, the true effect of treatment spill-over, or how varying degrees of personal engagement and organizational norms influenced participant views of the initiative.
Recommendations: Based on our review, five recommendations and considerations emerged that may inform the field of large-scale improvement initiatives in health systems moving forward:
Additional research is needed to better understand the social-cognitive dimensions of large-scale improvement and change.
More systematic approaches are needed to assess and evaluate the effects of large-scale initiatives.
More work needs to be done to understand the economics, infrastructure requirements, and major levers of large-scale spread.
More guidance on how to establish and evaluate effective learning networks and collaboration is needed.
Much of the work on large-scale improvement is fragmented and would be well served by the creation of a repository cataloging different approaches and examples of large-scale spread.
Conclusion: Our brief review of the literature on large-scale improvement initiatives in hospitals and health systems identified a tremendous amount of work being done around the world to improve the care patients receive. There is no doubt that our non-systematic review has missed some important contributions to the field. Nonetheless, it is clear that the frequency of large-scale improvement efforts—the epitome of an applied science—appears to be growing rapidly and is believed to represent the fastest way to reduce morbidity and mortality among large numbers of patients (far more than isolated or local interventions). In this sense, the pace at which we learn from each other must be quick and the quality of our information very good. By addressing some of the limitations of the field outlined in this review, we can move toward a more solid knowledge base and more effectual learning.
Introduction “The emerging health care funding challenge across the world, coupled with rising public expectations related to outcomes and quality of experience, requires health care leaders to make urgent and critical choices about which large-scale improvement approaches to adopt. The international health care movement has had enormous successes at the scale of individual services or system improvement, but has struggled to achieve large-scale spread with industry-level transformation of quality and cost.”
-- Jim Easton, National Director for Improvement and Efficiency NHS England
The movement to improve the quality of health care does not lack established interventions, powerful ideas, and examples of success and breakthrough results. Uptake of these advances, however, is limited, uneven, and slow.1 As a result, many patients receive less than basic care, thereby increasing the risk of negative outcomes for both patients and providers.2 A major challenge for global health systems is to spread these advances broadly and rapidly, adapting them for different care settings.
The goal of large-scale improvement in health care cannot focus solely on the eventual and rapid deployment of improved technologies and practices to achieve meaningful change and improved outcomes; those engaged in this work must address the issue of sustainable frameworks that stimulate continual learning and continuous improvement. The maturation and amalgamation of improvement science and the field of large-scale change puts this bold aim within reach. Indeed, the “science of improvement” and the “science of large-scale change and implementation” have gradually come together to form a nexus that now serves as the foundation for large-scale improvement initiatives in health care. While people like Rogers,3 Barabasi,4 Bandura,5 Dixon,6 and Cooperrider7 gave us frameworks and theories of large-scale change, people like Taylor,8 Deming,9 Juran and Godfrey,10 Shewhart,11 and Donabedian12 have given us something to spread—namely, specific and actionable models, ideas, strategies, and principles of improvement, quality, and collaboration that have been put to the test in all corners of the world. The question practitioners of large-scale change ask in a health care improvement context is not so much which interventions are the most appropriate for a particular setting, but rather how such interventions can be reliably and consistently delivered to all patients.
Simply put, those seeking to effect large-scale change in health care systems today must be equipped with the knowledge of improvement/operational science and large-scale change—and possess the skill to bring diverse stakeholders together to achieve a common end.
Against this backdrop, the goal of this paper is to provide a succinct review of the literature as it relates to the current thinking, practice, and knowledge base that informs large-scale improvement initiatives in health care. In this paper, the terms “large-scale improvement” and “scale up” refer to efforts that seek to stimulate positive and sustainable change in multi-state, regional, or national settings through the mobilization of hundreds or thousands of constituent organizations. The term “health care” refers broadly to local, regional, or national systems of organizing and delivering services for the prevention and treatment of disease and for the promotion of physical and mental well-being through hospitals, ambulatory, and home-care services.
One could argue that the current health care system in the United States is entering a phase of what Kuhn13 called “extraordinary science,” in which the conventions and rules of the past begin to rapidly deteriorate on the way to a new paradigm focused on quality, safety, equity, timeliness, accountability, collaboration, and learning, for which the conceptual infrastructure for large-scale change and rapid deployment of innovation in health systems is already established. Successful examples of such initiatives have begun to populate the literature, and this paper aims to capture and organize some of the lessons learned from those experiences to provide a clearer trajectory as we seek to close the gap between best practice and common practice.
Due to time constraints, we employed a largely non-systematic review of the literature followed by a modified Delphi technique,14 with three reviewers to organize themes that emerged during the review process. The three reviewers (RP, EB, CGM) have direct experience and training in the areas of improvement science and large-scale spread initiatives in hospitals and health systems at the local, regional, and national level and have published in these areas. This review was limited to large-scale spread efforts in hospitals and health care systems; the search method keyed on the following terms: large-scale, spread, scale-up, change, hospitals, health care, health systems, human factors, innovation, collaboration, quality improvement, learning networks, diffusion, improvement capability, capacity, re-engineering, outcomes measurement, evaluation, and social movements. Although there were no geographic limitations to the search, we excluded any works associated with large-scale spread in the public health sector or developing countries.
Briefly, the modified Delphi approach involved five separate phases. During phase 1, the three reviewers independently identified 10 significant published articles that informed the topic, using any sources available to them. During phase 2, all three reviewers reviewed the 30 articles identified during the initial search for significance, excluding duplicate articles (n=1) and creating the initial list of articles (n=30). During phase 3, we shared this list of articles with three additional experts in the field of large-scale change to determine if any articles should be added to the list. Following this expert review, we added 9 articles to the initial list (with none deleted), leading to a final list of 39 articles requiring summarization and review. During phase 4, we distributed the 13 articles to be reviewed evenly among the three reviewers (matched by interest and expertise). In order to approach the reviews somewhat consistently, we developed a standard review form (Table 1). During phase 5, we discussed each of the themes that emerged during the independent reviews and then used them to create a driver diagram.15 A driver diagram is a technique used to identify and explore key primary, secondary, and tertiary drivers (factors or concepts) related to a high-level aim (in our case, effective large-scale change). Lastly, we assessed each standard review form to identify tertiary drivers, and each tertiary driver was associated with the multiple references highlighted by the driver. Figure 1 presents the final driver diagram.
Standard Review Form 1. Title/Author
2. What type of work is this? (e.g., Report of an actual change initiative, Review of literature, A how-to/process/structure piece)
3. What was the stated goal/purpose/aim of the work? (e.g., To show that…, To summarize…, To outline a structure for…)
4. What were the major findings/insights of the authors?
5. Interpretation: How does this work contribute to the field of large-scale change in health care? (e.g., Does it link to other works you have read? Does it lead logically to other areas of inquiry? How does it link to the development of the field of large-scale change and improvement in health care? Were there any major limitations?)
Any discussion of large-scale spread and diffusion should acknowledge the tremendous impact of Everett M. Rogers and his model of how innovation diffuses over time and space. Many of the findings discussed below address, explicitly or implicitly, the foundation established by Rogers in his seminal book, Diffusion of Innovation, published in 1962.3 Briefly, Rogers proposed the idea that adopters of innovations or new ideas could be categorized as innovators (2.5%), early adopters (13.5%), early majority (34%), late majority (34%) and laggards (16%). This distribution of adopters, resembling the normal distribution curve, provides a common conceptual-mathematical framework and language for innovation researchers to use as they try to understand how new ideas are adopted in different populations and the rate at which this adoption occurs and is sustained. The general framework outlined by Rogers—embraced by the social and technology sciences—informs much of the work and research in large-scale improvement initiatives in hospitals and health systems and serves as a key theoretical and empirical referent.
Summary of Findings
Our findings are organized by primary and secondary drivers. The four primary drivers that emerged during our review include: Individual, Group, Organizational and System Factors,Planning and Infrastructure,The Process of Change, and Performance Measures and Evaluation (Figure 1). The primary and secondary drivers are not mutually exclusive; rather, they interact with each other in different contexts to create and define a spread initiative. The findings below are not exhaustive, but they do point to general themes, ideas, and concerns that may have a general applicability to others involved in large-scale spread initiatives in health-systems. Recognizing the need to customize any such principles to a particular setting will be the rule and not the exception.
Driver #1: Planning and Infrastructure
Large-scale change efforts require thoughtful planning and a robust infrastructure to create a strong foundation on which to build the work. A number of key elements accelerate the change efforts during the planning stage: a compelling vision or aim for the work; a carefully developed intervention; solid management of the overall effort; and sufficient resources to run the initiative, both centrally and within participating organizations.
Vision and Aim: Large-scale change initiatives appeal to constituents through the use of attractive visions and compelling aims. For example, in Jonkoping County, Sweden, quality improvement projects used the phrase, “a good life in an attractive county,” to create a convincing vision of a patient-centered (versus disease-centered) system.16 The Institute for Healthcare Improvement’s (IHI) 100,000 Lives Campaign and 5 Million Lives Campaign set ambitious and patient-centered aims (saving 100,000 lives from unnecessary mortality and avoiding 5 million instances of harm) to rally and inspire hospitals to take action against preventable mortality and harm.17, 18 Ganz’s work on social movements reinforces the value of goal setting and suggests that the challenge for organizers is to turn the high-level strategy into specific goals with real deadlines.19 Given the recent influx of health care efforts with and without specific aims and visions, the evidence base would be enhanced by comparative evaluations of large-scale change initiatives with and without aims to determine the specific contribution of this component to the overall effort.
Intervention: While a compelling vision is helpful for mobilizing organizations to act, large-scale efforts must provide explicit guidance to organizations on how, and not just what, to change. A key element of the “how” is the design of the intervention. The reviewed literature identifies a number of factors that make interventions more effective and more likely to spread within a social system. Greenhalgh et al. provide an extensive review of the characteristics of interventions that spread more readily.20 These characteristics include the often-cited elements of Everett Rogers’s diffusion model,3 such as relative advantage in effectiveness or cost-effectiveness over the existing process, compatibility with the values of adopters, simplicity of use, trialability (the ability of users to test the intervention before fully committing to its adoption), observability of the benefits, and reinvention (the ability for participants to modify the intervention to suit their needs). In addition, Greenhalgh et al. note other elements of the intervention to be considered, including the “soft periphery” (the ability to modify the intervention at the “edges”), degree of risk involved in adoption, the relevance of the intervention to everyday tasks, the knowledge required to use the intervention, and the level of support provided to participants during implementation.20
The elements of successful interventions in the Greenhalgh review are consistent with the elements of successful interventions in the large-scale change literature. For example, an analysis of a large VHA, Inc. (a network of community-owned health care providers and physicians in the United States) Coaching and Leadership Initiative (CLI) emphasized the importance of the cultural fit of the intervention (compatibility) and the technical merits or evidence base (relative advantage) over the other elements of the diffusion framework. In this paper, the authors cite the example of how an effort to use advanced-practice nurses to encourage vaginal births would be unsuccessful in a culture that highly values independent physician practice.21 Other large-scale change efforts reiterate the value of a “soft periphery” to allow for local adaptation of the intervention.17, 22 In one study, the intervention had both the “hard” core of audit tools that could not be modified and the “soft periphery” of plan-do-study-act (PDSA) cycles that allowed teams to adapt to the processes required for implementation. Some sites in the study provided feedback on the audit to staff right away, while another developed protocols to improve before sharing the data; in both cases, the intervention could be modified to be more consistent with the local culture and practices.22
Two commentary papers emphasize the importance of simplicity in large-scale change interventions because of the increased scope of the work and the risk that complex interventions will get lost at scale.23, 24A Guide to Planning for Large-Scale Change discusses the challenge IHI’s 100,000 Lives Campaign faced when it introduced an intervention on medication reconciliation that was too complex for the scale of the Campaign – thousands of hospitals across the United States.23 It is not clear from this paper whether the intervention was inherently too complex or the recommendations for implementation were not yet mature enough for this scale of work. Those planning for large-scale change have a number of tools at their disposal to correctly identify and scale interventions.23-25 In addition to identifying appropriate changes, health care change efforts require unique oversight structures and systems in order to be successful.
Overall Management: Whether mentioned explicitly in the literature or implied, large-scale change efforts require careful planning and exquisite management. While project management is noted as an important resource for local change efforts, the literature suggests that large-scale efforts in health care require a special management skill set.24 This idea is consistent with the social movement literature, which suggests a different leadership role (than organizational leadership) due to the “decentralized, self-governing, and voluntary mode” of these movements.26 For example, leaders of social movements need to be charismatic;19, 26 one paper on large-scale change in health care also noted the value of a charismatic figure.23 However, charismatic leadership alone was not sufficient to drive the social movement.19
Other than stating the general need for strong management, very few of the articles provided specific insight on the day-to-day management of the initiatives. In addition, little attention is paid in the literature to the need for major structural change within organizations and wider systems. For example, policy changes, measurement systems, and regulatory mechanisms may all need to be dissembled and then reassembled to support the transformed systems. Acknowledgement of the need to address major system levers in order to achieve radical and industry-level improvement is required in the planning and overall management of large-scale change. 27 One thought piece identified the importance for managers of large-scale change to link the changes to policy and organizational priorities on an ongoing basis.28 Berwick and Leape argue for strengthened regulation to drive compliance and a comprehensive national monitoring system with a single patient safety goal,29 and Bevan notes the need for strong governance of the innovation process.27 A further area for research or analysis is a more thorough study of the key elements of overall management of large-scale change. Whatever the management structure, large-scale change efforts cannot be run without sufficient, practical, and on-the-ground support.
Resources: The reviewed literature notes the positive relationship between sufficient resources and effective large-scale change. Resources include personnel, project management, time, and funding for the change effort. The literature also notes a need for sufficient resources to invest in scale-up infrastructure and in spread implementation, both within participating organizations and for the overall large-scale spread effort itself.24, 27, 30, 31 Likewise, inadequate resources were noted as a reason for failed or less effective spread efforts. In a review of one large-scale spread effort in primary care practices in Australia, the authors note that larger practices could more effectively devote resources to the change process, while many small practices found the entire project unrealistic because of the high demand on limited staff.22 In their literature review of diffusion of innovation in health service organizations, Greenhalgh et al. note that there was strong indirect and moderate direct evidence that dedicated and ongoing funds for implementation make the change more likely to be adopted and sustained.20 The need for dedicated resources within an organization is also a function of the design of the large-scale spread effort. McCannon et al. suggest planning efforts in such a way as to “remove ‘lack of resources’ as an excuse” for non-participation. The authors cite the value of the simple interventions within IHI’s 100,000 Lives Campaign that did not require participating hospitals to add staff or technology, simply to redesign existing processes.23 One change effort provided participating organizations with resources, specifically time from a centralized project manager, to lower the need for participating organizations to use internal resources.22 While outside help lowers the resource needs within participating organizations, support from the central change effort was only effective if sites devoted at least minimal resources to the project and participants supported the change.22
In addition to the resource needs for adopter organizations involved in change efforts, literature in this area cites the need for the team or organization managing the central change effort to be sufficiently resourced. In a review of four change efforts, one summary paper notes a number of central resourcing needs for effective efforts: site recruitment, program marketing, program education, and technical support on implementation and sustainability. The authors note that this infrastructure is costly but critical to success, and cite examples of both effective and less effective efforts.32 Another article notes the importance for the organization managing large-scale change to invest sufficient resources in the analytical and research capacity and capability.26 Articles on social movements note the need for long-term funding to support network building and connections between individuals through meetings and learning structures.26 Ganz et al. note the value of a structure for maximizing the use of resources. Ganz argues that within a campaign structure, the clear end point can increase support because participants can invest a set amount of time and energy in the initiative.19 In an evaluation of the Jonkoping County quality effort, the authors introduce the concept of an “investment threshold,” which argues that there is a set minimum level of investment in quality improvement infrastructure that needs to occur before an effort can obtain positive results.16 This concept is supported by other theoretical papers that highlight the need for change efforts to invest in skill development and quality training.27 While an interesting concept, much more research needs to be done to understand the necessary level of investment and how it varies in different contexts.
In terms of wider funding and the relationship between reimbursement and outcome of change, Leape and Berwick foresee that major partners such as the federal government will be more specific about measurable outcomes that will drive improvement in the future, and suggest pay-for-performance and payment incentives to engage clinicians more rapidly in compliance with patient safety best practice.29 Importantly, Leape and Berwick recommend redesign of payment structures to ensure that the current perverse system of payment for a “defective product” (unsafe care in this instance) is removed. Mandel proposes spread models that align financial rewards with large-scale improvement initiatives, and that partnering organizations share accountability for achieving population-based improvement goals rather than rewarding individual organizational success.33 Deming voices caution in undertaking improvement work where successful groups are penalized based on the poor performance of others, leading not to greater team work and shared learning but to resentment and strained relationships.9
Other Planning and Infrastructure Considerations: In addition to the key themes noted above, our review identified a number of other elements that appear to help with spread in large-scale change in the area of planning and infrastructure. Two studies stated that information technology was a key element in the change process. One article described the importance of information technology in a successful change effort in the Veterans Health Administration.34 Another evaluation and summary piece identified the lack of information systems as a barrier to large-scale change efforts and suggested its benefits included more accurate and timely information, better connections between patients and care providers, and better tracking of patients over time.31 Another study on a successful effort run by VHA, Inc. identified information technology as helpful to efforts, but noted that information technology cannot be successful without an organizational culture that supports the work.21 Information technology seems to be a helpful, but not sufficient, component of this work.
Several other important factors for planning large-scale change emerged from the literature. Environmental factors, such as community socioeconomic infrastructure, seem to play a role in helping or hindering change efforts.30 More research in identifying and analyzing which of these factors is most correlated to success would be a valuable contribution to the field. Another factor that accelerates change is advanced planning and infrastructure for sharing local success, tools, and stories with the broader participants in the change process. One study on spreading a system of improved access in the Veterans Health Administration’s outpatient clinics notes the value of developing a national infrastructure in order to share and strengthen social ties between participating sites. The article describes the elements of the infrastructure include establishing steering committees at the national and regional levels to lead the effort, identifying national and local communication strategies for building interest and attracting adopters, building a cadre of internal coaches to provide guidance and mentoring to clinics, and developing sophisticated measurement systems that produced reports at the national, regional, facility, and clinic level to track results and provide feedback.35 Another article described the value of stability among key team members, at both individual sites and the overall initiative level in the early planning and scale-up stages of the program.25 Studies also noted the importance of developing spread and sustainability strategies at the outset of any initiative.16, 21, 24-26 One study identified the value of creating clear communication plans for disseminating best practices at the outset of a large-scale change initiative.28 In order to be successful, large-scale change efforts in health care cannot underestimate the impact of strategic investment and allocation of resources, both in time and consideration, in the planning phase and the infrastructure needed to do this work.
Driver #2: Individual, Group, Organizational, and System Factors
All large-scale improvement initiatives involve humans in complex social settings. Therefore, understanding the cognitive dimension of spread—that is, how people individually and in groups think about and interact with the innovation, and the implications its adoption has for them personally and for their organization—is critical to developing and executing large-scale initiatives and also in evaluating their benefits.36, 37 Ferlie and Shortell argue for a four-level approach to change: individual, group, organization, and system.31 They suggest that a culture of teamwork, learning, and customer focus is the key to overcoming the multiple barriers to organizational change, and that hierarchical (multi-layered) cultures implement quality with difficulty. The authors also stress the need for a culture that values quality, practices reflection, and feeds this back into the system. These subsections are explored below.
Individual and Group Factors: In considering how individuals engage with innovation, the theme of personal values emerges very strongly in the health care spread literature30, 38 and in the exploration of social movements as a new way for health care to think about large-scale change.39 For example, Ganz discusses the concept of engaging the head, heart, and hands of those committing to social movements,19 and Bate et al. note the rational value-based decision to join the initiative, and the emotional context of belonging.40 Della Penna, in his study of inpatient palliative care, writes, “...the available literature on disseminating better practices in health care did not capture the sometimes intense sense of personal engagement that permeated all levels of the organization.”38 While motivation of individuals can influence the uptake of new knowledge, this broad personal engagement seemed to be a force stronger than individuals’ rational beliefs. Della Penna lists three factors that underpin this personal engagement: a strong evidence base that is viewed as highly credible by participants at many levels; a genuine belief that this new model provides better patient care and increased satisfaction; and a developing appetite for the innovation results in adopters welcoming it (“pull system”), rather than its being “pushed” by internal or external agents.38 ExpandNet adds these related factors: adopters perceive a need for the innovation and are motivated to implement it; the adopting organization has the appropriate implementation capacity, decision-making authority, and leadership; and the timing and circumstances are right.24
Another factor affecting an individual’s decision to adopt is the extent to which the change increases their workload. An increase of 10% or more caused by the innovation deters adoption,27 whereas adoption is more likely if the adopter perceives the innovation to relieve current workplace pressure.32 Greenhalgh et al. consider other influencing factors and challenge Rogers’s3 five widely cited adopter categories (early adopter to laggard) as “stereotypical and value laden...(they) fail to acknowledge the adopter as an actor who interacts purposefully and creatively with a complex innovation,” explaining influences such as general psychological antecedents (some people are more likely to try new things), context-specific psychological antecedents (some people are more able to use the innovation), the meaning of the innovation to the individual, the adoption decision, and how and when concerns at any stage of adoption are dealt with.20 This reinforces the need for those planning and leading large-scale change to be acutely aware of the relationships between individuals and the innovation, and the factors that can influence their decisions to adopt or reject the innovation. Greenhalgh et al. also make the point that the psychology of innovation and diffusion is not well researched within health care.20 Given its clear importance in adoption and spread, and the tremendous advances in cognitive psychology since the time of Rogers’s seminal work,41 perhaps the time is now ripe to explore the cognitive dimension of spread further and in greater detail.
Champions /Change Agents: Linked to the individual’s relationship with the adoption of change is the positive influence of people who model new behavior associated with adoption (change agents or clinical champions) and those who influence individual or group thinking about the innovation (opinion leaders). Both internal and external change agents among managers and clinicians are believed to be major contributors to effective diffusion of innovations,3, 20 although their roles differ.21 For example, the managerial champion’s role is to establish systems for implementing innovation, whereas the clinical champion’s role is to communicate technical knowledge about the innovation to clinical colleagues.35 Related to this, some clinical groups are more likely to adopt changes advocated by people from the same professional groups.22 In all cases, providing champions and change agents with time to concentrate on promoting the innovation and influencing its adoption can maximize its benefit.27 Clearly, managerial/clinical collaboration on innovation creates a synergy between subject matter experts, process owners, and various stakeholders and is one of the core principles of effective and sustainable change advocated by Deming.9
Leadership: Strong visible leadership is consistently referred to in the literature as a key factor in scale-up and spread, yet there is little definition of what this actually means, or evidence of spread and scale-up being adopted as key leadership skills. Though the word “leader” is widely used by researchers, its definition is unclear; there is a tendency to perceive chief executives as the only leaders in an organization, when inclusion of leaders such as directors of finance would be beneficial.42 Moreover, there is sparse discussion of the relationship between senior clinical leaders and administrative leaders, and the specific spread capabilities required by each.
Rooney and Leitch illustrate the key role that leaders can play in large-scale change, describing three factors that can move an entire country (Scotland) along in its quality journey: positive energy and attitude; senior policy and delivery leader support; and a standard national improvement model.43 The work in Jonkoping County, Sweden, also highlights the importance of leadership,16 where the persistence, energetic commitment, continuity, and style of the Chief Executive, Chief of Learning and Innovation, and Head of Department of Medicine, along with their constructive relationship with political figures and one another, were considered pivotal in regional transformation. Several articles considered specific leadership actions to support change efforts—for example, engaging staff and articulating the quality vision to the wider workforce,22, 38 identifying the target population, making the work a clear priority, committing time and resources to achieve the stated objectives, and aligning organizational goals.24, 35
In order to build a cadre of leaders equipped to deliver on leadership actions such as those above, the necessary leadership skills need to be identified and developed.19, 26, 27, 39, 40, 44 Green and Plsek are the only authors in the review to describe leadership competencies through a framework of 11 essential leadership skills, which they tested by coaching “diffusion executives.”21 The authors found that the senior leaders and their teams were able to achieve measurable improvement, and that the spread of changes became easier and faster with each improvement cycle. Developing sufficient leadership capability to effect large-scale transformation is in itself a scale-up challenge.
Capability and Capacity Development: In addition to leadership development, institutional capacity and capability building are widely described in the literature as crucial to effective and sustainable scale-up, and the need for special spread skills not found in traditional project management education is highlighted.24 But what constitutes adequate capability and capacity? The highest-performing health care organizations (in terms of cost and quality) in both the United States and United Kingdom have invested systematically in building improvement capability,27 yet there is little evidence of this in practice, with Bate et al. noting that fewer than 10-15% of the National Health Service (NHS) staff in England currently participate in formal improvement activities, although they estimate that to achieve the scale of improvement required to transform health care, 80-100% of NHS staff need to be involved in adoption and spread.40 This highlights the daunting challenge of effective scale-up and spread, and the need to more thoroughly understand whether capability exists and is underutilized42 or whether the gap is indeed this large.
There is reference in the literature to both the infrastructure needed to support organizational/system capability-building, and spread competencies. Matrix, a consulting group that wrote a report for the NHS Modernization Agency, identifies the following infrastructure factors: providing access to a range of training to develop appropriate skills; creating and embedding specific roles with a remit for taking forward the modernization agenda; recognizing the key role that middle mangers play in executing the strategic vision and ensuring that frontline views are heard.28 Bevan addresses a previous gap in the research on spread competencies, discussing the knowledge and skills requirements to enable the workforce to improve health care quality and productivity at scale by offering a set of seven skills: process and systems thinking; personal and organizational development; involving patients, careers, staff, and the public; initiating, sustaining and spreading change; delivering on cost and quality; problem solving/internal consultancy skills; and innovation for improvement.27 This was the most thorough and detailed capability framework within the papers reviewed, but does not go so far as to suggest which skills are required at which level of the organization, or to address formal student education. However, changes in undergraduate medical education to incorporate quality improvement training indicates a positive approach to equipping the next generation for large-scale improvement which can be enhanced through informal learning and social networks.45, 46
Learning Networks: In terms of informal education, the literature widely supports the value of continuous learning networks, which may vary in terms of scale and membership,28, 47 to maximize workforce improvement capability.19-21, 23-27, 30, 35, 38-40, 43 Structured learning environments such as virtual or face-to-face training sessions are advocated on the premise that intrinsic motivation and positive social relationships bolster energy for improvement and on a national or local scale, can deliver benefits such as amplified cooperation, a cadre of professionals and citizens skilled in implementation and spread, and a shorter time to full-scale implementation of the innovation.48 However, McCannon and Perla caution that it is often difficult to assess or attribute the specific contribution of learning networks to organizational capability and achieving large-scale improvement goals, which is similar to the reflections on social networks.48
Social Networks: As a means to support health care staff in generating improvement capability and capacity, both learning and social networks are seen as beneficial. The diffusion of innovation is affected by the structure and quality of social networks,20 which can generate intrinsic motivation and positive social relationships, and thereby impact on improvement energy.48 Robust social networks also aid spread and knowledge exchange among participating groups,21 which creates a sense of shared learning and overcoming struggles together.38 Of course, not all social networks are created equally or will experience the same degree of commitment, success, and sustainability; successful networks work best in a collaborative situation, and people need to feel comfortable sharing difficulties and challenges as well as successes.38 Network type and membership are believed to be critical factors for success; for instance, doctors may work in horizontal networks while nurses often work in vertical networks. Other network influencers are the degree of similarity amongst members; the effectiveness of opinion leaders and change champions in modeling adoption of innovation; recognition and reward of achievements; people working across organizational boundaries to minimize delays and problems; and formal dissemination programs.33 However, to draw any robust conclusions about the value of social networks and their contribution to organizational capability, more empirical research is needed to better understand and assess their impact.
Organizational and System Capability: The ability to plan and deliver innovation at the scale of the whole health care industry requires scale-up of both individual and organizational spread capability. Relationship building and investment in leadership and training are pivotal, 19, 26, 33, 39, 40 particularly as areas with the highest need for change often have the lowest existing capacity for introducing the intervention themselves,22 whereas “cosmopolitan” (outward-looking) organizations20 are more open to external influence, and “likeminded” organizations related either by proximity or by a network system are more likely to adopt innovation.20, 30 This has spread planning implications for geographically remote or poorly networked organizations, where alternative proactive networking systems would have to be developed.
In considering what health care could learn about spread from social movements in other sectors, Bate et al. give the example of a national structure to initiate and manage the organization with dispersed leadership through local “chapters,” using tactics suited to local circumstance.39 Pastor and Ortiz found that small organizations were unlikely to have the capacity to coordinate large-scale multi-stakeholder initiatives, and argued for identification and funding of large “anchor organizations” whose value is in leading smaller organizations in social mobilization.26 This approach was demonstrated in the development of “node” health care organizations responsible for supporting and disseminating good practice through multi-organizational learning networks during a national patient safety campaign.17, 18 The legacy of disseminated leadership models such as this is in building long-term organizational capability and effecting cultural change.
Organizational and System Culture: Culture, or “how things are done round here,” is a critical factor in an organization’s ability to respond to innovations and rapidly achieve spread. Both cultural characteristics and behaviors that relate to organizational spread are considered in the literature. Assessment of organizational characteristics can indicate whether the culture is likely to support the widespread adoption of innovation; these include enabling cross-functional teamwork; supporting pooled knowledge and teamwork toward shared results; a sense of urgency about the need to innovate coupled with a desire for rapid improvement; and an organizational commitment to keep focused on the pace of change.22 Organizational behaviors such as continual internal and external surveillance of health care innovations and assessment of their impact and fit with the current culture;44 the ability to translate research into practice and to coordinate spread across all health care disciplines;32 and the ability of the innovation to add to organizational knowledge21 all contribute to a culture of adoption, diffusion, and sustainability. Being able to assess these cultural characteristics and behaviors and identify likely problem areas is a critical component of the planning for large-scale change.
In terms of systems, two papers by ExpandNet highlight the importance of understanding and respecting local culture and being sensitive to cultural norms at all stages of spread initiatives.24, 25
Driver #3: The Process of Change
In addition to careful planning, large-scale change efforts need to carefully select the process of change. The literature identifies at least three dimensions of the process of change: (1) the extent to which the effort is actively pushed to participants, (2) the underlying change theory that drives the work (e.g., social movement theory or the Model for Improvement), and (3) the mechanism used to spread the intervention (e.g., campaign, collaborative, or extension agent model). On the first dimension, experts suggest that there are a range of ways that spread can occur, from “let it happen” (diffusion) to “help it happen” to “make it happen” (active dissemination).20 Nearly all of the reviewed literature fell in the range between “help it happen” and “make it happen.”
On the second dimension, the underlying change theory, nearly all of the change efforts used a clear model to drive the work or, at the very least, proposed a model for change to more effectively manage the large-scale change. The underlying theory in much of the reviewed literature was improvement theory, often associated with the Model for Improvement,15 and system thinking.17, 20, 23-25, 27, 29 One study noted the limitations of using quality improvement theory in an unsuccessful process redesign effort to decrease “time to the first available appointment” in clinics and the length of stay in hospitals. These limitations included difficult testing change ideas, difficulty using the “PDSA” format, and challenges with rapid testing. It is unclear, however, whether the challenge with the theory was also a result of other structural factors mentioned in the paper (e.g., lack of project team motivation).49 Three studies identified the value of social movement thinking, including the use of powerful narratives, in health care.16, 39, 40 While the power of the social movement theory as applied to health care is an area of interest and much commentary, none of the reviewed studies explicitly evaluated an actual attempt to use social movement in large-scale health care efforts, though IHI’s 100,000 Lives and 5 Million Lives Campaigns, described elsewhere in this document, applied some aspects of social movement organizing methodology.17, 18 While sometimes positioned in the literature as a choice between theories,39 the underlying theories of improvement science and social movement may not be mutually exclusive. As social movement theory becomes more widely applied in the health care setting, it would be helpful to evaluate its contribution to large-scale change efforts. None of the reviewed articles evaluating actual change efforts compared different theoretical approaches. Further research is needed to understand the impact of the underlying theory in driving large-scale change and ways in which the theories are complementary and contradictory.
The third dimension of the change process is the mechanism used to spread the change. Change efforts applied a variety of delivery mechanisms, from a small regional effort that used the sharing of data, learning, and site visits47 to large collaborative efforts,21, 34, 35, 49, 50 to very large campaign efforts.17, 18, 51 In many cases, there were examples of more and less effective efforts in the various delivery mechanisms. For example, the collaborative methodology was cited as producing significant results in a VHA, Inc. program,21 but unsuccessful in an effort to redesign processes within office practices and hospitals.49 One paper makes a strong case that, given its widespread use, there is a paucity of balanced and systematic literature on the effectiveness of the Collaborative methodology.52 While not explicitly mentioned in the literature, there are indirect references to the need to match the delivery mechanism to the problem. For example, a complex intervention that has not been fully tested might be more appropriate for the Collaborative methodology than a campaign structure. It is unclear whether the less successful change efforts were a failure of the mechanism or a failure to match the mechanism to the scope of the problem and the nature of the intervention. Evaluations of the delivery mechanism are increasingly complex, since many of the large-scale change efforts used hybrid or modified models for their change efforts.20, 22 Further research might work to understand the natural sequence or cycles during the change process and the nonlinear and flexible dynamics that seem to characterize most large-scale initiatives in health systems.48
As noted in the planning section of this paper, developing a spread plan at the outset of a project appears to be a factor in accelerating change efforts. The content of that spread plan varied among the various efforts. A number of studies, which were both evaluative and theoretical in nature, outlined frameworks for spread and sustainability in large-scale change efforts. 25, 26, 28, 32, 35, 42, 43, 53 While the content of these frameworks varied, there were a number of overlapping elements (e.g., the importance of engaged leadership). Because most of these elements are covered elsewhere in this review, we have not summarized the various frameworks here.
Driver #4: Performance Measurement and Evaluation
One of the premises of effective health care improvement is that its impact can be measured and understood, thereby providing a substrate for additional learning and development. Collecting data and linking this to the change being tested is important; clearly, this is true of large-scale spread initiatives. However, the frequency of uncontrolled studies and methodological limitations of the experimental studies reviewed was a concern among many authors, both in their own work and in their assessment of the work of others. We identified a tension between (1) the call for more controlled and rigorous approaches to assess, measure, and evaluate the factors associated with successful outcomes of large-scale improvement efforts and (2) the inherent complexity of doing just that.
Measurement and Feedback: The tension noted above is well described by the measurement and evaluation community, and the intent of isolating “pure factors” in social science research often leads a researcher into a hall of mirrors.36, 54, 55 As some authors pointed out, it was difficult to assess the fidelity of implementation of the program in relationship to its specifications, 49 the effect of contamination on the control group with the treatment or condition being tested, 56 or how varying degrees of personal engagement and organizational norms influenced participant views of the initiative.38 In their systematic review of the evidence of the impact of quality improvement collaboratives,50 Schouten et al. found that a number of studies demonstrated positive effects, but that the methodological flaws and the heterogeneity of the interventions makes it unclear if the quality improvement collaborative was responsible for the effect. The authors go on to state, “To understand how and why quality improvement collaboratives work it is necessary to look into the ‘black box’ of the intervention and study the determinants of success and failure.”50
Recognizing that the study of large-scale spread is a social science endeavor means that evaluators of such programs are forced to deal with a high degree of uncertainty and the complex interactions that define them.36 We noted that some reports in the literature reviewed had a different perspective on the effect of the intervention that could have been associated with the level of the analysis and the point of observation and data collection. In some cases, an initiative was deemed successful in relation to higher-level outcomes such as the quality-of-care indicators in the VA study, 34 but the linkage of these types of outcomes to front line activities and structural supports was not discussed in detail. In other cases, low-level success (execution of an initiative at the front lines) was not associated with the type of high-level outcome desired.47 Similarly, in papers we reviewed, systems with a highly regarded approach to the spread of improvement and innovation had limited quantitative evidence and data infrastructure to systematically understand their success, making it difficult for others to learn from their model and approach.16 In discussing effective teams, Green and Plsek note accountability for measurement and evaluation as “critical activities” in improving outcomes.21 They describe both the assessment method using a seven-point qualitative rating scale assessing spread capability (in terms of completeness, extent of spread, and expected sustainability), and the spread outcomes (improvement work became easier; improved cycle time on implementation within given time scale; improved ability to implement planned spread within given time scale and the overall improvement metrics). Dobbins et al. note that the scale of adoption has changed from the traditional “adopted/not adopted” to a continuum ranging from zero to full adoption, and this is a useful consideration in conjunction with the factors affecting degree of adoption and spread.30 They also call for research on the cost implications of altered clinical outcomes on health care resource allocation as an objective assessment of translating research into practice. For example, the impact of reduced hospital acquired infections could be calculated in cost savings through reduced length of stay, in assessing overall outcomes such as life expectancy / hospital mortality and societal factors such as time taken to return to work and contribution to GDP among work-aged people. Addressing these concerns would be a way to strengthen the methodological approaches currently employed in the assessment of the impact of spread. Moving toward more sophisticated and realistic forms of assessment as outlined by Pawson and Tilley—which asks the question, “What works, for whom, in what circumstances?”—might be helpful, particularly in light of the drive for industry-wide radical and measurable improvement.37 Although the realistic model advocated by Pawson and Tilley is appreciated by many, it requires training and skill to execute in a meaningful way.
Based on our review, a number of recommendations and considerations emerged that may inform the field of large-scale improvement initiatives in health systems moving forward:
Additional research is needed to better understand the social-cognitive dimensions of large-scale improvement and change. Many of the studies reviewed here emphasized the importance of understanding how the initiative in question was assimilated and understood from the vantage point of various stakeholders (front-line staff, middle mangers, and senior leaders) and how the initiative fit within the larger existing social structure and histories of the teams responsible to oversee and implement the program or initiative.
More systematic approaches are needed to assess and evaluate the effects of large-scale initiatives. This is a tall order and the complexity of understanding large socially mediated initiatives is not easy. That said, adopting more rigorous and realistic models of evaluation may allow for some aspects of the interventions to be better understood and replicated in other settings. If the true nature of the intervention is not well controlled or understood, then it becomes extremely difficult to know what really worked or did not work.
More work needs to be done to understand the economic and infrastructure requirements of large-scale spread. As large systems and governments begin to spread health care improvement initiatives at industrial scale (e.g., across large sections of a country), it is critical to understand what the “start-up” investment looks like and what the return on investment might be. As Ovretveit and Staines point out, it is possible that a quality improvement infrastructure threshold needs to be met before positive results are observed—what they refer to as an “investment threshold.”16 The second essential consideration is structural levers for change, such as policy frameworks, governance structures, reimbursement mechanisms, and overarching measurement systems. If these are ignored in the planning sequence, and/or the leadership capability and capacity does not exist to effectively address major barriers to change or outmoded infrastructure, the effect of large-scale spread will be limited.29
More guidance on how to establish effective learning networks and collaboration, and how to evaluate their impact is needed. A number of the studies we reviewed pointed to a differential effect of organizations participating in such networks; some thrived while others languished. The factors associated with optimal collaboration and sustainability of large learning networks is beginning to be better understood, but will require additional study and testing.
Much of the work on large-scale improvement is fragmented and would be well served by the creation of a repository cataloging different approaches and examples of large-scale spread. Much of what we know about the effects of large-scale improvement efforts appears to come from select organizations (such as the Institute for Healthcare Improvement) and through published reports. It would be helpful to develop and maintain a standard data registry that could house a wide number of studies and experiments from which secondary analyses could be performed and context-specific assessment tools developed.
Our brief review of the literature on large-scale improvement initiatives in hospitals and health systems identified a tremendous amount of work being done around the world to improve the care patients receive. There is no doubt that our non-systematic review has missed some important contributions to the field. Nonetheless, it is clear that the frequency of large-scale improvement efforts—the epitome of an applied science—appears to be growing rapidly and is believed to represent the fastest way to reduce morbidity and mortality among large numbers of patients (far more than isolated or local interventions). In this sense, the pace at which we learn from each other must be quick and the quality of our information very good. By addressing some of the limitations of the field outlined in this review, we can move toward a more solid knowledge base and more effectual learning.
The authors are indebted to Dr. Jane Roessner, who provided valuable assistance with different versions of this review. The authors also thank Mr. Jim Easton for providing insight regarding the challenges facing national leadership regarding large-scale spread in health systems and Mr. Joe McCannon, Dr. Brian Mittman, and Ms. Marie Schall for their valuable feedback during earlier stages of manuscript development.
1 Director, Analytics, UMass Memorial Medical Center, Center for Innovation and Transformational Change; Assistant Professor, UMass Medical School; George W. Merck/IHI Fellow 2008-2009
2 Associate Director for Quality Improvement, NHS Bolton, UK. Health Foundation/IHI Fellow 2009-10
3 Hospital Portfolio Operations Director, Institute for Healthcare Improvement