Mechanisms of Improvement
Framework to support the development and teaching of balancing measures Improvement Pause - how can it support improvement practice?
What we know
The complexity of the healthcare system means that eﬀorts to improve quality and safety often achieve only limited beneﬁts and can have unintended consequences, which may impact positively or negatively on care processes and outcomes. However, several systematic reviews have shown that there is little evidence that improvement programmes routinely evaluate the presence of unintended consequences either pre or post-implementation, with little specific guidance on how to best account for improvement consequences other than goals.
We have conducted three interconnected studies aiming to explore current understanding of measurement of unintended consequences in healthcare improvement, including how the wide range of consequences can be identified, evaluated and interpreted to achieve a more balanced accounting of the overall impact of improvement interventions.
Balancing measures or a balanced accounting of improvement impact: a qualitative analysis of individual and focus group interviews with improvement experts in Scotland
The first study comprised of a qualitative analysis of interview data carried out in two phases, with semi-structured interviews used to formulate a draft conceptual framework for considering all consequences of improvement which was then refined and elaborated during two focus groups. The framework proposes that a balanced accounting should consider goals (expected desirable consequences) and predictable trade-offs (expected undesirable consequences) early in the design of a quality improvement programme and pause to take stock of pleasant (unexpected desirable consequences) and unpleasant surprises (unexpected undesirable consequences) after a period of implementation.
A framework for ensuring a balanced accounting of the impact of antimicrobial stewardship interventions
Drawing on a Cochrane systematic review of Antimicrobial Stewardship Initiative (ASI) in hospitals, the second study aimed to pilot the previously developed framework and examine the range of outcomes measured in published evaluations. We conducted a narrative review and described a more structured framework with specific examples of the range of consequences that ASI can have, in terms of their desirability and the extent to which they were expected during planning. For instance, only 11 (10%) of 110 ITS studies included any information about unintended consequences, with 10 examining unexpected, undesirable outcomes (unpleasant surprises) using case-control, qualitative or cohort designs.
A balanced approach to identifying, prioritising and evaluating all potential consequences of quality improvement: modified Delphi study
The third study consisted of a two-round modified Delphi consultation with 72 experts in round one, and 60 (83.3%) in round two, which elaborates on the previous findings and focuses on exploring and developing consensus in relation to identifying, prioritizing, evaluating and interpreting all consequences of quality improvement, which can ultimately lead to a more balanced accounting of the impact and identified the concept of an ‘improvement pause’.
Translating new knowledge into practice
The next stage of this project is to engender understanding of improvement advisors of a new ‘’balanced accounting’ conceptual model that will improve identification of ‘’effective’’ balancing measures, support improvement advisors to enhance clinical team’s understanding of balancing measures, suggest strategies to incorporate the balanced accounting into the design of quality improvement projects and provide guidance in addressing the unique challenges associated with achieving it.
What this work explored
Exploratory work to adapt and refine the (including improving the language used) to enhance its utility and feasibility (ongoing) :
- Training materials to support teaching of balancing measures;
- Prompts for improvement advisors to use to help teams identify more meaningful balancing measures, i.e. 4-6 questions that influence measurement of unintended consequence.
- Test the impact of an ‘improvement pause’ to reflect on measurement plans and unintended consequences of improvement within the wider system.
The findings from this exploratory work have been refined into a tool to support improvement project reflection – expanding on the ‘Improvement Pause’ concept that was identified within the research. This approach is currently being tested by a cohort of the Scottish Quality and Safety Fellows – which was due to be completed by April 2020 (but had to be suspended due to COVID-19).
What this translational element of the study will add
Testing of the concept of the Improvement Pause – is exploring how the Pause can be applied and used, identify the overall role/impact of the Pause, how the Pause was understood by the improvement advisors and translated into frontline settings, how the Pause influenced the understanding and knowledge of unintended consequences of improvement and what are the implications for the wider use and application of this new concept/approach.
This work has been led by Dr Madalina Toma, Research Fellow at the School of Health Sciences, University of Dundee, in conjunction with colleagues within the School and at NHS Education for Scotland, who run the Scottish Quality and Safety Fellowship programme.
For more information please contact SISCC (firstname.lastname@example.org)
The Quality Measurement Framework
How Safe is Our Care
Safer Healthcare (book)
Science of Improvement: Establishing Measures
Introduction to Measurement for Improvement
Scottish Quality and Safety Fellowships