Evaluation of the Scottish Quality and Safety Fellowship (SQSF)


Knowledge into Action
A mixed-methods evaluation of a national fellowship programme to build quality improvement and safety skills and leadership capacity: The Scottish Quality and Safety Fellowship programme (SQSF)


What we know

SQSF is a 10-month programme established in 2008 to develop clinical leaders with advanced knowledge, technical ability and essential leadership skills to deliver improved healthcare at organisational and national levels. SQSF is managed by NHS Education for Scotland (NES), working in partnership with Healthcare Improvement Scotland (HIS) and NHSS.


Project aim

The aim of the evaluation was to:

          1)    explore the impact of SQSF and the influence of individual factors on the acquisition of quality improvement knowledge, skills and motivation for improvement;

          2)    explore the translation of the acquired quality improvement knowledge, skills and motivation into practical implementation of QI activities and the influence of organisational factors in achieving this; and

          3)    explore the impact of SQSF participation on capability and capacity building in health care systems and the influence of organisational and system level factors in achieving this.


What this research explored

Mixed methods theory-driven evaluation, including:

       1)    Qualitative analysis of two semi-structured telephone interviews with project leads involved in design and delivery of the SQSF, and 17 semi-structured telephone interviews  with SQSF Fellows conveniently and purposively selected across cohorts to reflect substantive heterogeneity in terms of gender, time elapsed since graduation, country base, care sector and clinical profession. Data analysis is in progress supported by a program theory which adapted the Kirkpatrick’s model for evaluating training programmes.  This  explores individual, organisational and SQSF programme drivers behind emerging changes in quality improvement knowledge, skills, psychosocial gains, translation of quality improvement learning into practice and professional role. It also considers capability and capacity building in colleagues and changes in capability and capacity in their organisation, structures & systems.

       2)    Documentary analysis of posters evidencing improvement projects designed and conducted during the fellowship, as a proxy measure for applying programme learning in the workplace. Data analysis is in progress and will inform a theoretical model to describe the process of conducting the projects, any facilitating structures that were in place to support successful delivery and how the anticipated outcomes were achieved. (e.g. improved system performance and experiential learning)

       3)    Quantitative analysis and thematic exploration of the free text reported perceptions, experiences and impacts of SQSF programme participation, as self-reported through: a) an online survey of participants from four cohorts (2014 to 2017), utilising a standardised 14-item questionnaire consisting of open questions and attitudinal statements 6-months after SQSF completion. A total of 69 completed post-fellowship questionnaires were returned for analysis, equating to a response rate of 57.0% (69/120) for the four cohorts with available data; and b) a 62-item online survey specially designed for this evaluation and distributed via the Questback survey software system to all 222 fellows who successfully completed the SQSF over 10 cohorts from 2008 to 2017. A total of 98 completed post-fellowship questionnaires were returned for analysis, equating to a response rate of 45.0% (98/220) for the ten cohorts with available data. Data analysis from the first survey is complete but comparison of findings with the current evaluation survey is in progress.

Follow up work-Data triangulation; Production of a series of infographics to disseminate the evaluation findings to all founders; Submission of poster abstract for The International Forum on Quality and Safety in Healthcare (Glasgow 2019), Two academic papers to be submitted to BMJ QS;


Collaborators

This project was led by Professor Paul Bowie, Programme Director (Safety & Improvement) NHS Education for Scotland, in conjunction with colleagues from the University of Dundee.     


For more information please contact SISCC (siscc@dundee.ac.uk)


Links:

A mixed-methods evaluation of a National Clinical Fellowship programme to build leadership capacity for Quality and Safety Improvement in Healthcare

Scottish Quality and Safety Fellowship Programme

Safety, skills and improvement research collaborative (SKIRC)