• Exploring variation in the use of feedback from national clinical audits: a realist investigation

      Alvarado, N.; McVey, L.; Greenhalgh, J.; Dowding, D.; Mamas, M.; Gale, C.; Doherty, P.; Randell, Rebecca (2020)
      Background National Clinical Audits (NCAs) are a well-established quality improvement strategy used in healthcare settings. Significant resources, including clinicians’ time, are invested in participating in NCAs, yet there is variation in the extent to which the resulting feedback stimulates quality improvement. The aim of this study was to explore the reasons behind this variation. Methods We used realist evaluation to interrogate how context shapes the mechanisms through which NCAs work (or not) to stimulate quality improvement. Fifty-four interviews were conducted with doctors, nurses, audit clerks and other staff working with NCAs across five healthcare providers in England. In line with realist principles we scrutinised the data to identify how and why providers responded to NCA feedback (mechanisms), the circumstances that supported or constrained provider responses (context), and what happened as a result of the interactions between mechanisms and context (outcomes). We summarised our findings as Context+Mechanism=Outcome configurations. Results We identified five mechanisms that explained interactions between providers and NCA feedback: reputation, professionalism, competition, incentives, and professional development. Underpinned by the mechanisms professionalism and incentives, feedback was used most routinely within clinical services resourced to maintain local databases, where data were stored before upload to NCA suppliers. Local databases enabled staff to access data easily, customise reports and integrate them within governance processes. Use of feedback generated in this way was further supported where staff supporting audit participation were trusted to collect timely and accurate data. Feedback produced by NCA suppliers, which included national comparator data, was used in a more limited capacity. Challenges accessing data from NCA supplier databases, concerns about the quality of data across participating organisations and timeliness were reported to constrain the perceived usefulness of this type of feedback as a tool for stimulating quality improvement. Conclusion The findings suggest that there are a number of mechanisms through which healthcare providers, in particular staff within clinical services, engage with NCA feedback, but that there is variation in the mode, frequency and impact of these interactions. Feedback was used most routinely within clinical services resourced to maintain local databases, where data were considered timely, trusted as accurate and could be easily accessed to customise reports for the needs of the service.
    • How, in what contexts, and why do quality dashboards lead to improvements in care quality in acute hospitals? Protocol for a realist feasibility evaluation

      Randell, Rebecca; Alvarado, N.; McVey, L.; Greenhalgh, J.; West, R.M.; Farrin, A.; Gale, C.; Parslow, R.; Keen, J.; Elshehaly, Mai; et al. (2020-02-25)
      National audits are used to monitor care quality and safety and are anticipated to reduce unexplained variations in quality by stimulating quality improvement (QI). However, variation within and between providers in the extent of engagement with national audits means that the potential for national audit data to inform QI is not being realised. This study will undertake a feasibility evaluation of QualDash, a quality dashboard designed to support clinical teams and managers to explore data from two national audits, the Myocardial Ischaemia National Audit Project (MINAP) and the Paediatric Intensive Care Audit Network (PICANet). Realist evaluation, which involves building, testing and refining theories of how an intervention works, provides an overall framework for this feasibility study. Realist hypotheses that describe how, in what contexts, and why QualDash is expected to provide benefit will be tested across five hospitals. A controlled interrupted time series analysis, using key MINAP and PICANet measures, will provide preliminary evidence of the impact of QualDash, while ethnographic observations and interviews over 12 months will provide initial insight into contexts and mechanisms that lead to those impacts. Feasibility outcomes include the extent to which MINAP and PICANet data are used, data completeness in the audits, and the extent to which participants perceive QualDash to be useful and express the intention to continue using it after the study period. The study has been approved by the University of Leeds School of Healthcare Research Ethics Committee. Study results will provide an initial understanding of how, in what contexts, and why quality dashboards lead to improvements in care quality. These will be disseminated to academic audiences, study participants, hospital IT departments and national audits. If the results show a trial is feasible, we will disseminate the QualDash software through a stepped wedge cluster randomised trial.