• Exploring variation in the use of feedback from national clinical audits: a realist investigation

      Alvarado, Natasha; McVey, Lynn; Greenhalgh, J.; Dowding, D.; Mamas, M.; Gale, C.; Doherty, P.; Randell, Rebecca (2020)
      Background National Clinical Audits (NCAs) are a well-established quality improvement strategy used in healthcare settings. Significant resources, including clinicians’ time, are invested in participating in NCAs, yet there is variation in the extent to which the resulting feedback stimulates quality improvement. The aim of this study was to explore the reasons behind this variation. Methods We used realist evaluation to interrogate how context shapes the mechanisms through which NCAs work (or not) to stimulate quality improvement. Fifty-four interviews were conducted with doctors, nurses, audit clerks and other staff working with NCAs across five healthcare providers in England. In line with realist principles we scrutinised the data to identify how and why providers responded to NCA feedback (mechanisms), the circumstances that supported or constrained provider responses (context), and what happened as a result of the interactions between mechanisms and context (outcomes). We summarised our findings as Context+Mechanism=Outcome configurations. Results We identified five mechanisms that explained interactions between providers and NCA feedback: reputation, professionalism, competition, incentives, and professional development. Underpinned by the mechanisms professionalism and incentives, feedback was used most routinely within clinical services resourced to maintain local databases, where data were stored before upload to NCA suppliers. Local databases enabled staff to access data easily, customise reports and integrate them within governance processes. Use of feedback generated in this way was further supported where staff supporting audit participation were trusted to collect timely and accurate data. Feedback produced by NCA suppliers, which included national comparator data, was used in a more limited capacity. Challenges accessing data from NCA supplier databases, concerns about the quality of data across participating organisations and timeliness were reported to constrain the perceived usefulness of this type of feedback as a tool for stimulating quality improvement. Conclusion The findings suggest that there are a number of mechanisms through which healthcare providers, in particular staff within clinical services, engage with NCA feedback, but that there is variation in the mode, frequency and impact of these interactions. Feedback was used most routinely within clinical services resourced to maintain local databases, where data were considered timely, trusted as accurate and could be easily accessed to customise reports for the needs of the service.
    • How, in what contexts, and why do quality dashboards lead to improvements in care quality in acute hospitals? Protocol for a realist feasibility evaluation

      Randell, Rebecca; Alvarado, Natasha; McVey, Lynn; Greenhalgh, J.; West, R.M.; Farrin, A.; Gale, C.; Parslow, R.; Keen, J.; Elshehaly, Mai; et al. (2020-02-25)
      National audits are used to monitor care quality and safety and are anticipated to reduce unexplained variations in quality by stimulating quality improvement (QI). However, variation within and between providers in the extent of engagement with national audits means that the potential for national audit data to inform QI is not being realised. This study will undertake a feasibility evaluation of QualDash, a quality dashboard designed to support clinical teams and managers to explore data from two national audits, the Myocardial Ischaemia National Audit Project (MINAP) and the Paediatric Intensive Care Audit Network (PICANet). Realist evaluation, which involves building, testing and refining theories of how an intervention works, provides an overall framework for this feasibility study. Realist hypotheses that describe how, in what contexts, and why QualDash is expected to provide benefit will be tested across five hospitals. A controlled interrupted time series analysis, using key MINAP and PICANet measures, will provide preliminary evidence of the impact of QualDash, while ethnographic observations and interviews over 12 months will provide initial insight into contexts and mechanisms that lead to those impacts. Feasibility outcomes include the extent to which MINAP and PICANet data are used, data completeness in the audits, and the extent to which participants perceive QualDash to be useful and express the intention to continue using it after the study period. The study has been approved by the University of Leeds School of Healthcare Research Ethics Committee. Study results will provide an initial understanding of how, in what contexts, and why quality dashboards lead to improvements in care quality. These will be disseminated to academic audiences, study participants, hospital IT departments and national audits. If the results show a trial is feasible, we will disseminate the QualDash software through a stepped wedge cluster randomised trial.
    • Institutional use of National Clinical Audits by healthcare providers

      McVey, Lynn; Alvarado, Natasha; Keen, J.; Greenhalgh, J.; Mamas, M.; Gale, C.; Doherty, P.; Feltbower, R.; Elshehaly, Mai; Dowding, D.; et al. (2021-02)
      Healthcare systems worldwide devote significant resources towards collecting data to support care quality assurance and improvement. In the United Kingdom, National Clinical Audits are intended to contribute to these objectives by providing public reports of data on healthcare treatment and outcomes, but their potential for quality improvement in particular is not realized fully among healthcare providers. Here, we aim to explore this outcome from the perspective of hospital boards and their quality committees: an under-studied area, given the emphasis in previous research on the audits' use by clinical teams. Methods: We carried out semi-structured, qualitative interviews with 54 staff in different clinical and management settings in five English National Health Service hospitals about their use of NCA data, and the circumstances that supported or constrained such use. We used Framework Analysis to identify themes within their responses. Results: We found that members and officers of hospitals' governing bodies perceived an imbalance between the benefits to their institutions from National Clinical Audits and the substantial resources consumed by participating in them. This led some to question the audits' legitimacy, which could limit scope for improvements based on audit data, proposed by clinical teams. Conclusions: Measures to enhance the audits' perceived legitimacy could help address these limitations. These include audit suppliers moving from an emphasis on cumulative, retrospective reports to real-time reporting, clearly presenting the “headline” outcomes important to institutional bodies and staff. Measures may also include further negotiation between hospitals, suppliers and their commissioners about the nature and volume of data the latter are expected to collect; wider use by hospitals of routine clinical data to populate audit data fields; and further development of interactive digital technologies to help staff explore and report audit data in meaningful ways.