Exploring variation in the use of feedback from national clinical audits: a realist investigation

Abstract

Background National Clinical Audits (NCAs) are a well-established quality improvement strategy used in healthcare settings. Significant resources, including clinicians’ time, are invested in participating in NCAs, yet there is variation in the extent to which the resulting feedback stimulates quality improvement. The aim of this study was to explore the reasons behind this variation. Methods We used realist evaluation to interrogate how context shapes the mechanisms through which NCAs work (or not) to stimulate quality improvement. Fifty-four interviews were conducted with doctors, nurses, audit clerks and other staff working with NCAs across five healthcare providers in England. In line with realist principles we scrutinised the data to identify how and why providers responded to NCA feedback (mechanisms), the circumstances that supported or constrained provider responses (context), and what happened as a result of the interactions between mechanisms and context (outcomes). We summarised our findings as Context+Mechanism=Outcome configurations. Results We identified five mechanisms that explained interactions between providers and NCA feedback: reputation, professionalism, competition, incentives, and professional development. Underpinned by the mechanisms professionalism and incentives, feedback was used most routinely within clinical services resourced to maintain local databases, where data were stored before upload to NCA suppliers. Local databases enabled staff to access data easily, customise reports and integrate them within governance processes. Use of feedback generated in this way was further supported where staff supporting audit participation were trusted to collect timely and accurate data. Feedback produced by NCA suppliers, which included national comparator data, was used in a more limited capacity. Challenges accessing data from NCA supplier databases, concerns about the quality of data across participating organisations and timeliness were reported to constrain the perceived usefulness of this type of feedback as a tool for stimulating quality improvement. Conclusion The findings suggest that there are a number of mechanisms through which healthcare providers, in particular staff within clinical services, engage with NCA feedback, but that there is variation in the mode, frequency and impact of these interactions. Feedback was used most routinely within clinical services resourced to maintain local databases, where data were considered timely, trusted as accurate and could be easily accessed to customise reports for the needs of the service.