Loading...
Methods for the analysis of ordinal response data in medical image quality assessment
Keeble, C. Baxter, P.D. Gislason-Lee, Amber J. Treadgold, L.A. Davies, A.G.
Keeble, C.
Baxter, P.D.
Gislason-Lee, Amber J.
Treadgold, L.A.
Davies, A.G.
Publication Date
2016-04
End of Embargo
Supervisor
Rights
© 2016 The Authors. Published by the British Institute of Radiology. Reproduced in accordance with the publisher's self-archiving policy.
Peer-Reviewed
Yes
Open Access status
openAccess
Accepted for publication
2016-03-10
Institution
Department
Awarded
Embargo end date
Collections
Additional title
Abstract
The assessment of image quality in medical imaging often requires observers to rate images for some metric or
detectability task. These subjective results are used in optimization, radiation dose reduction or system comparison
studies and may be compared to objective measures from a computer vision algorithm performing the same task. One
popular scoring approach is to use a Likert scale, then assign consecutive numbers to the categories. The mean of these
response values is then taken and used for comparison with the objective or second subjective response. Agreement is
often assessed using correlation coefficients. We highlight a number of weaknesses in this common approach, including
inappropriate analyses of ordinal data and the inability to properly account for correlations caused by repeated images or
observers. We suggest alternative data collection and analysis techniques such as amendments to the scale and
multilevel proportional odds models. We detail the suitability of each approach depending upon the data structure and
demonstrate each method using a medical imaging example. Whilst others have raised some of these issues, we
evaluated the entire study from data collection to analysis, suggested sources for software and further reading, and
provided a checklist plus flowchart for use with any ordinal data. We hope that raised awareness of the limitations of the
current approaches will encourage greater method consideration and the utilization of a more appropriate analysis. More
accurate comparisons between measures in medical imaging will lead to a more robust contribution to the imaging
literature and ultimately improved patient care.
Version
Published version
Citation
Keeble C, Baxter PD, Gislason-Lee AJ et al (2016) Methods for the analysis of ordinal response data in medical image quality assessment. British Journal of Radiology. 89(1063): 20160094.
Link to publisher’s version
Link to published version
Link to Version of Record
Type
Article