Reviewing studies with diverse designs: the development and evaluation of a new tool
Sirriyeh, R. (See also Harrison, R.) ; Lawton, R. ; ; Armitage, Gerry R.
Sirriyeh, R. (See also Harrison, R.)
Lawton, R.
Armitage, Gerry R.
Publication Date
2012-08
End of Embargo
Supervisor
Rights
Peer-Reviewed
Yes
Open Access status
closedAccess
Accepted for publication
2011-01-20
Institution
Department
Awarded
Embargo end date
Collections
Additional title
Abstract
RATIONALE, AIMS & Tools for the assessment of the quality of research studies tend to be specific to a particular research design (e.g. randomized controlled trials, or qualitative interviews). This makes it difficult to assess the quality of a body of research that addresses the same or a similar research question but using different approaches. The aim of this paper is to describe the development and preliminary evaluation of a quality assessment tool that can be applied to a methodologically diverse set of research articles. METHODS: The 16-item quality assessment tool (QATSDD) was assessed to determine its reliability and validity when used by health services researchers in the disciplines of psychology, sociology and nursing. Qualitative feedback was also gathered from mixed-methods health researchers regarding the comprehension, content, perceived value and usability of the tool. RESULTS: Reference to existing widely used quality assessment tools and experts in systematic review confirmed that the components of the tool represented the construct of 'good research technique' being assessed. Face validity was subsequently established through feedback from a sample of nine health researchers. Inter-rater reliability was established through substantial agreement between three reviewers when applying the tool to a set of three research papers (kappa = 71.5%), and good to substantial agreement between their scores at time 1 and after a 6-week interval at time 2 confirmed test-retest reliability. CONCLUSIONS: The QATSDD shows good reliability and validity for use in the quality assessment of a diversity of studies, and may be an extremely useful tool for reviewers to standardize and increase the rigour of their assessments in reviews of the published papers which include qualitative and quantitative work.
Version
No full-text in the repository
Citation
Sirriyeh R, Lawton R, Gardner P et al. (2012) Reviewing studies with diverse designs: the development and evaluation of a new tool. Journal of Evaluation in Clinical Practice. 18(4): 746-752.
Link to publisher’s version
Link to published version
Link to Version of Record
Type
Article