Developing and Evaluating Criteria to Help Reviewers of Biomedical Informatics Manuscripts: A Case Report.

Ammenwerth E, Wolff AC, Knaup P, Ulmer H, Skonetzki S, Van Bemmel JH, McCray AT, Haux R, Kulikowski C

J Am Med Inform Assoc. 2003 June.

---

Abstract:

Peer-reviewed publication of scientific research results represents the most important means of their communication. The authors have annually reviewed a large heterogeneous set of papers in order to product the International Medical Informatics Association (IMIA)Yearbook of Medical Informatics. To support an objective and high-quality review process, the authors attempted to provide reviewers with a set of refined quality criteria, comprised of 80 general criteria and an additional 60 criteria for specific types of manuscripts. Authors conducted a randomized controlled trial, with 18 reviewers, to evaluate application of the refined criteria on review outcomes. While the trial found that reviewers applying the criteria graded papers more strictly (lower overall scores), and that junior reviewers appreciated the availability of the criteria, there was no overall change in the inter-rater variability in reviewing the manuscripts. The authors describe their experience as a 'case report', and provide a reference to the refined quality review criteria, without claiming that the criteria represent a validated instrument for quantitative quality measurement.

Ammenwerth E, Wolff AC, Knaup P, Ulmer H, Skonetzki S, Van Bemmel JH, McCray AT, Haux R, Kulikowski C. Developing and Evaluating Criteria to Help Reviewers of Biomedical Informatics Manuscripts: A Case Report. 
J Am Med Inform Assoc. 2003 June.

PDF | PMID