What Is Peer Review?

Transcription

1 What Is Peer Review? Scrutiny by an independent third party Provides an independent source of assessment Editor still makes the final decision 1

2 What is Peer Review? Peer review is a somewhat flawed process little evidence that it works full of easily identified defects However, there is no obvious alternative least worst process 2

3 Management of Submissions Returned to author 3

4 Peer review process Relevance of the manuscript Recommendation about publication Specific and detailed feedback 4

5 Add to literature? Evidence based? Overview Content overlooked? Plagiarism concerns? Article type? Structured abstract? Structure Concise? Clear and logical? Reporting guidelines? 5

6 Existing literature considered? Introduction Clear research question? Sound? Ethics approval? Methods Appropriate to question? Ethical concerns? 6

7 Appropriate analysis? Results Do figures contribute? Based on the findings presented? Discussion and Conclusion Limitations stated? Alternative explanations? 7

8 For example Randomised controlled trial of magnetic bracelets for relieving pain in osteoarthritis of the hip and knee Tim Harlow, Colin Greaves, Adrian White, Liz Brown, Anna Hart and Edzard Ernst BMJ 2004;329; bmj/resourcesreviewers/training materials Example of a useful peer review Adapted from bmj/resources reviewers/training materials 8

9 Peer reviewer 1 Well structured Clear questions for the authors Detailed and thoughtful response to results Two examples of peer review which could be improved Adapted from bmj/resources reviewers/training materials 9

10 Peer reviewer 2 This is a very interesting paper which addresses the difficult issue of performing RCT's of appliances in patients with osteoarthritis. Osteoarthrtis is a common condition that leads to significant pain, disability for the patients and impacts heavily on health care resources. Magnets would be an excellent addition to the physicians armoury if thought to be effective. This is predominently due to the safety and cost effectiveness of such a treatment. The trial has been well designed and presented. Although I appreciate the constraint imposed by the word limit, a few extra details would make the paper much easier to interpret. What was the split of clinical vs radiographic cases?. It says that nurses arranged x rays if needed. Does that mean that they were all x rayed? Was there any difference in response between clinical and radiographic cases? Was there a differential response between grades of x ray changes where available? Lack of structure Spelling mistakes What percentage of people had monoarthrits and what percentage polyarticular OA? To try and address the issue of pacebo response, it would be worth looking at the treatment effect in dicotomised subgroups of belief if that were possible. The main difficulty of this paper is teasing out a true treatment effect from the placebo effect which is almost certainly present. Although iam sure that it will be impossible to fully apportion effects to each of these, further discusiion would be helpful. I would also be interested to see a discussion about the relaitive importance of this issue in clinical practice with a simple, safe treatment Peer reviewer 3 The methods seem sound, it appears to be wellpowered and the outcome measures are appropriate. Discussion and the conclusions are reasonable. Parts of this paper are written so badly my 12 year old daughter could have done a better job of writing it The stats are a little complex for me to give a definitive opinion, sorry, so I suggest you run them past a more knowledgeable statistician. Vague Rude, not constructive Appropriate comment about their limitations as a reviewer 10

11 Useful Not useful Well organised Poorly structured Clear questions for the author Well written States limitations Vague Many spelling mistakes Rude The Report Address what you were asked to do Professional Maintain confidentiality Declare conflicts of interest Pleasant and polite Helpful Scientific Timely Organised 11

12 So what have we learnt? Peer review is only one step in the editorial process Peer reviewers need to be familiar with the content but not extreme experts Peer reviews need to be thoughtful, structured, polite and ask discrete questions Interested in joining the AFP Peer review team? Send your expression of interest + CV afp@racgp.org.au 12

13 Prof Steve Margolis: Senior Medical Editor Dr Glenn Duns: Medical Editor Dr Sarah Metcalfe: Medical Editor Dr Kate Thornton: Publications Fellow 13