Author's response to reviews Title: Differences between patients' and clinicians' report of sleep disturbance: A field study in mental health care in Norway. Authors: Håvard Kallestad (havard.kallestad@ntnu.no) Bjarne Hansen (bjarne.hansen@ntnu.no) Knut Langsrud (knut.langsrud@ntnu.no) Torleif Ruud (torleif.ruud@medisin.uio.no) Gunnar Morken (gunnar.morken@ntnu.no) Tore C Stiles (tore.stiles@svt.ntnu.no) Rolf W Gråwe (rolf.w.grawe@rus-midt.no) Version: 2 Date: 30 September 2011 Author's response to reviews: see over
Dear Dr. Benca Thank you for considering and offering a constructive review of our paper entitled Differences between patients' and clinicians' report of sleep disturbance: A field study in mental health care in Norway. We have now carefully considered the comments of the two reviewers. Below we have specified each comment followed by a description of how we have handled each of the comments. We hope we have satisfactorily addressed these questions both in this letter and in the revised manuscript. We hope the manuscript is now suitable for publication in BMC psychiatry. Although we understand that it is your policy to receive the written consent of all patients in the study, please note that there were more than 42.000 patients included in this study. We have unfortunately not been able to present these consents. Please also note that we have added one person whom we would like to thank in the acknowledgements. On behalf of the authors, Håvard Kallestad.
Reviewer 1: David N Neubauer Reviewer's Comment: The authors conclude that sleep disturbance is imprecisely recognized and treated relative to the patients experience of sleep disturbance. They argue that there is a significant discrepancy between patient and clinician reports of sleep disturbance. These findings are less compelling. To assess patient-reported and clinician-reported sleep disturbances the investigators employed two different data sets that incorporated survey results including sleep-related items. Unfortunately, there were limitations inherent in the clinician survey instruments regarding the selection of mental and behavior problems, including sleep disturbance. It is not clear that this data truly represents the findings that would have resulted from a survey directly inquiring about sleep symptoms. The survey limitations weaken the usefulness of the calculated patient-clinician agreement and predictive values. : This is an important comment. We agree that we might have gotten different results if we had utilized a survey directly inquiring about sleep symptoms. We have therefore added that to the discussion of the limitations of the clinician rated sleep item. The study was not originally designed to measure sleep in mental health care. Thus, the survey instruments were not optimal. This discussion now reads (page 13): The study was not specifically designed to measure sleep disturbance in mental healthcare and we might have found a higher prevalence of clinician-rated sleep disturbance if we had utilized a survey directly inquiring about sleep symptoms. This could have resulted in higher agreement and different predictive values for clinician s evaluations. Reviewer's Comment: The authors also state, The low recognition of sleep disturbance raises questions concerning education and training of healthcare professionals. While this conclusion may be accurate, the study primarily shows that diagnostic coding of insomnia is low, not necessarily that recognition and treatment are low. As noted above, methodological problems limit an accurate assessment of physician identification of sleep problems. Moreover, there is simply no indication of how many patients were treated for their insomnia with alternate medications or how many received non-pharmacologic therapies.
We agree that there are no indications of how many patients who were treated with other therapies and we have included the following in the limitations (page 14): Finally, although we have assessed patients who received hypnotic medication, there might be a proportion of the patients who received alternate medication or nonpharmacological interventions for their sleep disturbance. We have also made changes to the Conclusions accordingly and deleted the conclusion about imprecise treatment. It was (page 14): When patients meet the criteria for a mental disorder, insomnia is almost never diagnosed, and sleep disturbance is imprecisely recognized and treated relative to the patients experience of sleep disturbance. It is now (page 14): ( ) sleep disturbance is imprecisely recognized relative to the patients experience ( ) However, we believe that our findings show that the agreement between clinicians and patients is low. We agree that the true prevalence of clinician rated sleep disturbance might be higher due to the above methodological issues. However, it is none the less a true finding that when clinicians do report sleep disturbance to be a major problem there is a noteworthy discrepancy between patients and clinicians reports. If we closely scrutinize the results, we see that, in agreement with the reviewer s point, the positive predictive value was higher in data-set 2 where the prevalence of clinician rated sleep disturbance was higher (Results p. 9, second paragraph under the heading Agreement ). However, this was not found for the patients whom regard their sleep disturbance as a prominent problem (results, p. 9 third paragraph under the heading Agreement ). Thus, the higher prevalence does not necessarily lead to higher precision of the patients who reports sleep disturbance as one of their most prominent problems. This is also interesting in the light of the ICD-10 criteria where it is recommended that insomnia should be coded as a comorbid diagnosis if it is one of the patient s most prominent problems. Reviewer 2: Philip Gehrman Reviewer's comment
Page 4, first paragraph of Methods. It is stated that information from clinical diagnostic evaluations was available for each data set. I m assuming this refers to just the diagnostic codes assigned, but it seems to imply that more detailed evaluation data were accessible. Please clarify. We agree that this might be ambiguous and we have changed the text accordingly so it now reads (page 4, first paragraph of the Methods section): Data-set 1 included demographic information and clinical diagnostic codes from an estimated 93% of all patients receiving treatment in mental healthcare, Reviewer s comment The authors acknowledge the potential for bias due to the low rates at which patients agreed to link their self-report information to their clinical data. Are there any variables available on which you could compare those who did and did not agree in order to assess for this possibility? Unfortunately, this is not available. However, we can compare the mean sum score of the HoNOS in this sample (mean score 10.2; sd = 4.9) with that of a comparable sample of 1688 patients from 8 psychotherapy out-patient clinics in the UK (mean score 8.9; sd = 5.2) (Audin et al., Brit J Psychiatry, 2001, 561-566). It seems our sample of patients had a little more severe problems than the UK sample. However, our sample consisted of about 11% in-patients and this might explain the higher HoNOS scores in our sample. Also, the mean score of the SCL-10 in the current sample was 2.4, which is well above a score of 1.85, which is recommended as a cut-off for mental disorders (Strand et al., Nord J Psychiatry, 2003, 113-118). Reviewer s comment Two limitations that are not sufficiently addressed either in the Methods or in the Discussion are 1. the reliance on single items to assess insomnia; We have now included a more thorough description of this in the methods section (page 6, second paragraph under the heading patient rated sleepdisturbance ):
Three items measure sleep disturbance on the SCL-90R. On the SCL-25 and the SCL- 10, these three have been reduced to one item measuring the severity of sleep disturbance. This item was used to assess patient rated level of sleep disturbance. And also in the Discussion section (pages 13-14): The use of a single item measuring sleep disturbance is another limitation. This means that it is not possible to discern if there were differences in agreement between sleep onset or sleep maintenance problems or if the patients experienced other kinds of sleep disturbance. Reviewer s Comment 2. is the fact that there were differences between the data collection for data sets 2 and 3 in terms of the form of the SCL used and the number of problems the clinician could rate on the HoNOS. The mention of these limitations does not sufficiently address the impact they may have on the conclusions that can be drawn from the data. We are uncertain about the comment about the SCL. All the items in the SCL-10 are found identically in the SCL-25, including the sleep item. We have therefore not addressed this specifically as a limitation. Regarding the use of the HoNOS, please see the above response to reviewer 1. We have now included a more thorough discussion of the limitations of the HoNOS. Reviewer Comment Related to this, please give them wording of the sleep item used from the SCL. The general text of the SCL: Please report how much each problem have disturbed you the last 14 days And then 10/25 symptoms are listed, including: Sleep problems (Rated: no disturbance, a little disturbance, quite severe disturbance, very severe disturbance).
Reviewer comment The Discussion was well-written and made appropriate interpretations of the data. As mentioned above, the limitations of the study need to be further emphasized. Please see our responses above. The limitations of the study have been emphasized. Reviewer comment Introduction, second paragraph. There is reference made to a study that tested if sleep disturbance is clinically recognized in patients with mental disorders [12]. What were the results of the study? We have included a more detailed description of the Berlin et al (1984) study (page 3, second paragraph): The authors found that 80 of 100 patients on a psychiatric consultation service had sleep disorders, but disturbed sleep was not mentioned in the records of 54% of these patients. However, the study was conducted in the early 1980s and had limited external validity. The state of sleep in psychiatry might be different 30 years later. Reviewer comment Methods. Were the data collection and study design, including the choice of questionnaires, specifically for the purpose of these analyses? Or were the data collected for other purposes and these analyses represent secondary analyses? The data collections were commissioned by the Norwegian Department of Health to evaluate the state of the National Mental Health Care System and were not designed specifically to evaluate sleep disturbance in patients with mental disorders. This is now stated in the second paragraph of the Methods section on p. 4. Reviewer comment
Results. The rationale for the chi-squared analyses is not clear. What were you hoping to learn? Because clinicians might be more likely to assess sleep disturbance in certain disorders, like depression or bipolar disorder, we conducted the chi-squared analyses to test if selected disorders also had higher levels of agreement. To make this more clear to the reader, we have changed this paragraph so the first paragraph on page 10 now reads: Because clinicians might be more likely to assess sleep disturbance in certain disorders, we conducted χ 2 for selected diagnostic groups (substance abuse, schizophrenia, bipolar disorder, depressive episode, recurrent depressive episode, anxiety disorders, adjustment disorders and personality disorders) in chapter V of the ICD-10 to test if there were differences in agreement in these patient groups. Reviewer comment I found the Venn diagrams to be very helpful. If space is a concern it would be reasonable to combine the data from data-sets 2 and 3 for a single figure. We have uploaded a Venn diagram where data-sets 2 and 3 are combined for a single figure if space is a concern. However, if this is not a concern, we recommend using both the original figures for the following reasons. 1. As noted above, clinician rated sleep disturbance was coded somewhat differently in the two data-sets and we therefore believe it is most true to the data to present them as two figures. 2. In Figure 2, from data-set 3, we have presented patients with patient rated sleep disturbance as one of their most prominent problems. This, in combination with figure 1, shows the finding that there was less agreement if the patient rated sleep disturbance one of the most prominent problems. One additional change not requested by the reviewers: Two sentences, on page 12 of the original manuscript have been edited to be more cautious and to avoid unnecessary repetition of findings: It was: However, our finding that when clinicians do evaluate sleep disturbance to be a
prominent problem, there is still a low agreement between patients and clinicians is therefore all the more disturbing. Especially as the clinicians predictive value was less than chance for the patients who reported sleep disturbance as one of their most prominent problems. It is now (page 13, at the end of the second paragraph) However, even when clinicians do evaluate sleep disturbance to be one of the most prominent problems, there is still a very low agreement between patients and clinicians.