Author s response to reviews Title: How efficient are Referral Hospitals in Uganda? A Data Envelopment Analysis and Tobit Regression Approach Authors: Paschal Mujasi (Pmujasi@yahoo.co.uk) Eyob Asbu (zeyob@yahoo.com) Jaume Puig-Junoy (jaume.puig@upf.edu) Version: 1 Date: 05 Mar 2016 Author s response to reviews: Reviewer #1 The authors studied hospital efficiency in Uganda. The study can be potentially important to Uganda healthcare sector as well as other countries alike. The adopted methodology seems appropriate. And the findings are interesting. Issue 1: However, the writing and organization of this article need to be significantly improved. The current version is much longer than necessary. For example, there is way too much information on the background of efficiency evaluation. The authors only need to provide brief discussions and relevant references as opposed to repeating the literature. Thanks for this feedback. The article has been re-organized, moving some of the information from one section to another (e.g. conceptual framework and discussion of efficiency concepts into the background section. Sub-headings have also been introduced mainly in the background section. Most of the background information on the efficiency concepts has been removed, leaving the minimum required to orient the reader on these concepts. Issue 2: The description of analysis methodology also needs to be significantly improved. Please more clearly describe the steps taken and remove unnecessary background information and discussions.
The information on the analysis section has been re-organized into two sections (First Stage DEA and Second Stage Econometric analysis), describing in some detail the steps involved in each stage. Also most of the description information of the analytic tests conducted has been removed. Issue 3: The hospitals are from different regions of the country. I assume that there is significant across-region variation. It is not completely clear how such variation is accounted for. If not, please explain why. Response During the second stage Tobit regression analysis, the contextual factors considered included distance of the hospital from the capital city (proxy of whether hospital is urban or rural) and total catchment population for the hospital. These help to account for some of the cross region variations. The final model presented in the article was selected based on the significance of the Chi Square for the model and did not contain these cross regional variation. Initially, the contextual factors not in the final model had not been included in the article. These have now been included (Table 1). Also the article has been updated under the Methods Section to explain that several models including various combination of external factors (including cross regional factors) were run but only one containing a few of the factors was selected. This is shown in highlighted text. Issue 4: The results also need to be better presented. For example Table 5 only provides the raw results. It is not how you present in a published paper. The table has been updated by removing some of the raw outputs and only maintaining those highlighted in the results section Issue 5: The problem of small sample size (although understandable) needs to be more extensively discussed. In the same line, sensitivity analysis may need to be conducted.
The issues of small sample size has been discussed in the limitations sections, highlighting the problem of small sample size in DEA (may result in many hospitals becoming efficient by default as a result of not having a comparator from within the small sample) and also how it was mitigated in this study (minimization of the input and output variables and inclusion of a high proportion of the population of regional referral hospitals in the sample). We have also highlighted a prior study that had a sample size not much different from that in our study. The addition is in blue text. Agreed; it would have been nice to conduct Sensitivity Analysis say by using alternative outputs and inputs for the DEA or Corrected Ordinary Least Squares instead of Maximum Likelihood for the Tobit analysis to test the robustness of our analysis. Absence of complete compressive data on all other inputs and outputs precludes sensitivity analysis for the DEA. Reviewer #2: TOPIC: How efficient are Referral Hospitals in Uganda? A Data Envelopment Analysis and Tobit Regression Approach. General Comment: The study is an interesting one, as far as measurement of hospital efficiency in one of sub- Saharan African countries is concerned. However, the paper was unable to fairly address its research questions and there are wrong inferences from the results. Specific Comments: 1. Background: Issue 1: The assumption that only 26% of Uganda's health expenditures are attributed to hospitals is most unlikely to be correct. This is not even true for most high and medium income countries.
The quoted figure of 26% is from Uganda s most recently published National Health Accounts (FY 2008/09 and FY 2009/10)[1]. The figure refers to proportion of TOTAL Health Expenditure (from all sources) and not just Government spending in terms of health budget. It includes for example out of pocket health spending (which is a high proportion-21% of Percapita health expenditure [1] due to lack of health insurance schemes) and DONOR direct spending (a significant proportion of health spending) which is mainly for public health programs and is mostly targeted at primary health facilities. Also, given that the number of hospitals in the country is very low (129 [3], for a population of about 34.856million [4]) it is not surprising that only about 26% of Uganda s TOTAL health expenditures are attributed to hospitals. Not surprisingly, this figure is likely to be higher in most high and medium countries either due to more availability of hospitals and health insurance. This as earlier indicated is not the case in Uganda which is still a low income country. We have included in the manuscript information on the proportion of Government Current Health Expenditure (GCHE) which is spent of hospitals; and indeed this is higher at 37% [2]. This information is included in blue text in the background section Issue 2: The study was not designed to address its first research question, which says: "were the regional referral and large private not for profit (PNFP) hospitals in Uganda relatively technically efficient in FY 2012/2013?''. To effectively answer this the study need to compare their efficiency overtime using additional two or more Financial years (FYs), either by using window DEA or a DEA based malmquist productivity index efficiency change over time. We used the word relatively since DEA by its nature measures relative efficiency (compared to a group in a homogeneous sample) and not absolute efficiency. Thus, a hospital efficient in this study may turn out to be inefficient if other hospitals are added to the sample. It is however true that the use of the phrase relatively technically efficient in FY 2012/2013 may imply comparison to previous financial years and thus necessitate the use of other techniques as mentioned by the reviewer. However, comparisons over time periods was not the intent of the study.
The study question has thus been rephrased to read: What was the technical efficiency of regional referral and large private not for profit (PNFP) hospitals in Uganda in FY 2012/2013? We believe this better reflects the intentions of the study which was to estimate technical efficiency at a point in time for the hospitals in our sample. 2. Method: Issue 1: The inclusion of theoretical framework of DEA and of hospital as a production unit fit better not under methodology but in the background section of the presentation. Thanks for this suggestion. These aspects have been moved to the background section. Accordingly subsections for these have been created in the background section Issue 2: None of the inputs/outputs variables used for the study reflect the primary attribute or function of referral hospital, which include specialist treatments. The inputs selected include beds and staff and it s difficult to imagine any hospital including referral hospitals not using these. Of course the caliber of staff at this level is likely to be more qualified and specialized; and it would be preferable if possible, to categories the staff (say into specialists Vs generalists) as opposed to lumping them as we did in this study. This is a weakness we highlight in our discussion Regarding the outputs, we considered what is actually reported by the referral hospitals to the Ministry of Health and is thus a reflection of the actual outputs of the hospitals. It is also important to note that in Uganda given the limited number of hospitals, the referral hospitals besides providing specialist treatment also provide the services offered by general hospitals. This fact is already highlighted in the background section.
Also classifications are sometimes misleading. In Africa, many hospitals classified as regional referral hospitals actually operate as district hospitals for various reasons, such as absence of gate-keeper mechanism and loose referral systems, lack of subsidiary health facilities tending to the simple clinical cases etc. We report in the manuscript that data was assembled for 5 outputs (OPD visits, in-patient days, deliveries, major operations and immunizations). Based on completeness of available data, final selection was limited to 2 outputs: outpatient department (OPD) visits and in-patient days as reported by the hospitals to MOH. We presume these capture most of the hospital activities. For example surgeries, which are a major output for the referral hospitals are subsumed in inpatient and outpatient care (if day case surgeries). We thus believe that these more comprehensively and realistically capture most of the activities of the hospitals in our sample Issue 3: No mention of how overall (constant return to scale) technical efficiency was computed. As mentioned in the manuscript, we adopted the Variable Returns to Scale (VRS) model under the assumption that in practice there are variable returns to scale and not all hospitals are operating at an optimal scale. The use of the CRS specification when all firms are not operating at the optimal scale results in measures of TE which are confounded by scale efficiencies. The use of the VRS specification permits the calculation of technical efficiency devoid of these Scale efficiency effects. Thus we focused on estimating VRS technical efficiency scores since this seemed more realistic for the study context and did not see the need to estimate CRS technical efficiency (overall technical efficiency) Issue 4: The explanatory or independent variables used (see table 1) were mainly institutional; no contextual or external environmental factors such as population of catchment area, distance from city centre, ownership, etc were used for second stage DEA (Tobit regression). Thus, the study could not answer the third research question well enough.
Indeed during the second stage Tobit regression analysis, the contextual factors considered were more than those earlier presented in Table 1 and included population size of the catchment area (expressed as a categorical variables). More contextual factors would have been preferable. However, we have two constraints: (i) lack of data due to under-developed information systems; and (ii) the ensuing problem of degrees of freedom due to the small sample size We had not mentioned in our manuscript that the model presented was selected based on the significance of the Chi Square for the various models we run and so contained only some of the considered external factors. Accordingly the contextual factors not in the final model had not been included in the article and in Table 1. This was an omission. All the explanatory factors considered have all now been added Table 1 including the catchment population and distance from the city. Also the manuscript has been updated under the Methods Section to explain that several models including various combination of external factors were run but only one containing a few of the factors was selected. This is shown in highlighted text. Issue 5: The choice of output orientation for the study is faulty; since hospital mangers are better positioned to influence productivity or efficiency through appropriate input mix. Responses: Whereas it may be true that in a number of contexts hospital managers are better positioned to influence productivity or efficiency through appropriate input mix thus justifying use of an in-put orientation, as explained in the manuscript, this is not necessarily the case in Uganda. As we mention in our manuscript, the choice of using an output-oriented model was guided by the fact that the hospitals in our sample have a more or less fixed quantity of inputs and managers are expected to produce as much output as possible. Even when inputs such as beds and staff are underutilized, it is not within the managers power to dispose of them. Thus the hospital managers have no control over the size of the hospitals they run. Additionally, in the Ugandan context, the staffing capacity of each hospital is determined centrally by the MOH or hospital managing authority, and thus individual hospital managers do not have any control over the size of the health workforce which is another of the inputs considered in our study. Also,
given the existence of unmet need and low quality of care, we wanted to investigate the potential efficiency savings that can be used to expand care and/or improve quality. Indeed according to Coelli [5], where DMUs are given a fixed quantity of resources (inputs) and asked to produce as much output as possible, an output orientation is more appropriate. 3. Result: Issue 1: The variation in input (hospital beds: 100-482; and medical staff: 97-453) is wide, against the allusion under study population that the hospitals have same capacity. This observation is correct; the text has been amended accordingly by removing the phrase similar capacity. What is now reflected is that the hospitals are similar in terms of range of services offered and this is what qualifies them as referral hospitals Issue 2: The repeated use of "variable return to scale technical efficiency" can simply be replaced with ''pure technical efficiency" in the report. Thanks for this suggestion, the manuscript has been updated accordingly Issue 3: Most of the inferences made under result should be moved to discussion section. Response Thanks for the suggestion. Specifically the inference on the required output increases has been removed and moved to the discussion section. We have however maintained text that we believe just explains or interprets the results. Issue 4: The mean pure technical and scale efficiencies of inefficient hospitals were not computed, hence the deduction made for the inefficient hospitals using general mean (for TE and SE) are wrong.
It would have been helpful if the reviewer specifically highlighted some of the deductions referred to. From our perspective, no deductions were made for the inefficient hospitals based on the general mean (TE and SE) for the whole sample. For example, a key deduction made for the inefficient hospitals was about input and output increases/decreases required and as explained in the manuscript these were calculated using individual hospital efficiency scores. 4. Discussion: Issue 1: The paper failed to highlight the important inferences and outcome of the research under this section. Some presumptions were included as part of the discussion and these were outside the outcome of the study. To the contrary, our view is that the discussion section presents the main inferences from the study, specifically in terms of the variations in hospital efficiency, the required output increases, factors explaining the inefficiency, possible policy action. However, the study also provides some commentary in terms of contextualizing this study to studies conducted elsewhere and also the demonstration of the application of DEA to routinely available data to assess hospital efficiency by policy makers. References 1. Government of Uganda. National Health Accounts FY 2008/09 and FY 2009/10. Ministry of Health 2013 2. Government of Uganda. National Health Accounts: Key Messages (FY 2010/11-2011/12). Planning Department, Ministry of Health 2015 3. Government of Uganda. Health sector strategic and investment plan- Promoting people s health to enhance socio-economic development 2010/11-2014/15. Ministry of Health 2010
4. Government of Uganda. National Housing and Population Census. Uganda Bureau of Statistics (UBOS) 2014 5. Coelli TJ: A guide to DEAP Version 2.1: a data envelopment analysis computer 1. programme. CEPA Working Paper No. 8/96. Armidale: University of New England; 1996.