EUROPEAN COMMISSION EUROSTAT Directorate F: Social statistics Unit F-4: Income and living conditions; Quality of life Doc. LC-ILC/194/17/EN estat.f.4 (2017) WORKING GROUP ON INCOME AND LIVING CONDITIONS HOUSEHOLD BUDGET SURVEY 28 SEPTEMBER 2017 10.00 am BECH building (room Quetelet) - Luxembourg Transmission, processing and publication of HBS 2015 data
1 Introduction This document describes the state of play of the HBS 2015 wave processing, which encompass all operations from micro-data to EuroBase indicators etc., going from the transmission of files to Eurostat to dissemination. This document will serve as a basis for discussions with Member States in order to identify ways to improve the whole HBS process. The Working Group is invited to give feedback, and especially suggestions to improve the process. 2 HBS 2015 Wave 2.1 Data transmission Eurostat expects a reduction in the number of transmissions and re-transmissions of microdata files due to Countries respecting more closely the guidelines contained in the HBS 2015 Transmission document 1, as well as a reduction in the number of file-types transmitted and the absolute number of individual files, which will ensure a more efficient processing by Eurostat. An overview of the HBS 2015 data transmissions is presented in Table 1. To summarise: Different file-types : 3 (SAS, TSV, TEXT) Number of files : 138 (counting re-transmissions) Number of transmissions : 17 Countries sent 1; 4 sent 2; 1 sent 3 and 1 sent 4 transmissions 2.2 Timeliness According to the HBS WG meeting in June 2014, the national HBS data sets for the 2015 Wave should have been sent to Eurostat not later than December 31 st, 2016. This deadline for the transmission of clean micro-data files was necessary so that Eurostat could publish aggregated tables on the Eurostat Website by the end of 2017. By mid-september, 2017, HBS 2015 micro-data has been provided by 18 EU countries (BE, BG, CZ, DE, EE, EL, ES, HR, LV, HU, AT, NL, PL, PT, RO, SI, SK, LU) and 5 non-eu countries (MK, ME, RS, NO, XK). See Table 1. No tabular data has been transmitted so far for this Wave. Eurostat is awaiting HBS micro-data transmissions from 10 EU countries (DK, IE, CY, MT, IT, LT, SE, UK, FI, FR) and also from some non-eu countries. Could these countries please indicate when they are likely to finally deliver these datasets? 1 Description of the data transmission for Household Budget Survey (HBS) for the Reference Year 2015. Version 3 from 17th May 2016 CIRCABC Household Budget Survey (HBS) user group, Library, Round 2015 https://circabc.europa.eu
Table 1 Overview of HBS 2015 data transmissions Country Received Format No of files No of transmissions 1 CZ-Czech Republic 07/10/2016 TSV 4 1 2 EL-Greece 09/11/2016 TSV 4 1 3 DE-Germany 05/12/2016 SAS 3 1 4 BG-Bulgaria 05/12/2016 TSV 3 1 5 HR Croatia 16/12/2016 SAS 3 2 6 SI-Slovenia 22/12/2016 SAS 4 1 7 BE-Belgium 23/12/2016 SAS 4 1 8 ES-Spain 27/12/2016 SAS 3 1 9 RO-Romania 27/12/2016 SAS 3 1 10 LV-Latvia 30/12/2016 SAS 4 2 11 PL-Poland 05/01/2017 TSV 10 4 12 EE-Estonia 26/01/2017 SAS 4 1 13 NL-Netherlands 24/02/2017 TSV 3 2 14 HU-Hungary 31/03/2017 SAS 4 3 15 SK-Slovakia 30/05/2017 SAS 4 1 16 AT-Austria 08/06/2017 SAS 4 2 17 PT-Portugal 28/08/2017 SAS 4 1 18 LU-Luxembourg 08/09/2017 SAS 3 1 DK-Denmark Promised to send the data by the end of April 2017 IE-Ireland Promised to send the data during 2017 Q1 CY-Cyprus Promised to send the data during 2017 Q2 MT-Malta Promised to send the data during 2017 Q3 IT-Italy Promised to send the data during 2017 Q3 LT-Lithuania Promised to send the data during 2017 Q4 SE-Sweden No information UK-United Kingdom FI-Finland FR-France Promised to send the data during 2017 Q4 Promised to send the data during 2018 Q1 Promised to send the data during 2018 Q4 19 RS- Serbia 28/12/2016 TSV 4 1 20 MK-FYR-Macedonia 26/01/2017 SAS 4 1 21 ME-Montenegro 31/01/2017 SAS 3 1 22 NO-Norway 08/05/2017 SAS 3 1 23 XK-Kosovo 07/09/2017 TXT 3 1 2.3 Data Processing The data for 13 countries (BE, CZ, DE, EE, EL, ES, HR, LV, NL, RO, SI, HU, SK) have been validated fully, Indicators for Eurobase have been generated and the results have been approved by the corresponding NSI for publishing on the Eurostat Website.
2.4 Data quality and comparability issues in the HBS data files 2.4.1 Different HBS reference years Some countries have a different data collection year for the HBS 2015 Wave. A distinguishing feature of the EU-wide HBS is that country data are output-harmonized by adjusting expenditure/income variables to make them refer to the agreed Eurostat Reference year (1988, 1994, 1999, 2005, 2010, 2015). Just a reminder, that the next HBS Waves should take place in 2020 and 2025. The use of different HBS data collection years, as expected, creates comparability problems, both between individual countries for the HBS Wave and also when attempting to make comparisons with data from other statistical domains like EU-SILC or EU-LFS, where microdata are being collected more regularly. While most of the countries transmitted micro-data using 2015 as Survey Year, the HBS micro-data from Belgium and Croatia are from 2014, while the micro-data from Germany are from 2013. See Table 2. Table 2: HBS Data Collection years for the HBS 2015 wave Belgium Bulgaria Czech Republic Germany Estonia Greece Spain Croatia Luxembourg Latvia Hungary Netherlands Austria Poland Portugal Romania Slovenia Slovakia 2013 2014 2015 2016 Eurostat encourages countries to organize future HBS Surveys, so that they take place during the agreed Reference Year.
2.4.3 Non-Response items in the HBS and uncollected variables. A common problem in population surveys is the failure to collect complete information due to respondents unwillingness or lack of capability to provide a requested piece of information. The HBS micro-dataset facilitates high-quality cross country comparisons of private consumption in the EU countries. The list of variables and also the survey methodology is largely pre-defined, however there are significant differences across country surveys which impair cross country comparability. 2.4.3.1 Missing imputed rent For the HBS 2015 wave, all the countries should provide the imputed rent variable, as per ESA guidelines and as previously agreed for the HBS 2010 wave. This is even easier since this variable is mandatory for EU-SILC (EU Statistics on Income and Living Conditions). So far, only the Czech Republic has not provided imputed rent for the HBS 2015 wave. 2.4.3.2 Reporting of expenditure and income Expenditure and income amounts reported to Eurostat should be: in national currency (NAC), annualised and un-weighted The weighting of the sample should be provided separately by Member States (HA10), to be applied by Eurostat Household's consumption expenditure is to be supplied broken down by the COICOP 2013 (Classification of Individual Consumption by Purpose at five-digit level). In countries where it is not possible to obtain very detailed income data, the minimum objective is to provide Eurostat with the net disposable income of the household. Net disposable income is calculated using the total monetary net income (HH095) plus the three non-monetary income variables (HH012, HH023 and HH032). 2.4.3.3 Some comments on derived variables required for the generation of Eurobase Indicators. If variables which are key dimensions for some of the Eurostat HBS Tables in EuroBase, are not provided by a minority of countries, then the quality of the tables is affected and it is impossible to produce accurate EU-/EA- Aggregates. e.g. HC24, HD20, HH032, HB074, HB05, HB062, HC04 and HA09. 2.4.3.4 Some comments on other variables. Countries ought to make an effort to supply the HJ variables (Cross-border Expenditure); Income variables i.e. HH012 & HH023; and HQ variables (Consumption in quantities).
2.5. Dissemination The data dissemination foreseen by Eurostat for the HBS 2015 wave is the following: 2.5.1 Publishing the first version of the aggregated tables based on micro-data from less than 28 Member States on the Eurostat Website by Oct 2017. All tables for a given country will be uploaded to the Eurostat Website as soon as they are validated by the respective NSI. EU- /EA- Aggregates cannot be produced until data has been received by all Member States. 2.5.2 Preparation and publication of the "EU-wide HBS 2015 Quality Report" by Eurostat by the end of 2018 Q1. 2.5.3 Finally, preparation of Anonymised Micro-Datasets for Scientific Research. Conclusions The Working Group is invited to take note of these points: The Working Group is invited to give feedback, and especially suggestions to make the processing of HBS micro-data more efficient and shorten the time delay for publishing EU Indicators. Eurostat encourages countries to organize future HBS Surveys, during the agreed Reference Year. Eurostat asks countries to respect more closely the guidelines contained in the HBS Transmission document. Eurostat recommends using standardised file formats (SAS) for the transmission of the HBS data to Eurostat. Countries are reminded that amounts reported to Eurostat should be in national currency (NAC), annualised and un-weighted. Eurostat requests that countries aim to reduce the frequency of missing values in the HBS micro-datasets transmitted to Eurostat. Eurostat requests that all countries provide the imputed rent variable. The EU countries that have not managed to deliver micro-data for the HBS 2015 Wave yet, will notify Eurostat when they will finally send these data.