Capturing and analysing the impact of MRC funded research

Transcription

1 Capturing and analysing the impact of MRC funded research Ian Viney Head of Evaluation, MRC Strategy Group March 2011

2 MRC Evaluation Programme Progress so far Capture and analyse the results of MRC funding Develop approaches to assess progress, productivity and quality of MRC research output at an aggregated (portfolio) level Provide information to strengthen the case for medical research Develop quantitative and qualitative information about the impact of MRC research

3 MRC Evaluation Programme Priorities for 2011/12 Strengthen strategy development and decision making Support discussions at Strategy Board, measure progress with the strategic plan Improve understanding of economic/academic/societal impact and what might lead to it Identify improved indicators, encourage research on the science of science Support partnership working Collaborate with other funding agencies internationally

4 Funding for Research and Training MRC Evaluation Framework INPUT OUTPUTS/OUTCOMES Generation of new knowledge Development of collaborative networks Leveraging funding Dissemination of research Research materials Intellectual property/licensing New products/interventions Changes to policy IMPACTS Improvements to health (living longer and with better quality of life) Improving the performance of existing businesses Attracting R&D investment from global business Improving public policy and public services Creating new businesses Delivering highly skilled people to the labour market Academic impact (effects on further research including other disciplines)

5 MRC e-val Capturing output/outcome/impact MRC e-val is an online database used to gather information about outputs/outcomes/impacts arising from MRC research MRC e-val is building a robust dataset of evidenced output which allows progress/productivity and quality of MRC research to be analysed, and this information to be utilised in decision making MRC e-val is used each year to collect updated, structured feedback from researchers throughout the lifetime of their MRC funding, and beyond Take a holistic approach, include papers and patents, but aim for a much wider range of output types E-Val data can set case studies in context of the productivity and quality of all MRC output. Comprehensive data obtained across the whole MRC portfolio A baseline dataset against which progress and trends can be prospectively tracked over time

6 MRC e-val 2010 Feedback from all researchers that have held MRC funding since 2006 (3714 MRC awards to over 2800 scientists) Includes fellowships, grants and Unit/Institute programmes Significant new system developments between 2009 and 2010 have made data entry and administration of the system easier, speeding up data cleaning, analysis and publication of results Submitted responses for 91.4% of funding agreements (3339/3655) obtained in e-val These responses correspond to 92.3% of MRC spend between 2006/07 and 2009/10 ( 1.88/ 2.0 billion in RDW) STFC and Chief Scientist Office (CSO) Scotland have implemented e-val

7

8

9 MRC e-val 2010 dataset Example Outputs/Outcomes/Impacts 37,500 publications reported (30,000 papers between ) Details of 5,000 active collaborations involving 6,000 partners Over 200 published patents (roughly 30% are licensed) Over 50 spin out companies with an evidenced link to MRC research since 2006 Over 130 citations in policy documents, including 30 citations in NICE guidelines since new products and interventions in development, around 40 launched onto the market since 2006, including around 10 new drugs 340m of inward investment to UK research and development from overseas and private sector research funding between 2006 and 2010

10 Impact Profile of MRC Publications (Rebased Impact to end 2009 Data & analysis: Evidence, Thomson Reuters)

11 Network analysis of collaborations within MRC Regenerative Medicine portfolio

12 MRC Research has contributed to the creation and growth of new companies

13 Examples of changes in policy/practice (2009) Martin Dennis, Edinburgh findings of the CLOTS trial; stockings not effective at preventing venous thromboembolism in acute stroke patients - impact on NICE and international clinical guidelines and treatment affecting approximately 80,000 people a year in the UK Paresh Vyas, MRC Molecular Haematology Unit (British Journal of Haematology paper) recommended newborns with Down s Syndrome should have a full blood count to screen for a preleukaemic condition now most do (approximately 750 each year in the UK). Simon Thompson, MRC Biostatistics Unit, helped to develop the basis for the UK national screening programme for abdominal aortic aneurysms (AAA). AAAs affect 4% of men aged (approximately 80,000). Ray Waters, University of Cardiff, is deputy chair of the Committee on Medical Aspects of Radiation in the Environment (COMARE) - which has advised on the risks of sunbeds and made recommendations regarding legislation for the use of sun parlours the misuse of which is estimated to lead to 100 additional deaths a year from skin cancer in the UK.

14 Influences on policy/practice Pathways to Impact Potential pathways Researchers participate in policy setting processes and/or Research is cited in policy documents Specific pathway Research Citation in policy paper document Approaches for analysis Informs changes to Clinical practice/ economy etc. ROD PubMed Over 200 guideline issuing organisations NLM/UKPMC digitising NICE guidance E.g. ERNIE database

15 Products/interventions (2010)

16 Results from MRC e-val Published summaries of 6 sections from MRC e-val 2009 on the MRC website Reports e.g. Impact of MRC Research used as briefing for BIS/new ministers early in 2010

17 University/MRC Unit/Institute dashboard contrasting local MRC e-val results with the whole MRC portfolio In addition to spreadsheets with all raw data, arranged by section and principal investigator

18 Capturing output in 2011/12 MRC will continue to use MRC e-val to gather evidence of progress, productivity and quality of MRC output. Data gathering is planned again for October 2011 CSO (Scotland) is piloting e-val and STFC are currently gathering data using e-val NIHR have just completed an output survey using a different approach RCUK is re-developing ESRC s Society Today system to collect data for AHRC, EPSRC, ESRC and BBSRC MRC is in discussion with medical research charities to pilot the next phase of output gathering approaches which will encourage adoption of a federated approach between output datasets

19 Medical Research: What s it worth? (2008) Consistent time series for medical research funding in CVD and MH from Clear conceptual framework relating to GDP gain from spillovers Estimation from literature of the magnitude of this GDP return Development and application to CVD and MH, of a bottom-up approach to estimate health gain in terms of QALYs Analysis of UK guidelines (5 CVD and 12 MH) to provide indicators of lags and proportion of benefits attributable to UK Suggestions for developing research agenda Strong quantitative argument for investment in medical research