The new CRP Portfolio: initial impressions from the ISPC Recalling the CRP evaluation process in Phase I SRF needs to give more direction

Transcription

1 27 April 2016 The new CRP Portfolio: initial impressions from the ISPC The agenda for FC15 includes an item on the new portfolio. The purpose of this document is to remind the FC of the process that has taken place so far and of earlier comments made by the ISPC to illustrate the progress that has been made and some continuing concerns and the portfolio level. It is too early in the review process to comment on individual CRPs. Recalling the CRP evaluation process in Phase I: Over a two and half year period from 2010 to 2013, the ISPC reviewed the first phase of CRP proposals. Proposals for the extension of all 15 CRPs were simultaneously reviewed by the ISPC in 2014, at which time the ISPC also produced its first portfolio commentary. In that commentary, we noted: The transformation of the CGIAR to a more programmatic way of structuring the research, where the whole is bigger than the sum of the parts is challenging, but we feel that the Annual Reports and Extension Proposals provide significant evidence of delivery and change under the new structure. And summarised our concerns related to System-level delivery as follows: In summary, we have highlighted a number of areas where the SRF needs to give more direction if the CRPs are to deliver more value as a System, in particular these include a clear Theory of Change, a prioritization framework, clear guidance on the role of systems programs in the delivery pathways and on cross-crp roles, for example in relation to policy and nutrition. The SRF was subsequently revised in 2014 and 2015, and whilst this has provided clearer guidance on cross-crp roles, it still lacks an overall Theory of Change or clear prioritization framework. Responsibility for leading the development of a Prioritization Framework is a task which has been passed to the ISPC (by the FC) and it is planned to start this once the roles and responsibilities of the different entities in the new governance model have been completed. The issue of the need for a further refinement of the System s Theory of Change has not, however, been discussed with the FC but could be taken forward as part of the proposed work on Foresight at the System level. Recalling the process for Phase II pre-proposals: The Centers had relatively little time to finalize their pre-proposals for Phase II between the approval of the SRF (May 2015) and the date of submission (mid-august2015), yet the ISPC observed in their commentary on the portfolio at that time that:

2 There was strong alignment with the SRF, particularly with respect to addressing the grand challenges, their focus on the eight CGIAR research priorities and the alignment with SLOs, Cross-cutting themes, IDOs and sub-idos. Every sub-ido is being addressed by at least one CRP. The ISPC also noted, however, variability between CRPs: There was significant evidence of lessons learnt from within CRPs (during Phase 1), from the Evaluation reports and from earlier ISPC commentaries, although this was better in some CRPs than others. Progress was observed in some CRPs regarding Theories of Change and Impact Pathways but others still have some way to go. This could be attributed in part, to significant changes in the remits of some CRPs (e.g. Livestock and Fish had separated and the Systems CRPs had been ended with some elements being incorporated into other CRPs). All CRPs were given overall ratings, with 8 being rated as B and 4 being rated as C. The ISPC Chair and senior members of the Secretariat met with the leaders of the latter 4 CRPs in October 2015 to clarify the areas of concern and also commented on partial re-submissions of those CRPs in January/February The ISPC did not comment formally on the Genebank CRP suggesting it was not appropriate for consideration against CRP criteria. At the November 2015 meeting in Rome, it was agreed that Genebanks should be submitted as a platform. Competitive expressions of interest were also submitted at the pre-proposal stage for cross- System initiatives on Gender, Capacity Development, Genetic Resources Policy (GRP) and Big Data. The ISPC supported the development of a full-proposal for a distinct (but timebound) initiative on Big Data but suggested that cross-system initiatives on Gender and Capacity Development should be led from within existing initiatives and expressed major concerns about the GRP initiative. An expression of interest on Genetic Gains was submitted by a group of Centers in June 2015, but not reviewed by the ISPC since it was not part of the Call. Significant donor interest in such an initiative led, however, to further discussion with the ISPC and FC and a meeting on the nature of such a platform was held in San Diego in January The ISPC provided formal feedback on the report of that workshop on 7 February Process for ISPC Review of Phase II full proposals: In December 2015 the FC agreed to the request from the CGIAR CRP II Working Group to give a green light to proceed with the portfolio of 12 CRP pre-proposals and 3 platforms (with a note stating that funding for a comprehensive platform on Big Data will be awarded in the base budget or in the uplift budget scenario will be assessed on the basis of the proposal received as laid out in the Guidance for full proposals published on 19 December 2015, subject to a series of caveats laid out in the WG documents dated 30 November The original pre-proposals, the ISPC reviews (in four cases 2 reviews), and the caveats provide the starting point for the reviews for this phase of the evaluation. In other words, this current review of the portfolio is focusing primarily on the extent to which there is evidence of further progress towards the development of a coherent, system-wide integrated portfolio of research, in response to the advice already given.

3 To aid the ISPC, and to ensure adequate coverage of the disciplines represented in the CRPs, the ISPC has commissioned 53 external experts to provide expert reviews of individual CRPs or cross-cutting aspects of all CRPs. Council members are between them also reviewing all CRP and platform proposals and ensuring continuity with previous advice, including that provided in the CRP Evaluation reports. ISPC will discuss all reviews collectively (equivalent to a moderating panel) at a face to face meeting prior to our open meeting in Lima at the end of May.. This is a 2-stage process, with the ISPC scheduled to provide written reviews of individual CRPs on 16 June and giving verbal feedback at the portfolio level on 17 June, to be followed later by a written review of the portfolio..initial impressions: alignment with the Results Framework The revisions agreed in the WG paper of 30 November 2015 resulted in decreases in the total budget from US $8.556 billion to US $5.226 billion and in the number of flagships from 72 at the pre-proposal stage to 57 in the full proposals. On average flagships are targeting 7.3 sub- IDOs which is only slightly up on the 6.9 average at the pre-proposal stage. All sub-idos are being targeted, with the biggest investment being aligned with SLO1 (US $2.061 billion) followed by SLO3 (US $807 million) and SLO2 ($515 million). The biggest investments per sub-ido are targeted at two of the sub-idos which were top priorities in the donor survey conducted by the ISPC last year ( Closed yield gaps through improved agronomic and animal husbandry practices ; and Enhanced genetic gains ).The biggest decrease from the preproposal phase was in Increased conservation and use of genetic resources. The only sub- IDO at the low end of the donors priorities that received significant investment was optimised consumption of diverse nutrient-rich foods but that investment was markedly down from the pre-proposal stage. Another point to note is that almost all CRPs have included budget lines for impact assessment, generally at the FP level, but the proportion of the total CRP budget which has been allocated to impact assessment is quite variable ranging from 0.2 % to over 5%. For the entire portfolio the estimate is about 3%. With respect to integration across the portfolio, initial impressions from reading some CRPs, supplemented by presentations at GCARD3 and in Addis Ababa on site integration suggest that progress has been made with respect to both: a) pathways by which Agri-food system CRPs and Integrating CRPs plan to influence each other s research programs and b) greater cross-crp discussion with stakeholders in key target countries though this progress has also highlighted risks. The ISPC will comment on these at a later stage. As noted in our earlier commentaries, Theories of Change, a clear idea of program direction and strong (but open) CRP leadership are key to integration and there are signs of further evolution in these respects. Yet, it is expected that some CRPs (and platforms) will be further ahead than others given the different starting points. Request for FC agreement on nature of advice At the pre-proposal stage, the ISPC rated CRPs in terms of A, B or C, defined in terms of the degrees of revision required. At this stage in the process, ratings should reflect the degree of further external monitoring which the ISPC considers each CRP and platform requires to give the funders confidence in the contribution that that CRP or platform can make to delivery of

4 the SLOs. Thus the ISPC anticipates assigning individual CRPs and platforms to one of 3 categories described below, subject to agreement by the FC that these proposed actions are aligned with the outcome of the Transition discussions. GREEN: where the ISPC feels the CRP or platform proposal is mature, with evidence of good performance to date and that the leadership team have a sound strategy and theory of change which they will use to ensure delivery. This would likely require minimal revision between 16 June and 31 July 2016 and then the ISPC would recommend to the System Council (in November 2016) that such CRPs/platforms should be approved to proceed with no further input from the ISPC, but annual monitoring by the SO. YELLOW: where the ISPC considers that considerable progress has been made, key points would be listed and a plan as to how they would be addressed would be expected to be provided in a revised submission by 31 July The ISPC would then outline how progress should be monitored in the first annual reports of any CRPs or platforms in this category. This monitoring would then be done by whichever entity is assigned that role depending on the outcome of the Transition Team discussions. As currently understood by the ISPC, Annual reports of CRPs and platforms in this category could be passed to the ISPC to provide assurance to the System on the quality of the monitoring process, at least in the first year. AMBER: If a CRP or platform is assigned to this category, the ISPC is signalling that it feels that further face to face contact with the CRP leadership is required, to help them understand how they could make a greater contribution to the portfolio as a whole. The exact nature and duration of ISPC intervention would depend on the nature of the issues identified. This might be expected to last longer than the first Annual report as suggested for category B. It is important to note, however, that the ISPC does NOT recommend that these categories should be used to inform funding decisions, since the assignments are more likely to be associated with the maturity of the CRP or platform rather than its potential performance and centrality to portfolio delivery. (As agreed in the discussions of the Science Working Group, the System still needs to agree on criteria for ranking performance according to science quality and feasibility of impact. It has been agreed that the ISPC will lead and facilitate such a consultative process, but progress on this is awaiting confirmation of the roles and responsibilities of the System entities). Overall Summary The ISPC would like to congratulate Center Director-Generals, CRP leaders and scientists from across the System, together with their partners and stakeholders for the amount of time and effort invested in responding to reviewers comments and making progress towards achieving a more coherent and integrated portfolio targeted on delivery of IDOs which will in time and with the involvement of partners contribute to delivery of SLOs and hence SDGs. The ISPC would also like to emphasize the importance of stable budgets for successful delivery of research for development. The sudden decrease in Window 1 and 2 funding during 2015 presented a considerable challenge to CRP leaders, DGs and Center Boards and threatened the continuity of research already underway and the confidence of partners. The ISPC recognizes the pressure which donor budgets are under, but it hopes that the increased dialogue between donors and the CGIAR research community will help in managing the impact of the inevitable fluctuations. The ISPC looks forward to playing its role in leading System-wide agreement on

5 key criteria. It is to everyone s credit that confidence in and commitment to the concept of an integrated portfolio was maintained and that all the expected full proposals were submitted on time. The improved (relative to phase I) quality of the proposals and the degree of consistency of format between them is much appreciated by the ISPC, who would also like to recognise the contribution of the CO over the last few years in proposing common definitions of terminology and introducing various measures to provide more standardisation across the System. Constructive critiques of each CRP and platform full proposal are under development and will be made available to the CO (and through them to those who submitted the proposals) on 16 June The aim of these critiques will be to provide guidance on where the proposals can be further improved as the whole System moves forward to our common goal of increasing the contribution of research to delivery of the SLOs.