In partnership with The Centre for Development Impact (a joint initiative of the Institute of Development Studies, Itad Ltd and the University of East Anglia)
As part of the International Year of Evaluation 2015 this meeting provided an opportunity to identify good impact evaluation practices and to build new evaluation coalitions for market-oriented development initiatives.
Over the past 15 years there have been dramatic changes in the way in which middle income and low income countries have achieved economic growth and political and social change. These changes have been driven by new contexts, new actors and new ideas, in which market-oriented development approaches have become increasingly prominent. Evaluation practice, often shaped by a public sector orientation, has not kept pace, which raises important questions about the role of evaluation, both in understanding the impact of new activities on society, as well as in supporting democratic accountability to citizens.
Key points from the meeting:
- Global trends suggest a number of major challenges are being faced, including: a growing inequality of wealth, voice and power; an accelerating challenge to achieve sustainability alongside growing consumer demand and within planetary boundaries; and a need to build more inclusive and secure societies that are better able to deal with conflict, terrorism, epidemics or other threats.
- The old aid narrative, founded on a north-south model of cooperation, is shattered. Global problems extend beyond states, with a greater level of complexity and interconnectedness. The political landscape has shifted, with an emerging multi-polar world beyond traditional powers. New actors and alliances exist, and are increasingly possible with new technology and hybrid arrangements with the private sector and others.
- Alternative resource flows and investments dwarf international aid, with an emerging group of investors who are developing products that have a social and/or environmental conscience alongside a financial return. This is a growing market, and while investors presently draw on a range of advisors and services, there is an underdeveloped demand / supply relationship for data and evidence on social impact, including who pays for it.
- Within this context there is a potential demand for the skills of evaluators. There is also an urgent need to build the field including a need to better articulate the demand for evaluative evidence, its associated costs and the respective institutional architecture. In particular, there appears to be a mismatch between investor’s understanding of what evaluation can provide, and evaluator’s understanding of how best to meet investor, and potentially societal, demands for evidence. It may also be that building this field has to be initially subsidised if it cannot be extracted from individual investment deals.
- To address this apparent latent demand requires new evaluative approaches consisting of adapted and innovative services, methodologies and tools. While there is extensive ongoing work on social metrics and indices, emerging strands for further development might include: blending monitoring and evaluation through the use of continuous data capture and evaluative analysis; better integrating ex ante evaluative thinking around social impact into risk analysis and investment decision-making; extending analysis beyond simplistic notions of social change (for example numbers of jobs created) towards transformational impact (for example broader economic impact); plus, shifting analysis beyond the effects of the direct investment in small or medium sized companies, and more towards better capturing effects, at the household or individual level. Capturing insights through this latter approach can also contribute towards empowering the voice of the individual.
- Some urgency is called for. In many ways, the changes are already occurring, although it is a dynamic space that is being partly filled by more traditional advisors on risk analysis, due diligence and investment decisions (such as auditors, accountants, lawyers and management consultants). The research and evaluation community, particularly in international development, needs to step-up to meet the challenges of demand and practice. Various ways to speed up learning were suggested, for example developing communities of practice that would exchange knowledge and experience, and the setting up of a Social Innovation Fund that would support experiments and innovation. Learning should be promoted beyond the evaluation community and include actors already serving market-oriented initiatives.
- One way forward to help better shape demand and supply in this space would be to promote a diversity of platforms for dialogue between the evaluation profession and market-oriented initiatives. Rather than creating a single platform, the emphasis should be on convening multiple coalitions and dialogues using an open and non-transactional approach. This could include discussions about language, failure, and the values used to make evaluative investment judgements.
- A mapping exercise was proposed to provide a better understanding of the current and potential actors, and the institutional and regulatory architecture, both present and future. Existing networks should be invited to contribute and collaborate: the Centre for Development Impact was invited to continue to coordinate initiatives, and suggestions made to link this to the global evaluation agenda 2016-2020, and the Sustainable Development Goals.
Videos
A selection of interviews with key participants from the conference, produced by the Centre for Development Impact and filmed at Wilton Park.
