news

HAI hosts Monitoring and Evaluation Seminar

– Supporting Learning and Change in International Development

With recent changing trends and developments in aid delivery and aid modalities, we need to constantly strengthen and improve the ways we capture our performance stories, the systems used to learn about why changes occur, and how, through learning we can enhance the impact of our desired changes.

On 20 June the HAI Development Impact Group facilitated a Seminar ‘Supporting Learning and Change in International Development’ with the aim of sharing knowledge and experiences on how to improve M&E for enhanced accountability, impact assessment and learning.

The interest for the event was overwhelming with more than 60 people attending, from 7 government departments, managing contractors, individual consultants, aid practitioners, NGOs, and development researchers.

The four presenters, Graham Rady (AusAID), Stewart Norup (HAI Beijing Office), Paul Crawford (Aid-IT) and Tracey Delaney (Clear Horizon), provided outstanding presentations and stories on M&E from four different perspectives.

Graham Rady introduced the experiences from AusAID with their new Quality Framework, providing insight into the guiding principles of a new and simplified framework for quality assessment of development initiatives. The new framework focuses on strengthening both requirements of accountability and learning, which provide a challenge for M&E systems, and the resources and capacity levels to support M&E. The Australian White Paper on Aid, the DAC Paris Declaration and other current policies on ODA delivery emphasise the expectations of M&E systems to provide evidence of enhanced efficiency and effectiveness. Graham stressed that good M&E systems are imperative for AusAID to document development effectiveness and impact.

Stewart Norup gave a brief introduction to the China Australia Governance Program (CAGP) funded by AusAID. He explained the challenges in designing the program’s M&E system including:

• measure performance against the broad objective of “enhancing good governance as a means to alleviate poverty in China”;
• dealing with the complexity of the governance context in China; and
• the challenge of attribution versus contribution.

Stewart described how CAGP was able to capture performance stories by using theories of change i.e. the theory of how inputs and activities led to specific outputs, that in turn led to intermediate, substantive and long-term outcomes.  Also presented and discussed were lessons learned from one of the most successful CAGP projects - a 360º Performance Appraisal and Balanced Scorecard System – and how institutional ownership and commitment was essential to performance outcomes.

Paul Crawford provided a brilliant presentation, helping us visualise the complexity of change and demonstrating the links between outputs, outcomes and impact. He illustrated through volunteers how implementation teams ‘bounced off’ surrounding boundary partners, who ultimately impacted on targeted beneficiary groups.  Paul presented how the AusAID-funded Philippines Australia Community Assistance Program (PACAP) has approached the performance data collection of outputs, outcomes and impacts using a systems and IT-based approach. The system is operating with three different levels of data:  management data for accountability, intervention data for demonstrating effects, and development data aiming at showing real development impact. Paul emphasised the importance of using more qualitative methods such as the Most Significant Change technique to strengthen the systems approach. His presentation also noted the complexity of demonstrating PACAP’s attribution versus contribution, arguing that any project has a sphere over which it has direct control (outputs), a sphere that can be influenced (outcome level), but also a sphere of concern (impact level) largely determined by a range of externalities.

In a very participatory and interactive approach, Tracey Delaney demonstrated the Most Significant Change (MSC) technique. MSC is one of the latest methods in M&E that, using participatory techniques, captures the most significant changes according to the beneficiaries and people affected by development programs and projects. Conventional M&E systems struggle to capture qualitative data on change and impact, that according to Tracey, is where MSC can complement already established M&E systems. The technique doesn’t work with traditional indicators, but relies on people’s stories of experienced changes. Tracey emphasised that even though stories might be subjective or lack accuracy, they provide a richer picture of actual changes and provide a good basis for further discussion about specific activities and future direction. In this way, the aim of the MSC method is to enhance learning and project improvement.

The seminar provided a wealth of diverse experiences and lessons learned on M&E, provoking good discussions and interaction between the presenters and the audience. With increased demand on the international development community to demonstrate efficiency and effectiveness, the seminar also highlighted the need to further strengthen capacity and resourcing of M&E in aid programs.

The HAI Development Impact Group would like to take this opportunity to once again thank the four presenters and the audience for making this day a great learning experience for all involved.

For further reading on DIG activities and to download the four power point presentations, please visit our web site www.hassall.com.au/international/Development Impact Group

Jorgen Jensen

Manager

Development Impact Group

jjensen@hassall.com.au