ANZEA Online Workshop

Evaluation and Value for Money: An approach using rubrics and mixed methods
Julian King, Director of Julian King & Associates Ltd and a member of the Kinnect Group

(two 3-hour sessions)

Venue: Via Zoom. Details will be emailed to registrants just prior to the seminar start time

 

Event overview

It is important for good resource allocation, accountability, learning and improvement that policies, programs and other initiatives undergo rigorous evaluations of value for money (VfM). Many evaluators, however, lack training and experience in this area. This workshop offers a set of techniques to address that gap. Combining strengths of evaluation and economics, the approach builds on theory and practice that will already be familiar to many evaluators, though there are also evaluators and commissioners to whom these techniques may be new.

 

This workshop provides practical guidance, underpinned by a sound theoretical framework, for evaluating VfM. It focuses on unpacking a process of explicit evaluative reasoning (using rubrics) and the use of mixed methods (qualitative, quantitative, and economic).

 

A sequence of steps will be shared which help evaluators and commissioners develop and use context-specific definitions of good performance and VfM (i.e. VfM rubrics). These definitions provide a system for ensuring the evaluation: is aligned with the design and context of the policy, program or initiative; collects and analyses appropriate evidence; draws sound conclusions; and answers VfM questions.

Exercises and examples will be shared throughout the workshop.

 

The workshop includes a brief overview of economic methods of evaluation (e.g. cost-benefit analysis) including considerations for determining when to use them in a VfM assessment, and when to combine them with other evidence and values.

 

This workshop doesn’t provide detailed instruction in the design and implementation of economic evaluations, for which there are many courses already on offer.

 

The workshop is offered as two 3-hour sessions, to be held on consecutive weeks. Registration covers both sessions with the expectation of attendance on both dates for completion of the workshop.

 

Participants will learn how to:

  • Frame an evaluative question about VfM
  • Develop rubrics, comprising criteria (aspects of performance) and standards (levels of performance), to provide a transparent basis for making sound judgements about VfM
  • Combine multiple sources of evidence to support nuanced evaluative judgements
  • Incorporate economic evaluation within a VfM framework, where feasible and appropriate
  • Interpret the evidence, using criteria and standards, to make sound evaluative judgements
  • Present a clear and robust performance story, guided by the criteria and standards

 

Presenter's background

Julian King is the Director of Julian King & Associates Ltd and a member of the Kinnect Group. He specialises in evaluation and value for money. Julian’s PhD research developed the ‘Value for Investment’ approach that will be shared in this workshop. The approach is used globally to evaluate complex and hard-to-measure programs and policy reforms.

 

Oxford Policy Management is a significant early adopter of the approach, as described in this open-source guide to VfM assessment: https://www.julianking.co.nz/wp-content/uploads/2018/02/OPM-approach-to-assessing-value-for-money.pdf. The approach has also been picked up by government agencies, consulting firms and NGOs around the world and is taught as part of the University of Melbourne’s online Master of Evaluation degree. Julian received the 2021 Australian Evaluation Society

Evaluation Systems Award, recognising the adoption and use of the approach as a “flexible and collaborative approach that can be applied to all domains and program types”. https://www.aes.asn.au/awards-winner-spotlight/1003899-evaluation-systems-award-recipient-2021.

 

Julian has delivered workshops on this approach for evaluation societies and private sector organisations in Australia, New Zealand, Europe, USA, Africa, Asia, and UK.

 

Part 1: Wednesday, 4 May 2022, 9am – 12 pm NZST

AND

Part 2: Wednesday, 11 May 2022, 9am – 12 pm NZST

 

Register online by: 1 May 2022

*please note that registration is limited to 30 participants

 

 

Cost: 

Early-bird $250 +gst [only available to the first 10 ANZEA members who register before 11th April]

Members: $295 +gst

Non-Members: $375 +gst*

 

 

*Non-members who register for this event will receive a limited-time 30% discount on their ANZEA membership if they sign up within four weeks of the first session. Ensuring the best prices for future workshops and the opportunity to join upcoming free member-only events. Please get in touch with our manager after registering at info@anzea.org.nz to access this discount.

 

**This is a paid event organised by the Aotearoa New Zealand EvaluationAssociation. All prices are in NZD.

Seminar start times

  • New Zealand (NZST): 9 am

 

For other time zones please go to https://www.timeanddate.com/worldclock/converter.html

 

Please ensure you have access to your email address just prior to the start time to access Zoom details. Please also check your email address is correct on your registration form before submitting. Thanks.