Using AI in Evaluation – Evidence, Practice and Principles for Responsible Use

This report examines how Artificial Intelligence (AI) tools are used in policy evaluation, bringing together two phases of work conducted for UKRI. It combines a rapid review of academic and grey literature with semi-structured interviews with evaluators across consultancy, government and academic settings. It sets what the published evidence says AI can do against how evaluators are actually using it. The report aims to provide UKRI with evidence-based insights to inform principles for the responsible and effective integration of AI into its evaluation processes. 

Using a funding evaluation cycle as its organising framework, the report examines AI use in: 

1. choosing and prioritising intervention areas;
2. assessing project proposals;
3. programme monitoring and process evaluations;
4. impact evaluation and value for money assessments.

The report finds that evaluator practice is in some areas moving faster than the published evidence, particularly in qualitative synthesis. There is also a strong consensus in favour of keeping a human in the loop. Drawing on both strands of evidence, it proposes ten principles for responsible AI use in evaluation and sets out how UKRI, as a commissioner of evaluation, can create the conditions for safe, transparent and accountable adoption. 

If all this sounds intriguing then please download full report using the link on the right-hand side.