Participants highlighted the indispensable role of humans in ensuring the quality and reliability of output produced by AI and continually improving its effectiveness. The role of AI as complementing rather than replacing humans was also discussed, as was the enduring importance of human relationships and skillsets that cannot be replicated by technology.
“AI needs some guardrails, albeit at relatively high level, to make it designed by people and for people”
Human-in-the-loop. The challenges inherent in using AI for evaluation and evidence synthesis underscore the importance of retaining human oversight in its design and implementation. Although this is particularly crucial during the early stages of the incorporation of AI into the evidence ecosystem, robust systems of human supervision must be maintained as tools become more widely used across the sector. Participants emphasised the need to find a balance between achieving maximum efficiency and the importance of critical oversight, with an agreed distinction between the role of humans and AI seen as a first step towards achieving this.
AI as supplementing human intelligence. Concerns over the impact of AI on job security for those working at different points in the evidence cycle were noted. However, participants firmly positioned AI as complementing rather than replacing human intelligence. Rather than being viewed as a threat to employment, participants emphasised the potential for AI to increase the number of evidence-related jobs by allowing people’s skills to be enhanced through technological advancement; AI tools were seen as not only maximising the efficiency and impact of the work of researchers, but also as enabling those with differing degrees of skills and experience to understand and interact with complex evidence as part of their role.
The ongoing importance of human relationships. As AI tools continue to become more sophisticated, evidence suggests that their ability to respond to complex demands and mimic human interaction is increasing. However, participants emphasised the irreplaceable function of human relationships in producing evidence and maximising its uptake. The ability of experienced evaluators to interpret and communicate evidence pragmatically was flagged as a key example of the ongoing value of human relationships in the evidence cycle, as was the capacity of evidence brokers to react to rapidly changing political contexts and maintain relationships with decisionmakers in complex and nuanced environments.