Effective KM is crucial to extracting maximum value from existing evaluation evidence. It is also a necessary precursor to the effective use of AI for evidence synthesis, with the ability of researchers and evaluators to realise the full potential of AI dependent on the quality of underlying systems of KM. Although a range of barriers were recognised, participants highlighted the criticality of good KM practices and offered a series of suggestions for overcoming these barriers.
“We can be doing so much more with the evidence that’s already out there”
Defining ‘knowledge management’. For some researchers effective KM means getting information to evidence users and decision makers at the right time, while for others it means developing an organisational culture around evidence sharing and reuse. KM is also seen as a process for managing risk, preserving knowledge, and creating space for learning and connections. There was broad agreement that KM represents a wide-ranging concept that intersects with evaluation and evidence synthesis to create an environment conducive to the production, curation and use of high-quality evidence. However, participants acknowledged the need for further work to clarify and contextualise the term to allow for a productive dialogue on the central role of KM.
The role of AI in evidence mapping and synthesis. Evidence mapping represents a foundational step in the development of an effective KM system. It enables an understanding of where gaps exist in a current body of evidence and provides an indicator of which geographies or methodologies may be underrepresented in an evidence base. Such exercises also identify areas where evidence is being over-produced, allowing researchers to avoid duplicating efforts and target increasingly limited resources more effectively. AI was recognised as having the capacity to perform mapping tasks to a high standard, and its use here was seen as both low-risk and scalable. While evidence and gap maps are increasingly utilised by research organisations, AI tools were also viewed as having the potential to support the future creation of ‘living’ synthesis products which are automatically updated to take account of emerging research.
AI and KM in maximising evidence use. Robust KM systems can also enhance the capacity of AI to contribute to evidence uptake. When used in combination with high-quality KM platforms, AI tools can improve the quality and applicability of insights derived from evidence synthesis products by allowing researchers to enter tailored prompts and refine results to focus on specific geographic or thematic areas. AI tools which interact with reliable KM systems were viewed as enabling policymakers to navigate complex knowledge bases, identify relevant findings in a timely manner, and more easily incorporate evidence into the decision-making process.
Promoting good knowledge management. Participants offered a series of suggestions for developing good KM practices. Drawing up a blueprint detailing internal processes, principles, systems, and partnerships was advocated, with the aim of codifying how KM is approached and communicating this widely. The creation of an organisation-wide knowledge agenda was also offered as a step towards creating a positive KM culture, with each team summarising their needs and priorities and setting out the actions and resources required to achieve these. More broadly, developing a cross-organisational maturity model for effective KM was recommended, focusing on detailing what success in this area would look like, setting achievement targets, and designing metrics to track progress.