In any area of rapid growth, the risk of fragmentation and duplication of efforts is high. For the sector to maximise the potential advantages of incorporating AI into evaluation and evidence synthesis, participants acknowledged a duty to foster links between institutions, to pool resources, and to formalise collaboration to advance the responsible and efficient use of AI and KM.
“How can we collectively use the power of AI to do something that couldn’t be done before?”
Collaboration between institutions. Although organisations are working to develop tools that suit their specific needs, much of this work is taking place in isolation. Participants called for the dismantling of institutional silos and the creation of an ecosystem that spans the organisational level to ensure that the work of individual institutions complements rather than duplicates the efforts of others. Suggestions for collaboration included sharing the results of experiments with existing off-the-shelf AI tools, discussing the strengths and weaknesses of different approaches to developing and using bespoke software, and uniting to build skills and capability across the sector.
Connecting evidence synthesis efforts. While the volume of evidence synthesis work being undertaken has increased significantly in recent years, a co-ordinated system linking these individual efforts together is lacking. As a result, there is currently a patchwork of evidence synthesis products and repositories that provide only a partial picture of the evidence base. Meanwhile, these products are often static, quickly becoming outdated upon the production of new evidence. Participants suggested pooling evidence that currently sits on institutional platforms, creating links between existing evidence repositories to form a ‘repository of repositories’, and working collaboratively towards the development of open and accessible living synthesis products.
Creating multi-disciplinary alliances. Participants also underlined the importance of collaborating with experts from other disciplines with alternative perspectives and complimentary skill sets. This included working with technical specialists to tailor emerging AI products to suit researcher needs, with governments to co-generate evidence and promote the use of AI in synthesis and dissemination, and with policymakers to ensure that their priorities are understood and catered for by emerging technologies. Several examples of successful collaboration were spotlighted including the World Bank’s Impact AI initiative and the work of the Deep Learning Indaba, while participants suggested establishing regular hackathon-style events incorporating technical experts and policymakers and highlighted the importance of identifying key future partners.
Developing a common voice. Participants flagged the importance of jointly advocating for the use of AI technology, establishing agreed principles for its responsible use, and building capacity across the sector. Participants noted the lack of an agreed lexicon to discuss these issues, with collaboration seen as an important first step in developing shared definitions and holding productive conversations.
Emergent networks. Participants highlighted the existence of several networks designed to facilitate co-operation in this space. These include the Evidence Synthesis Infrastructure Collaborative (ESIC), a collective supported by the Wellcome Trust and designed to advance the use of AI and KM in evidence synthesis; OECD’s DAC Network on Development Evaluation (EvalNet), a network of organisations working to strengthen global evaluation systems and capacities; ALIVE, an alliance of organisations and individuals from across the evidence ecosystem building and testing a collaborative approach to living evidence partnerships; and the MERL Tech initiative, a collaborative working with social sector organizations to achieve thoughtful tech-enabled program design and implementation and convening communities of practice to share learning and advance the responsible use of technological advancements in monitoring and evaluation. Participants were also encouraged to strengthen and expand existing informal connections between individuals and organisations working on AI in evaluation synthesis.