Artificial intelligence (AI) holds massive potential for the humanitarian sector – from computer vision models that automate damage assessment in the immediate aftermath of a crisis, to chatbots that help vulnerable communities and frontline workers.

Despite this, the sector is struggling with limited technical expertise and low AI literacy. Findings from our research with Sphere indicate that 57% of humanitarians either aren’t aware of or don’t have any guidance about responsible AI in their organisations. This opens up humanitarian organisations to significant risk if things go wrong.

Since August 2024, Nesta’s Centre for Collective Intelligence Design has been exploring the value of an AI Safety Label – an independent assurance process for auditing and assuring humanitarian AI tools. Our process involved three key elements: 

1. a technical review, 
2. an organisational capability assessment, and 
3. community feedback. 

Our aim was to design an approach that would boost confidence for non-technical users whilst ensuring organisations were taking a responsible approach to AI deployment.

What makes our approach unique in the landscape of AI assurance is that it involves communities in assessing the social acceptability of risks of AI tools. We see the AI Safety Label as a community trustmark that can help organisations navigate risks while also fulfilling their accountability commitments to crisis-affected communities.

In November 2024, we set out to test the feasibility of our community trustmark approach: could we have meaningful conversations with crisis-affected communities about AI tools that could impact their lives? And would their recommendations provide useful information for humanitarian organisations as they decide whether or not to use a given AI tool? 

We travelled to Antakya in southern Türkiye to gather community input on the use of Automated Damage Assessment (ADA), an AI tool developed by the Red Cross to identify damaged buildings using satellite images following natural disasters. Antakya was devastated by a series of earthquakes in early 2023 – it was reported that more than 80% of all buildings collapsed. 

Working with a local expert facilitator, we held two deliberative workshops with a diverse group of 24 residents. Our approach draws on a collective intelligence method known as deliberative polling. It combines easy-to-understand, interactive animations, facilitated group discussion, expert Q&A and polling. The sessions are delivered through our bespoke collective intelligence platform Zeitgeist.

We know that people are more able to meaningfully engage with complex questions about technology when they’re asked about specific use cases, rather than abstract concepts. Building on this idea, we created bite-size animations about:

• current approaches to damage assessment
• what AI is and how it works
• how the ADA tool works and how it can support damage assessment
• limitations of the ADA tool and their impacts
• safety measures that could help mitigate risks posed by the tool

In between demonstrations of the bite-size content, participants discussed how they felt about humanitarians using a tool like ADA in their local context, drawing on their recent experiences of the earthquake. They weighed up the benefits of the tool (eg, faster, safer damage assessment) and the risks (eg, underestimation of damage leading to unfair allocation of resources). Throughout these discussions, we captured their views through interactive polls and recordings.

Too often, we’ve heard that it’s too difficult to involve communities in complex decisions about technology or that when it’s done, it’s hard to do it well. Our previous work on localising AI for crisis response and the results from this project show this isn’t true. It is possible to engage frontline communities in meaningful conversations about AI. In contrast to expectations that “communities will say no”, these processes can open up a mandate to experiment with AI as long as they have the appropriate guardrails in place. These were our key takeaways from the deliberative workshops:

The people we spoke to saw the potential of humanitarians using AI

Despite the risks, 96% of participants were positive about the use of the tool to improve future response operations in their local context. 

They also made practical recommendations about the safety measures organisations should prioritise to help mitigate risks 

The top two mitigations they wanted organisations to implement first were staff training and ongoing impact monitoring. 

They were pragmatic about the potential of AI tools to address human blindspots

Conversations about responsible AI often end up assessing tools in comparison to an idealised, perfect version of the technology. In contrast, our experience shows that people are pragmatic – they compare the technology to the status quo and can be discerning about the limitations of human processes that might be enhanced by AI. In Antakya, workshop participants spoke about the impacts of the earthquake on frontline responders, who were often local people who had themselves been affected.

They wanted to have a say in decisions about the use of AI by humanitarians

Most importantly, workshop participants told us they enjoyed taking part (96%) and thought it was important that humanitarians involved people like them in decisions about using AI (92%). 

We used the results from the workshop to create a prototype AI Safety Label for the ADA tool to support damage assessment in southern Türkiye. The Label summarises the community input on the acceptability of different risks and the safety measures they want to see implemented, as well as a final recommendation about whether the tool should be used. It also documents the results of a technical evaluation of the tool carried out by the Red Cross. 

These are the first steps in our efforts to develop a robust and replicable assurance process that foregrounds community voices in decisions about how AI tools are used in high-risk settings. We believe that an independently-verified community trustmark is a critical gap in current conversations about AI assurance. 

All the humanitarians we spoke to agreed that practical guidance was desperately needed. They also agreed that community accountability was an important part of getting AI implementation right in the sector. But they also couldn’t see their organisations signing up to a community trustmark unless there was strong endorsement by donors or demonstrator pilots, with big humanitarian organisations leading the way. 

We believe the sector needs to move in this direction as part of ongoing conversations about standards and principles for responsible AI. Over the next year, we’ll continue developing this concept, building on our work in the humanitarian sector and expanding to other contexts, such as the UK public sector. In the meantime, look out for upcoming events about standards and responsible AI for humanitarians from our partners Sphere and CDAC Network.

Acknowledgements

We’re grateful to our advisors Kate Conroy, Kasia Chmielinski and Mallory Durran for their guidance throughout this work and our partners at Sphere, Data Friendly Space and CDAC Network. This project was made possible with the support of the UK Humanitarian Innovation Hub.