Following the news that Babylon, the company behind the NHS GP at Hand app, says its follow-up software achieves medical exam scores that are on-par with human doctors, the executive director of Nesta’s Health Lab has the following reaction.

Halima Khan, Executive Director of Nesta’s Health Lab commented:

This is a significant achievement from Babylon, but needs careful interpretation - especially because apps which use Artificial Intelligence for diagnosis and triage are developing quickly and could be commonplace within five years. They could make the health system simpler, more accessible, more responsive and more sustainable. If developed correctly, AI could help solve a major issue: how to reduce the unnecessary use of health services. But there are also risks to manage.

Many of these risks relate to implementing AI beyond its competence. Today’s AI performs very well at narrow and well defined tasks - looking at a medical image, or answering an exam question. This does not mean that AI can bring to bear the full range of skills and knowledge - physiological, clinical, psychological, interpersonal, linguistic and so on - required, for example, in a consultation with an older patient with multiple and interdependent health and social care needs.

Being able to pass an exam is not the same as being able to be a good doctor.

The best safeguard against inappropriate implementation of AI is if patients and doctors are put in the ‘driving seat’ of its development, and soon. If not the technology risks creating more of a barrier than an open door, offering opaque advice and dehumanising healthcare. It could generate a flood of unnecessary demand from false positives or risk-adverse advice, and could widen health inequalities - for example by disempowering those without access to technology.

To avoid this, we need to ensure that both patients and healthcare professionals are at the heart of how AI-based services are developed and provided.

About Nesta’s report on AI in healthcare

Nesta’s recent report on AI in healthcare, Confronting Dr Robot , explores the potential ways in which AI is likely to be implemented in the health system over the coming years.

The report sets out that AI advice and triage could be appealing to both health services under pressure, and to patients who are already routinely using online technologies to search their symptoms. But it also warns there are big risks in using machines beyond their competency or in ways that undermine the relationship between patients and doctors.

Setting out how AI could be integrated into health, the report states that it could extract complex data in real time and inform patients that they need to seek urgent care - for example, by assessing breathing sounds of people with congestive heart failure to spot signs of deterioration. A number of health trackers and apps are available now for those with health concerns. Harnessing the power of AI to evaluate this data could make the health system more dynamic and responsive. However, there is a risk that it might replace individualised conversations or generate unnecessary concerns.

Halima Khan is a leading expert in innovation for health. She is Executive Director of the Health Lab at Nesta - the global innovation foundation. Nesta’s Health Lab envisions a people powered health system, and is working to test and develop new relationships, networks and technologies which empower people to improve their health and wellbeing. Halima works with health and care services, civil society and social business to develop and scale innovations and new ideas to tackle many of our biggest health challenges.