NHS England

Introduction

The INSTANT project (Implementing New Solutions to Test and Assure New Technologies) was a small research team within NHS England focused on the testing and assurance of healthcare related AI (artificial intelligence systems). NHS England is currently undergoing a wholesale restructure, which will involve the team transitioning from being a research team to becoming a formal ‘Community of Practice’ responsible for supporting and advising NHS colleagues who are trying to test or assure AI systems.

The National Innovation Centre for Data collaborated with Adam Byfield, Principal Technical Assurance Specialist, Anusha Jose, Lead Practitioner within Solution Assurance and Benjamin Wallace, NHS Digital Academy. The team, which involved Dr Matt Edwards and Dr Antonia Kontaratou embarked on an ambitious project aimed to establish a framework for the assurance of artificial intelligence (AI) within healthcare settings.

The motivation

The team's motivation stemmed from the NHS’s proactive foresight of the significant potential AI holds for improving healthcare delivery, coupled with the complexities of assuring AI technologies.

Anusha highlighted the necessity of adapting traditional assurance methodologies for AI: "Our Head of Assurance Paul Butterworth had identified that AI is coming, and the current technique of assuring IT systems would not necessarily work. We sought the expert knowledge of the National Innovation Centre for Data to provide a framework for understanding risk and establishing a foundation for training colleagues".

The challenge

The project's primary challenge involved developing an in-depth understanding of AI, its potential risks, and how these could be mitigated within the healthcare context.

Assessing risk and applying AI assurance within this context is not an easy task. Unlike other sectors benefitting from AI advances such as the financial sector, the NHS is heavily regulated, and thorough assurance across IT systems is vital.

Anusha emphasised the importance of assuring AI systems: “(The financial sector) can take that one percent risk factor, but when we talk about AI in healthcare and rolling out something to a million people... that one percent error could be a lot of people's lives. We must be extremely careful."

This challenge was not only technical but also conceptual, requiring the team to reframe and adapt their approach to assurance for technologies that learn and evolve.

Building towards AI assurance

With the National Innovation Centre for Data’s support, the team undertook a comprehensive strategy to develop AI-specific learning materials and adapt their Assurance Framework to include AI technologies.

Much of the project involved formal presentations that covered the risks associated with AI in software, as well as pair programming. This was to understand aspects of AI in computer vision, audio, and natural language processing applications, such as chatbots.

The team explored tools and techniques for developing AI systems that can be justifiably trusted.

Any system, artificial or human, is trustworthy if the system is aligned, competent, reliable and open. It is important to know that the system:

shares your goals (is aligned),
can achieve those goals (is competent),
can continue to achieve them (is reliable), and
can provide reasons for its behaviour (is open).

Reflecting on unexpected learnings from the project, Adam said “We were surprised to find that, from our point of view, there appears to be a significant gap in the testing and assurance of AI. This has since been confirmed by our various other academic and regulatory partners and has become a key topic in current AI development.”

This effort included identifying unique risks associated with AI, developing models from scratch, and developing training content for NHS England testing and assurance colleagues.

While the team have so far only assured a small number of AI systems, Adam is confident in the foundation established through this project. He said, "We are starting to get enquiries about assuring AI systems. We are now confident in our approach to this emerging technology.”

Business Impact

Although still in the pilot stages of applying the AI Assurance Framework to clinical AI models, the initiative has prepared the team for future AI projects.

The team has leveraged their pioneering discovery sessions with the National Innovation Centre for Data to develop an internal training course on AI theory and assurance. This course has been successfully delivered to over fifty colleagues across the entire department.

Anusha Jose expressed optimism about the project’s impact: "(Assuring AI systems) is something that was always the plan. When people are asking the right questions and when we are getting the right information and happy about it, we can be sure that we are on the right track."

Anusha also described the excitement within the department about AI following the project “Everyone has heard a lot about AI, especially with tools like ChatGPT. Not everyone quite yet grasps how AI systems are built or how they work. We wanted to assure interpretability, safety, transparency and so on. What is the background of AI models? How do they work? Getting to learn more was interesting and colleagues across the department have been extremely excited to train and upskill on the subject."

"It has been both a privilege and a joy to collaborate with NHS England on their journey towards AI Assurance. As AI Assurance is an emerging field, I am grateful for the opportunity to support NHS England in bring the potentials benefits of AI to the NHS. I eagerly await updates on the project's progress and the resulting impact."

Matt Edwards, Senior Data Scientist, National Innovation Centre for Data

"Throughout our work with the National Innovation Centre for Data, we have made significant strides in understanding and managing AI-specific risks. This project has been instrumental in developing an effective risk log and assurance framework tailored for AI systems, marking a significant advancement in our approach to healthcare technology."

Anusha Jose, Lead Practioner, Solution Assurance, NHS England

A unique offering

The NHS England team has since made significant strides in advancing the assurance of AI technologies within the healthcare sector.

Adam highlights the practical benefits of their work with the National Innovation Centre for Data: "The work to which this project contributed will definitely save us significant time in the future as we are now prepared to test and assure AI systems when they appear. It has effectively expanded our service offer meaning that we should now be able to test and assure new technologies in the same way and to the same timescales as traditional systems."

Moreover, the collaboration was marked by a strong and productive working relationship, which Adam described as "Excellent throughout. The team were always responsive and accommodating to our scheduling needs and the working relationship was very pleasant and productive." He also appreciated the networking opportunities provided, such as the Data Innovation Showcase, which allowed valuable interactions with National Innovation Centre for Data (NICD) colleagues, the public and peers across a diverse range of industries.

Adam further emphasised the unique expertise brought by the National Innovation Centre for Data. (NICD) to the project: "The National Innovation Centre for Data colleagues were incredibly knowledgeable about AI, including cutting-edge developments. They were also always keen to go and quickly and thoroughly research topics about which they did not know as much. This meant there weren’t really any relevant AI-related questions they couldn’t answer for us."
The adaptability and enjoyment derived from these interactions were also noted, "You were a pleasure to work with and were very accommodating to our needs, even when our scheduling requirements meant having to change plans etc. The sessions were highly productive and valuable but also very enjoyable."

Adam concluded with a recognition of the unique value offered by the National Innovation Centre for Data, stating, "I’m not aware of any other organisation that offers this kind of service. I don’t know where else I could have got this from."

By preparing and laying the groundwork for future AI integration, the team have ensured that NHS England remains at the forefront of healthcare innovation, ready to leverage innovative technologies effectively and safely.

"Our collaboration with the NHS England testing and assurance team was excellent. It was a great opportunity to work towards developing assurance techniques for AI in healthcare technologies."

Antonia Kontaratou, Data Scientist, National Innovation Centre for Data

"The National Innovation Centre for Data were invaluable in helping us establish a solid foundation of knowledge of different types of AI on which to base our AI assurance project. The working relationship we have developed with them is friendly but also highly effective and our project would not have been able to deliver as it has so far without their support."

Adam Byfield, Principal Technical Assurance Specialist (Functional and Integration), Solution Assurance, NHS England

Our Discovery workshop

Our Discovery workshops enable you to explore the potential of your data and understand the benefit you could gain before committing to a full-scale project.

Explore