Naimuri
Safety and Security. Image Credits: Canva.
The company
Naimuri is a software and data engineering company specialising in data intelligence capabilities that help make the UK a safer place by bringing together talented specialists with expertise and a passion for technology.
The challenge
The advent of AI technologies brought Naimuri to the forefront of delivering secure, safe AI solutions that mitigate risks and ensure reliability.
Uncertainty quantification in AI is about measuring how confident an AI system is in its predictions or decisions.
The project's ambition was to explore approaches for quantifying uncertainty in deep neural networks, so that their outputs can be trusted by users and meet stringent security standards.
The goal
Reflecting on the goal of the project, James Ramsden, Data Science Capability Lead at Naimuri said “Our ambition was to develop AI systems that not only performed exceptionally but could also communicate their reliability to users.”
In collaboration with the National Innovation Centre for Data (NICD), they focused on enhancing their capabilities in uncertainty quantification. A key use case in this initiative involved applying natural language processing applications.
The collaboration sought to enhance Naimuri's ability to deliver AI systems with a clear understanding of their operational reliability and the associated risks.
Representation of a neural network. Image Credits: Canva.
The result
The collaboration yielded six prototype AI systems aware of their own uncertainty, highlighting Naimuri's enhanced capabilities in delivering reliable AI solutions.
The success of these prototypes has led to follow-on projects worth approximately £1.4 million, enhancing Naimuri's strategic position.
James highlighted the adaptability of their collaboration with the National Innovation Centre for Data, stating "We applied similar methods that we were implementing in other use cases as part of this project to enhance compute efficiency, but we also expanded our horizons into the realm of natural language processing (NLP)."
Emphasising the success of these approaches, James said "We discovered approaches to uncertainty quantification that were novel to us and to the sector. These methodologies were integrated back into our computer vision projects, enhancing their effectiveness and reliability. Our customer described the final report that we delivered for the project as a turnkey text that they're going to keep on their desk at all times." He concluded, "These were extremely successful outcomes."
Image credit: Canva
Enhancing AI reliability and trustworthiness
Data Science Tools and Techniques
In the pursuit of developing a solution that enhances AI reliability and trustworthiness, Naimuri, in collaboration with the National Innovation Centre for Data, harnessed a variety of advanced data science tools and techniques.
Central to this effort were open-source Transformer models such as DistilBERT sourced from Hugging Face, a platform renowned for its vast collection of pre-trained models tailored for natural language processing tasks. DistilBERT, known for its efficiency and performance, is a streamlined version of the more complex BERT model.
Model training was carried out using the Trainer class within the Hugging Face’s Transformers package. The Trainer class provides an API for feature-complete training in PyTorch.
This selection of tools facilitated the creation of a comprehensive code repository. This repository is a treasure trove of resources, including:
- learning materials on natural language processing,
- code for fine-tuning Transformer models for text classification,
- guidelines for employing multi-billion parameter language models in inference tasks on a single Graphics Processing Unit (GPU),
- and instructional content for training a Transformer model from scratch.
Each component of the repository exhibits the practical application of these sophisticated tools in addressing real-world challenges, ensuring that Naimuri's AI systems are both reliable and efficient.
"Working with the National Innovation Centre for Data was an iterative and dynamic exchange, driven by a two-way conversation that greatly enhanced the project's success. Working closely with Mac Misura and Matt Edwards, allowed us to deeply engage with the mathematical and theoretical aspects of AI."
James Ramsden, Data Science Capability Lead, Naimuri
Data centres are where cloud servers are located. Image credits: Canva
Business impact
This project led to the formation of a dedicated AI assurance segment within Naimuri's data science team, underpinning the need for a solid mathematical basis in AI development. It contributed significant internal and external recognition, securing Naimuri's role as a leader in AI solutions.
Naimuri continues to leverage the insights and methodologies developed during this collaboration, embedding them into ongoing and future projects. Commenting on the impact of the project, James said “The bigger follow-on work that we are doing now is still one small part of a huge strategy. Doing so well on that one smaller research project with the National Innovation Centre for Data (NICD) has meant that we are in a great strategic position to be to be working in that area with that existing customer, as well as other customers.”
Naimuri is actively advancing AI assurance, focusing on upskilling its team and pioneering technology applications in key sectors. The firm's current efforts include developing taxonomies for data team roles and analysing risk factors to refine mitigation strategies.
Although today's approach encompasses AI assurance broadly, Naimuri envisions a more defined future structure. The company plans to delineate roles among AI Assurance Engineers, Data Scientists, and Platform Developers, each tasked with specific AI assurance responsibilities.
Looking toward the future, James said “AI assurance stands as a vast and perplexing domain, incorporating every aspect of data science within Naimuri's operations, which are segmented into five distinct areas."
James concludes "As it stands, the roles of AI assurance engineers, the tools they would use, and the specific requirements from our customers are yet to be clearly defined. This means that presently, AI assurance encompasses everything from building cloud infrastructure and software components to developing the methods and tools needed for ensuring AI's reliability.”
"The collaboration with the National Innovation Centre for Data was a cornerstone in our journey towards AI assurance. It not only advanced our technical capabilities, but also shaped our approach to developing trustworthy AI systems for critical applications."
James Ramsden, Data Science Capability Lead, Naimuri
To find out more about Naimuri, visit their website.
You can read more of our case studies and sign up to our newsletter to keep up to date with our latest news, events and developments.
Our Discovery workshop
Our Discovery workshops enable you to explore the potential of your data and understand the benefit you could gain before committing to a full-scale project.