Insights North East | HaSS Faculty | Newcastle University
Background
With over 3,500 academics, Newcastle University is a powerhouse of expertise. But navigating that wealth of knowledge has long been a challenge. Identifying relevant research has proven to be a complex task, especially when responding to external opportunities such as requests to help policy makers. Insights North East (INE) and The Faculty of Humanities and Social Sciences (HaSS) and the University's Business Development team partnered with the National Innovation Centre for Data (NICD) to address this challenge through the development of a novel, AI-powered search tool.
The result? An application that helps surface relevant expertise quickly and more accurately, improving internal collaboration, reducing unconscious bias, and supporting engagement with industry as well as regional and national policy initiatives.
The challenge
James Annesley, Associate Dean for Research and Innovation at HaSS, and Jonny Gray, HaSS’s Business Development Manager, spend time brokering relationships between academics and external partners. Whether for funding bids, consultancy opportunities, or public policy collaborations, being able to identify the right expertise quickly is critical.
But traditional approaches, such as relying on personal networks, using static web directories, or searching via manual platforms, had major limitations.
“We often fall back on who we already know,” said James. “That introduces unconscious bias and can leave out early career researchers or those outside our immediate circles.”
Jonny agreed:
“We’ve long needed a more intuitive, scalable way to locate expertise—especially across disciplines. The university’s website and internal systems just weren’t up to the task.”
INE, a knowledge-exchange partnership between universities and local government organisations, funded by Research England, also had a strong need for such a tool. As a regional policy platform, INE regularly requires quick access to relevant expertise across faculties, not just in HaSS. Furthermore, INE must identify research expertise based on policy demands. Translating research outcomes into actionable insights and aligning research expertise with policy questions has proven to be a complex task.
In the past, INE relied on information available on academic profiles published on university webpages, manually building its own database. However, these webpages are often not regularly updated and may lack important information.
The vision
NICD came up with an innovative way of addressing this need using AI – it would build a tool that could automatically mine the university’s repository of academic publications and return contextually relevant results in response to natural language queries.
The ultimate goal was to streamline processes, reduce search time, promote diverse engagement with funding and collaboration opportunities, and uncover lesser-known pockets of expertise.
What we did
NICD ran a Discovery project from July 2024 to February 2025. Working closely with the HaSS Faculty, INE, Newcastle’s Business Development team, and ESRC IAA teams, NICD designed and delivered a custom semantic search tool.
Key stages included:
- Project scoping and stakeholder engagement to clarify goals and expectations.
- Tool and technique research to explore state-of-the-art methods in large language models (LLMs) and semantic search to select the best approach for this task.
- PDF parsing and metadata analysis of ~13,000 academic publications from Newcastle University’s research repository.
- Embedding model experimentation to generate high-dimensional vector representations of text, enabling semantic similarity comparisons.
- Cloud deployment of a containerised Django web application, hosted on Microsoft Azure, with user authentication.
- Automated updates using Azure Functions to maintain up-to-date content in the project database and generate an end-to-end pipeline.
- User interface development to ensure ease of use for non-technical users.
The final output was an intuitive, scalable, and secure application that allows users to type a natural language question, such as "Who’s researching climate impacts on public health?", and receive a ranked list of relevant academic papers based on the semantic meaning of their publications. Testing by users has shown that this performs much better for them than traditional keyword search.
“I really enjoyed working on this project and contributing to the development of a tool that brings together semantic search and LLMs to develop a real-world application used by Newcastle University staff. It was a valuable learning experience in LLMs, deployment and data engineering best practices. Our NICD team also put a lot of thought into potential future enhancements to evolve this tool into a more powerful solution.”
Dr Antonia Kontaratou, Data Scientist, National Innovation Centre for Data
“The project required NLP, AI engineering and data engineering skills. It was great to work on an applied NLP project that resulted in application that is used by colleagues at Newcastle University.”
Louise Braithwaite, Data Scientist, National Innovation Centre for Data
The results
An application that users have begun using to:
- Find collaborators for large research bids.
- Identify speakers for policy briefings and workshops.
- Respond to requests from policy-makers and industry.
- Explore potential interdisciplinary matches.
It has already surfaced previously unknown or overlooked researchers, often early-career academics, and helped reduce reliance on informal networks.
“It’s not a magic bullet,” said James, “but it gives us a different and more inclusive way to solve a long-standing problem. It’s helped me find people I wouldn’t otherwise have known to contact.”
“Some of the results validated what we already knew,” said Jonny. “But it also revealed new people doing important work, people who hadn’t been on our radar. That’s incredibly valuable.”
“The project resulted in a valuable tool that greatly simplifies the process of finding expertise within the university. By enabling more targeted inquiries on the preprints, we are now able to access more relevant and accurate work on specific topics. This is crucial for accessing the region's research capacity, not only for the academic community but also for external partners.”
Nayara Albrecht, Research Associate, Insights North East
The impact
While still a prototype, the tool is already proving valuable and holds significant potential for long-term impact. By automating the discovery of academic expertise using NLP and LLMs, the solution addresses several key issues:
- Faster response times to internal and external requests.
- Increased visibility for a broader range of researchers.
- Improved interdisciplinary collaboration across faculties.
- Streamlined engagement with policy stakeholders and civic partners.
- Reduced workload for staff trying to manually identify experts.
“It doesn’t duplicate effort,” said Jonny. “Once your papers are in the repository, the tool does the rest. That’s a big win in terms of usability."
Working with NICD
“The communication was seamless,” said Jonny. “We got clear updates and never felt out of the loop. And we always knew what the team was working on.”
“For someone like me, with no technical background, it was brilliant,” added James. “The team were professional, responsive, and respectful. I learned a lot—and the results speak for themselves.”
“Working with NICD was an excellent experience. The data scientists Louise Braithwaite and Dr Antonia Kontaratou provided outstanding guidance throughout the entire process, effectively engaging stakeholders through regular meetings and detailed reports. They also led insightful sessions that thoroughly explored the technical aspects of the work they developed, ensuring a comprehensive understanding for all involved.”
Nayara Albrecht, Research Associate, Insights North East
What's next?
The MVP is now being tested and used internally by selected university teams. While additional features could be added in a future phase, the tool in its current form is already delivering value.
“We’re open to a future phase,” said James. “But even if this remains a lightweight internal tool, it’s a real achievement. And it could be a model for other universities too.”
The NICD team is now exploring how this solution could be adapted or scaled for other institutions or sectors looking to surface and connect expertise.
“There’s real potential here,” said Jonny. “This could become a vital piece of university infrastructure, not just for us, but for others facing the same challenge.”
Discover more about Newcastle University and Insights North East by visiting their websites.
You can read more of our case studies and sign up to our newsletter to keep up to date with our latest news, events and developments.