At the NHSE, DHSC and NHS BSA Joint Data Science Conference, we ran a workshop on breaking AI agents on purpose. This is that workshop in article form: try the attacks yourself, then learn what defends against them.
This article introduces an application of the team's synthetic clinical notes dataset, published on Hugging Face. It is a very small agent example, built on purpose to demonstrate the open-source LangGraph framework and to show how little it takes to get started. This agent is engineered to handle autonomous summaries over clinical notes.
“There is no joy in possession without sharing.” — Erasmus
A spotlight on how the project 'Using LLM as a Judge to evaluate Gen AI Search' was marketed effectively by Will Poulett.
We have been exploring using an LLM to evaluate and assess LLM summaries. This utilises the speed and language understanding of LLMs to score summaries, but how much trust can we put into an 'LLM-as-a-Judge'?