Putting public engagement at the heart of research

Congenital heart disease is the most common type of birth defect and covers a range of disorders, from a small hole in the heart to more severe conditions where a child needs heart surgery to survive. Around 3500 children under the age of 16 years undergo heart surgery in the UK each year and overall, 98% of children survive for at least 30 days afterwards.

Access to reliable information on the outcomes of cardiac surgery is understandably important to these children’s parents, and it is also of interest to clinicians, the media and the general public.

Since 2000, information on the outcomes of all UK paediatric heart surgery has been recorded for the National Congenital Heart Disease Audit. In 2010 the NIHR Health Services and Delivery Research Programme funded researchers at University College London and Great Ormond Street to develop a statistical formula and bespoke Excel software to let hospitals easily monitor survival outcomes for children within 30-days of undergoing cardiac surgery.

Referred to as PRAiS (Partial Risk Adjustment in Surgery), this method considered not only the surgical procedure carried out, but also other factors that may affect a child’s outcome, such as the severity of their heart defects, type of operation, age and any additional medical conditions. These additional details are used for “risk adjustment”, to account for the fact that hospitals operating on children who are more severely ill are likely to have lower survival rates than hospitals operating on less sick children. 

In 2013, all UK paediatric cardiac surgical units and the national audit body began using the PRAiS software. The research team has since been awarded the Lyn Thomas Impact Medal in recognition of the significant impact of its development on children with heart conditions and their families.

However, the use of PRAiS monitoring initiated the events that resulted in the temporary suspension of surgery at one unit over concerns about its performance in surgery after inaccurate data were used.

The high level of interest in the closure and misinterpretation of the use of PRAiS highlighted the importance of developing a resource to help the public interpret the outcome data of children’s heart surgery and led to the follow-up PRAiS II study.

Although 98% of children survive for at least 30 days after cardiac surgery, death following surgery is the main outcome reported and has often been interpreted as a measure of a hospital’s performance. For reporting of these outcomes to be fair to the clinical teams, the PRAiS research team recognised that the model needed to take greater account of children’s additional health problems as these also affect a child’s chance of survival.

In 2015 the NIHR provided funding to Christina Pagel, Professor in Operational Research at UCL and member of the original PRAiS research team, for the follow-up study,PRAiS II The aim was to improve the model to take better account of children’s specific medical conditions, and to distribute the updated software to all UK units free of charge.

Dr Trevor Richens, consultant in congenital heart disease at University Hospital Southampton, said: “The introduction of PRAiS II has ensured units are not penalised for operating on the sickest patients or doing the most complex operations, and has become the gold standard for ensuring all UK units deliver excellent outcomes.”

Alongside improving the statistical formula for the PRAiS model, Professor Pagel wanted to communicate to the public how the survival data should be interpreted. The project team of mathematicians and psychologists from UCL, the University of Cambridge and King’s College London, alongside Sense about Science (a charity>promoting the public understanding of science), began work on an online resource to achieve this. Professor Pagel said: “The NHS routinely monitors the outcomes from surgeries, particularly for specialised areas like congenital heart disease, but the reports are often hard to find and then hard to read. I strongly believe that for the NHS to be transparent about outcomes, data need to be easy to find and easy to understand, not just available.”

In a novel approach to building a website to explain clinical outcomes, parents of children with heart conditions and other members of the public attended workshops to co-develop content for the new Understanding Children’s Heart Surgery Outcomes website. This aimed to clearly explain the complex and sensitive topics of children’s heart surgery and survival data. Their input helped the team develop meaningful content and key messages to make clear what the survival data could and could not tell people about the outcome of children’s cardiac surgery. 

Alex Smith, a participant in the study, said: "When my daughter, Georgina, was diagnosed with a heart condition, there wasn't anywhere to go to find clear and concise information about children's heart surgery and hospitals where their children were being, or going to be, treated.” 

"This project allowed me, along with a group of parents, to help researchers shape the website by looking at the information and language, and how data was being displayed, to make it easier for people to understand."

The NIHR and Sense about Science both emphasise the importance of involving potential users in research. Representing Sense about Science, Emily Jesper-Mir commented on the importance of also sharing information with the public. Equipping people with the tools, information or questions to scrutinise data is key to Sense about Science’s approach. Displaying these data on the website provides a shared picture which can properly inform debates. In our experience of public engagement partnerships, there are no subjects that are too sensitive or too complex if you engage people honestly.”

The carefully reviewed website content helped to explain that because hospitals treat different patients, hospitals should not be ranked by their survival rates. For example, a high-performing hospital may have a lower survival rate because they treat the most complex cases.

The website was launched in 2016 with the endorsement of charities, clinicians, NHS Choices and the national audit, with an editorial in The Lancet editorial recommending that other areas of medicine that needed to explain risk to the public should use it as an example. 

The Children’s Heart Federation is a charity that supports families of children with heart conditions, which helped recruit parents to participate in the workshops. They said: "The Understanding Children's Heart Surgery Outcomes website provides important and rather complicated statistics in a clear, accessible way. It is good to be able to share this information with parents who approach us with concerns about their child’s heart surgery.”

From his perspective as a doctor treating children with heart conditions, Dr Richens noted:“Public understanding of these outcome measures has benefited enormously from the introduction of the Understanding Children’s Heart Surgery Outcomes website. There is no doubt families benefit from the explanations provided on the website, having a better understanding of the risks associated with the procedures their children are going through."

The NIHR recognises the valuable insights that patients and the public offer to its research, which also helps to make the research more accessible and relevant for everyone. Following the team’s experience of co-developing the website, the NIHR encouraged them to extend their collaboration with Sense about Science to produce a best practice guide. 

The Five-step guide to involving the public in communicating research gives advice to researchers on involving the public to shape and co-design how research findings or information are presented. Emily Jesper-Mir explained that the guide “coaches research teams to develop thoughtful, responsive and reflective public engagement plans, giving researchers the confidence to involve people at the earliest stages of research, and to listen, identify and reach out to audiences.”

The guide has been widely circulated and adopted by a number of research communities, including the Economic and Social Research Council and the British Neuroscience Association.

When describing her experience of public engagement, Professor Pagel reflected that: “Pretty much every aspect of the website was improved through the involvement of families and the wider public. There were many times when they pointed out ways the charts or text could be misinterpreted that I would never have seen in a million years.” 

“I learned so much and now the idea of building this sort of resource without the close involvement of the public seems like madness.”  

Further reading:

• The study was funded by NIHR’s Health Services and Delivery Research Programme. 
• More information about the study is available on the NIHR’s Funding & Awards website
• Read more making a difference stories