There is a risk of deskilling health and social care staff through the overreliance on artificial intelligence tools, the health and safety watchdog has warned.
The Health Information and Quality Authority (Hiqa) on Wednesday published the State’s first national guidance on the responsible and safe use of AI in health and social care services.
The guidance acknowledges AI is being used to address challenges in the sectors, such as rising costs, increased demand due to an ageing population and increased prevalence of chronic and complex disease.
Innovations are now being used for a variety of tasks, including streamlining administrative responsibilities, supporting diagnostics by identifying patterns in scans and predicting medical outcomes.
“AI is a promising tool that can be integrated into the delivery of health and social care services to support staff and help address some of these challenges,” the report said.
However, it warned there are also risks associated with the use of AI that must be guarded against.
“Examples of potential risks include the use of data without people’s consent, unintentional breaches of people’s personal data, staff uncritically accepting AI outputs without human oversight, staff misinterpreting unreliable recommendations from unvalidated or biased AI tools and deskilling of staff due to overreliance on AI tools,” the report said.
The national guidance has four principles: accountability, a human rights-based approach, safety and wellbeing, and responsiveness.
According to the guidance, any new technology must be deemed safe and trustworthy before use.
[ How accurate are AI chatbots when we ask them a medical question?Opens in new window ]
There also must be robust governance and clear lines of accountability embedded into services when AI is used to ensure care remains person-centred, it added.
It also highlighted the importance of AI being used to augment rather than replace human judgment and professional decision-making.
This may include staff directly overseeing decisions in real time or allowing the tool to operate autonomously with the ability of it to be overridden by a member of staff when needed.
Transparency is also an important component of AI use in the health and social care services, according to Hiqa.
“Transparent services inform people using services about when, where and for what purpose AI tools are used in their care, how they are used to support decisions, and what impact they might have on the care, safety, security and privacy of people using the service,” the guidance states.
It added that there must be traceability of AI usage to enable “swift action” if something goes wrong.
Rachel Flynn, Hiqa’s director of health information and standards, said the guidance provides “practical direction to help organisations and staff harness these opportunities while ensuring that patient safety, transparency and accountability remain at the forefront”.
The guidance was developed in collaboration with the Department of Health, the HSE and a variety of partners across the health and social care sector.
The European Union has also introduced rules about the use of AI, which will be applied on a phased basis in the coming years.
The guidance comes following the publication of the Government’s first national strategy for the use of AI in the system, with Minister for Health Jennifer Carroll MacNeill describing the technology as “a rare opportunity to reshape how care is delivered”.
The HSE and the Department of Health have consistently maintained that the use of AI in healthcare will not serve as a replacement for clinicians, and will instead be a tool for them.
The executive also said it would enable healthcare staff to spend less time on administrative tasks, and more time with patients.
