Impact of AI on the regulation of medical products
Published 30 April 2024
© Crown copyright 2024
This publication is licensed under the terms of the Open Government Licence v3.0 except where otherwise stated. To view this licence, visit nationalarchives.gov.uk/doc/open-government-licence/version/3 or write to the Information Policy Team, The National Archives, Kew, London TW9 4DU, or email: psi@nationalarchives.gov.uk.
Where we have identified any third party copyright information you will need to obtain permission from the copyright holders concerned.
This publication is available at https://www.gov.uk/government/publications/impact-of-ai-on-the-regulation-of-medical-products/impact-of-ai-on-the-regulation-of-medical-products
Introduction
The Medicines and Healthcare products Regulatory Agency (MHRA) is the independent regulator of medicines, medical devices and blood components for transfusion in the UK. We operate in a statutory framework set by HM Government. Our responsibilities are to;
- ensure medicines, medical devices and blood components for transfusion meet applicable standards of safety, quality and efficacy (effectiveness)
- secure safe supply chains for medicines, medical devices and blood components
- promote international standardisation and harmonisation to assure the effectiveness and safety of biological medicines
- educate the public and healthcare professionals about the risks and benefits of medicines, medical devices and blood components, leading to safer and more effective use
- enable innovation and research and development that is beneficial to public health
- collaborate with partners in the UK and internationally to support our mission to enable the earliest access to safe medicines and medical devices and to protect public health.
As a science-led organisation, the MHRA has a key role to play in enabling the UK to be a science and technology superpower by 2030. The agency is considering the opportunities and risks of Artificial Intelligence (AI), which has been identified as one of the five critical technologies in the UK Science and Technology Framework, from three different perspectives.
1. As a regulator of AI products
Where AI is used for a medical purpose, it is very likely to come within the definition of a general medical device, meaning it must meet the requirements of the UK Medical Devices Regulations 2002 (as amended) in order to be on the market in the UK. The MHRA is currently undertaking a programme of regulatory reform for medical devices. This includes ensuring there is proportionate regulation of AIaMD which takes into account the risks of these products without stifling the potential they have to transform healthcare.
2. As a public service organisation delivering time critical decisions
AI offers us the opportunity to improve the efficiency of the services we provide, across all our regulatory functions from regulatory science, through market access for medicines and medical devices, to post market surveillance and enforcement. More efficient regulatory services will lead to earlier access to medical products for UK patients and enable us to focus our talented resources more on priority activities that demand their advanced skills, such as enabling innovation and engaging with patients.
3. As an organisation that makes evidence-based decisions that impact on public and patient safety, where that evidence is often supplied by third parties
As a regulator of medical products we consider submissions, inspect premises and examine data relating to the products, protocols and practices we regulate. Increasingly, we expect AI to feature in how those we regulate undertake their activities and generate evidence. We need to ensure we understand the impact of that in order to continue to regulate effectively.
The Pro-innovation approach to the regulation of AI
The MHRA welcomed the publication of the Pro-innovation approach to the regulation of AI and has taken significant steps, in the 12 months since its publication, to adopt its recommendations into the work we do. The approach introduced five key principles.
Table 1: Five key principles for regulatory use of AI
| Principle | High level summary |
|---|---|
| Safety, security and robustness | AI systems should function in a robust, secure and safe way throughout the AI life cycle, and risks should be continually identified, assessed and managed. |
| Appropriate transparency and explainability | AI systems should be appropriately transparent and explainable. |
| Fairness | AI systems should not undermine the legal rights of individuals or organisations, discriminate unfairly against individuals or create unfair market outcomes. |
| Accountability and governance | Governance measures should be in place to ensure effective oversight of the supply and use of AI systems, with clear lines of accountability established across the AI life cycle. |
| Contestability and redress | Where appropriate, users, impacted third parties and actors in the AI life cycle should be able to contest an AI decision or outcome that is harmful or creates material risk of harm. |
The approach set an expectation that regulators would:
- Assess the cross-cutting principles and apply them to AI use cases that fall within their remit.
- Issue relevant guidance on how the principles interact with existing legislation to support industry to apply the principles.
- Support businesses operating within the remits of multiple regulators by collaborating and producing clear and consistent guidance, including joint guidance where appropriate.
- Monitor and evaluate their own implementation of the framework and their own effectiveness at regulating AI within their remits.
This publication provides an update on that work and is the MHRA response to the Secretary of State letter of 1 February 2024.
Part 1 – The MHRA as a regulator of AI products
Current Regulations
The Medical Devices Regulations 2002 govern medical devices in Great Britain. A medical device is defined in those regulations as an instrument, apparatus, appliance, material or other article which is intended to be used for human beings for a medical purpose. That medical purpose can be the diagnosis, prevention or treatment of disease of injury, the replacement or modification of anatomy of physiological process, or the control of contraception. Software, and its subset AI, used for a medical purpose, is within the remit of the UK MDR 2002. This includes whether it is used alone or in conjunction with other medical devices.
AI as a medical device (AIaMD) products must conform to the provisions of the UK MDR 2002, (or for a transitional period, with similar EU law), before they can be placed on the market in GB. These regulations cover the lifecycle of the product from clinical investigation pre-market, through assessing conformity with regulations, registering the product with the MHRA and placing it on the market. It includes legal requirements for manufacturers to mitigate risk and address safety and performance of devices throughout the product lifecycle, mandating post-market surveillance and post market clinical feedback activities. Manufacturers are expected to have established vigilance systems and processes for continuous monitoring and reporting of safety incidents directly to the MHRA.
The regulations include clear responsibilities and accountabilities for manufacturers of medical devices, the MHRA and conformity assessment bodies. There are also requirements for manufacturers to have appropriate Quality Management Systems (QMS), which ensure that there is traceability and accountability for decisions, actions and works undertaken by a manufacturer and individuals in relation to the ideation of a device through to development/ manufacture, certification/approval and market release.
Regulatory Reform
About AI Airlock
The MHRA’s AI-Airlock is a regulatory sandbox for AIaMD which will launch in pilot form in spring 2024.
This pioneering work is a collaborative project which will enable us to start to identify and address the novel regulatory challenges for AIaMD and answer previously unanswered questions.
Using real-world products and challenges, the AI-Airlock will bring together the expertise of key partners including the UK Approved Bodies, the NHS and other regulators.
The use of AI has developed considerably since 2002 so the regulations have been supplemented by guidance including the comprehensive Medical devices: software applications (apps), which has evolved over time in line with caselaw and developments in the sector.
International alignment is critical for businesses that operate in a global environment. The UK is a full member of the International Medical Devices Regulators Forum, (IMDRF), which is committed to greater harmonisation of regulation across the world. The UK is the co-chair of the IMDRF working group on AI and Machine Learning enabled medical devices.
Whilst the current regulations map well to the five principles of the AI White Paper, the UK leaving the EU presented an opportunity to update those regulations. The current UK MDR uses a risk-based classification system. This classifies products from Class 1 (low risk) to Class III (highest risk) devices. The higher the risk class, the greater the scrutiny required. This system will continue in the reformed regulations however many AI products, currently in the lowest risk classification meaning they can be placed on the market without an independent assessment of conformity, will be up-classified. This will protect users and patients through greater scrutiny throughout the product lifecycle.
For AIaMD, we are particularly mindful of the need to adopt a proportionate approach which takes account of the unique challenges of these types of products. In 2021 we published our approach in our Software and AI as a Medical Devices Change Programme – Roadmap. We will use principles supplemented by guidance to avoid constraining innovation, where this can be done safely. This will include clear guidance on cyber security due for publication by spring 2025.
Principle 2 of the AI White Paper considers transparency and explainability of AI. Current regulations set requirements for usability, labelling and the information manufacturers must supply for intended users. This should include how the device works. All devices, including AI devices are mandated to be designed taking into account the intended users, whether they are healthcare professionals or home users and manufacturers must provide a clear statement of the purpose of the device. For software products, including AI, this presents particular challenges, so the MHRA has provided guidance to support manufacturers in this process: Crafting an intended purpose in the context of Software as a Medical Device.
A key risk for AIaMD is the human/device interface. Usability engineering processes implemented by manufacturers, address human factors and ergonomic risks of devices. Existing MHRA guidance on applying human factors to medical devices will be supplemented by further detailed guidance specifically for AIaMD products, in spring 2025.
Principle 3 of the AI White Paper addresses fairness. This is critical for the MHRA as was highlighted in the recent publication of the Independent Review of Equity in Medical Devices. The MHRA is fully committed to ensuring equitable access to safe, effective, and high-quality medical devices for all individuals who use them. As Dame Margaret Whitehead notes, the advance of AI brings with it not only great potential benefits to society but also possible harm through inherent bias against certain groups in the population, notably women, ethnic minorities and disadvantaged socio-economic groups.
Again, in this area, the MHRA looks to take an internationally aligned position, encouraging manufacturers to refer to ISO/IEC TR 24027:2021 Information technology, Artificial intelligence (AI), Bias in AI systems and AI aided decision making and to IMDRF guidance document N65. The MHRA also worked as part of an international initiative called STANDING Together, involving patients, researchers, healthcare professionals as well as industry experts and regulators, culminating in the publication of recommendations for diversity, inclusivity and generalisability in AI health technologies and health datasets.
The existing regulations set obligations for manufacturers, conformity assessment bodies and the MHRA, but these will be strengthened and clarified in the new regulations, including for other economic operators in the supply chain, thus strengthening accountability and governance in line with principle 4 of the AI White Paper. Within the MHRA our governance includes a Software and AI Expert Advisory Group which consists of external experts in the field. This is a subgroup of the Interim Devices Working Group.
For AIaMD, accountability and governance are also applicable to the datasets used in the creation of AI models and potential changes that occur to the model in post market use. In collaboration with partners in the US Food and Drug Administration (FDA) and Health Canada, the MHRA has recently published guidance on principles of Predetermined Change Control Plans (PCCP) to enable full traceability and accountability of manufacturers for how AI models meet intended use as well as the impact of changes. Further, we intend to introduce PCCPs in the future core regulations, initially on a non-mandatory basis, to better govern the full-lifecycle management of AIaMD products, providing enhanced accountability for products. The ability to monitor AIaMD product changes will also assist with the contestability and redress aspects of AIaMDs (Principle 5 of the AI White Paper).
The MHRA Yellow Card scheme enables anyone to report concerns to the MHRA about a medicine or device, including one incorporating AI. Current regulations also place legal requirements for manufacturers to report incidents to the agency, and these obligations will be strengthened for medical devices, by new regulations which we aim to put in place by the summer. In a patient pathway there may be multiple products and healthcare professionals working together. Where there are safety concerns the MHRA works with other regulators such as the Care Quality Commission (CQC), the Health Research Authority (HRA) and the Human Fertilisation and Embryology Authority (HFEA), as needed. We continue to work closely with other system partners including via the AI and Digital Regulations Service for health and social care to provide support for both developers and adopters of AI technology within healthcare.
MHRA guidance in place and planned for Software (including AI) as a Medical Device
Table 2: Guidance already in place and planned
Resources and budgets allocated to this work
We currently have approximately three full-time equivalent (FTE) employees working on AI as a Medical Device workstreams. This is forecast to rise to ~7.5 FTE through the year 24/25. This is funded (2 FTE) from the £10m government funding provided to the MHRA in the 2023 budget. In addition, the AI Airlock, a regulatory sandbox for AI medical devices in healthcare, is being developed with funding from the Department of Health and Social Care AI Lab.
Part 2 – The MHRA as a public service organisation delivering time-critical decisions
AI offers us the opportunity to improve the efficiency of the services we provide, across all our regulatory functions from regulatory science, through market access for medicines and medical devices, to post market surveillance and enforcement. More efficient regulatory services will lead to earlier access to medical products for UK patients, enabling us to focus our talented resources on priority activities that demand their advanced skills, such as enabling innovation and engaging with patients.
Like many other organisations, we are early in our journey of discovering the potential of AI in the way we deliver our services. As we progress this work, we are aligning to key government strategies including the National AI Strategy and NHS National Strategy for AI in Health and Social Care. Senior leaders in our Digital and Technology Group are also members of cross government groups that contribute to the development and implementation of strategies, best practice and roadmaps. To ensure that any efficiency gain is not at the expense of the quality of the decisions made, and that the reason for those decisions can be explained at a future date should that become necessary, the five principles of the AI White Paper will also be adhered to as we develop our approaches.
Improving the quality of applications for medicines licences
MHRA assessors perform an initial assessment on the documents that are submitted as part of the applications for marketing authorisation or approval.
The initial assessment involves checking the completeness, consistency, and quality of the documents, and identifying any gaps, errors, or discrepancies against a set of criteria or standards that are relevant for the type and category of the product. The MHRA is exploring the use of supervised machine learning to assist the assessors in this task. Supervised machine learning is a type of AI that can learn from labelled data and make predictions or classifications based on the learned patterns or rules.
The MHRA aims to use supervised machine learning to do an initial assessment on the documents and provide a score or a recommendation for each criterion or standard.
This application would reduce the need for human involvement in these early assessment stages, enabling human expertise to be better targeted to make the critical assessment of whether the benefits of the product outweigh the risks.
As we design this approach we are drawing on learning from public and private sector organisations who have deployed similar approaches including in Canada, Australia and Switzerland.
Approach to expanding and optimising AI in the delivery of regulatory services
Safety, security and robustness, including cyber security is the first principle of the AI White Paper and achieving this requires investment in getting the basics right.
Data structures, classification, security and sharing rules which are fit for purpose, will be essential for the successful expansion of AI into new use-cases. The AI White Paper also highlights the need for fairness, ensuring AI systems do not discriminate unfairly against individuals.
For the MHRA this means well characterised training and validation datasets across all use cases. We are developing an MHRA data strategy which will include a theme on safely and responsibly applying advanced analytics and AI within the business. Within this, there is a deliverable on exploring the role of Large Language Models (LLM) and Generative AI across the business including the capabilities we will require across tech, people and partnerships.
The application of machine learning and AI to real world data (RWD) to enhance our ability to understand the relationship between exposure to medical products and clinical outcomes
To enable the effective use of AI in this way, we need timely access to RWD of known quality which is representative of the population exposed to the product of interest.
Addressing these challenges will require cohesive and coordinated efforts across the UK health data ecosystem to address longstanding challenges of siloing and fragmentation.
The forthcoming Sudlow Review will provide context and insight into the opportunities and challenges within the UK health data ecosystem.
Deriving actionable and robust insights from scientific data also requires clearly-framed hypotheses and sound methodology.
Whilst there is considerable potential in the use of Generative AI and Large Language Models, there are significant uncertainties around best practice for their use, and focus on these modalities should not be at the expense of well-characterised, statistically-robust approaches to advanced analytics and machine learning.
We will be actively exploring the role of these analytical approaches to enhance and extend the generation of actionable insights from RWD over the coming years.
Similarly, we will explore the roles of such methodologies, such as Bayesian approaches, within our vigilance systems including our spontaneous reporting systems.
Determining the best use-cases requires business as well as technology input and may best emerge through experimentation in a safe environment. To that end we are working with a developer of a generative AI tool that helps users write documents or analyse data by providing suggestions and feedback, providing access to selected users across our business to enable them to explore the potential to support them in the work they do.
This is not the only tool we are working with and so far we have 12 potential use cases to start prototyping the different technologies and paradigms such as Generative-AI, LLM, Machine Learning etc. Use cases range from productivity through to decision making cross all areas of the business including, communications, customer service and helpdesk, as well as across the regulatory activities for the whole product lifecycle.
We have updated our 12 month technology roadmap with emphasis on three strategic themes: innovation, eradication of legacy, and cybersecurity which are fundamental to the successful deployment of AI. We also have a refreshed plan for information governance and data protection policies as well as for new ways of working.
Protecting consumers from fraudulent medical products
The online sale of medicines and medical devices is a growing phenomenon, offering convenience, anonymity, and lower prices for consumers in search of, for example cheap diet pills and cosmetic products, as frequently reported in the news. It also poses significant risks, as many online pharmacies are unregulated, illegal, or fraudulent.
Fake products are a serious threat to public health and safety, as they can contain harmful or ineffective ingredients, or be mislabelled or counterfeit. They can cause adverse reactions, treatment failure, antimicrobial resistance, and even death.
The illicit trade that exists online in all commodities is truly global, with illegally trading websites generally based overseas and beyond the reach of UK legislation. There is no single UK regulator for the internet. Tackling it requires partnership working.
The MHRA monitors online channels and, where possible, works with partners to disrupt illegal trade.
We are currently developing a Medicines Website Checking tool which will allow members of the public to report a website, social media platform or online marketplace they suspect of selling fake or illegal medicines and medical devices to the MHRA. This will allow us to investigate, and where appropriate take enforcement action, and update our list of unsafe websites for the public to check against.
In parallel we are starting work with other global regulators and technology partners to begin prototyping products with AI in this area. This follows on from proof of concepts work with private sector organisations who have done more advanced work in these areas which, along with support from technology partners, has expedited our learning.
We have prioritised the development of our cyber strategy and implementation plan and budget to defend against AI and utilise AI to strengthen cyber resilience. Working closely with national cyber security centre (and global equivalents) and aligning with relevant frameworks. Increased cyber visibility is on our corporate risk register and is now a regular item on senior governance forum agendas.
The MHRA was one of the first regulators to use AI to increase the efficiency and effectiveness of our vigilance systems. This was initially for COVID-19 vaccine suspected adverse reaction reports as an additional quality assurance step to ensure that information in free text is coded to the structured fields used for signal detection. In excess of 100,000 previous vaccine reports were used for training and validation, with rules applied over the top of the technology to ensure adequate control of the system.
Building on that into further use-cases, the principles of the AI White Paper run through our approaches. Expanding the use of AI, machine learning, and advanced analytical approaches to improve how we deliver timely, proportionate, and scientifically-robust regulatory decisions in future will require those decisions to remain transparent and explainable, subject to clear governance, and to avoid inadvertently creating or exacerbating health inequalities, they also need to be fair. Security, including cyber security, is central to how we operate as an organisation that holds patient and commercial data. This requires us to consider the AI White Paper principles not only for the AI tools we use, but also with respect to the data those tools are used upon.
Resources and budgets allocated to this work
The resource deployed to this second theme is primarily in our Digital Technology and Safety and Surveillance groups and is integrated with other work rather than dedicated only to AI. Across the organisation we have in the region of 15 FTE engaged in this work.
Part 3 – The MHRA as an organisation that makes evidence-based decisions that impact on public and patient safety, where that evidence is often supplied by third parties
As a regulator of medical products we consider submissions, inspect premises and examine data relating to the products, protocols and practices we regulate. Increasingly we expect to AI to feature in how those we regulate undertake their activities and generate evidence. We need to ensure we understand the impact of that in order to continue to regulate effectively.
Since 2020, we have made extensive efforts both to engage the pharmaceutical industry in their use of AI for vigilance purposes and to optimise our own systems using the technology. There is a robust legal framework for pharmacovigilance activities and whilst there is no specific reference to AI within the regulations, there are requirements around quality management systems to provide assurance about the standards under which data is processed.
We are also collaborating with international regulatory bodies and pharmaceutical industry through the Council for International Organizations of Medical Sciences to develop best practice in use of AI across organisations, with a view to international alignment of expectations to avoid burden on both industry and regulators.
Many of the changes our customers make will not impact on how we regulate. The questions that we as the regulator need to ask to determine whether a product is safe do not change, when the nature of the evidence we consider changes, for instance the use of in silico data in the place of animal models. What may change is the pace at which new medicines can be developed for example through the use of AI to reduce the costs and time invested in failed molecules. It is also likely to impact on clinical trial design and is an enabler of personalised medicines. The MHRA’s horizon scanning, scientific advice and accelerated access pathways for innovative products, as well as our strong focus on regulatory science, including our recently announced discovery on Centres of Excellence for Regulatory Science, will enable us to ensure our regulatory pathways are sufficiently agile and robust to respond to these changes.
Conclusion
The MHRA recognises the transformative potential of AI in shaping the future of healthcare and the significant contribution AI can make across all its regulatory functions. We have adopted the five key principles set out in the Pro-innovation approach to the regulation of AI and we are determined to embrace emerging technologies with vigilance and foresight, to ensure that we use them effectively for the greater good of public health.
We are committed to taking a proportionate approach to regulating AI medical products, which takes into account the unique challenges of these products. Our collaboration with the IMDRF and in particular our partnership with the US FDA and Health Canada, will ensure safe and responsible use of AI in medical products and keep the UK at the forefront of international best practice.
As a public service organisation delivering time critical decisions, and as an organisation that makes evidence-based decisions that impact on public and patient safety, we are committed to exploring opportunities to make best use of AI. We are committed to embracing AI responsibly, in alignment with the five key principles, to protecting public health and advancing innovation in the healthcare landscape.