This article was originally published as a guest column in Outsourced Pharma.
Considering the feverish pace of innovation in the field of AI/ML and the inevitable impact this family of technologies has on drug development, an overview of the approaches to AI/ML regulation by the leading medical product regulatory authorities, the FDA and European Medicines Agency (EMA), is timely. Below, we outline the documents and guidances the two regulators have released thus far, comparing and contrasting their areas of focus and concern.
A Comparison of the Definitions of AI and ML
Despite the lack of a universally accepted definition of AI among experts,1 both regulatory agencies have settled on a working definition of AI.
In its definition, FDA acknowledges the breadth and multidisciplinary nature of the field, defining AI as “[a] branch of computer science, statistics, and engineering that uses algorithms or models to perform tasks and exhibit behaviors such as learning, making decisions, and making predictions.”2 Meanwhile, FDA identifies ML as a subset of AI that allows “[m]odels to be developed by ML training algorithms through analysis of data, without being explicitly programmed.”3
EMA, however, takes a more mechanistic approach, defining AI as “systems displaying intelligent behavior by analyzing data and taking actions with some degree of autonomy to achieve specific goals.”4 Meanwhile, EMA’s definition of ML — “models [that] are trained from data without explicit programming” — mirrors FDA’s ML definition.
The FDA’s Approach
In May 2023, FDA began to consider the implications of AI/ML technologies for drug development with the publication of two discussion papers: Artificial Intelligence in Drug Manufacturing5 and Using Artificial Intelligence and Machine Learning in the Development of Drug and Biological Products.6 These two discussion papers highlight the agency’s areas of concern related to the incorporation of AI/ML in drug development and manufacturing.
Chief among these concerns are the governance, accountability, and transparency of AI/ML systems. For ML models, transparency and accountability are particularly challenging considering they are sub symbolic, or a “stack of equations — a thicket of often hard-to-interpret operations on numbers.”7 Thus, the nature of these systems makes its outputs difficult to interpret, presenting obvious regulatory challenges. To address these challenges, the FDA emphasizes the importance of “tracking and recording … key steps and decisions, including the rationale for any deviations and procedures that enable vigilant oversight and auditing.”8 The problem of transparency and accountability is further compounded by competitive concerns, as many of these models are proprietary.
Data quality is another concern the FDA addresses in its discussion papers, noting that the application of AI/ML systems in drug manufacturing can significantly increase the frequency and volume of data exchanges in the manufacturing process, thereby exponentially increasing the quantity of data. This increase in data output may require new considerations relating to data storage, retention, and security. In terms of data input, sponsors must be cognizant of any preexisting biases in the training data, as ML systems can easily duplicate or even amplify these biases.
The FDA also highlights reliability as another area of focus and concern. As recent experiences with large language models may attest, some AI systems are prone to hallucination, “a phenomenon where AI generates a convincing but completely made-up answer.”9 Indeed, in a recent study on AI hallucination, a group of researchers prompted a chatbot to generate a list of research proposals with reliable references. Of the 178 references provided by the chatbot, 69 did not have a digital object identifier (DOI), while 28 did not turn up on internet searches.10 Thus, FDA’s concern about reliability seems well founded, especially in the context of a drug development program.
The EMA’s Approach
Following the FDA’s recent publications, the EMA released a reflection paper11 advocating for a risk-based approach that considers patient safety and the reliability of development data. In April 2021, the European Union (EU) introduced a coordinated plan and a regulation proposal for AI, aimed at promoting innovation and ensuring AI benefits society. The reflection paper is an extension of this plan, outlining considerations for AI usage in drug development and emphasizing regulatory oversight based on risk assessment. It highlights three key concerns, specifically, the need for:12
risk-based oversight,
the establishment of strong governance for AI deployments, and
guidelines covering data reliability, transparency, and patient monitoring.
The paper categorizes the risk of AI application in drug development stages. AI use in early drug discovery is deemed low risk, while its use in clinical trials spans various risk levels depending on factors like human oversight and potential impact on regulatory decisions. To manage risks, the paper recommends transparent AI models (the idea to fully trace information flow within a ML model), cautious handling of issues like overfitting (the result of non-optimal modeling practices wherein you learn details from training data that cannot be generalized to new data), and appropriate performance assessment metrics. Ethical and privacy issues, such as human agency and oversight, technical robustness and safety, privacy and data governance, transparency, accountability, societal and environmental well-being, diversity, non-discrimination, and fairness, are addressed and outlined.
Specific considerations for AI usage include ensuring accurate AI-generated text through quality review procedures and high-risk AI decisions in precision medicine settings, AI use in manufacturing adhering to quality risk management principles, and the importance of regulatory interactions during development. The reflection paper acknowledges that it is not an exhaustive source of regulatory insight on AI but serves as a starting point for further discussions. Stakeholders can provide feedback until Dec. 31, 2023.
Conclusion
While both the FDA and the EMA strive to provide a framework that balances innovation and patient safety, nuances emerge in their respective approaches. Stakeholder input and evolving industry practices are critical to shaping future regulatory guidelines. Collaboration among regulators, manufacturers, and researchers will be pivotal in fostering a transparent, accountable, and efficient AI ecosystem that enhances the development and deployment of medical products for the betterment of global health.
FDA, “Using Artificial Intelligence and Machine Learning in the Development of Drug and Biological Products,” (May 2023), https://www.fda.gov/media/167973/download.
FDA, “Artificial Intelligence in Drug Manufacturing,” May 2023, https://www.fda.gov/media/165743/download.
FDA, “Using Artificial Intelligence and Machine Learning in the Development of Drug and Biological Products,” May 2023, https://www.fda.gov/media/167973/download.
Mitchell, Melanie. Artificial Intelligence: A Guide for Thinking Humans, p. 12.
FDA, “Using Artificial Intelligence and Machine Learning in the Development of Drug and Biological Products,” p. 20.
Athaluri SA, Manthena SV, Kesapragada VSRKM, Yarlagadda V, Dave T, Duddumpudi RTS. Exploring the Boundaries of Reality: Investigating the Phenomenon of Artificial Intelligence Hallucination in Scientific Writing Through ChatGPT References. Cureus. 2023 Apr 11;15(4):e37432. doi: 10.7759/cureus.37432. PMID: 37182055; PMCID: PMC10173677.
Ibid.
European Medicines Agency. (2023, July 13). Reflection Paper on the Use of Artificial Intelligence (AI) in the Medicinal Product Lifecycle. European Medicines Agency. https://www.ema.europa.eu/en/documents/scientific-guideline/draft-reflection-paper-use-artificial-intelligence-ai-medicinal-product-lifecycle_en.pdf.
European Medicines Agency. (2021, August 16). Artificial Intelligence in Medicine Regulation. European Medicines Agency. https://www.ema.europa.eu/en/news/artificial-intelligence-medicine-regulation.
This article was originally published as a guest column in Clinical Leader.
The Food and Drug Omnibus Reform Act (FDORA),1 signed into law by President Biden on Dec. 29, 2022, which we initially wrote about in February with regard to inspections, also includes numerous provisions intended to modernize clinical trials. These provisions enact and expand upon FDA initiatives over the last decade and could go a long way toward advancing clinical research. The provisions are broadly divided into two categories: (1) encouraging the enrollment of more diverse patient populations in clinical studies; and (2) facilitating novel clinical trial designs and methods for conducting clinical trials, in part by continuing certain flexibilities initiated during the COVID-19 public health emergency.
Clinical Trial Diversity
The diversity (or lack thereof) of patients enrolled in clinical trials has long been a topic of consideration and concern for those in the public health field. There is evidence that minorities, women, the elderly, children, and other demographic subgroups are underrepresented in clinical research, leading to gaps in understanding the safety and efficacy of certain drugs used in those populations.
In 2012, the Food and Drug Administration Safety and Innovation Act (FDASIA) recognized this gap and required the FDA to report on and create an Action Plan to improve the analysis of demographic subgroup data in clinical trials, expand the inclusion of such data in labeling, and make such data more available and transparent to the public.2 In the decade following FDASIA, the FDA took numerous steps to implement this FDASIA provision, including issuing the required report in August 2013, followed by the Action Plan in 2014, and convening a public meeting in between.3
The FDA’s efforts culminated in a draft guidance released in April 2022 (contemplated in the 2014 Action Plan), which encourages the inclusion of diverse patient populations in research conducted to support marketing applications for medical products.4 The draft guidance, titled Diversity Plans to Improve Enrollment of Participants from Underrepresented Racial and Ethnic Populations in Clinical Trials, describes when diversity action plans are recommended, the timing for submission of such plans, and the suggested content.
FDORA sections 3601-3604 further promote clinical trial diversity by enacting into law a requirement for diversity action plans as well as requiring additional public communications and information gathering to facilitate diversity in clinical trials, as described below.5
§ 3601. Diversity action plans for clinical studies
Amends the FD&C Act to require a “diversity action plan” for all Phase 3 or other pivotal studies (excluding bioequivalence and bioavailability studies) for drugs and biological products and for many medical devices.
The diversity action plan must include the sponsor’s goals for enrollment in the clinical study, the rationale for such goals, and an explanation of how it intends to meet the goals. The plan must be submitted with the clinical study protocol.
The FDA may waive the requirement for a diversity action plan in certain circumstances, including “based on what is known or what can be determined about the prevalence or incidence of the disease or condition for which the new drug is under investigation” or if conducting the study in accordance with a diversity action plan “would otherwise be impracticable” or is against the interests of public health during a public health emergency.
Applies to clinical trials for which enrollment begins 180 days after publication of the final guidance described in § 3602. See § 3602(c).
§ 3602. Guidance on diversity action plans for clinical studies
Requires the FDA to issue or update guidance (e.g., the April 2022 draft guidance) on the requirement for clinical trial diversity action plans, including on their format and content, modifications to such plans, public posting by a sponsor of key information from a diversity action plan that would be useful to patients and providers, criteria the FDA will use when determining whether to grant a waiver for the requirement to follow a diversity action plan, and regular reporting to the FDA on the sponsor’s progress in achieving the goals described in the diversity action plan.
Draft guidance is due 12 months after enactment of FDORA (by Dec. 29, 2023); final guidance must be published within nine months of the close of the comment period of the draft guidance.
§ 3603. Public workshops to enhance clinical study diversity
Requires the FDA, in consultation with drug and medical device sponsors, clinical research organizations, academia, patients, and other stakeholders, to convene one or more public workshops to solicit input from stakeholders on “increasing the enrollment of historically underrepresented populations in clinical studies and encouraging clinical study participation that reflects the prevalence of the disease or condition among demographic subgroups[.]”
The FDA must establish a public docket in association with the meeting and must publish a report on the topics discussed at the meeting(s) within 180 days after the public comment period closes.
§ 3604. Annual summary report on progress to increase diversity in clinical studies
Requires the FDA to publish an annual report beginning two years after enactment of FDORA (Dec. 29, 2024) summarizing, in aggregate, diversity action plans received by the FDA, whether the clinical studies conducted under diversity action plans met the enrollment goals in the action plans, and if not, the reasons provided for why not.
Clinical Trial Flexibilities
Similar to efforts surrounding clinical trial diversity, the FDA and other stakeholders have been considering novel clinical trial designs for at least a decade, including whether the FDA’s regulatory framework must be modified to accommodate the oversight of such trials and allow evidence generated from those studies to satisfy the “substantial evidence” statutory standard.6
The FDA held a public hearing to solicit input on a broad effort to “encourag[e] the use of innovative models that may enhance the effectiveness and efficiency of the clinical trial enterprise”7 in 2012 and was subsequently granted new authorities related to novel clinical trial design and real-world evidence (RWE)/real-world data (RWD) in the 21st Century Cures Act of 2016.8 These efforts continued in PDUFA VI (FY18–FY22) and PDUFA VII (FY23–FY27), both of which include commitments related to RWE/RWD and to advancing the FDA’s capacity to review complex innovative trial designs, including Bayesian and adaptive protocols.9 As a result, the FDA has issued several guidance documents reflecting updated thinking on clinical trial design, trial conduct and compliance with good clinical practice (GCP), and the collection and analysis of data.10
A number of these initiatives were further advanced during the COVID-19 pandemic through flexibilities the FDA exercised to protect the safety of patients and investigators and to ensure adherence to GCP, while also minimizing disruptions to clinical research. The efforts are described in a guidance the FDA issued early in the pandemic that allowed for certain accommodations in the conduct of clinical trials.11 For example, the guidance described when it could be appropriate to conduct remote clinical visits for study participants, remotely monitor clinical sites, and ship investigational product to local providers for administration. The flexibilities outlined in the guidance were intended to address challenges associated with quarantines, travel restrictions, supply chain interruptions, and other disruptions associated with COVID-19.
With the termination of the public health emergency on May 11, 2023, many of the FDA’s guidances issued during the pandemic will sunset.12 Notably, the FDA has indicated that the guidance on the conduct of clinical trials will remain in effect for 180 days after the end of the PHE as a transitional period.13 This dovetails with a provision in FDORA that requires the FDA to hold a public meeting to better understand the effectiveness of the clinical trial flexibilities the FDA exercised during the pandemic. Two other sections of FDORA related to decentralized clinical trials, novel clinical trial designs and digital health technology in clinical trials, continue the progress on clinical trial modernization. Below, we describe these three provisions of FDORA in more detail.
§ 3605. Public meeting on clinical study flexibilities initiated in response to COVID-19 pandemic
Within 180 days from the end of the public health emergency (by Nov. 7, 2023), the FDA must hold a public meeting to discuss FDA recommendations during COVID-19 to mitigate disruption of clinical studies, including as described in the guidance.
The FDA must issue a report summarizing the discussion at the public meeting. Topics to be included for discussion at the meeting include, but are not limited to:
the frequency with which sponsors availed themselves of the flexibilities in the guidance;
characteristics of the sponsors, studies, and patient populations impacted by such recommendations;
impact of those flexibilities on patient access to clinical studies, especially underrepresented patient populations; and
recommendations for incorporating these flexibilities into new guidance to improve clinical study enrollment and diversity.
§ 3606. Decentralized clinical studies
Within one year after enactment of FDORA (by Dec. 29, 2023), the FDA must issue or update draft guidance to advance the use of decentralized clinical trials. The guidance must address numerous aspects of decentralized trials, including recommendations related to:
collecting data remotely, considering the security and privacy of such options (e.g., digital health technology, telehealth, local providers and labs, and patient experience data);
minimizing or reducing burdens on subject enrollment and participation (e.g., digital health technology, telehealth, local providers/labs, home visits, direct engagement with study participants, direct shipping of investigational drug to the study participants, electronic informed consent, and partnerships with community organizations);
evaluating the protocol design of decentralized trials and data collected in decentralized trials, including whether evaluations will be different than for non-decentralized trials;
using decentralized clinical trials to maximize participant diversity;
validating digital technologies and establishing appropriate clinical endpoints for use in decentralized trials;
combining centralized and decentralized clinical trial approaches; and
oversight of decentralized clinical trial sites, including remote oversight.
§ 3607. Modernizing clinical trials
Within one year after enactment of FDORA (by Dec. 29, 2023), the FDA must issue or update draft guidance regarding appropriate use of digital health technologies in clinical trials to improve diversity in recruiting and retention of participants and to facilitate novel clinical trial designs; final guidance must be published within 18 months after the close of comment period on the draft guidance.
Within one year after enactment of FDORA (by Dec. 29, 2023), the FDA must issue or update draft guidance on the use of “seamless, concurrent, and other innovative clinical trial designs” to support development and review of new drugs; final guidance must be published 18 months after the close of comment period on the draft guidance.
Directs the FDA to work with foreign regulators to facilitate international harmonization of the regulation and use of decentralized trials, digital health technology, and seamless, concurrent, and other adaptive or innovative clinical trial designs.
The FDA has already begun to implement the FDORA clinical trials provisions with the issuance of the required guidance on decentralized clinical trials in early May, described in more detail in a separate article.14 Given the short timeframes, we expect the other guidances (diversity action plans, digital health technology, modernizing clinical trials) to be forthcoming shortly and for the FDA to implement the remaining provisions, such as public meetings and reports, in a timely manner. The FDORA guidances are likely to build on the agency’s efforts over the last 10 years, including the already existing guidances discussed in this article. You would be well served to familiarize yourself with the current draft guidances and FDA reports and activities to prepare for the future.
The FD&C Act (section 505(d)) requires a sponsor to demonstrate with “substantial evidence” that a drug is safe and effective, and defines substantial evidence as “evidence consisting of adequate and well-controlled investigations, including clinical investigations[.]” FDA has traditionally required two randomized, controlled clinical trials as substantial evidence, but has recently expanded its view of what may meet the statutory standard. See e.g., Demonstrating Substantial Evidence of Effectiveness for Human Drug and Biological Products, Draft Guidance, Dec. 2019, https://www.fda.gov/media/133660/download.
FDA Guidance on Conduct of Clinical Trials of Medical Products during the COVID-19 Public Health Emergency, March 2020, updated Aug 2021, https://www.fda.gov/media/136238/download.
In early May, two weeks prior to the expiration of the COVID-19 Public Health Emergency declaration, FDA released the draft guidance Decentralized Clinical Trials for Drugs, Biological Products, and Devices.1 The timing of this draft guidance is notable as the document builds upon many of the recommendations FDA provided in the March 2020 guidance Conduct of Clinical Trials of Medical Products During the COVID-19 Public Health Emergency2 (which we covered back in 2020 for Clinical Leaderhere) which were intended to help sponsors continue their trials throughout the COVID-19 crisis. During the public health emergency, with many trial participants and personnel in isolation, per local COVID-19 control policies, FDA detailed several approaches sponsors could take to maintain continuity of their clinical trials, such as the use of electronic informed consent, use of digital health technologies to conduct assessments, use of local healthcare providers to administer trial procedures, and the direct shipment of investigational products to trial participants, among other measures. Thus, the May 2023 guidance on decentralized trials marks a continuation of FDA’s thinking on the decentralization of clinical trials.
The draft guidance fulfills Section 3606(a) of the Food and Drug Omnibus Reform Act (FDORA)3 requiring the agency to issue guidance on decentralized clinical trials (DCTs) by Dec. 29, 2023. Authored by all three medical product review centers, as well as the Oncology Center of Excellence, the draft guidance recommends a risk-based approach to the conduct and oversight of decentralized clinical trials.
The draft guidance identifies DCTS as trials wherein some or all trial-related activities occur outside of traditional clinical trial sites. More specifically, DCTs incorporate the use of local healthcare facilities, local healthcare providers (HCPs), and local clinical laboratory facilities; visits to trial participants’ homes; and direct distribution of the investigational product to the trial participant. Trials where some activities are conducted at the traditional trial site while other activities, such as follow-up assessments, are conducted remotely are termed “hybrid” clinical trials by the agency.
As noted, FDA recommends a risk-based approach when considering a decentralized trial design – noting that investigational products that are “simple to administer or use, have well-characterized safety profiles, and do not require complex medical assessments”4 are the most appropriate and well suited for evaluation in DCTs. The FDA cautions sponsors to be mindful of potential differences in data accuracy and consistency between DCT and conventional trials conducted at a physical site. While these variances may not impact the validity of trial results that seek to establish superiority of one treatment over another, they could impact the accuracy of results in a trial that aims to establish non-inferiority. For example, the effectiveness of a drug tested in a DCT may not be identical to the effectiveness of the same drug tested in a traditional trial that employs an active control, as evaluations performed by local healthcare providers in DCTs may be less precise and more variable than those conducted by dedicated trial personnel at site-based trials.
Beyond defining DCTs and identifying the appropriate situations for their use, the draft guidance provides key recommendations on the innovative approaches sponsors can use to decentralize trials and move trials outside of traditional sites. The draft guidance also includes some important considerations on safety and data security that are likely to arise in remote contexts.
Remote Trial Visits
The draft guidance affirms that telehealth visits can be used instead of in-person trial site visits, especially if no in-person interaction is needed. This was an innovation that FDA introduced in the March 2020 Conduct of Clinical Trials guidance, although it should be noted that the need for alternatives to in-person site visits to facilitate enrollment and conduct of clinical trials was recognized by FDA well before the pandemic. For example, a 2015 Federal Register notice sought comment and recommendations on innovative mechanisms to increase clinical trial enrollment, such as the use of telehealth visits.5
In addition to telehealth visits, the draft guidance also provides sponsors with the option to use local HCPs, who are not officially trial personnel. Importantly, the services local HCPs provide should not differ from the services they are qualified to perform in clinical practice. In addition, the activities local HCPs may provide should not require unique or detailed knowledge of the trial protocol or the investigational product.
Should telehealth technologies or local HCPs be used to facilitate decentralized trials, the trial protocol should detail how adverse events will be remotely identified, evaluated, and managed. Additionally, in the interest of trial efficiency and patient experience, sponsors are responsible for training trial personnel on the technology used to conduct a telehealth visit.
Digital Health Technologies
Although digital health technologies (DHTs) are among most common tools used in DCTs, the draft guidance does not focus extensively on this topic as it is well covered in the December 2021 draft guidance Digital Health Technologies for Remote Data Acquisition in Clinical Investigations6[Editor’s note: Covered by Clinical Leader here.] However, the draft guidance does note that sponsors may permit trial participants to use their own DHTs in trials, as long as the sponsor also provides the same DHTs to other participants, so they are not excluded.
Direct Shipment of Investigational Products
The draft guidance confirms that the direct distribution of the investigational product (IP) to trial participants at their homes or other remote location is acceptable. However, should this method distribution be used, sponsors should describe in the protocol how the physical integrity and stability of the IP will be maintained during shipment. Similarly, the protocol should describe how investigators will track and document the receipt of IP by trial participants, as well as how participants should dispose of unused IPs and how this should be documented. IPs that are considered good candidates for direct shipment include those with long shelf lives and good stability profiles.
The draft guidance also notes that sponsors may also use a central distribution service to ship IP directly to trial participants. The investigator, however, must still control the release of the IP by the distributor, as well as monitor receipt and use by trial participants, as specified in the protocol.
Administration of the Investigational Product
As discussed above, sponsors should consider the nature of the IP when determining whether to administer it outside of traditional trial sites. FDA advises that IPs that involve complex administration procedures, have a high-risk safety profile, or are in early stages of development may require in-person supervision by the investigator at a trial site. Similarly, investigators should also consider the safety profile (e.g., risk of hypersensitivity, abuse potential) in determining the type of local care that participants may need to have access to if an adverse event occurs.
Informed Consent and Institutional Review Board Oversight
While discussed extensively in the March 2020 Conduct of Clinical Trials guidance, the DCT draft guidance also addresses the use of remote informed consent, albeit briefly. Specifically, the draft guidance confirms that investigators may obtain electronic informed consent remotely provided that all regulations under 21 CFR Part 50 are met. FDA also recommends that investigators have a central Institutional Review Board (IRB) to facilitate the review of the protocol and the informed consent documents.
Roles and Responsibilities
FDA notes that sponsors’ responsibilities are the same in DCTs as they are in traditional site-based trials. Due to the potential of multiple sources of data collection, sponsors should ensure their data management plan specifies data origin, data flow, and the methods used for data collection and includes a list of vendors involved in data collection, handling, and management. The draft guidance also recommends sponsors detail all operational aspects of a DCT in a trial protocol, while case report forms should identify when and where data is collected.
FDA recognizes that DCTs add complexity to the investigator’s role in overseeing trial conduct, as some decentralized features necessitate additional training and careful coordination of remote activities. FDA reiterates that local HCPs may perform trial-related procedures at a participant’s home or other local healthcare facility. However, local HCPs need not and should not be listed as sub-investigators on Form FDA 1572. The draft guidance also states that some trial protocols may permit the use of clinical laboratory facilities close to the trial participant, although designated clinical laboratory facilities are preferred to minimize variability.
Safety Monitoring
As with site-based trials, safety monitoring plans should describe how participants are expected to respond to and report adverse events, specifically noting where to seek local medical care if needed. In addition, the draft guidance also notes that trial participants should be able to contact trial personnel to report adverse events and arrange for telehealth visits if necessary. Lastly, the safety monitoring report should also describe the information collected by a digital health tool – detailing how the information will be used and monitored and how personnel or participants should respond to electronic alerts.
Software Considerations
The FDA notes that software can be used for multiple purposes in a DCT, including managing electronic informed consent, capturing and storing reports, managing electronic case forms, scheduling trial visits, and tracking IPs shipped directly to participants. Software programs used to produce and process trial records are subject to 21CFR Part 11 and must ensure data reliability, security, privacy, and confidentiality. Real-time video interactions, however, such as telehealth visits, are considered by FDA as live exchanges of information between trial personnel and trial participants and thus are not considered electronic records and subject to 21 CFR Part 11.
The FDA is accepting public comments on the draft guidance until August 1, 2023. All written comments should be identified with the docket number FDA-2022-D-2870.
Thor, S., Vetter, T., Marcal, A., Kweder, S. EMA-FDA Parallel Scientific Advice: Optimizing Development of Medicines in the Global Age. Ther Innov Regul Sci 57, 656–661 (2023). https://doi.org/10.1007/s43441-023-00501-9.
This article was published in Therapeutic Innovation & Regulatory Science on March 4, 2023. It is reprinted here under a Creative Commons Attribution 4.0 International License and can be downloaded through the button at right.
Abstract
As medicines development continues towards a globalized approach, both the pharmaceutical industry and regulatory agencies increasingly seek opportunities to proactively engage early in product development. The parallel scientific advice program shared by the European Medicines Agency (EMA) and the US Food and Drug Administration (FDA) provides a mechanism for experts to concurrently engage in scientific discourse with sponsors on key issues during the development phase of new medicinal products (drugs, biologicals, vaccines, and advanced therapies).
Introduction
Regulators at both the European Medicines Agency (EMA) and the US Food and Drug Administration (FDA) support and foster increasingly globalized approaches to medicines development. Covering a broad range of relevant topics in medicines development, both Agencies participate in multilateral fora such as the International Council on Harmonization (ICH), International Coalition of Medicines Regulatory Authorities (ICMRA), and the World Health Organization (WHO) to address topics such as standards setting and policy convergence at the global level. On a smaller scale, the two Agencies lead more than 30 technical working groups or “clusters” where members exchange perspectives and experiences on regulatory science topics.1 The cluster meetings are opportunities for regulatory experts to discuss amongst themselves challenges and difficult applications of regulatory science and policy based on the priorities of the Agencies and are not intended to serve as a forum for advising sponsors. There are situations, however, in which a developer can benefit from scientific advice on a product development program from both Agencies concurrently, and where convergent advice on the same or similar product-based scientific questions could benefit public health and facilitate patient access to needed therapies. To meet this need, EMA and FDA established a sponsor-initiated, product-specific exchange: the parallel scientific advice (PSA) program.2
PSA provides a mechanism for EMA and FDA experts, upon request by the applicant, to concurrently advise sponsors on scientific issues during the development of new medicinal products (drugs, biologicals, vaccines, and advanced therapies). Importantly, as part of the process the two agencies engage with each other to compare perspectives in advance of and during the actual interaction with the sponsor. This voluntary program was launched in 20053 with four goals: increase dialogue between the two agencies and sponsors from the beginning of the lifecycle of a new product; provide a deeper understanding of the bases of regulatory decisions; optimize product development; and avoid unnecessary testing.
To initiate a PSA request, the applicant, herein referred to as ‘sponsor’, emails a request to each Agency.4 The request is expected to be brief and state the rationale for why the PSA would be beneficial, the proposed scientific questions to the Agencies, and desired goals for the meeting. If both Agencies agree to accept the request, the sponsor can move forward with preparing a full meeting package according to EMA’s Scientific Advice Working Party (SAWP) procedure schedule.5 A bilateral meeting between EMA and FDA takes place approximately 35 days after EMA validates the meeting package. After the bilateral meeting, preliminary feedback from each Agency is shared with the sponsor in writing. This could include preliminary responses to the sponsor’s questions or requests for the sponsor to clarify or expand a concept or proposed pathway. At approximately 65 days after validation, a trilateral meeting with the sponsor, EMA, and FDA is held. Written advice from each Agency to the sponsor follows this meeting, from EMA within ten days and within 30 days from FDA.
In 2022, we, scientists overseeing PSA at EMA and FDA, conducted a program review covering the five years from 2017 through 2021. The review included more intensive examination of a sub-cohort of submissions in calendar year 2020 to examine how well timelines were met. This paper shares the results and insights from our review and describes best practices for sponsors considering PSA.
Methods
We independently conducted records searches in FDA and EMA files for PSA procedures requested in calendar years 2017 through 2021. The records were then merged and reviewed for accuracy and completeness. The requests were first categorized by whether they were accepted and, if not accepted, the reason. We also stratified the requests by the therapeutic area of each application’s subject product, and whether any accepted requests were later withdrawn by the sponsor. Further, we examined detailed timelines of procedural steps from the seven PSAs accepted during 2020. We selected 2020 for this sub-cohort year because when we began the records review in January 2022, the 2020 calendar year was the most recent year when all procedures had been completed and therefore had all aspects of their timelines fully characterized. For these we noted the dates of each request, acceptance, meeting package validation, and provision of the EMA Final Advice Letter.
Results
The 5-year review identified a total of 37 PSA requests (see Table 1). Of these, 26 (70%) were accepted to participate. Even when requests are accepted, there were times when the sponsor chose not to proceed with submitting a meeting package or formally withdrew the request. This happened four times over the 5-year period, leaving 22 completed PSA procedures, ranging from four to seven per year, as shown in Fig. 1. In no case was a request accepted and later one or both Agencies decided to discontinue the process. We note that the COVID-19 global pandemic was ongoing during the 2020 and 2021 years of this dataset. Though regulatory operations shifted to a nearly entirely virtual environment during that time, this shift did not affect the PSA program as virtual operations were already a necessary component of PSA. Further, the number of accepted requests did not decline during the pandemic years, despite both Agencies needing to shift many resources to address COVID-19 related public health needs.
Table 1, PSA Requests 2017–2021
Total requests
37
Accepted requests
26 (70%)
Withdrawn/package not submitted
4 (15%)
Completed procedures
22
Figure 1
PSA requested and accepted decisions by year (2017–2021).
Of the accepted requests, the majority were in the therapeutic area that combines submissions for Gastroenterology, Inborn Errors of Metabolism, Rare Diseases, and Medical Genetics. We combined these into a single category for purposes of this report because during the period of our cohort FDA shifted its organizational structure and categorization of submissions. As shown in Fig. 2, Oncology, Anti-infectives, Cardiology/Nephrology, and Neurology were also areas with multiple PSA requests. Other therapeutic areas included accepted requests in Ophthalmology, Dermatology, Cardio-metabolic diseases, Pulmonology, Rheumatology, Advanced Therapies, and Hematology.
Figure 2
Accepted PSA requests (N = 26) by product category 2017–2021.
As previously stated, to be accepted for PSA both Agencies must agree to the request. Over the five years of our review cohort, eleven requests were not accepted (see Fig. 3). Four requests were not accepted because they were made very early in development, such as when the product had not been the subject of a pre-Investigational New Drug (pre-IND) application or IND application at FDA. Another four requests were not accepted because the request had a device component, which would not have been within EMA’s advice remit at the time (though this remit has since changed, and the EMA no longer discourages PSA submissions for products containing a device component). The other three denials involved circumstances where one or both agencies felt that PSA was not a good option for other more varied or nuanced reasons.
Figure 3
PSA requests: reason for not accepted (N = 11*) *At the time of these requests, EMA did not accept PSAs with a medical device component.
Timeline data from the 2020 PSA cohort is displayed in Fig. 4. There was an average of 13 calendar days between the PSA request and the Agencies’ acceptance. Then the PSAs spent an average of 67 days in the phase of meeting package preparation and validation. Once the meeting package is validated, the Agencies begin review. During this review time multiple milestone events take place, specifically a bilateral meeting of FDA and EMA to discuss their respective reviews, followed by issuance of draft comments and further questions to the sponsor and then a trilateral meeting of the Agencies with the sponsor. A final advice letter (FAL) from EMA is issued in follow-up to the trilateral within ten days, and FDA’s meeting minutes are provided within 30 days. For six of the seven PSAs in the 2020 cohort, the average Agency review time was 79 days. There was one outlying PSA with a review time of 105 days. This PSA occurred over the period when the EMA SAWP has its annual August recess. As this is a predictable outlier that will always increase review time duration by one month, we did not include that PSA in the average for Agency review time. When we include this outlier, the average time spent under Agency review for the seven PSAs in the 2020 cohort is 83 days. Subsequent to our analysis of the 2020 cohort, we revised and published a timeline that describes each phase of PSA (Table 2).
Figure 4
Selected Milestones for PSA Procedures in 2020. *PSA occurred over EMA SAWP August recess; not included in average. **Sponsor requested a pre-submission meeting with EMA.
Table 2, PSA Timeline
Discussion
Adding Value
For more than 15 years, PSA has been an opportunity for sponsors who are developing medicines across regulatory regions. It allows a sponsor to submit the same background and supporting material to both FDA and EMA and seek their respective advice simultaneously on the same issues. The Agencies do not commit to harmonizing advice, as each has its own regulatory frameworks. However, during a bilateral meeting they can discuss the sponsor’s questions and focus on sharing information and their perspectives in order to identify areas of convergence and divergence. In sharing their respective preliminary feedback with the sponsor in writing, including requests for further clarification and discussion, the sponsor is provided an opportunity to plan for more in-depth discussion during the subsequent trilateral meeting.
Our observation is that bringing EMA, FDA, and sponsor perspectives to a PSA trilateral setting provides a rich opportunity for all. It is common for PSA trilateral discussions to result in convergence in advice on approaches to a product’s development even though full harmonization is not always possible. And in cases of divergence, the trilateral meeting is an opportunity for the sponsor to offer proposals for how to meet both regulators’ requirements without having to explain each regulator’s perspective to the other. Even when Agencies maintain differing perspectives, an important benefit of PSA is that all parties in the process understand the reason(s) for the divergence.
Increasing Awareness and Understanding
Typically, sponsors pursue a more traditional model of seeking advice from each Agency independently, often in series, which requires expending resources on preparing for two separate meetings where the scientific questions are often nearly identical and the burden of having to articulate one Agency’s views to the other is carried by the sponsor. When discussing PSA at a 2017 public workshop on expedited programs and regulatory harmonization, participants noted that the PSA process is not well understood by sponsors, especially the expected timelines of PSA procedures.6 In our observation neither our Agencies nor industry have promoted it widely and little has been written about this process. We have sought to increase awareness and understanding through public presentations,7 collaborating with sponsors on educational efforts,8 the publication of a new timetable,9 and this review.
Data from our 5-year review show that uptake of the PSA pathway has been limited- just four to seven procedures annually over the last five years. As described in the General Principles for PSA,2 PSA procedures are designed to generally correspond with the EMA’s SAWP timeline3 and the FDA Type B meeting10 timeline. Results from our 2020 cohort were consistent with these timelines. The cohort showed an average acceptance turnaround time of PSA requests at 13 calendar days; FDA Type B meeting requests are allowed up to 21 days for a response. The average review time for the cohort was 79 days, which is consistent with previously published SAWP PSA timetables predicting 75 days from the validation of the PSA meeting package to receipt of final advice.
The time from acceptance of the PSA to the validation of the meeting package varied from 47 to 93 days, with a mean of 67 days. Variation in time spent in this phase is largely within the sponsor’s control. For example, this phase may be quite short if the sponsor quickly submits a robust meeting package after their PSA request is accepted. It may be longer if the sponsor submits a deficient meeting package or requests a pre-submission meeting with EMA. The latter was the case with Product 5, shown in Fig. 4, which spent 93 days in the validation phase. Also, in some cases the sponsor delays the submission of their meeting package, for example when they are awaiting additional data.
Looking Ahead
We have been overseeing, coordinating, and participating in the PSA program, some of us for more than a decade. Although not easy to quantify, our experience has been that once underway the outcome of the process is remarkably productive and positive for all parties. The interactions between the two regulators are critical and serve as a form of peer discussion, an opportunity to expand thinking and explore ways to address common challenges in drug development together, especially in areas where there is little experience or thorny scientific issues at hand. Products discussed under PSA are often products with no simple path forward. Therefore, EMA and FDA exploring alternative or innovative approaches together adds great value to the advice ultimately rendered to the sponsor. Such potential for value underpinned the launch of an FDA-EMA PSA pilot for complex generic products in 2021, with the hope that PSA will be a tool for optimizing global development of products for which traditional bioequivalence methods are challenging.11
Based on our experience and the analyses presented here, we suggest a few strategies to sponsors who are considering PSA. First, consider the timing of your request. It is strongly recommended to have begun the pre-IND or IND process at FDA on your product before requesting PSA, so that there is a baseline for reference. With the foundations and background of your product’s development plan already understood, your PSA questions can be focused on the specifics of global development that merit consideration for convergence. If timing is important to you, we further suggest that you factor into your planning the August recess of the SAWP and approximately two weeks for the Agencies’ review of your PSA request.
Second, research existing guidance on the topic to see where you can expect there is alignment across the two Agencies and where there is not. Some areas where PSA may be most appropriate are for innovative products or new scientific or regulatory concepts that have not been the subject of published guidance. Examples include advanced therapies, biosimilars, or use of novel/surrogate endpoints. Innovative manufacturing and non-clinical concepts and questions are also appropriate.
Third, consider the public health benefit of your product. PSA requires extra investment of resources from both Agencies, so the program’s focus is on products that address unmet medical needs, rare diseases, pediatric populations, or other areas of importance to patients and public health. In fact, the majority of accepted requests during the cohort period have been for rare disease therapies, pediatric populations, or advanced therapy medicinal products. Be sure to explain your product’s potential public health benefits in your request letter.
Finally, make the best possible use of the trilateral meeting. It is key to prioritize and address the issues raised in the preliminary feedback from FDA and EMA in a well-structured presentation enabling thorough and efficient discussion. This 90-min meeting is your avenue for probing both Agencies on opportunities for convergence. Hence, make sure you focus on the most critical scientific questions, and prepare proposals and rationales that address the issues noted in the preliminary feedback you received from each Agency.
Conclusion
PSA is a longstanding EMA and FDA collaboration that continues to have strong support within both Agencies. The PSA program offers an opportunity for companies to simultaneously consult international regulators for advice on the development of important medical products, with the intent of optimizing development and deepening their understanding of regulatory decision making. Our experience has shown that the PSA program can provide timely and insightful advice on the most challenging aspects of global development. Sponsors wishing to seek PSA should consult the General Principles for Parallel Scientific Advice2 for further guidance.
Notes
Tania Teixeira, Sandra L. Kweder, and Agnes Saint-Raymond. Are the European Medicines Agency, US Food and Drug Administration, and Other International Regulators Talking to Each Other?
GENERAL PRINCIPLES EMA-FDA PARALLEL SCIENTIFIC ADVICE. July 2021.
PSA Pilot Program Launch 2005.
Email addresses: EMAinternational@ema.europa.eu and US-FDA-EUR@fda.hhs.gov.
EMA Scientific Advice Working Party (SAWP).
Elizabeth Richardson, Gregory Daniel, David R. Joy, Sandra L. Kweder, Diane M. Maloney, Miranda J. Raggio, and Jonathan P. Jarow. Regional Approaches to Expedited Drug Development and Review: Can Regulatory Harmonization Improve Outcomes?
FDA Small Business and Industry Alliance Webinar: FDA-EMA Parallel Scientific Advice Program. March 2022.
Parallel Scientific Advice: Increasing International Dialogue Early in the Product Lifecycle. Drug Information Association Global Annual Meeting. May 2021.
PSA Timetable.
Guidance for Industry: Formal Meetings Between the FDA and Sponsors or Applicants. May 2009.
FDA-EMA PSA Pilot Complex Generics.
Authors and Affiliations
Shannon Thor, Europe Office, US Food and Drug Administration, Silver Spring, Maryland, USA
Thorsten Vetter, Scientific Advice Office, European Medicines Agency, Amsterdam, The Netherlands
Anabela Marcal, International Affairs Department, European Medicines Agency, Amsterdam, The Netherlands
Sandra Kweder, Office of Global Policy and Strategy, US Food and Drug Administration, Silver Spring, Maryland, USA (Sandra Kweder’s contributions were during the time of FDA employment.)
This article was originally published as a guest column inClinical Leader.
Over the past month, the FDA has continued to build out its real-world evidence program with the release of the final guidance, Submitting Documents Using Real-World Data and Real-World Evidence to FDA for Drug and Biological Products. The brief process-focused guidance describes the recommended approach sponsors should use to identify if and how real-world data (RWD) and real-world evidence (RWE) are incorporated in their regulatory submissions. The FDA notes that this guidance specifically applies to investigational new drug applications (INDs), new drug applications (NDAs), and biologics license applications (BLAs) that contain RWD and RWE. The guidance does not, however, address or comment on review standards for submissions containing RWD or RWE.
Focus on Product Safety and Effectiveness
In this guidance, the FDA signals that its primary focus regarding RWD and RWE is on their use to support product effectiveness and safety. Particularly, the FDA advises that sponsors only flag a submission’s inclusion of RWD and RWE if they are intended to support a regulatory decision on a product’s safety and efficacy. As a rule of thumb, the FDA suggests “sponsors and applicants not identify submissions that contain RWD/RWE if those data are not intended to support product labeling.”1 Examples of submission types that do not need to be flagged for incorporating RWD or RWE include those that use RWD to generate hypotheses or studies that include RWD in exploratory models or simulations, studies that use RWD to validate an endpoint, and studies that use RWE to qualify a Drug Development Tool (DDT).
These guiderails will likely have implications for the type of public data that both sponsors and stakeholders will have access to on the use and regulatory acceptance of RWE. The FDA notes that the internal tracking of these submissions is intended to inform the “FDA’s understanding of the scope and use of RWD and RWE submitted to support regulatory decisions…”2 However, this focus likely limits the scope of the FDA’s public reporting of RWE submissions. Under PDUFA VII, the FDA has committed to reporting aggregate and anonymized information on RWE submissions to CDER and CBER starting no later than June 2024. Thus, sponsors and stakeholders can expect the future report to focus only on RWE submissions supporting product labeling rather than other uses.
Key Data Points for Tracking
The sample submission checklist included in the appendix titled Sample Presentation of Information to be Included with Submissions Containing RWD/RWE provides further insight into the data points the FDA is looking to collect on RWD and RWE use. In addition to the product name and targeted disease or medical condition, the FDA suggests sponsors identify their purpose for using RWD or RWE as part of the submission, the study design using RWD to generate RWE, and RWD sources used to generate RWE. Per the sample list of purposes for RWD and RWE, the FDA anticipates the most common uses of RWD and RWE will be:
to support safety and/or effectiveness for a product not previously approved by the FDA;
to support labeling changes for an approved product; and
to support or satisfy a postmarketing requirement or postmarketing commitment (PMR/PMC).
In both the body and appendix of the guidance, the FDA also lists the anticipated study designs likely to involve RWD to support a regulatory approval. The list broadly aligns with the current clinical trial landscape and includes randomized controlled trials with pragmatic elements, single-arm trials using RWD in an external control arm, and non-interventional or observational studies. The FDA is also interested in collecting data on the types of RWD sources sponsors use to generate RWE. The FDA’s sample list of RWD sources, which is not intended to be exhaustive, includes electronic health records data, medical claims data, product or disease registry data, and data collected from digital health technologies in non-research settings.
As noted, the submission checklist included in the appendix is helpful in providing a glimpse into the FDA’s current understanding of the RWE landscape and a preview of the data points the agency is looking to track on RWD and RWE use for regulatory purposes. Just as important, however, are the data points that the FDA did not include in the submission checklist. Notably, the FDA does not request information on the data standard sponsors used to structure large data sets. To date, the agency has remained silent on preferred common data standards. However, per the FDA’s 2021 draft guidance titled Data Standards for Drug and Biological Product Submissions Containing Real-World Data,3 “FDA plans to issue further guidance and/or to update the Catalog with standards for study data that are derived from RWD sources.”4
The guidance was authored by the Office of Medical Policy in the Center for Drug Evaluation and Research (CDER), in consultation with the Center for Biologics Evaluation and Research (CBER) and the Oncology Center of Excellence (OCE). Notably, the Center for Devices and Radiological Health (CDRH) is not listed as a co-author, signaling that the FDA may not be taking an “inter-center” approach to RWE regulatory policy. This is further underscored by the fact that the 2017 guidance on RWE use for medical device applications, Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices,5 was authored by only CDRH and CBER, not CDER. As of now, the FDA has not provided recommendations on how to flag device submissions as using or containing RWD or RWE.
This article was originally published as a guest column in Clinical Leader.
The FDA’s release of four draft guidances on real-world evidence (RWE) at the tail end of 2021 is a productive step toward implementing FDA’s RWE program. The draft guidances are expansive, providing recommendations to sponsors seeking to use real-world data (RWD) derived from medical claims data, electronic health records (EHRs), and registries to support regulatory submissions. However, despite the scope of these documents, a survey of comments letters submitted in response to the draft guidances reveals a widespread need, specifically among drug developers, to understand how the agency will implement these guidances in practice. In an attempt to solicit concrete examples, several sponsors requested the agency develop an online dashboard that publishes all of the agency’s decisions, both positive and negative, on regulatory submissions that incorporate RWE.
Despite the lack of a central catalog of all RWE-related regulatory decisions, there are a handful of product approvals and FDA reviews that highlight both best practices and common pitfalls sponsors face when incorporating RWE in regulatory submissions. These decisions show that the use of RWE to support regulatory decision-making is often challenged by both methodological and process issues. In terms of process, the primary mistake made by sponsors is failing to share a prespecified protocol and statistical analysis plan (SAP) with the agency prior to submission. From a methodological perspective, unsuccessful RWE submissions are often plagued by missing data, small patient populations, and a lack of objective endpoints. Approvals based on or supported by RWE, such as Ibrance (palbociclib) for male breast cancer and Prograf (tacrolimus) to prevent organ rejection post lung transplant, demonstrate how to successfully address the myriad challenges of working with RWD.
1. Failing to Share a Prespecified Protocol and SAP
As noted, the most common mistake sponsors make when seeking to leverage observational studies to support or demonstrate a product’s safety and efficacy is failing to share a prespecified protocol and SAP with the agency. The FDA emphasizes the importance of up-front transparency to guard against this risk, noting in its real-world evidence framework that multiple analyses in electronic data sets can be done quickly and inexpensively, “making it possible to conduct numerous retrospective studies until the desired result is obtained….”1 The four recent guidances also recognize this as a potential risk to undermining study validity and recommend sponsors:
“Provide draft versions of their proposed protocol and statistical analysis plan (SAP) for Agency review and comment, prior to finalizing these documents and before conducting the study analyses.”2
2. Missing Data
One of the most common methodological deficiencies identified in FDA reviews of RWE studies is missing data. Unlike clinical trials, wherein patient visits and data collection are prespecified, “real-world” patient visits are often irregular and data collection uneven, leading to spotty and inconsistent electronic health record (EHR) data. In turn, incomplete EHR data can make it difficult to define baseline characteristics for observational cohorts, obfuscating efforts to establish comparability with an active trial cohort. This problem has hobbled several attempts to use RWE studies to support oncology product approvals. In these cases, the sponsor was unable to provide complete documentation on essential baseline characteristics such as previous treatment regimens, tumor stage, and Eastern Cooperative Oncology Group (ECOG) scores. FDA reviewers noted that the failure to capture previous treatment regimens introduces confounding bias, while missing dates, such as the date of diagnosis or start of first treatment, can introduce bias into the study in favor of the investigational product.
In successful RWE applications, the majority of these important clinical details are well documented. For example, FDA notes in the supplemental review of palbociclib for male breast cancer that “In this study, real world tumor response data was generally available for several lines of therapy.”3 The reviewer also notes that real-world data sets were complete and included birth date, metastatic sites, and first metastatic treatment. Unstructured source data in the form of clinician assessments and radiology reports were also included in the submission.4
3. Small Patient Cohorts
Incomplete EHR data has significant downstream effects on a sponsor’s ability to use RWE for regulatory purposes. In an attempt to achieve comparability with active trial cohorts, sponsors often have to disqualify patients with incomplete data elements from analyses, further whittling down a data set that may already be pretty small. For oncology and rare disease applications, small patient populations are already a common issue, which the use of RWE can compound. This has hindered several attempts to use RWE to support approval. Even in successful RWE applications, like palbociclib, the reviewer still called out the small sample size as a key deficiency.
To address the issue of small patient populations, FDA has in recent guidance identified both data linkage and the combining of data as possible solutions. However, the guidance recognizes that these methods also introduce a new set of methodological problems, notably data heterogeneity. In the draft guidance Real-World Data: Assessing Electronic Health Records and Medical Claims Data To Support Regulatory Decision-Making for Drug and Biological Products (September 2021), FDA states:
“For studies that require combining data from multiple data sources or study sites, FDA recommends demonstrating whether and how data from different sources can be obtained and integrated with acceptable quality, given the potential for heterogeneity in population characteristics, clinical practices, and coding across data sources.”5
In addition to undermining attempts to combine two different data sets, differences in population characteristics, clinical practice, and coding have derailed efforts to use observational cohorts to define standard clinical practice in the context of superiority trials. Again, many attempts to use RWE for oncology indications have failed due to differences in treatment locations and treatment classifications which prevented sponsors from being able to draw direct comparisons between the investigational product and the standard of care.
4. Lack of Objective Endpoints
A third common roadblock to the acceptance of RWE studies for regulatory purposes is the difficulty in uniformly capturing clinical outcomes in EHR or claims data. In recent guidance, FDA emphasized the enormous impact that variability in physician practice, specifically in regard to diagnoses and coding, have on capturing clinical outcomes. To address this risk, FDA recommends sponsors use RWD sources to capture outcomes with more objective “well-defined diagnostic criteria,” such as death, stroke, or myocardial infarction.
Using RWE for oncology indications is particularly tricky as many commonly used oncology endpoints are subjective and time-bound. Tumor response rates and progression free survival (PFS), for example, are inherently subjective and rely on visual assessments of radiographic images, often generating wide variability. These outcomes are also difficult to validate, as most EHR records do not incorporate the radiographic image. Meanwhile, time to treatment discontinuation (TTD) and overall survival (OS) is particularly subject to confounding bias due to treatments beyond disease progression. As noted, FDA’s answer to these issues is to encourage sponsors to use RWD for conditions or diseases with straightforward and objective outcomes. It is telling, then, that the only drug to have received approval based on an observational study is tacrolimus for lung transplant recipients, wherein the primary endpoint was post-transplant survival.
Considering the methodological issues sponsors face in working with RWD, FDA’s cautious stepwise approach to implementing the RWE program is understandable, but it is frustrating to some. As drug developers and the agency continue to wrestle with these methodological issues, we can expect that successful RWE regulatory submissions will continue to be for drugs with well-established safety and efficacy profiles.
Maria T. Nguyen, Kevin Wright, Barbara Fuller, and Lashawn Griffiths, sNDA 207103-S-008, Multi-Discipline Review, 5 March 2019, 50, https://www.accessdata.fda.gov/drugsatfda_docs/nda/2019/207103Orig1s008.pdf.
The Food and Drug Administration (FDA or the Agency) user fee programs help to provide funding for the Agency to achieve its mission of protecting the public health and providing safe and effective medical products to patients in the United States.1 The user fee programs provide the FDA with financial support to meet specified performance goals and commitments related to medical product submissions. These specific performance goals and commitments are negotiated and agreed upon between the FDA and industry in the years leading up to the reauthorizations and must be sent to Congress for review and approval.
All medical product user fees are renegotiated every five years. As the Prescription Drug User Fee Amendments (PDUFA VI), the Biosimilar User Fee Amendments (BsUFA II), and the Medical Device User Fee Amendments (MDUFA IV) are also set to expire on September 30, 2022, it is expected that all three user fee program extensions will be part of the same legislative package.2,3 The next set of authorizations for these programs will cover Fiscal Years 2023 to 2027.
Below is a summary of the current status of the PDUFA, BsUFA, and MDUFA negotiations as well as a description of the key commitments being discussed between the FDA and industry.
PDUFA
PDUFA VII will build upon the progress of past programs by providing the FDA with the resources and tools it needs to keep pace with advances in drug development. Considering recent scientific breakthroughs in cell and gene therapy and the bolus of investigational new drug (IND) applications for advanced biological therapies received by the Agency, PDUFA VII seeks to provide the Center for Biologics Evaluation and Research (CBER) with the funding and authority to hire additional staff to meet this demand. With negotiations being held virtually against the backdrop of the COVID-19 pandemic, PDUFA VII also aims to formalize some of the lessons learned from the pandemic, including guidance and workshops on the use of alternative tools to assess manufacturing facilities and additional resources to support the broader use of digital health tools (DHTs). The commitment letter,4 which was made public in late August 2021, includes the following commitments:
Strengthen Scientific Dialogue. Recognizing the need for further dialogue between the Agency and sponsors, the FDA will formalize the INTERACT meeting framework as well as establish a similar meeting type for the Center for Drug Evaluation and Research (CDER). Both industry and the FDA also discussed the possibility of sharing metrics related to all PDUFA meeting types and associated interactions.
Promote Innovation. To shorten review timelines for certain approved therapies and support efficacy endpoint development for rare diseases, PDUFA VII will establish the Split Real Time Application Review (STAR), modeled after CDER’s Real Time Oncology Review (RTOR).5
Support Advanced Biological Therapies. As noted, CBER will likely be provided dedicated resources to ensure the timely review of all applications for innovative biological therapies. Negotiations also focused on including the patient voice in gene therapy development programs as well as the potential for a workshop on how sponsors could leverage prior knowledge to accelerate gene therapy development. The FDA and industry also discussed a proposal on potential guidance dedicated to clarifying evidentiary standards for the RMAT program.6
Modernize Evidence Generation and Drug Development Tools.To advance the use of real-world evidence (RWE), both industry and the Agency discussed the establishment of a pilot program to develop new methods for using real-world data (RWD) in regulatory decision-making, including in the review of applications.7 Both industry and the FDA expressed an interest in continuing the Model Informed Drug Development (MIDD) paired meeting program. Issuance of guidance related to Complex Innovative Designs (CID) is also included in the commitment letter.8
Advance IT Infrastructure.PDUFA VII will also modernize data and information technology (IT) capacity and capabilities, including the adoption of cloud-based technologies as described in the FDA’s Technology Modernization Action Plan as well as technology convergence across the review centers more broadly.9 Both industry and the FDA also discussed potential programs and initiatives that could inform the evaluation of DHT-generated data.10
Next Steps
The initial draft of the PDUFA commitment letter has been published and is currently open to comment.11 The letter will also be reviewed and discussed at a public meeting on September 28, 2021.
PDUFA VII Negotiation Process
BsUFA
With negotiations recently completed on the heels of the PDUFA VII negotiations, BSUFA III will likely include several commitments that were agreed to in PDUFA VII, such as the development of guidance on alternative tools to assess manufacturing facilities, the dedication of resources to modernize the FDA’s data and information technology capabilities, and resources to enhance hiring and retention. Additional focus areas from the BSUFA III negotiations are outlined below:
Greater Collaboration Between Sponsors and the FDA. To improve collaboration between the Agency and sponsors, the commitment letter will likely include a new Biosimilar Biological Product Development (BPD) meeting type in which the Agency can provide focused targeted feedback. Modifications to the timelines and processes for Type 4 meetings are also likely.12
Regulatory Science Program. To facilitate more efficient biosimilar and interchangeable product development, as well as enhance regulatory decision-making, both the FDA and industry appeared to have agreed to funding a BsUFA Regulatory Science Program, modeled on the successful GDUFA Regulatory Science Program.13
Expedited Application Review. During negotiations, industry and the FDA discussed best practices for application review and opportunities for implementing those best practices into FDA documents and procedures.14
Next Steps
The BsUFA commitment letter is currently being reviewed by the Department of Health and Human Services (HHS), the Office of Personnel Management, and the Biden Administration. The BsUFA letter will also be open to public comment and discussed in a public meeting, likely to be held in mid-fall 2021, before being sent to Capitol Hill.
BsUFA III Negotiation Process
MDUFA
The MDUFA V negotiations have also been shaped by the experience of the FDA, industry, and other stakeholder groups during the COVID-19 pandemic. Both the FDA and industry want to keep elements of the regulatory flexibility that was provided during the pandemic as well as the increased amount of interaction between sponsors and the Agency. Additionally, for MDUFA V, both groups want to carry over many of the commitments and goals that were set during MDUFA IV. This approach aims to help maintain the status quo and ensure stability and continuity for the premarket review program in light of the many adjustments that the Agency had to make to manage the additional workload brought on by the COVID-19 pandemic. This approach will also provide additional time to achieve some of the commitments outlined in MDUFA IV that have not yet been met.15
Several proposals made by each group during the MDUFA V negotiations are still under discussion. These proposals include the following:
Hiring Targets and Vacancies.Throughout the negotiations there has been significant discussion regarding the number of vacancies for full-time equivalent (FTE) staff positions tied to user fees and the salaries for these positions. As part of MDUFA V, industry has proposed that annual specific numerical hiring targets be set in order to increase the formality of these goals, including increased transparency and prioritization by the Agency.16
Review of MDUFA IV One-Time Costs. Industry has proposed a review of funding for one-time costs from MDUFA IV including renewing funding for certain programs but not others. More specifically, industry would like to continue funding for “initiatives for patient engagement, recruitment, retention, and the independent assessment” while not renewing as part of MDUFA V funding “the investment to stand up time reporting” or “the IT investment to support digital health.” Other programs would require further discussion in order to be extended with MDUFA V funding, including “IT enhancements for premarket review work; real-world evidence; standards conformity assessment; and third party review.” In addition, industry has proposed to reinstate 5th year offset fees as the carryover balance has grown to a significant level.17
Device Safety. The FDA has presented a proposal to enhance its capabilities related to postmarket surveillance to allow the Agency “to more accurately and precisely identify the scope of potential concerns, to more efficiently resolve device performance and patient safety issues, and to provide timely and clear communications with patients and healthcare providers.”18 Although industry supports these enhanced capabilities, they noted during the negotiations that MDUFA funding has statutorily been limited to only premarket activities, and this significant change for industry-based funding would require statutory changes.
The Total Product Life Cycle Advisory Program (TAP).The FDA has presented the TAP program as a new model for frequent and rapid FDA interaction with sponsors that would also provide valuable feedback from external stakeholders such as payers and providers. The FDA described the program as including the hiring of new premarket review staff with different levels of expertise as well as inviting external stakeholder groups to participate in the process. The FDA also explained that the program aims to provide a more iterative engagement process with sponsors that could lead to higher quality submissions and fewer review cycles. Industry has voiced serious concern that many elements of the TAP program as currently outlined, such as convening private payers, seem to go beyond the scope of MDUFA and could require statutory changes. Industry is also concerned that this model could lead to more complex reviews and thereby increased program costs. Lastly, industry noted that there are already several FDA premarket programs that provide sponsors with increased engagement with the FDA as well as opportunities to discuss coverage with payers.19
Next Steps
The FDA and industry will continue to meet throughout the coming year to develop an agreed-upon commitment letter to present to Congress. All stakeholder groups are encouraged to participate in the ongoing public negotiations. The most recent FDA-industry meeting was scheduled for May 19, 2021, but meeting minutes have not yet been published.20 Public records also show that several additional stakeholder consultation meetings were scheduled through the end of August.21
2. Although not discussed in this memo, the Generic Drug User Fee Act (GDUFA) has the same end date and will likely be a part of the same legislative package as PDUFA, BsUFA, and MDUFA.
8. PDUFA Reauthorization Meeting Summary, FDA and Industry Negotiation Regulatory Decision Tools Subgroup, 1 December 2021, https://www.fda.gov/media/146389/download.
Recently the Food and Drug Administration (FDA or the Agency), led primarily by the Center for Devices and Radiological Health (CDRH), has been supporting multiple initiatives to promote the development of digital health technologies and to encourage the use of digital health tools (DHTs) in medical product development. In 2017, for example, the FDA published the Digital Health Innovation Action Plan, in which the FDA committed to hiring digital health experts and provided a timeline for issuing new digital health-focused guidances. This was followed by the launch of the Digital Health Center of Excellence (DHCOE) in August 2020. Sitting within CDRH, the DHCOE is intended to serve as a “one-stop shop” for all digital health-related inquires and will coordinate digital health activities both across the Agency and with other regulators internationally.
In the face of the COVID-19 pandemic, as the clinical trial enterprise seeks to maintain continuity while ensuring the safety of trial participants, the FDA’s digital health initiatives seem particularly well-timed. Indeed, during the past few months the use of DHTs have been critical in ensuring the continuity of disrupted trials by supporting remote data collection. During the pandemic, DHTs have been used to facilitate enrollment screening, conduct real-time safety monitoring, evaluate dose effects, and conduct endpoint assessments. In a recent survey of 245 clinical trial investigators, participants reported that 57% of patient interactions during ongoing trials were now conducted remotely.1 The same survey found that 77% of investigators expected the use of DHTs for remote patient monitoring to increase, while 54% expected an increase in use of DHTs to conduct electronic clinical outcomes assessments (eCOAs) and electronic patient reported outcome assessments (ePROs).2 This represents a marked increase over past DHT usage in medical product development, with DHTs only being used in 13% of registrational trials for the time period 2000 to 2018, according to recent research.3
A Regulatory Framework for DHTs
Acknowledging this trend, the FDA has requested funding in the PDUFA VII negotiations to support the development of a DHT framework to strengthen review capabilities of DHT- generated data in submissions, build staff capacity in digital health, and develop IT capacities to store and use DHT data.4 The FDA also shared in the summary report on the COVID-19 Pandemic Recovery and Preparedness Plan (PREPP) Initiative5 that that Agency is currently developing a guidance, which will be released this year, on the use of DHTs to capture study- related data directly from patients.
In the short term, the FDA has addressed the use of DHTs in the guidance “Conduct of Clinical Trials During the COVID-19 Public Health Emergency,”6 providing sponsors with recommendations on how to shift from in-person to remote assessments during the pandemic. In the guidance, the FDA provides sponsors with the flexibility to change the method of administration of clinical outcome assessments (COAs), specifically identifying performance outcomes (PerfO) assessments, interview-based clinician-reported outcome (ClinRO) assessments, PRO assessments, and observer-reported outcome (ObsRO) assessments as COAs that can be conducted remotely. In considering this change in administration, the FDA states that the change must be documented, and any additional variables related to the change should be included in the clinical trial data. The Agency also emphasizes the importance of prioritizing trial participant safety and privacy, maintaining data quality and integrity, and ensuring that both trial personnel and participants are appropriately trained on using any new technologies that are introduced.
DHTs in Clinical Trials
The emerging regulatory framework for DHTs in clinical trials is being built upon a regulatory paradigm that is already, in many respects, supportive of the remote collection of data in trials. As early as September 2013, the Agency addressed the use of DHTs to capture trial data in the guidance “Electronic Source Data in Clinical Investigations.”7 In the guidance, the FDA encourages electronic data capture, noting that it eliminates unnecessary duplication of data, reduces the possibility for transcription errors, and promotes real-time access for data review.
Although the FDA has not issued guidance specifically addressing the use of DHTs in clinical trials since 2013, the Agency has signaled an openness to the use of DHTs so long as they are fit for purpose, private and secure, and compliant with 21 CFR part 11.8 In public meetings and through work with the Clinical Trials Transformation Initiative (CTTI), the FDA has discussed how sponsors could best use digital technology to evaluate clinical benefit in trial participants. For example, prior to selecting a DHT, the FDA states that the concept being measured must be clinically meaningful, measuring what patients would like to see improved through treatment.9 Similarly, in selecting a technology, the sponsor should assess the reliability of the DHT through verification in the laboratory and validation in the field. On a practical level, sponsors also need to consider the technical aptitude and willingness to use a DHT among the population they are studying.10 As DHTs generate large volumes of data, sponsors should pre-specify how the data will be analyzed according to the data characteristics (e.g., intensity, frequency, event, etc.).11 Lastly, as previously noted, DHTs and DHT-generated data must be secure and participants’ privacy must be protected.
4. U.S. Food and Drug Administration, “Prescription Drug User Fee Act (PDUFA) Reauthorization: FDA and Industry Digital Health and Informatics,” 30 Sept. 2020, https://www.fda.gov/media/143237/download.
6. FDA, Guidance for Industry, “Conduct of Clinical Trials of Medical Products During the COVID-19 Public Health Emergency,” March 2020, https://www.fda.gov/media/136238/download.
8. 21 CFR part 11 establishes the criteria under which electronic records and electronic signatures are considered by the FDA to be trustworthy, reliable, and generally equivalent to paper records and handwritten signatures.
9. FDA, Draft Guidance for Industry, “Patient-Focused Drug Development: Methods to Identify What is Important to Patient,” October 2019, https://www.fda.gov/media/131230/download.
10. John Concato, Comments at FDA/CMS Summit, December 7, 2020.
Greenleaf’s experts are available for consultation on strategic and technical FDA regulatory matters. Please provide the information below and a team member will be in contact.
