The Cures Act of December 2016 added section 505F to the Federal Food, Drug, and Cosmetic Act (FD&C Act), requiring FDA to create a framework for the use of real-world evidence (RWE) in regulatory decision-making. In response, the draft FDA guideline entitled, “Considerations for the Use of Real-World Data and Real-World Evidence to Support Regulatory Decision-Making for Drug and Biological Products,” issued in August 2023, provides very interesting and useful information for consideration in the planning of a marketing application (NDA/BLA) for a drug or biologic product based on RWE.
This guidance focusses on the use of RWE to support a new indication for an already approved drug or biological product or to help support post-approval study requirements. Applications for new products are not in the scope of this guidance.
Before discussing the initiatives included in this guidance, it is important to have an understanding of the definitions of real-world data (RWD) and RWE.
RWD are data relating to patient health status and/or the delivery of health care, routinely collected from a variety of sources such as electronic health records (EHRs); medical claims and billing data; data from product and disease registries; patient-generated data, including from in‑home-use settings; and data gathered from other sources, such as mobile devices, that can inform on health status. Basically, this is the raw data from which we want to extract the information to be used to form RWE. RWD is analogous to the patient records that are used as source data in traditional prospective interventional clinical studies, the difference being that the data are not prospectively generated and collected.
RWE is the clinical evidence about the usage and potential benefits or risks of a medical product derived from analysis of RWD. It is the result of the extraction, standardization and analysis of the RWD. When we look at it this way, RWE is analogous to the statistically derived tables and the interpretation of the analyses presented in Clinical Study Reports (CSRs) of traditional prospective interventional clinical studies.
While FDA has been using RWE for some time to support safety, and does this routinely in its post-marketing safety surveillance activities including the sentinel system, the use of RWE to support drug approvals has been limited and has been primarily focused on oncology and rare diseases.
This new guidance outlines what one needs to consider when deciding to use RWE in support of a new indication, how to determine if it is suitable to use RWE, the nature of early interactions with FDA, and the planning and execution of data collection and analysis.
How do I know if my RWE is good enough?
When using RWE as substantial evidence of efficacy in a marketing application, not only is the amount and quality of the data important, equally important are the methodologies used to generate the evidence and the reliability and relevance of the underlying RWD. The same systematic approaches that one follows in designing a prospective clinical study must be used in designing a program where RWD will be used to generate RWE.
The guidance focusses on RWE clinical study designs that are non-interventional. Interventional studies (also referred to as clinical trials) are studies in which participants, either healthy volunteers or volunteers with the condition or disease being studied, are assigned to one or more interventions, according to a study protocol, to evaluate the effects of those interventions on subsequent health-related outcomes.
In a non-interventional study (also referred to as an observational study) patients received the marketed drug of interest during routine medical practice and are not assigned to an intervention according to a protocol. Another type of observational study is a natural history study (NHS) where untreated patients are observed and important data on those patients is collected in accordance with a protocol. These NHS studies are used as external control arms in drug and biologic approvals. As such, non-interventional studies are not clinical investigations as defined under 21CFR 312.3 and do not require an IND. This, however, does not mean that other safeguards, such as informed consent, which may be limited to consent to use the patient’s data, IRB approval, and data privacy measures are not needed.
As one reads through the guidance, it becomes clear that the same methodical steps that one goes through in setting up and executing an interventional study are also required for non-interventional studies.
FDA recommends that Sponsors collaborate with FDA in the early stages of designing a non-interventional study intended for use in a marketing application. It is important to discuss with the relevant FDA review division the expectations regarding access to the RWD used in the development program. Sponsors must ensure that they are able to submit to FDA patient-level data for any analyses that will be part of the final report included in the marketing application.
The protocol and statistical analysis plan (SAP) should be provided to FDA for review prior to finalization and data analysis. If and when they occur, revisions to the protocol should be dated and the rationale for each change provided to the agency.
Transparency regarding data collection and analysis is very important. How a particular set of RWD was selected, where the data was taken from, how the data was collected and by whom become very important and need to be documented prior to the analysis of the data. The guidance also suggests that the collection protocol be posted on public websites such as clinicaltrials.gov.
One of the reasons there is a focus on having a collection plan (protocol) and analysis plan (SAP) signed by the Sponsor before the start of data collection is to ensure that the analyses planned and agreed to with the agency are conducted as outlined; any ad hoc analyses need to be appropriately identified in the final report. Emphasis is placed on this because the Sponsor has the ability to conduct numerous retrospective analyses until the desired result is obtained and then submit only favorable results as if they were the result of a single study with a prespecified protocol.
The first part of the design of an RWE study includes a feasibility assessment to determine if the data source or database is fit for use.
How and why the source was chosen, as well as, the data to be extracted are important pieces of information to include in the final report. Also to be included are patient characteristics of the source population and the study populations, together with analyses performed per SAP and any additional exploratory analyses.
There should be complete audit trails and agreements so that FDA will have access to the RWD to verify the results. If certain RWD are owned and controlled by other entities, agreements should be put in place to ensure that relevant patient-level data can be provided to FDA, and that the source data necessary to verify the RWD are made available for inspection, as applicable.
Sponsors should ensure that RWD and associated programming codes and algorithms submitted to FDA are documented, well-annotated, and complete, allowing FDA to replicate the analyses using the same dataset and analytic approach.
If you are asking whether there is a need for study monitoring, the guidance makes it clear that the answer is yes.
Monitoring should begin with the data extraction from RWD sources and focus on the protection of human subjects and on maintaining data integrity. At a minimum, study monitoring should include the following:
− Ensure that the RWD required by the protocol are accurate and consistent with the source records.
− Ensure that prespecified plans, protocol, and study procedures were followed.
− Ensure that deviations from the prespecified plans, protocol and study procedures are identified and documented, and when necessary, promptly evaluated and remediated according to the significance of the deviations that have been identified.
What is expected for safety reporting?
One should not forget the requirements for safety reporting. For the studies that examine the use of a drug or biologic in routine medical practice, requirements for post-marketing safety reporting of relevant adverse events must be followed. The expectation here is that, if as part of collecting RWD to support a new indication, one identifies an AE which meets the requirements for reporting, the appropriate reporting steps, per regulations, should be followed. To assure that this is done, it may be useful to add these processes to the protocol.
All electronic systems used in this process should be 21CRF part 11 compliant.
What are the responsibilities of the Sponsor and other parties?
FDA considers the Sponsor the responsible party throughout the process. All activities related to the design, conduct, analysis, and oversight of the study, including selection of qualified researchers, assuring that the study is conducted in accordance with the final protocol and SAP, documenting deviations, and assessing whether study records are adequate and whether FDA has access to study records for verification, are part of these responsibilities.
As part of study oversite, a log of researchers involved, including names, qualifications, roles, and activities conducted should be retained by the Sponsor.
If third parties (e.g., data service providers or contract research organizations) are contracted to perform certain study-related tasks, this should also be documented and this information made available to FDA upon request.
In conclusion, this very informative guidance provides important information to consider when designing, implementing and analyzing RWD to be included in an efficacy supplement based on RWE. After reading the guidance and other supporting information, it becomes clear that using RWE to support a new indication is not a simple undertaking and certainly is not easier or faster than conducting a prospectively designed study. However, there are cases when the use of RWE to support an efficacy supplement may be the most appropriate approach.
For those situations where a randomized controlled study may be difficult to do, this guidance provides information on how best to leverage RWD to gain approval of a supplement to support a new use or fulfill a post-marketing requirement of an approved drug or biologic.
