As discussed in ICH E9 Statistical Principles for Clinical Trials, the primary analysis of a clinical trial should include all randomized subjects in the groups to which they were randomized (intention-to-treat principle). This reduces bias and provides estimates of treatment effects that are more likely to mirror those observed in practice. However, rarely in longitudinal studies is there complete follow-up of all randomized subjects. Methods for addressing missing data in longitudinal studies have been written about extensively for many years, but the focus has increased more in the past decade with the publication of the EMA Guideline on Missing Data in Confirmatory Clinical Trials and the National Research Council’s treatise on The Prevention and Treatment of Missing Data in Clinical Trials both in 2010 and the release of ICH E9 (R1) Addendum on Estimands and Sensitivity Analysis in Clinical Trials in 2017. Case studies and new methodologies addressing the handling of missing data are published on a regular basis. Yet for all the attention this topic receives, and perhaps because of the abundance of information on this topic, clinical researchers often do not know what data imputation approach is best for their particular clinical trial.
In this and subsequent posts, some of the most frequently used data imputation methods in clinical trials will be reviewed, including the assumptions specific to each, their limitations and advantages, and most importantly, guidance on the questions researchers should ask themselves when deciding if a particular method is appropriate for their trial. The reason missing data is such an interesting and challenging problem is that there is no optimal method for all trials. As will be discussed in more detail, the method selected has to be appropriate for the trial indication, the reasons for missingness, and the type of missingness.
Let’s start with what has historically been one of the most popular methods. Last observation carried forward (LOCF) is a methodology that has been used for many years in clinical trials, but is used less frequently now, in part because of criticisms that arose from its misuse, but also because more sophisticated approaches with better mathematical properties have come into favor.
LOCF is an imputation method used in longitudinal studies primarily when missing data is due to patient dropout. For example, in a clinical trial with monthly assessments over one year, if a patient discontinues after completing the 8-month visit, then the 8-month value will be carried forward to months 9, 10, 11, and 12. But what is the underlying assumption that is generally made when applying this method? It is that patients discontinuing early will not have as good a response as patients that complete the full course of treatment. In this regard, carrying forward their last observation is viewed as a conservative approach. The problem is that this assumption does not hold for many clinical trials.
Consider a clinical trial to test a new treatment for Alzheimer’s disease. Without any treatment, patients will progressively worsen over time, some at a greater rate than others. A promising treatment for Alzheimer’s would be one that slows its progression over time, even if it does not cure it. This means that a patient who discontinues from a trial and is on the experimental treatment may have better clinical endpoints at the time of dropout, then he would have had at the end of the trial had he completed. If the primary endpoint is assessed at the end of the trial, then the treatment effect will be overestimated for those for whom LOCF is applied, and consequently the average treatment effect for the study will be overestimated.
This is one of the main drawbacks of LOCF. It ignores the natural history of the disease under study. Conversely, in a condition which spontaneously improves over time without treatment, such as depression, the treatment effect may be underestimated if LOCF is applied in situations where patients in the experimental group withdraw earlier and more frequently. While underestimating the treatment effect is less of a regulatory concern, not bringing a potentially effective treatment to market due to a failed trial is a disservice to the patients who could benefit from it. Since few disease states are static, the effect of LOCF in relation to the disease under study should be considered carefully before applying LOCF.
Another reason LOCF is often not a good method for predicting future outcomes is that it does not account for individual patient trajectories prior to dropout. Even among patients with similar medical conditions receiving the same treatment, responses and drop out reasons will vary. Some may drop out because they have improved and do not see the need to continue, while others may drop out because their condition has worsened. Still others may drop out for reasons unrelated to treatment efficacy, and may have experienced little change in their condition as of the time of discontinuation. Depending on when these patients drop out and which treatment groups they are in, LOCF may not provide good estimates of where these patients would have ended up had they finished the trial.
Having reviewed the underlying assumptions and limitations of LOCF, are there situations where the use of LOCF may be appropriate or even beneficial? Some advantages of LOCF are that it is easy to apply, does not require complex modeling, and is conceptually easy to understand. In cases where the number of missing values is very small, LOCF may be a tool to consider because these advantages may outweigh any small biases that may occur from its application. Additionally, although LOCF may not be selected as the primary method of imputation, when it can be argued that it is a conservative approach, it may be a reasonable choice as one of several sensitivity analyses including worst observation carried forward.
In subsequent posts, we will review some of the more sophisticated missing data imputation methods which are extensively used now, including mixed models for repeated measures (MMRM) and multiple imputation.
