abstract. The 21st Century Cures Act includes a set of provisions affecting the FDA drug approval process. One of those provisions requires the FDA to issue guidance on how drug manufacturers, patient organizations, and others can collect and submit “patient-experience data” to the FDA and how the Agency will use this information in approving new drugs. This Essay examines the FDA’s implementation of these statutory requirements in light of past problems with patient-involvement initiatives at the FDA. It argues that not all patient-involvement initiatives are “good.” Past FDA approval decisions illustrate that there are ways of involving patients in drug development and approval that are ineffective and harmful to patient health. Patient involvement can be ineffective when practices are ill-designed to permit patient input to legitimately affect drug development and the FDA’s decision-making process. And patient involvement can be harmful to patient health when the practices merely serve to pressure FDA decision-makers to bend the Agency’s approval standards to approve drugs that do not work.

Although Congress’s patient-experience-data requirements pose a threat of institutionalizing past problematic practices at the FDA, this Essay concludes that the Agency’s early implementation efforts are encouraging. The Agency’s draft guidance documents and transcripts of meetings with stakeholders on patient-experience data suggest that the Agency is attuned to these problems and is using the directives as an opportunity to hone its patient-involvement mechanisms. Specifically, the Agency appears committed to three important goals in interpreting the concept of “patient-experience data” that support the objective of good patient involvement: (1) fostering an evidence-based approach to collecting patient input, (2) ensuring that patient-involvement mechanisms and tools are tailored to the particular research or policy question at issue, and (3) encouraging patient involvement in the earliest stages of drug development.

Introduction

Two years ago, Congress passed the 21st Century Cures Act, a massive health-care bill with provisions affecting a broad range of issues related to medical research, development, and treatment.1 The Act promised much-needed funding for opioid abuse prevention and treatment, mental-health issues, and research programs at the National Institutes of Health.2 But these potential benefits came at a cost: the Act included a controversial set of provisions affecting the FDA's drug-approval process.3 Consumer-protection groups, physicians, and others fiercely opposed the Act, arguing that the legislation’s changes would weaken the FDA’s drug-approval standards.4

The FDA’s drug-approval process serves a vital role in ensuring the safety and efficacy of drugs in the United States. A manufacturer can only market a new drug if it can prove to the Agency that there is “substantial evidence” from “well-controlled investigations” that the drug is safe and that it works.5 After a drug is approved, the FDA can require manufacturers to carry out postapproval follow-up studies to confirm that the drug provides the intended benefit to patients and does not pose any unjustified safety risks.6

The problematic Cures Act provisions are designed to alter the kinds of data that drug manufacturers can rely on to secure FDA approval or to meet postapproval requirements.7 These provisions, critics argue, push the FDA to approve drugs faster but with less reliable evidence of effectiveness.8 Although purportedly designed to get “twenty-first century cures” to patients faster, these changes risk making it difficult for doctors and patients to know which treatments work and may expose patients to harmful side effects of treatment based on unsubstantiated therapeutic benefits. For example, one section of the Act directs the FDA to consider allowing manufacturers to rely on “real world evidence,” in place of more reliable evidence from randomized controlled trials, to gain FDA approval to market an existing drug for a new purpose.9 Another provision directs the Agency to “maximize” the use of surrogate markers in approving new drugs for rare diseases.10 Surrogate markers are objective laboratory measurements—like cholesterol levels, blood pressure, or tumor size—intended to provide an early indication of whether a drug will improve survival rates or how a patient functions and feels.11 But not all surrogate markers accurately predict clinical benefit.12 In the past, drugs approved using surrogate endpoints—rather than traditional clinical endpoints that directly measure a drug’s effect on symptoms and survival—have been pulled from the market years later, after further studies reveal that the drugs fail to benefit patients.13

And the Act codifies a new category of data that the FDA must consider during the drug-approval process, dubbed “patient-experience data.”14 The Act defines patient-experience data expansively, encompassing any data that “are intended to provide information about patients’ experiences with a disease or condition, including—(A) the impact of such disease or condition, or a related therapy, on patients’ lives; and (B) patient preferences with respect to treatment of such disease or condition.”15 Patient-experience data can be “collected by any persons (including patients, family members and caregivers of patients, patient advocacy organizations, disease research foundations, researchers, and drug manufacturers).”16 Congress framed the use of patient-experience data as part of its broader intent to encourage “patient-focused” drug-development reform by increasing the patient voice in both the drug-development and regulatory-approval processes.17

The promotion of patient involvement in drug development and approvals is undoubtedly valuable. Over the past two decades, the drug-development-and-approval process has faced the compelling criticism that it focuses too narrowly on what physicians and scientists think patients want (or should want) from treatment and what physicians and researchers think will improve patients’ quality of life, rather than on what patients actually desire from treatment.18 This criticism in the drug-development context reflects a broader modern critique of the medical profession as problematically paternalistic, which has spawned a shift in the professional paradigm from a physician-driven practice of medicine toward “patient-centered medicine.”19 In the drug-development context, this has led to the recognition that, although doctors bring medical expertise and broader clinical experience to the table, patients are “experts in what it is like to live with their condition” and are thus “uniquely positioned to inform the understanding of the therapeutic context for drug development and evaluation.”20 Both kinds of expertise are necessary to develop treatments that best improve patients’ health and well-being. Given this understanding, bringing the patient perspective to bear on the drug-development-and-approval process—like which endpoints are measured in clinical trials, what factors constitute the “benefits” and “harms” of treatment options, and how much risk is an acceptable tradeoff for predicted benefit—is an important goal.

But Congress’s broad framing of patient-experience data raises concerns about the way the patient perspective will affect the FDA’s approval process.21 In the past, some drug manufacturers that failed to provide sufficient evidence from clinical trials that a drug works have nonetheless succeeded in marshalling evocative patient testimony and advocacy to secure FDA approval. Gregg Gonsalves and Diana Zuckerman recall the pressure that AIDS activists in the 1980s put on FDA to speed drug approval and allow access to experimental therapies, one of the first campaigns by patients to affect the FDA’s drug-approval process.22 They lament that AIDS activists’ efforts to speed drug approvals inadvertently “grease[d] the wheels for a deregulatory agenda at the FDA” pushed largely by conservative think tanks and the drug industry.23 These groups “often invok[ed] the legacy of AIDS activists and the rights of patients”24 to promote a number of expedited approval pathways allowing drugs to be approved with less and lower-quality data supporting efficacy.25 Alongside these new approval pathways, the FDA has responded to congressional demands to increase patient involvement in the drug-approval process.26 The Agency now offers a diverse array of ways that patients and caregivers can get involved in FDA decision-making, like the Patient Representative Program, which places patients on FDA decision-making committees; the Patient Focused Drug Development Initiative, which holds disease-specific meetings to hear the patient perspective on a disease and its current treatments; and the Patient Network, which links patients to important FDA resources.27

These existing patient-involvement mechanisms at the FDA have not always served patients well. Just months before President Obama signed the Cures Act into law, the FDA approved eteplirsen (Exondys 51), a drug produced by Sarepta Therapeutics to treat Duchenne muscular dystrophy.28 The FDA approved the drug even though both the Agency’s own review team and an independent advisory committee concluded that eteplirsen lacked substantial evidence of effectiveness.29 Despite the limited data submitted by Sarepta, an outpouring of patient and caregiver testimony before the advisory committee, in addition to sustained lobbying of the FDA and Congress, provided the impetus for approval.30 There was substantial outcry from the scientific community; observers lamented that the episode “show[ed] that a drug company, harnessing the desperation of patients, c[an] bulldoze its way to market at a price of $300,000 a year” with a drug that was “studied in an uncontrolled fashion in a handful of patients and showed no clinical benefit.”31

The story of flibanserin’s approval provides another example of how some avenues for patient involvement can undermine rather than support the Agency’s scientific-review process. Sprout Pharmaceuticals sought FDA approval of flibanserin (Addyi), better known as the “pink Viagra,” as a treatment for reduced libido in women.32 The FDA rejected the application twice, noting that the drug’s minimal benefits did not outweigh the risks it posed to patients.33 The third time Sprout sought FDA approval, it did not present any new evidence of the drug’s effectiveness.34 Instead, it provided funding to Even the Score, an advocacy organization that launched an intense lobbying campaign framing the FDA’s past failures to approve the drug as sexist and paid for patients to travel to testify in front of the FDA’s advisory committee about the effects of reduced libido on their lives and marriages and their need for treatment.35 The strategy worked: the advisory committee was swayed by the patient testimony and recommended approval, and the Agency ultimately adopted the committee’s recommendation.36

Past success in leveraging patient involvement at the FDA to secure drug approvals helps explain the industry’s support of the Cures Act, which was critical to the Act’s success. PhRMA, the drug industry’s trade association, increased its quarterly lobbying budget from $3.96 million to $5.44 million as the House Energy and Commerce Committee prepared to introduce the legislation, while the drug and medical-device industries held a top spot on the list of business sectors contributing to the Committee’s chair in the preceding election cycle.37 Over 1,100 lobbyists were registered as working on the Act shortly before its passage, a number “staggering even by the standards of Washington.”38 Many of these lobbyists represented “patient-advocacy organizations.” They argued that the Act’s provisions would speed FDA approval of new, lifesaving treatments for patients in need and enhance the patient voice in the drug-approval process.39 But, as in the flibanserin case, recent studies exposing the extent of industry funding of such groups raise concerns about whose interests some patient-advocacy organizations represent. One study examined 104 of the largest patient-advocacy organizations and found that at least eighty-three percent received financial support from the pharmaceutical industry.40 Another found that fifty-one out of sixty-eight cancer-patient advocacy groups reported sponsorship by one or more drug manufacturers.41 And a study examining the patient groups that the FDA consulted to provide the “patient voice” during recent user-fee negotiations with industry found that ninety-three percent received funding from drug companies, while one-third had executives, directors, or other personnel from pharmaceutical companies on their boards.42

In light of concerns about how the concept of patient involvement has been harnessed in the past and may be used in the future, this Essay examines the patient-involvement provisions of the Cures Act and how the FDA has begun to implement them. It argues that, as past practices at the FDA illustrate, not all patient-involvement initiatives are “good.” There are ways of involving patients in drug development that are ineffective—because the practices fail to actually or accurately reflect patient perspectives—and that are potentially harmful to patient health—because the practices merely serve to weaken approval standards. Yet, despite the risk that the Cures Act’s patient-involvement provisions might lead to a less robust drug review process, this Essay concludes that the FDA’s early efforts to implement the patient-involvement provisions of the Cures Act show that the Agency is committed to eschewing bad patient-involvement practices. Rather than interpreting Congress’s new patient-experience-data directives in a way that would weaken drug-approval standards, the FDA is using the directives as an opportunity to hone its patient-involvement mechanisms so that they shape the drug-development-and-review process at the right moments and in legitimate ways.

I. past problems with patient involvement at the fda

The eteplirsen and flibanserin examples discussed above illustrate two problems that have plagued patient involvement at the FDA since the Agency first formalized patient-involvement mechanisms in the drug-approval process in the 1990s.43 First, the practice of patient involvement can be ineffective, when the method of patient involvement does not allow patient perspectives and experiences to meaningfully affect the drug development or approval process. This is often a design problem: failing to match the patient-involvement method with the goal or reason for soliciting patient input. Consider an example from the approval of eteplirsen. Duchenne muscular dystrophy results in progressive muscle degeneration and weakness, and patients often lose the ability to walk in their teenage years. The eteplirsen clinical trial failed to show that the drug improved patients’ rates of retention of the ability to walk—the clinical endpoint measured in the trial.44 But at the public hearing before the advisory committee, many patients and their caregivers testified that they had witnessed the drug’s effect on smaller but nevertheless meaningful activities, like feeding oneself or holding a book.45

While heart-wrenching, this kind of patient-experience data was ill-suited to the regulatory task at hand: the advisory committee to which they were appealing had to determine whether Sarepta had provided substantial evidence of efficacy through adequate and well-controlled investigations. Patient testimony that the drug might have had other effects that were not tested suggest that Sarepta might have been wise to consult patients earlier in the process and choose different clinical endpoints; but it could not provide reliable evidence that the drug worked. In other words, the patient-involvement method was poorly tailored to the decision it was meant to influence. It did not provide meaningful and legitimate input into the regulatory decision-making process.46

This kind of mismatch is troubling in part because it represents a missed opportunity to involve patients at the right point along the drug-development continuum. Here, the testimony suggests that consulting patients earlier might have affected the clinical trial design, the knowledge that Sarepta generated about the drug’s effects, and ultimately the review team’s and advisory committee’s evaluations of the application. But the mismatch is also problematic because it represents a kind of process harm. Patients rightly expect that their participation in various involvement mechanisms for the purpose of capturing the patient voice and experience will actually feed into the Agency’s decisions. When a mechanism’s design renders patients’ input irrelevant, patients are misled. Superficially involving patients in the drug-approval process without any possibility that their testimony will meaningfully inform the Agency’s decision gives the impression that the Agency is consulting them only as a political maneuver: it may look as though the Agency cares more about appearing to incorporate the views and experiences of affected persons than actually doing so.

Second, the practice of patient involvement can be harmful to patient health when it serves as a vehicle for pressuring the Agency to approve drugs, without substantial evidence of effectiveness, under the guise of listening to or respecting patient perspectives. Paradoxically, although patient involvement was ill-designed to legitimately affect the approval process in the eteplirsen and flibanserin cases, there is little doubt that it played a significant role in their approvals. On flibanserin’s third run through the FDA’s approval process, the drug manufacturer produced no additional evidence that the drug worked;47 the only factors that convinced the FDA to change course appear to be the outpouring of testimony from patients and a media campaign that framed the FDA as discriminating against libido-enhancing drugs for women.48 With respect to eteplirsen, many critics, including some within the FDA’s review team itself, concluded that the drug was only approved in light of the overwhelming pressure from patients and patient-advocacy organizations at the public hearing and through other interactions with the FDA.49

But patients ultimately bear the cost when the FDA signs off on drugs that do not work, particularly if those drugs involve serious side effects or require forgoing other effective treatment. Channeling patient input through avenues like public testimony at media-heavy hearings may increase pressure on decision-makers to bend the FDA’s approval standards, absent any other way to hear and attend to patient perspectives. As discussed above, drug manufacturers have been quick to fund and organize patient advocacy both through institutional channels, like hearings or meetings, and through political avenues. It is likely that the industry will similarly attempt to capitalize on the Agency’s new focus on patient-experience data. Indeed, on the heels of the 21st Century Cures Act, Evidera—a unit within global contract research organization Pharmaceutical Product Development, LLC—announced the expansion of its patient-centered research services to “enable [its] clients to effectively navigate this new world of patient-focused drug development,” promising to “elicit[] patient preference information that meets the needs of regulatory and payer decision-makers.”50 While stimulating this kind of research is important, industry-funded patient-experience studies based on biased methods could be used to push for approval of drugs that otherwise lack sufficient evidence of efficacy.

In sum, past patient-involvement practices at the FDA have proved at times both ineffective—ill-designed to channel patient experiences and preferences into the drug development process—and harmful—by creating channels to place pressure on FDA committees to approve drugs that don’t work, thereby exposing patients to harms of treatment that are not outweighed by the benefits. The concern is that the patient-experience-data provisions of the Cures Act may endorse and increase these problems in drug approvals. The next Part assesses the FDA’s early efforts to implement these provisions and concludes that, despite this threat, the Agency is attuned to these concerns and is interpreting the statutory concept of patient-experience data to support meaningful patient input and to avoid these past problems.

II. implementing the patient-involvement provisions of the 21st century cures act

When Congress codified the basic concept of patient-experience data, it delegated to the FDA the task of fleshing out this definition and explaining how such data would be collected and used. The Act mandates that within six months of its enactment, the FDA must begin “mak[ing] public a brief statement regarding the patient experience data and related information, if any, submitted and reviewed” when it approves a new drug application.51 It also requires the FDA to issue a series of guidance documents “regarding the collection of patient experience data, and the use of such data and related information in drug development.”52 These guidance documents must address a range of issues relating to patient-experience data, roughly falling into three categories: (1) how to collect this data,53 (2) how to submit it to the FDA,54 and (3) how the FDA will use the data.55

In May 2017, the FDA released its “Plan for Issuance of Patient-Focused Drug Development Guidance.”56 The plan sets out a five-year time frame for issuing the guidance documents mandated by the Act.57 In the plan, the FDA explains that the Agency will produce four guidance documents detailing the “methods and approaches that can be used by drug sponsors, patient advocacy groups and others to more systematically collect and rigorously measure disease and treatment impacts that matter most to patients.”58 The first guidance document will describe approaches for collecting patient and caregiver input on the burden of disease and existing therapies, including methods for collecting input throughout the drug-development process and considerations for data collection and analysis.59 The second will address methods for identifying what set of impacts are most important to patients and how that data might ultimately feed into drug development and regulatory decision-making.60 The third will address the development of measures to facilitate patient input in clinical trials.61 And the fourth will address clinical outcome assessments, including technologies for collecting and analyzing patient-perspective information and methods for incorporating clinical outcome assessments as endpoints appropriate for regulatory decision-making.62 Based on the development of these four initial documents, the FDA will release guidance explaining how patient-experience data will affect the FDA’s decision-making process and its framework for benefit-risk assessment.63 For each guidance document, the FDA plans to hold a public workshop to receive feedback from stakeholders on proposed ideas.64

To date, the FDA has held public workshops on the first three guidance documents65 and has issued draft guidance on the first document.66 Discussion documents that track the draft guidance the FDA plans to promulgate are released in advance of these public workshops in order to provide participants with an opportunity to comment on the FDA’s proposals before the draft guidance is officially released.67 These discussion documents thus provide a first look at the FDA’s understanding of patient-experience data and the role it will play in patient involvement at the Agency. Similarly, transcripts from these public workshops detail the FDA’s dialogue with a variety of stakeholders, including industry representatives, patients, doctors, and researchers, as the Agency seeks to justify its vision for patient input in the drug development and approval process.68 Taken together, these sources reveal three core commitments that the FDA has made in interpreting the concept of patient experience data under the Act: (1) fostering an evidence-based approach to collecting patient input, (2) ensuring that patient-involvement mechanisms and tools are tailored to the particular research or policy question at issue, and (3) encouraging patient involvement in the earliest stages of drug development. Each commitment pushes back on past practices of patient involvement at the Agency that were problematic, suggesting the FDA’s resistance to interpreting and implementing these statutory provisions in a way that would institutionalize the problems seen in the eteplirsen and flibanserin examples.

A. Toward a “Science of Patient Input”

A central message of the FDA’s early implementation efforts is that it will use the patient experience directives to develop more systematic and rigorous methods for collecting patient input. The Agency frames the use of patient-experience data as an evolution of its past patient-involvement practices, which it recognizes relied on more anecdotal and nonrepresentative accounts of patient values and experiences. For example, a primary method of patient involvement at the Agency before the Cures Act was a series of “patient focused drug development meetings” that the Agency held, a forerunner attempt at promoting the patient voice in drug approvals.69 Under the FDA Safety and Innovation Act, the Agency held twenty-four disease-specific meetings that brought together representatives from the FDA, patients, drug developers, doctors, and researchers for “an opportunity to hear the patient’s voice.”70Meetings focused on a range of diseases, including autism, psoriasis, breast cancer, and narcolepsy.71 At these meetings, the Agency heard from patient and caregiver participants about the most burdensome symptoms of the disease and the adequacy of current treatments. After each meeting, the FDA produced a “Voice of the Patient” report summarizing input it received during the meeting.72

In its Cures Act implementation plan, the Agency recognized that these meetings and the resulting reports provided substantial insight into the nature of the studied diseases and the shortcomings of current treatment.73 But the FDA also recognized that this method of collecting patient experiences has its limits. Patient panelists can speak to their own experience of a disease and treatments, but the Agency has no way of knowing if those perspectives are representative. Many of the diseases studied affect diverse populations that may react differently to treatments and that may have different health values. And, even when these kinds of meetings elicit new priorities in treating the studied disease, they do not provide an opportunity to measure the effects that specific treatments have on those outcomes. Thus, the FDA explains that one of the broader lessons from holding the series of disease-specific meetings was “that a more systematic and rigorous approach to collecting the patient’s perspective and patient experience data was needed to better advance patient-focused drug development.”74

The Agency casts the Cures Act provisions as an opportunity to “bridge from important early-stage efforts to gain patients’ narrative perspectives on the clinical context (e.g., meetings with patients), to development and use of methodologically-sound data collection tools in clinical trials.”75 At the Guidance 1 workshop, the FDA explained the importance of developing objective approaches to collecting representative patient experiences and values. Although the Agency recognized that independent “narrative anecdotal accounts are very important and powerful,” it stressed that individual patient stories cannot “substitute for data collected for a whole population in terms of how we can use it in decision-making.”76 Thus, during its guidance-development meetings, the FDA indicated to stakeholder attendees that it hoped to move beyond the practice of soliciting patient input at disease-specific patient meetings or during committee hearings and towards a “science of patient input.”77 In a glossary of terms prepared for the purpose of standardizing terminology for the guidance series, the FDA defined the science of patient input as “[m]ethods and approaches of systematically obtaining, analyzing, and using information that captures patients’ experiences, perspectives, needs, and priorities in support of the development and evaluation of medical products.”78

The Agency’s guidance documents are intended to support this more systematic and representative approach to patient input. The first document focuses on how to select patients from whom to collect patient-experience data by both properly defining the target population and selecting an appropriate sampling method to ensure that the patient participants are representative of the broader target population.79 Once that population is defined, the second guidance document will explain ways to pose unbiased, nonleading questions to participants in order to elicit the impact of their disease and what matters most to them in treatment. Framing questions appropriately is important to avoid “results that inadequately or incompletely identify what is important to patients.”80 And the third guidance document “will address how to refine the list of important impacts and concepts elicited from patients, as described in Guidance 2, to develop potential study instruments,” called clinical outcome assessments (COAs).81 COAs are used in clinical trials to measure the effect that a drug has on “an outcome that describes or reflects how an individual feels, functions or survives.”82 Ensuring that these clinical trial tools accurately test effects on the outcomes that patients themselves have identified as the most important will help provide the best evidence to FDA decision-makers of a treatment’s value.

This evidence-based approach to patient experiences and values sharply contrasts with the anecdotal patient and caregiver testimony that supported the eteplirsen and flibinaserin approvals. By supporting the systematic collection of patient views in a nonbiased manner, the FDA’s approach to patient involvement as a “science” indicates how patient perspectives can be appropriately integrated into the drug development and approval process.

B. “Fit-for-Purpose” Methods of Patient Involvement

The Agency’s early work also emphasizes that different methods for collecting patient-experience data will be appropriate for different stages of the drug-development-and-approval process. As required by the Cures Act, the FDA has added a section on “Patient [E]xperience Data” to the review and decision templates the Agency uses during new drug reviews.83 When the FDA reviews new drug applications, it now includes a checklist of the different types of patient-experience data that the review team considered. The checklist includes seven possible categories of patient-experience data: (1) clinical-outcome assessment data; (2) qualitative studies—like patient, caregiver, focus group, or expert interviews; (3) patient-focused drug-development reports or reports from other stakeholder meetings; (4) observational-survey studies; (5) natural-history studies; (6) patient-preference studies; and (7) input from meetings with patient stakeholders.84 This list suggests a broad range of sources and methods for collecting patient-experience data. But it does not explain (1) what research questions these data can appropriately answer or (2) what kinds of methodologies can ensure that patient-experience data collected in these various categories is rigorous and sufficiently reliable for the Agency to use in its decision-making.

The Agency has begun trying to answer these questions through its Cures Act guidance. During the first guidance workshop, the FDA attempted to delineate the various points at which patient-experience data might influence the drug-development process. In the prediscovery or discovery phase, a researcher “might . . . identify[] what the disease impact and treatment burdens are that patients and their families are most concerned about”85 and then select data-collection instruments relevant to those impacts and burdens and test them to ensure they are suitable for clinical trials.86 Next, during the clinical trials, the researcher tries “to assess whether the changes in those clinical outcome assessments during the studies are . . . clinically meaningful to the patient.”87 Finally, after the FDA has approved a drug, the researcher may “collect[] information post-market to really understand the degree to which those benefits and risks . . . reported on during the clinical-development phase are consistent with what’s happening in the larger population post-approval.”88

Each of these points in the drug-development process, as the FDA’s explanation suggests, calls for different kinds of patient engagement. Focus groups or a series of qualitative interviews with patients may be appropriate at the early stages of determining what outcomes patients most hope for from treatment. Seeking input at this exploratory stage of the process calls for methods that will elicit a broad spectrum of patient views and experiences. For example, the FDA has recognized that attaining an appropriate sample size and sampling method in this context is different from the later context of showing that a drug has a statistically significant effect on whatever outcomes earlier patient research identified. As one of the panelists at the workshop for the second guidance document noted:

It is really important to understand that the sample size in qualitative research has a completely different purpose than the sample size in quantitative research. And the guidance observes this, that sample size for qualitative research is intended to prevent discovery failure. In other words, not include a voice or perspective that is very important in that context. It has nothing to do with statistical significance.89

By contrast, the concept of statistical significance is crucial in showing the effect a drug has on the meaningful outcomes identified in this early-stage research. The tools developed to measure changes in the outcomes that patients care about must be sensitive enough and suitable for capturing the size of the treatment effect relative to a control arm in a clinical trial. And clinical-trial participants responding to those tools must be representative of the drug’s target population if we are to know whether the drug will have the demonstrated effect on patients when brought to market.

Although patient-experience data can be collected in a broad range of research contexts—including “clinical trials, observational studies, advisory boards, public meetings, and other novel settings (e.g., online patient communities)”—the “level of rigor needed for patient experience data generation . . . will depend on the intended use.”90 And the tools for collecting patient-experience data must be “fit-for-purpose.”91 On this point, one FDA official recognized that “qualitative and narrative anecdotal accounts are very important and very powerful,” but stressed that “they won’t be a substitute for data collected for a whole population in terms of how we can use [them] in decision-making.”92 She clarified that these kinds of data are appropriate for different purposes: while “narrative data, including [information] extracted from . . . interviews”93 could be used—like the patient-focused drug development meetings and reports—”to []give us general insight about how patients feel, the clinical context for regulatory decision-making, [or] a general sense of the burden of disease and burden of treatment that are available today,”94 the new guidance aims to help patient advocates and sponsors “collect . . . data that can be used to actually measure the performance of a particular product that’s under development or investigation.”95

Recognizing that different kinds of patient involvement are appropriate at different stages of the drug-development-and-approval process is crucial. In the past, failure to tailor the patient-involvement mechanism to the relevant regulatory question has undermined the FDA’s approval process: the anecdotal accounts of patients testifying before the FDA’s advisory committees in the eteplirsen and flibanserin cases were not suited to answering the question whether the drug manufacturer had produced substantial evidence of the drug’s efficacy in well-controlled trials. What the committees needed at that point was robust evidence showing a statistically significant improvement provided by the drug, so that the committees could be sure, in recommending approval, that their decisions would serve all patients in the target population—not just the ones testifying in favor of approval. Patient testimony at such hearings still has a role to play in contextualizing the effect that a drug demonstrates and the lived experience of patients with the disease at issue, but it cannot substitute for representative, unbiased data collected during clinical trials in showing the effects of a treatment.

C. Upstream Patient Involvement

Finally, the FDA has stressed in its guidance work to date that involving patients earlier in the drug-development process will improve the effect that drugs have on patients and help ensure that the Agency has the reliable data it needs in making approval decisions. A key element of this guidance is that patient input will often have greatest effect if considered earlier and throughout the drug-development process, rather than just at the later stages when an FDA review team or advisory committee is assessing a drug sponsor’s submission. The first draft guidance document stresses that “[p]atient experience data may be collected throughout medical product development, beginning early in development (e.g., discovery) or independent of any specific medical-product development program (precompetitive setting).”96

Absent these early efforts to ascertain what outcomes would be most meaningful to patients, drug manufacturers may develop less effective treatments or fail to measure a drug’s effect on the outcomes that matter most to patients. For example, at the public hearing on the second guidance document, one FDA official explained that the “lack of a thoughtful approach to measurement” can leave clinicians without “a patient-centered instrument” capable of “assess[ing] what is important to patients.”97 In such a case, use of the tool “can lead to content validity problems or misleading content, such that the tool doesn’t accurately assess the target concept,” which ultimately “may compromise [the FDA’s] ability to accurately describe the clinical benefit.”98 Alternatively, poor upstream patient involvement may result in the design of a measurement tool with “poor ability to detect change,” which in turn “may compromise the ability to detect a treatment effect when one exists.”99

In other words, it is vital to consult patients well before clinical trials begin and before selecting or designing clinical outcome assessments that will be used in those trials. It is impossible to show clinical benefit if the tools selected are insufficient to measure a drug’s influence on the outcome, or if the outcome selected is not particularly important to patients in the first place. The eteplirsen case provides a useful example. There, the manufacturer decided to measure the drug’s effect on patients’ retention of the ability to walk. The clinical trial did not produce substantial evidence that eteplirsen improved walking ability.100 But at the public hearing, patients and caregivers repeatedly stressed that smaller functional improvements were deeply valuable—like being able to grasp a spoon to eat or brush one’s teeth.101 Although individual patients in the clinical trial and their families testified that they experienced improvements on these dimensions, the manufacturer’s data did not bear this out—if only because the effects on these functions simply were not tested during the clinical trial. Had the manufacturer consulted patients early in the study-design process, it may have opted to measure the drug’s effects on these functions, which may in turn have altered the evidence submitted to the FDA.

Conclusion: A Promising Start

The FDA’s implementation of the 21st Century Cures Act is still in its earliest stages. Nevertheless, its work on the patient-involvement provisions to date is encouraging. The Agency’s framing of the concept of patient-experience data and how to collect it suggests that with this new phase of patient involvement at the Agency, the FDA may be able to avoid two problems that have plagued attempts at patient involvement in the past: ineffective involvement and involvement mechanisms that produce harmful effects on patient health. Good patient involvement ensures that patients actually have the opportunity to affect the course of development or review, and that patient input legitimately influences the course of drug development and approval and does not serve merely to erode standards of efficacy. In the past, the practice of patient involvement at the FDA has failed at times to meet these guiding principles. The Agency’s insistence in its Cures Act work on (1) developing a science of patient input; (2) requiring tools for collecting patient-experience data that are fit for purpose; and (3) increasing upstream patient involvement in the drug development process support the goal of distinguishing valuable patient-involvement mechanisms from bad ones. The ultimate test of these commitments will be whether the FDA adopts them in practice and not just in theory: whether the Agency applies these concepts in case-specific drug approvals while holding drug manufacturers to the Agency’s safety and effectiveness standards. But for now, the Agency’s plans appear well designed to improve patient involvement in the drug-approval process.

J.D., Yale Law School; D.Phil., Public Health, University of Oxford. Many thanks to Aaron Kesselheim for supervising this research project as part of his FDA Law and Policy class, and to Amy Kapczynski, Gregg Gonsalves, and the other participants in Yale’s Global Health Justice Partnership (GHJP) Research and Advocacy Fellows Program for conversations and feedback on these ideas. Thanks also to Peter Kallis, Abigail Pershing, Cody Poplin, Zoe Jacoby, and the editors of the Yale Law Journal for their careful editing and thoughtful suggestions.