Clinical Evaluations – Methods

Chuck Simonton, MD

Michael John, MPH

In the United States, the evaluation and approval of medical devices is regulated by the Food and Drug Administration (FDA) (1) through its Center for Devices and Radiological Health (CDRH), which has a Division of Cardiovascular Devices (DCD). Within DCD, the Interventional Cardiovascular Devices Branch, or ICDB, has lead review responsibilities for coronary stents, including bare-metal stents (BMS), drug-eluting stents (DES), and bioresorbable scaffolds (BRS).

Classification of medical devices by FDA (2), categorized as Class I, Class II, and Class III, is based on the risks associated with the device.

Class III devices are generally the highest-risk devices and are therefore subject to the highest level of regulatory control/evidence. Class III devices must typically be approved by FDA before they are marketed.  Coronary Drug Eluting Stents (DES) are classified as Class III and follow the regulatory pathway for premarket approval (PMA) (3). This requires a comprehensive evaluation including bench testing, preclinical animal studies, and clinical evaluation, so that the device demonstrates a reasonable assurance of safety and effectiveness and that the potential benefits outweigh the potential risks. The term ‘effectiveness’ means that the device will provide clinically significant benefits, and thus evaluation focuses on clinical outcomes such as a reduction in cardiac death, target vessel myocardial infarction (MI) or target vessel revascularization.

Clinical data for a PMA are typically obtained from feasibility studies followed by a larger, ‘pivotal’ trial. To conduct a clinical trial in the United States to assess the safety and/or effectiveness of a Class III device such as DES and thereby provide evidence that will support a PMA, the sponsor must obtain FDA approval of an Investigational Device Exemption (IDE). Since the FDA requires robust preclinical safety data before the initiation of human trials, first-in-human (FIH) early device implants may start outside the United States, but the U.S. early feasibility study pathway (see below) has provided a new mechanism for industry to bring early clinical studies back to the U.S. as a basis for data needed to start a pivotal IDE study.

For permanently implanted Class III devices such as DES and devices for occluding intracardiac shunts, the FDA typically requires 5 years of clinical follow-up. Post-approval clinical studies that collect and report real-world outcomes associated with the use of novel devices are also commonly required. The FDA has had a policy of ‘global transparency’ since 1994 and all medical device reports concerning significant adverse events are available online (4,5).

United States  device manufacturers and those outside the United States (OUS) can meet with FDA staff at multiple times during the development of a device through the FDA’s Pre-Submission, or Q-Sub, Program (which is a request for feedback structure from the FDA) (6). These informal meetings are particularly useful for the sponsors and the FDA to reach consensus on preclinical device testing protocols, key elements of IDE clinical trials, and requirements for PMA submissions. Following submission of a pre-submission package, FDA has up to 90 days to provide written feedback, or if requested, schedule and hold a meeting directly with the manufacturer. Although not binding, FDA feedback represents the agency’s current thinking on the information presented, and FDA routinely adheres to the feedback and agreements documented unless new information arises before the formal submission of an IDE or PMA. FDA encourages early and often interaction, especially in the case of novel devices. It is highly recommended to use these meetings to clarify and strategize the submission content for both IDE and PMA to gain alignment with the FDA on data required for approval.

FDA defines different types of studies (7):

An early feasibility study (EFS) is a limited clinical investigation of a device early in development, typically before the device design has been finalized, for a specific indication (e.g., an innovative device for a new or established intended use or a marketed device for a novel clinical application). It may be used to evaluate the device design concept with respect to initial clinical safety and device functionality in a small number of subjects (generally fewer than 10 initial subjects) when this information cannot practically be provided through additional nonclinical assessments or appropriate nonclinical tests are unavailable. Information obtained from an EFS may guide device modifications. Importantly, an EFS does not necessarily involve the first clinical use of a device.

An FIH study is a type of study in which a device for a specific indication is evaluated for the first time in human subjects.

A traditional feasibility study is a clinical investigation that is commonly used to capture preliminary safety and effectiveness information on a near-final or final device design to adequately plan an appropriate pivotal study. Because the study of a near-final or final device design takes place later in development than an EFS, FDA would expect to see more nonclinical (or prior clinical) data in a traditional feasibility study IDE application.

A pivotal study is a clinical investigation designed to collect definitive evidence of the safety and effectiveness of a device for a specified intended use, typically in a statistically justified number of subjects. It may or may not be preceded by an early and/or a traditional feasibility study.

Initial clinical testing of novel devices has been moving to non-U.S. sites and device innovation may follow overseas or devices are being developed exclusively for non-U.S. markets. There appears to be palpable lag time in the availability of some beneficial medical devices for U.S. patients compared to overseas patients.

In 2013, FDA issued a draft guidance document “Investigational Device Exemptions (IDEs) for Early Feasibility Medical Device Clinical Studies, Including Certain First in Human (FIH) Studies”  (7). Here are some pertinent excerpts:

— Early feasibility studies are intended to allow for early clinical evaluation of devices to provide proof of principle and initial clinical safety data. These studies may be appropriate early in device development when clinical experience is necessary because nonclinical testing methods are not available or adequate to provide the information needed to advance the developmental process.

— FDA recognizes the value of encouraging medical device innovation to address clinical needs and improve patient care, particularly when alternative treatments or assessments are unavailable, ineffective, or associated with substantial risks to patient safety. This guidance has been developed to facilitate the early clinical evaluation of medical devices in the United States under the IDE regulations, using risk mitigation strategies that appropriately protect human subjects in early feasibility studies.

— FDA approval of an IDE application for an early feasibility study, including certain first in human studies, may be based on less nonclinical data than would be expected for a traditional feasibility or a pivotal study. This is because early feasibility studies are only appropriate when additional nonclinical testing would not provide the information needed to advance the developmental process.

— The IDE application should clearly state that the proposed study is an early feasibility study and provide justification for conducting this type of study.

— Data from an early feasibility study may lead to device modifications and be used to refine the bench, analytical, and in vivo animal studies and future clinical study protocols.

— If the device design is near-final or final, and the results of the early feasibility study support the initial safety of the device and proof of principle, it may be more appropriate for the sponsor to pursue either a traditional feasibility study or a pivotal study.

The regulatory approval pathway for a coronary stent in the United States is well established.  FDA has published guidance for non-clinical and clinical testing of both BMS and DES, which are also broadly applicable to the newer BRS.  The three most relevant guidance documents include:

— Non-Clinical Engineering Tests and Recommended Labeling for Intravascular Stents and Associated Delivery Systems April 18, 2010 and Select Updates for Non-Clinical Engineering Tests and Recommended Labeling for Intravascular Stents and Associated Delivery Systems – August 2013

— Coronary Drug-Eluting Stents-Nonclinical and Clinical Studies – March 2008

— Draft Guidance: Coronary Drug-Eluting Stents — Nonclinical and Clinical Studies -Companion Document – April 2008 (3)

The regulatory standard for clinical evidence in the United States requires demonstration of a reasonable safety and effectiveness of the device and demonstration that the expected benefits likely outweigh the expected risk (8). New coronary stents were initially expected to be evaluated clinically and demonstrate superiority on the current treatment standard (BMS over percutaneous transluminal coronary angioplasty [PTCA] and DES over BMS) to meet this standard. The first BMS were approved based on clinical evidence showing superiority to PTCA with respect to target vessel revascularization (TVR) and other safety measures. The first DES, the Cypher® sirolimus-eluting coronary stent, gained FDA approval in 2002 based primarily on clinical evidence from a randomized controlled trial (RCT) powered to show superiority to BMS (9). Cypher was approved based on the pivotal SIRIUS (Sirolimus-Eluting Stent in De Novo Native Coronary Lesions) trial in 1,058 subjects (10), with supporting data coming from an FIH study of 45 subjects and the RAVEL (Randomized Study with the Sirolimus-Coated Bx Velocity Balloon-Expandable Stent in the Treatment of Patients with de Novo Native Coronary Artery Lesions) trial in 238 subjects (11). Additionally, clinical pharmacokinetic data of the DES was obtained in a small PK study of 19 subjects. The SIRIUS trial evaluated both angiographic and clinical outcomes, showing significant reduction in target vessel failure (TVF) and angiographic and lesions characteristics over the control BMS.

Subsequent devices in each device type followed a similar path to develop clinical evidence, but rather than superiority to the previous standard of care, these devices were shown to be noninferior in safety and effectiveness outcomes to a currently approved device of the same type. For example, the XIENCE V everolimus-eluting coronary stent system (12) was evaluated OUS in the SPIRIT (Clinical Evaluation of the Xience V Everolimus Eluting Coronary Stent System in the Treatment of Patients with de novo Native Coronary Artery Lesions) FIRST (13) and SPIRIT II (14) trials to establish safety and performance.  These data provided reasonable assurance of safety and expected benefit, allowing FDA to grant approval for the sponsor to conduct the pivotal SPIRIT III, an RCT conducted under an IDE in the United States (15). PMA approval for XIENCE V was granted based on demonstration of noninferiority with in-segment late loss at 8 months and TVF at 9 months versus an approved first-generation DES.

As defined in the draft DES guidance, the current pathway to approval for a novel DES would require demonstration of safety and effectiveness through clinical evaluation in an RCT that shows noninferiority to an approved DES in the now preferred primary endpoint of target lesion failure (TLF) at 12 months.  While TLF is the primary endpoint, FDA does consider other endpoints such as stent thrombosis and MI considering the totality of the information prior to approval.

To orchestrate a set of consensus definitions, four academic research organizations involved in the design and management of current DES clinical trials combined efforts in an informal collaboration termed the Academic Research Consortium (ARC), which was published in 2007 (16).

ARC determined that for DES study endpoints, the endpoint definitions should support the characterization of device effectiveness or safety. Safety endpoints represent any adverse outcome whether specifically related to the use of the device or not, and effectiveness endpoints refer specifically to maintenance of coronary artery luminal patency. The endpoint definitions should relate to the pathophysiological mechanism(s) most likely responsible for the clinical outcome.

ARC stated that DES-related safety issues are governed to some degree by time. Adverse outcomes within 30 days of implantation are generally considered temporally related to the procedure. In the setting of a progressive entity such as coronary disease, the later that adverse events occur, the more likely they are to represent an interaction between the device and the disease or to represent new disease activity altogether. This is not always the case, however: for example, periprocedural MI or sudden death within 30 days in elective patients may clearly be device- or procedure-related, whereas in patients with acute or evolving MI such a relationship may not be clear.

DES are implanted for the treatment of obstructive coronary artery disease and their effectiveness is measured by the relief of such flow-limiting obstructions, initially through structural mechanisms and later with preservation of the luminal dimension through inhibition of neointimal hyperplasia or restenosis. Effectiveness clinical endpoints are designed to assess clinically significant restenosis, assessed objectively as a requirement for ischemia-driven repeat revascularization, either of the stented segment itself (target lesion revascularization [TLR]) or of the stented vessel or its side branches (TVR) (17). Target vessel failure, proposed as any TVR, death, or MI attributed to the target vessel, is an even broader metric of failed effectiveness and adjusts for the potential bias introduced when patients who die or sustain an MI before the end of the TLR endpoint time are considered to be free from TLR.

Apart from clinical endpoints evaluating safety and effectiveness, the initial evaluation of a new DES is typically conducted in a small single-arm study with intensive invasive imaging.

Imaging-derived measures of restenosis, such as percent diameter stenosis and late lumen loss, are potentially powerful effectiveness endpoints (3). Such outcome measures have the advantage of providing quantitative data to compare specific parameters of stent performance, such as suppression of neointimal hyperplasia, as well as for comparisons to existing stents. Furthermore, they can provide additional effectiveness data, even in patients who have not developed a major clinical adverse event, and consequently have the potential to increase the sensitivity of outcome measures between treatments.

Imaging endpoints are commonly measured as continuous variables and this powerful discriminatory advantage can be apparent with sample sizes considerably smaller than typically needed for clinical endpoints. However, in FDA’s viewpoint, the use of these potential imaging measures as primary endpoints does not preclude the need for evidence of safety through evaluation of a clinical endpoint, such as death, MI, and/or TLR, either individually or as a composite. The use of an imaging endpoint as the sole primary effectiveness endpoint in pivotal DES trials is currently acceptable only for certain second-generation DESs, such as iterative modifications from currently approved DESs and/or indication expansion, in specific patient populations or in specific vessel or lesion types. For a novel DES, clinical studies performed to support regulatory approval should include at least one study of sufficient size that has as its primary endpoint a clinical endpoint and is appropriately powered for statistical demonstration of superiority or noninferiority against an appropriate control.

The aforementioned SPIRIT program provides an example of a second-generation DES development. In SPIRIT II, an imaging surrogate such as 6-month angiographic in-stent late loss was still an acceptable primary evaluation of safety and effectiveness of XIENCE to TAXUS^TM EXPRESS2^TM, both of which were drug-eluting metallic stents (14). The U.S. pivotal study, SPIRIT III, similarly included a primary imaging endpoint, and also a coprimary clinical endpoint to enable a larger and more robust study. Approval of XIENCE depended on demonstrating noninferiority in both the primary endpoints of angiographic in-segment late loss at 240 days and TVF (cardiac death, all MI, or TVR) at 270 days, against the same TAXUS^TM control (15).  The in-segment late loss correlated well with the clinical outcomes (18). Approval for subsequent XIENCE size extensions (XIENCE 2.25, SPIRIT Small Vessel) and a slightly modified design with 33/38 stent lengths (XIENCE PRIME 33/38, XIENCE PRIME) likewise had 1-year TLF as the primary endpoint.

Though TLF continues to be the most common and the recommended primary endpoint in DES trials, companies continue to explore use of other endpoints/outcomes to establish performance of product iterations.  For example, the Medtronic Resolute Onyx Core (2.25 mm – 4.0 mm) Clinical Study used in-stent late lumen loss as measured by quantitative coronary angiography (time frame: 8 months) and in-stent late lumen loss at 8 months post-procedure as measured by quantitative coronary angiography as the primary outcome measures (19). FDA continues to work with industry to ensure that the most appropriate and “ least burdensome” approach to obtaining valid scientific evidence necessary to support effectiveness of the device to support approval, including the use of real-world evidence and foreign data where appropriate, is utilized (20).

Generally speaking, bioresorbable scaffolds are treated the same as DES with respect to the burden of clinical evidence required for approval in the United States, as long as no new questions of safety are raised specific to the design, materials, or degradation of the BRS that cannot adequately be addressed with nonclinical testing. The DES guidances apply, and thus these devices can be approvable by showing noninferiority in TVF at 12 months versus an approved DES in an RCT. To date, one BRS has been approved by FDA; the Absorb GT1™ BVS System was approved in 2016 based on the pivotal trial ABSORB III, which showed noninferiority of TLF at 12 months for Absorb to XIENCE in a 2,000-patient RCT (21). Additional supporting data came from OUS studies ABSORB Cohort A, ABSORB Cohort B, ABSORB II, and ABSORB Japan (22).

Due to the novelty of the device and concept, the initial performance of Absorb BRS was tightly evaluated clinically including multiple imaging modalities. Thirty patients with simple de novo native coronary artery disease were evaluated in Cohort A.  Follow-up included invasive angiography (e.g., angiographic in-scaffold lumen loss), intravascular ultrasound (IVUS) (e.g., scaffold area, minimum lumen area), virtual histology – IVUS (VH-IVUS) and optical coherence tomography at 6 months and 2 years (23-25). Despite low event rates, a second-generation of Absorb BVS with greater radial support was developed. The revision used the same polymer (PLLA) and had the same strut thickness but with a different processing and a redesigned backbone allowing for more uniform vessel support. The Absorb Cohort B trial tested this generation in 101 patients, including angiographic in-scaffold lumen loss at 6 months, 2 years, and 3 years, which remained unchanged  (26,27).  IVUS imaging demonstrated lumen preservation by an increase in the mean scaffold area which was an important finding supporting further clinical development. Additionally, the results of the ABSORB Cohort B trial proved the dynamic nature of vessel wall changes following implantation of a fully bioresorbable which cannot be achieved by a metallic platform (27).

It should be noted that as of September 14, 2017, Abbott halted global sales of its first-generation Absorb BVS due to what it described as low commercial sales. The company is continuing its work on a second-generation bioresorbable stent with a lower profile that is easier to deliver. Other companies are developing BRS technologies as well.

Following an approval of a DES or BRS, manufacturers often make design changes to improve the performance of the device. All design changes must be evaluated to determine if the existing nonclinical or clinical evidence may be leveraged in support of the new generation device. Key device factors that have the potential to impact clinical outcomes include, but may not be limited to: the implant; coating polymer or active pharmaceutical ingredient materials; stent design; strut thickness; coating formulation; and manufacturing processes, particularly coating, stent retention, and sterilization processes. Substantial changes to any of these elements could trigger the need for additional clinical evidence to be generated to confirm the device is still safe and effective for the intended use. Additionally, new stent sizes outside of the workhorse matrix studied in the pivotal trial or the expansion of the indication to a new population are likely to require new clinical data prior to approval. For example, the second generation XIENCE DES, the XIENCE PRIME Stent System, was evaluated in the SPIRIT PRIME trial with ~400 patients in a core size registry and ~100 patients in a long lesion registry (to support an expanded size matrix) (28). Both study arms evaluated TLF at 12 months against a prespecified performance goal derived from historical data. Newer-generation design iterations, which affected only the delivery system, were approved by leveraging the existing datasets, without the need for additional prospective clinical data  (29,30).

The FDA may accept clinical studies conducted partially or fully outside the United States in support of safety and efficacy claims for all medical devices, including coronary stents. All device studies conducted under an IDE are governed by the FDA informed consent and institutional review board requirements (21 CFR part 812 IDE regulations) (31). Under 21 CFR 814.15(a) and (b), FDA will accept a foreign clinical study involving a medical device not conducted under an IDE only if the study conforms to whichever of the following provides greater protection of the human subjects: the ethical principles contained in the 1983 version of the Declaration of Helsinki or the laws and regulations of the country in which the research was conducted.

While FDA has a longstanding practice regarding scientifically sound data from studies conducted OUS, in the draft guidance originally issued April 2015 (but since withdrawn) titled “Acceptance of Medical Device Clinical Data from Studies Conducted Outside the United States,” FDA acknowledged that “certain challenges exist in using data derived from foreign studies of devices to support an FDA marketing authorization. These challenges may include differences between the study population and the intended U.S. patient population, difficulties in extrapolating from different endpoints used to support OUS review standards, and even differences in disease characteristics and treatment standards.” As such, FDA has required most or all of the pivotal data supporting coronary stents to be derived under IDEs with a majority of the patients being enrolled at U.S. sites. Currently, FDA has recommended that at least 50% of patients being enrolled in an IDE intended to support a U.S. approval be enrolled at U.S. sites; however, FDA has started to accept high-quality OUS data when the data are shown to reflect the U.S. patient population and practice. This trend could open the door to more global clinical study programs designed to support global approvals more efficiently, without the need of country-specific data.