Requirement for New Technology to be Included in the Guidelines: “The Ultimate Goal”

Alice Jacobs, MD

To reach the “ultimate goal” to have new technology included in clinical practice guideline recommendations, it is important to understand the history and evolution of the guideline process and methodology. The American College of Cardiology (ACC) and the American Heart Association (AHA) embarked on a joint effort in 1981 in response to the U. S. government’s request to review the evidence informing the practice of pacemaker insertion due to a concern over inappropriate overuse of the procedure. The first ACC/AHA guideline was published in 1984 and recommended indications for pacemakers. To date, there are more than 24 guidelines available with more than 4,000 recommendations; there are also derivative products consisting of performance measures and appropriate use criteria.

The ACC/AHA Task Force on Practice Guidelines, which is composed of members with varying content expertise, oversees and directs the guideline development process and methodology.

The process begins with selection of a topic for a new, revised, or updated guideline by the Task Force with the input of guideline writing committees and ACC and AHA council and committee members. It further follows a review of annual scientific meetings and recently published late-breaking clinical trials with the potential to influence clinical practice. Selected organizations and professional societies with similar interests are invited to participate as partners or collaborators, fostering consistency in the clinical community and enhanced uptake of the guideline recommendations. A writing committee chair and potential members are identified with due consideration of a detailed and specific policy on relationships with industry and other entities shared by both organizations and a desire to balance intellectual and practice perspectives in addition to sex, race, ethnicity, and geographic location. The writing committee outlines the document content, performs a detailed and specific evidence review and acquisition, and drafts recommendations with limited preliminary text based on the strength and quality of the evidence using the time-honored Class of Recommendation (COR) and Level of Evidence (LOE) schema.

The COR relates to the strength and the risk-benefit ratio of a diagnostic test or therapeutic strategy, medication, device, procedure, or other intervention.  Class I recommendations are considered strong, Class IIa moderate, and Class IIb weak, a ranking more analogous to the grade system. Class III has been separated into Class III:  No Benefit (moderate) and Class III: Harm (strong).

Suggested phrases for each recommendation class assures that the strength of the recommendation is well recognized by the community; recommendations based on comparative effectiveness are also depicted using specific phrases to compare interventions.  For example, a Class I recommendation noting a procedure or treatment is useful and effective is depicted with the phrases “should be,” “is recommended,” “is indicated,” or “is useful/effective/beneficial.” Suggested phrases for a Class II recommendation include “should be considered” and for Class IIb “may be considered”.

The LOE represents the certainty or precision of the intervention effect and the quality of the supporting scientific evidence based on the type, quantity, and consistency of clinical trials and other relevant evidence. The LOE is also based upon objectively defined criteria to assess potential bias in addition to their relevance and fidelity. Level of evidence A represents the highest quality implicit in the guideline methodology and is supported by multiple randomized trials or meta-analyses. Limited populations evaluated, observational studies, or cohort and case-controlled studies support level of evidence B.  Level of evidence C, which is prevalent in the guidelines, represents limited published data, case studies, standard of care, or expert opinion.

Once recommendations and limited text are drafted, the vetting process begins. In the absence of new evidence, prior recommendations in ACC/AHA guidelines do not change. Overlapping recommendations across multiple documents often exist and reconciliation can be tedious and time consuming. The process is facilitated at a consensus conference where members of multiple related writing committee gather to vet and review similar or the same recommendations. The first consensus conference was held in 2009 and attended by members of the percutaneous coronary intervention (PCI), coronary artery bypass graft surgery (CABG), and ST-elevation myocardial infarction writing committees when all of their respective guidelines were under revision (1). Moreover, with input from the stable ischemic heart disease (SIHD) writing committee, the attendees crafted a combined section on coronary revascularization for inclusion in all three guidelines (PCI, CABG, and SIHD). The consensus conference approach aims to expedite the development process while broadening input on related recommendations and facilitating consensus with subsequent enhanced implementation at the point of care.

Although usually approved by consensus, every recommendation is ratified by a majority of votes by the members of the writing committee; those members with relevant relationships with industry are required to recuse themselves from voting on sections to which their specific relationship applies. The final draft document then undergoes peer review by ACC and AHA official reviewers, content experts (many of whom serve on ACC and AHA councils and committees), participating organizations, and the Task Force (Figure 1). The peer review process has been expanded to encompass a broad spectrum of relevant stakeholders, including lay representatives and governmental agencies. Each peer reviewer’s comment is reviewed by the chair and members of the writing committee and receives a written response. It is not unusual to receive more than 1,000 comments from peer reviewers.

Figure 1. ACC/AHA Guideline Review Process

A lead reviewer from the Task Force confirms that there is appropriate reconciliation of all peer reviewer comments. Changes in recommendations require approval by a formal vote of the members of the writing committee according to the voting policy. The document must then be approved by the ACC Board of Trustees, the AHA Science Advisory and Coordinating Committee, and the appropriate body of partnering or collaborating organizations; additional comments and concerns are again addressed by the writing committee chair and leadership of the Task Force. Finally, after thorough vetting within the academic and clinical communities and the leadership of the ACC and AHA, the clinical practice guideline is jointly published in the Journal of the American College of Cardiology, Circulation, and the flagship journals of partnering and collaborating organizations as appropriate. Once published, guideline recommendations become the official policy of both the ACC and AHA, informing strategic initiatives, advocacy, educational programs, and services.

Therefore, it is critically important that the process is not only robust but also impeccable.  When investigated, the data affirm that treatment according to guideline recommendations is associated with improved patient outcomes. For instance, in a study linking guideline adherence and mortality in CRUSADE, a quality improvement registry, it was shown that for every 10% increase in guideline adherence, there was a 10% decrease in in-hospital mortality (adjusted odds ratio: 0.9; 95% confidence interval: 0.84-0.97) (1). Similar data from another quality improvement registry (ADHERE) evaluated heart failure performance indicators (2). Patients treated at leading centers in terms of conformity to discharge instructions, measurement of left ventricular function, angiotensin-converting enzyme inhibitor use, and smoking cessation had a significantly shorter length of stay and, importantly, a decrease in mortality.

In view of the necessarily time-consuming guideline development process and methodology, there is an ongoing challenge to lead rather than lag behind clinical practice as the continuous stream of new evidence is evaluated. During the past several years, multiple initiatives have been undertaken to streamline the process and expedite the time to publication.  These include a focus on recommendation tables, shortened text, and links to evidence tables with links to abstracts and references. Moreover, there has been an emphasis on Focused Updates, which present recommendations informed by new data such as from published late-breaking clinical trials that suggest improved patient outcomes (or possibility of harm).

In 2011, the Institute of Medicine (IOM) released its report “Clinical Practice Guidelines We Can Trust” (3) and “Finding What Works in Health Care” (4) and proposed eight standards for developing trustworthy clinical practice guidelines. In fact, the ACC/AHA guidelines were hailed as being compliant with most of these standards including established transparency, management of conflict of interest, creation of multidisciplinary guideline development groups, establishment of evidence foundations for rating the strength of recommendations, articulation of recommendation, and regular guideline updates. However, two standards were not met: routinely perform systematic evidence reviews and regularly request external review of the final document.

In response to the IOM, a guideline methodology summit was held, supported by five invited working groups where the 21 recommendations were reviewed along with 82 supporting documents on recommendations of the IOM with respect to the ACC/AHA current methods and process to decide whether changes should be made. The report recommended maintaining the current methods in addition to creating an Evidence Review Committee (ERC) separate from the writing committee, selective performance of systematic evidence reviews using a PICOTS (Population, Intervention, Comparator, Outcomes, Timing, and Setting) format to develop critical questions, including patient representatives in the process, and expanding the review process to outside organizations such as the Food and Drug Administration (FDA) and the Agency for Healthcare Research and Quality (5).

The ERC includes methodologists, epidemiologists, clinicians, and biostatisticians tasked with formal systematic review of available evidence and responsibility for identification, abstraction, and quality assessment of the evidence. Once the PICOTS question is generated, it undergoes a period of public comment that may be particularly pertinent when addressing new technology. The evidence review of the specific question is published simultaneously with the guideline. When a recommendation within a guideline is informed by a systematic evidence review, it is so noted with a superscript SR.

The writing committee and ERC work in parallel: the writing committee performs literature searches, analyzes evidence, and generates evidence tables while the ERC does so working with a partner such as the National Institutes of Health (6).  When it comes to relationships with industry (RWI), the Chair of the writing committee and 50% or more of the committee members can have no relevant relationships, while no one involved in the ERC may have any RWI (7,8).  Quality assessment is performed using the Cochrane Risk of Bias for randomized controlled trials while the Newcastle-Ottawa Scale is used for observational studies, case versus controls, and exposed and nonexposed cohorts. An example of an ERC assessment of quality considering risk of bias, relevance, and fidelity for randomized trials of dual antiplatelet therapy is shown in Figure 2 (9).

Figure 2. Evidence Review Committee Assessment

The COR/LOE schema has now been modified (Figure 3) to include a more granular assessment of the level of evidence, where LOE B is reviewed separately for randomized (LOE B-R) and nonrandomized (LOE B-NR) studies. LOE C has been separated into limited data (LOE C-LD) and expert opinion (LOE C-EO). Noteworthy is that the guidelines have been criticized for the prevalence of LOE C, but often LOE C is designated when trial evidence is lacking, and these are circumstances for which the community needs the most guidance.

Perhaps most germane to new technology and devices is the concept of cost and value. In the early years, guideline writing committees were specifically instructed not to consider cost but to focus only on the evidence. More recently, fueled by the rising cost of cardiovascular care and variation in care absent differences in clinical acuity (suggesting overuse or underuse), the Task Force on practice guidelines has embarked on assessment of cost and value based on recommendations emanating from an ACC/AHA statement on cost/value methodology in practice guidelines and performance measures (10). Considerations included the highly dynamic cost of therapy, available technology, and practice patterns as well as the costs associated with unintended harm or complications. Moreover, the evaluation of burden of cost is based on varied perspectives including the individual, third party, stakeholder, or society.  Level of value was defined as high when there are better outcomes at lower cost or an incremental cost-effectiveness ratio of <$50,000 per quality-adjusted life-year (QALY) gained. Intermediate value is defined as $50,000 to $100,000 per QALY gained and low value ≥$150,000 per QALY gained.

 Figure 3. ACC/AHA Guidelines: COR/LOE Schema

Finally, to incorporate new technology into guidelines, several issues must be addressed. First, the device must have FDA approval; writing committees may write text around new technology and devices not yet approved by FDA but not recommendations. Several potential touch points include the period of public comment of a PICOTS question awaiting a systematic evidence review and participation in the external review. When planning the evaluation of new technology, consideration should be given to the best designed study to achieve the strongest COR and LOE, clearly articulating the risk/benefit ratio and the quality of the study including bias, relevance, and fidelity. Cost and value considerations are very appropriate for new devices particularly since devices can be tracked. Noteworthy is the concept of time in the marketplace; new devices (or drugs) often need time in use in clinical practice for potential unintended or unexpected outcomes to be understood.

The ultimate goal is to move new, evidence-based, guideline recommended life-saving therapies into the broad community. This continues to present a challenge. The AHA’s Guideline Transformation and Optimization program is designed to decrease the time to implementation of guideline recommendations by increasing knowledge and awareness for three target audiences: healthcare professionals, patients and their caregivers, and hospital systems (11). Additional challenges and opportunities include augmentation of the research base to close the gaps in evidence (perhaps most apparent to guideline writing groups), creation of “living” guideline documents that lead clinical practice, and incorporation of comparative effectiveness, cost, cost effectiveness, value, and patient preference into guideline documents. What will always remain is the steadfast commitment to provide guidance to clinicians to improve the quality of care and outcomes of patients with cardiovascular disease.