Levels of Evidence
The purpose of this page is to create a reference to describe our methodology for assigning levels of evidence to regimens.
Important note: Our intent is not to provide clinical decision support. Rather, our goal is to faithfully reproduce findings of clinical trials. Efficacy and toxicity information, in particular, is sometimes presented by authors in a confusing or ambivalent manner. As such, we try to illustrate ambiguities when they happen, and take no responsibility for your decision to choose a particular treatment regimen. Please read our disclaimer for further information.
A bit of background: We have taken a simplified approach to information visualization, based on a three-color "traffic signal" metric. In order to account for color-blindness, text is also included within each colored box. The colors we use are:
Green box text Yellow box text Red box text
See the sections below for a discussion of the various metrics we use.
Evidence
Generally, a regimen should be evaluated in a randomized fashion with an adequate patient sample to be considered a "green" regimen. We have defined adequate as 20 or more patients per arm. Non-randomized studies and randomized studies with fewer than 20 patients per arm are considered to be "yellow" regimens. Finally, case reports, retrospective series, and non-randomized studies with fewer than 20 patients enrolled are considered to be "red" regimens. Of course, there are finer gradations of the quality of evidence so this simplified scheme should be taken with a grain of salt.
Evidence is thus reported using one of the three labels:
Strong evidence
Moderate evidence
Weak evidence
Examples
A trial with strong evidence: R-CHOP for untreated follicular lymphoma
Study | Evidence |
Flinn et al. 2014 (BRIGHT) | Phase III |
A trial with moderate evidence: bortezomib & rituximab for untreated follicular lymphoma
Study | Evidence |
Evens et al. 2014 | Phase II |
A trial with weak evidence: cladribine for aggressive systemic mastocytosis
Study | Evidence |
Lim et al. 2009 | Retrospective |
Frequently asked questions
Q: What is the current status of evidence labeling on hemonc.org?
A: Most of the 1,300+ chemotherapy regimens and their variants have a level of evidence label.
Q: If a randomized trial has more than two arms, will they all be labeled the same?
A: No, it depends on how many patients are in each arm of the trial. For arms that have more than 20 patients, the label is green. For arms with fewer than 20, the label is yellow.
Efficacy
Defined generally, efficacy is the presence or absence of a positive effect on the study population. Efficacy can be reported ranging from a weak surrogate measure (e.g. response rate) to direct measure of overall survival. Currently, we are not labeling efficacy by the quality of the outcome measure, but rather by comparative efficacy. As such, we only currently report efficacy for randomized trials; many non-randomized trials report efficacy compared to historical controls. However, in the rapidly developing field of oncology, this approach is rife with bias and as such we do not report on comparison to historical controls, at this time.
Efficacy is thus reported using one of the three labels:
Increased efficacy
Equivalent efficacy
Decreased efficacy
More details
To be completed.
Examples
A treatment regimen with increased efficacy: BR for untreated follicular lymphoma
In this trial there are two arms: BR and R-CHOP.
Study | Evidence | Comparator | Efficacy |
Rummel et al. 2013 (StiL NHL1) | Phase III | R-CHOP | Increased PFS |
A treatment regimen with increased efficacy: R-CHOP for untreated follicular lymphoma
In this trial there are three arms: R-CHOP, R-CVP, and R-FM.
Study | Evidence | Comparator | Efficacy |
Federico et al. 2013 (FOLL05) | Phase III | R-CVP;R-FM | Increased PFS |
A treatment regimen with equivalent efficacy: MCP for untreated follicular lymphoma
Study | Evidence | Comparator | Efficacy |
Nickenig et al. 2006 | Phase III | CHOP | Equivalent OS |
A treatment regimen with decreased efficacy: MCP for untreated follicular lymphoma
Note that this is the same regimen as above; it is the comparative efficacy that is different.
Study | Evidence | Comparator | Efficacy |
Herold et al. 2007 | Phase III | R-MCP | Worsened OS |
Frequently asked questions
Q: What is the current status of efficacy labeling on hemonc.org?
A: The follicular lymphoma page is now completely labeled for efficacy.
Q: How do we choose to label efficacy when multiple outcomes are reported?
A: Often, a trial will report on multiple outcomes, such as overall response rate, progression-free survival, and overall survival. In this case, we generally look to the PRIMARY endpoint, as defined in the published methods. However, if a secondary endpoint shows differential efficacy and is less "surrogate" than the primary endpoint, we will label by that endpoint.
Q: How do you distinguish between a failed trial and a formal non-inferiority or equivalence study?
A: We do not currently distinguish these; both of them would be labeled yellow. We are looking into ways to convey this information appropriately.
Q: Do you have a hierarchy of surrogacy?
A: Yes, this is the hierarchy that we use to determine the strength of an outcome measure:
Strong outcomes
- Overall survival (OS)
Intermediate outcomes
- Disease-free interval (DFI)
- Disease-free survival (DFS)
- Event-free survival (EFS)
- Progression-free survival (PFS)
- Time to next treatment (TTNT)
- Time to treatment failure (TTTF)
Weak outcomes
- Response rate (RR)
- Complete response rate (CR rate)
- Partial response rate (PR rate)
- Very good partial response rate (VGPR rate)
- Overall response rate (ORR) No specific definition. Generally, a sum of CR + PR rates. Sometimes also includes stable disease.
Q: Do you consider quality of life (QoL) measures in efficacy?
A: Very few RCTs report on QoL measures, and as such we do not currently include them in the consideration. This may change in the future.
Toxicity
Defined generally, toxicity is the presence or absence of a negative effect (harm) on the study population. This is often also referred to as safety. As with efficacy, we only report comparative toxicity.
Toxicity is thus reported using one of the three labels:
Decreased toxicity
Equivalent toxicity
Increased toxicity
Examples
A treatment regimen with increased toxicity: R-CHOP for untreated follicular lymphoma
Study | Evidence | Comparator | Efficacy | Toxicity |
Hiddemann et al. 2005 | Phase III | CHOP | Increased OS | Increased toxicity |
Frequently asked questions
Q: What is the current status of toxicity labeling on hemonc.org?
A: A few regimens are currently labeled for toxicity; we are focusing current efforts on labeling efficacy.
Q: Are you basing the label on the reported CTCAE measures?
A: CTCAE measures are extremely valuable in that they are structured and thus reproducible. However, it is often hard to compare them directly. For example, if one regimen has grade 4 lab-based toxicity and the other has grade 2 gastrointestinal toxicity, which is the more toxic? In general, we plan to use the authors' interpretation of overall toxicity and tolerability when labeling.
Q: Do you plan to incorporate patient-reported outcomes?
A: As shown in numerous publications, patient reports of toxicity are more accurate than clinician assessments. However, they have not until recently been standardized. Now that the PRO-CTCAE is available, we expect to see more of these in the future and will incorporate them into the toxicity assessment.
Example code (for contributors)
Case report Red label code: <span style="background:#ff0000; padding:3px 6px 3px 6px; border-color:black; border-width:2px; border-style:solid;">Case report</span>
Phase II Yellow label code: <span style="background:#EEEE00; padding:3px 6px 3px 6px; border-color:black; border-width:2px; border-style:solid;">Phase II</span>
Phase III Green label code: <span style="background:#00CD00; padding:3px 6px 3px 6px; border-color:black; border-width:2px; border-style:solid;">Phase III</span>