Skip to main content
  • More from ADA
    • Diabetes
    • Diabetes Care
    • Diabetes Spectrum
    • ADA Standards of Medical Care
    • ADA Standards of Medical Care, Abridged
    • ADA Scientific Sessions Abstracts
    • BMJ Open Diabetes Research & Care
  • Subscribe
  • Log in
  • Log out
  • My Cart
  • Follow ada on Twitter
  • RSS
  • Visit ada on Facebook
Clinical Diabetes

Advanced Search

Main menu

  • Home
  • Current
    • Current Issue
    • Online Ahead of Print
    • ADA Standards of Medical Care
    • ADA Standards of Medical Care, Abridged
  • Browse
    • Issue Archive
    • Saved Searches
    • COVID-19 Article Collection
    • Quality Improvement Sucess Stories
    • ADA Standards of Medical Care
    • ADA Standards of Medical Care, Abridged
  • Info
    • About the Journal
    • About the Editors
    • ADA Journal Policies
    • Instructions for Authors
  • Advertising
  • Reprints/Reuse
  • Subscriptions
    • Individual Subscriptions
    • Institutional Subscriptions and Site Licenses
    • Access Institutional Usage Reports
    • Purchase Single Issues
  • Alerts
    • E­mail Alerts
    • RSS Feeds
  • Podcasts
    • Diabetes Core Update
    • Special Podcast Series: Therapeutic Inertia
    • Special Podcast Series: Influenza Podcasts
    • Special Podcast Series: SGLT2 Inhibitors
    • Special Podcast Series: COVID-19
  • Submit
    • Submit a Manuscript
    • Submit Cover Art
    • Instructions for Authors
    • ADA Journal Policies
  • More from ADA
    • Diabetes
    • Diabetes Care
    • Diabetes Spectrum
    • ADA Standards of Medical Care
    • ADA Standards of Medical Care, Abridged
    • ADA Scientific Sessions Abstracts
    • BMJ Open Diabetes Research & Care

User menu

  • Subscribe
  • Log in
  • Log out
  • My Cart

Search

  • Advanced search
Clinical Diabetes
  • Home
  • Current
    • Current Issue
    • Online Ahead of Print
    • ADA Standards of Medical Care
    • ADA Standards of Medical Care, Abridged
  • Browse
    • Issue Archive
    • Saved Searches
    • COVID-19 Article Collection
    • Quality Improvement Sucess Stories
    • ADA Standards of Medical Care
    • ADA Standards of Medical Care, Abridged
  • Info
    • About the Journal
    • About the Editors
    • ADA Journal Policies
    • Instructions for Authors
  • Advertising
  • Reprints/Reuse
  • Subscriptions
    • Individual Subscriptions
    • Institutional Subscriptions and Site Licenses
    • Access Institutional Usage Reports
    • Purchase Single Issues
  • Alerts
    • E­mail Alerts
    • RSS Feeds
  • Podcasts
    • Diabetes Core Update
    • Special Podcast Series: Therapeutic Inertia
    • Special Podcast Series: Influenza Podcasts
    • Special Podcast Series: SGLT2 Inhibitors
    • Special Podcast Series: COVID-19
  • Submit
    • Submit a Manuscript
    • Submit Cover Art
    • Instructions for Authors
    • ADA Journal Policies
Departments

“Negative” Results: An Editor's Dilemma

  1. Tom A. Elasy, MD, MPH, Editor-in-Chief
    Clinical Diabetes 2009 Dec; 27(1): 1-3. https://doi.org/10.2337/diaclin.27.1.1
    PreviousNext
    • Article
    • Info & Metrics
    • PDF
    Loading

    Publishing negative studies is difficult.

    Many physicians and readers believe that journals are systematically disposed to publishing positive studies (journals, after all, are responsive to their audience, and most readers are less interested in that which does not work) and that investigators are systematically disposed to only publishing their positive findings (newly minted physicians in the past 15 years have likely learned of funnel plots as a means to detect this bias). Although these two beliefs have merit, my hesitancy is more methodological: it is more difficult to carefully evaluate negative studies.

    Many medical schools now teach some structured program about reviewing the medical literature. Drawing heavily on a series titled “User's Guide to the Medical Literature” that was published in the Journal of the American Medical Association, many physicians have received some systematic training in evaluating results. The degree to which this cohort of physicians who have been exposed to this training has filtered sufficiently into the “reviewer pool” to affect journal review quality is uncertain. Unfortunately, this training does less to help in reviewing negative studies because the curriculum employed in medical schools is heavily focused on evaluating studies that reach a positive conclusion.

    Negative findings, without addressing the nuances between equivalency versus no difference, have very specific review concerns. The most well-known issue is power. Power refers to the chance that a study can detect a difference if a difference actually exists; this is also occasionally called the ability to avoid a Type 2 error. While this is often provided in randomized controlled trials, it is rarely found in observational studies.1

    Although power is positively correlated with sample size, it can also be modified by the effect size selection. Ideally, investigators choose the minimally important clinical difference as the effect size. Unless someone is a content expert, it is difficult to know whether the effect size chosen was appropriate. In addition, many studies are frequently powered to only one primary outcome. If the results report negative findings, it may be that they were not powered for that outcome.

    Beyond power, which only addresses the likelihood that one has avoided an error due to chance, there are several additional considerations. Many examples exist2 of studies that enroll patients who are not the ones most likely to benefit from the intervention and then declare the intervention not effective. Conversely, the Hawthorne effect may be so powerful in the control arm of a study that a signal is not detected between the groups.

    Measurement is also an extremely common reason for negative studies. I am convinced that most studies that look at quality of life fail to find differences because we cannot adequately measure quality of life. Similar measurement concerns plague studies in which dietary variables are of interest because of the difficulty of measuring dietary variables in large, free-living population studies.

    The proper delivery of an intervention is an additional concern often referred to as intervention fidelity. This is particularly of concern in diabetes, where so many interventions target self-care behaviors and involve some form of education or more advanced methods to alter behaviors. Unless the intervention was delivered as intended—information that is often unavailable to the reviewer—one may falsely conclude that the negative study is a reflection of an inadequate intervention as opposed to inadequate delivery.

    Data management is yet another concern in negative studies. Failure to provide quality control for data input and management can result in negative studies if those errors are non-differentially distributed. An anonymous quote from several centuries ago reminds us that, “The government are very keen on amassing statistics. They collect them, add them, raise them to the nth power, take the cube root, and prepare wonderful diagrams. But you must never forget that every one of these figures comes in the first instance from the village watchman, who just puts down what he damn well pleases.”

    And these are but a few of the issues.

    In this issue of Clinical Diabetes, an important “negative” study is reported in our Landmark Studies department (p. 17). Michael Pignone, MD, MPH, reports on a randomized clinical trial investigating the efficacy of self-monitoring of blood glucose (SMBG) in 184 subjects with newly diagnosed type 2 diabetes. The study assigned subjects to a fairly comprehensive management program that either included SMBG or not. The study concluded that SMBG provided no benefit with regard to glycemic control—a “negative” study.

    This is an important observation that is somewhat at variance with accepted wisdom. Several considerations should provide pause before these results are translated into practice. First, the population studied was apparently remarkably easy to control; < 10% required more than one medication to achieve control at 12 months, and nearly one-third required no medication at all. The less complex the population, the more likely that the elimination of one component (here SMBG) of an intense multidimensional intervention will have minimal effect on the outcome of interest. Second, the intervention that the control group received was indeed multifactorial and robust. Third, subjects were asked to monitor their blood glucose eight times per week and advised on a response to abnormal values. It is hard to know how well this instruction was delivered without some form of fidelity analysis, let alone whether the nature of the “advice” was sufficiently robust to influence any behavioral change. In addition, it is possible that monitoring only about once per day is insufficient to allow someone to detect patterns of response to their medications, diet, activity, illness, or other factors. It is generally my approach to have individuals monitor more frequently early on during the learning phase and then cut back once they have stabilized.

    Despite some of these limitations, we still published a review of this study in this issue. But it was difficult, because negative studies are seemingly just a little harder to evaluate. No work is perfect or clear, and this work clearly demonstrates common issues that affect negative studies. Still, it appears that individuals with newly diagnosed type 2 diabetes in this study, who were enrolled in an intensive, multidimensional diabetes control program and who readily achieved control with no medications or just one medication, did not gain additional glycemic benefit from performing SMBG about once a day during the first year of diagnosis based on the advice given in this study.

    Footnotes

    • American Diabetes Association(R) Inc., 2009

    REFERENCES

    1. ↵
      1. Hebert RS,
      2. Wright SM,
      3. Dittus RS,
      4. Elasy TA
      : Prominent medical journals often provide insufficient information to assess the validity of studies with negative results. J Negat Results Biomed 1:1-7, 2002
      OpenUrlCrossRefPubMed
    2. ↵
      1. O'Malley PG,
      2. Feuerstein IM,
      3. Taylor AJ
      : Impact of electron beam tomography, with or without case management, on motivation, behavioral change, and cardiovascular risk profile: a randomized controlled trial. JAMA 289:2215-2223, 2003
      OpenUrlCrossRefPubMedWeb of Science
    View Abstract
    PreviousNext
    Back to top

    In this Issue

    December 2009, 27(1)
    • Table of Contents
    • Index by Author
    Sign up to receive current issue alerts
    View Selected Citations (0)
    Print
    Download PDF
    Article Alerts
    Sign In to Email Alerts with your Email Address
    Email Article

    Thank you for your interest in spreading the word about Clinical Diabetes.

    NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

    Enter multiple addresses on separate lines or separate them with commas.
    “Negative” Results: An Editor's Dilemma
    (Your Name) has forwarded a page to you from Clinical Diabetes
    (Your Name) thought you would like to see this page from the Clinical Diabetes web site.
    CAPTCHA
    This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
    Citation Tools
    “Negative” Results: An Editor's Dilemma
    Clinical Diabetes Dec 2009, 27 (1) 1-3; DOI: 10.2337/diaclin.27.1.1

    Citation Manager Formats

    • BibTeX
    • Bookends
    • EasyBib
    • EndNote (tagged)
    • EndNote 8 (xml)
    • Medlars
    • Mendeley
    • Papers
    • RefWorks Tagged
    • Ref Manager
    • RIS
    • Zotero
    Add to Selected Citations
    Share

    “Negative” Results: An Editor's Dilemma
    Clinical Diabetes Dec 2009, 27 (1) 1-3; DOI: 10.2337/diaclin.27.1.1
    del.icio.us logo Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
    • Tweet Widget
    • Facebook Like
    • Google Plus One

    Jump to section

    • Article
      • Footnotes
      • REFERENCES
    • Info & Metrics
    • PDF

    Related Articles

    Cited By...

    More in this TOC Section

    Departments

    • Community Primary Care Diabetes Pathway
    • Increasing Attendance at Scheduled Appointments for Group Classes at a Diabetes Education Center
    • Case Reports on Diabetes-Related Outcomes for Pregnant Women in the National Diabetes Prevention Program
    Show more Departments

    Editorial

    • Telemedicine: The 2020 House Call
    • Black Diabetic Lives Matter
    • COVID-19 and Diabetes
    Show more Editorial

    Similar Articles

    Navigate

    • Current Issue
    • Papers in Press
    • Abridged Standards of Care
    • Archives
    • Submit
    • Subscribe
    • Email Alerts
    • RSS Feeds

    More Information

    • About the Journal
    • Instructions for Authors
    • Journal Policies
    • Reprints and Permissions
    • Advertising
    • Privacy Policy: ADA Journals
    • Copyright Notice/Public Access Policy
    • Contact Us

    Other ADA Resources

    • Diabetes
    • Diabetes Care
    • Diabetes Spectrum
    • Scientific Sessions Abstracts
    • Standards of Medical Care in Diabetes
    • BMJ Open - Diabetes Research & Care
    • Professional Books
    • Diabetes Forecast

     

    • DiabetesJournals.org
    • Diabetes Core Update
    • ADA's DiabetesPro
    • ADA Member Directory
    • Diabetes.org

    © 2021 by the American Diabetes Association. Clinical Diabetes Print ISSN: 0891-8929, Online ISSN: 1945-4953.