How to Recognize Credible Sources


Pt 1. How to Recognize Credible Sources: Everyone should know how to critically analyze an article or study, especially when the it concerns their health. www.thedistractedmom.com

Fish Oil Is a Potentially Effective Natural Alternative for ADHD

 

Common Household Pesticide Linked with Higher Rates of ADHD

 

ADHD Experts Re-evaluate Zeal for Drugs

 

As concerned parents or adults affected by ADHD and related disorders, we see a lot of headlines designed to grab our attention. Many of them sound like they are reporting new scientific findings, and some of them are… but others are peddling their wares and hoping to influence consumers or impress them with false authority. Every responsible consumer should know how to critically analyze an article or study, especially when the it concerns their health and money.

Many of these press releases and articles do not link to any studies at all, but rather reference “research” in a nebulous way. Others may use buzz words and scare tactics or rely on implied authority. Others may appear like legitimate reporting but may actually be selling something.  The ‘Fish Oil’ headline above, for example, looks like a press release, but scroll down in the “article,” and you’d see that it’s an ad for a supplement and that it doesn’t even link to any research at all!

 How to Spot a Dubious Source

 

When evaluating the credibility of a source, first examine the author’s credentials and institution affiliations.  Some are well-respected (major universities and government institutions), but others can appear well-respected but are agenda-driven (the Discovery Institute).

When evaluating an author making any big claims,  you should examine not just their degree, but also if they have any credentials from the field or any peer-reviewed published research.

An interesting example of an author with a lot of perceived clout but with dubious credibility is Dr. Perlmutter, author of Grain Brain. He has been something of a sensation for the last couple of years, making headlines with his recommendations for a gluten-free diet, even claiming that ADHD and Autism may be prevented with the use of probiotics and treated with a grain elimination diet.

“[Dr. Perlmutter] contributed extensively to the world medical literature,” reads his website, “with publications appearing in The Journal of Neurosurgery, The Southern Medical Journal, Journal of Applied Nutrition, and Archives of Neurology.” Earlier bios also list the Journal of the American Medical Association.

Yet a closer look at his publications reveals that Perlmutter hasn’t actually conducted much research. In the scientific community, full-length peer-reviewed articles, especially widely cited ones, are the gold-standard of significant research. But his contribution to JAMA — an extremely prestigious medical journal — is actually just a letter to the editor.  The Southern Medical Journal? A case report and a clinical brief, both co-authored with his father when the younger Perlmutter was still a medical student.  Archives of Neurology? Another case report. (Case reports and clinical briefs are short discussions around 1-4 pages long.)  As for the prestigious-sounding Journal of Applied Nutrition, you’ll be hard-pressed to find a legitimate nutrition scientist who’s even heard of it.”                  -via Alan Leninovitz for The Science of Us

When you consider the above, it may seem wise to take Dr. Perlmutter’s recommendations with a grain of salt, no?

A little digging may find that an author may have an interest in promoting a particular viewpoint or even a product. Dr. Perlmutter happens to direct readers to his $8,500 brain detoxification retreat, and among many other products, he sells $160 “Senior Empowerment Packs,” and he also used to endorse his own expensive hyperbaric oxygen chamber treatments!

An important point to remember: if an author has affiliations with a particular organization with an agenda, then what they have to say about matters relating to that topic must be understood to be influenced by their interest in the matter.

Questions to Ask:

  • Does this information come from an unbiased source?
  • Is the information presented factual and relevant?
  • Are the claims backed up by statistically significant studies and cited?

If the article only presents one side on the matter and ignores or completely discounts all other viewpoints, it’s biased. It may be a valid opinion piece, but it is not unbiased and it is not research.

A couple more examples to consider:

There has been a lot of buzz on Twitter lately about Neurocore, a company claiming to treat ADHD and Autism with neurofeedback. The site even makes a claim to raise a person’s IQ “an average of 12 points” using their technique! Their Chief Science Officer is Timothy Royer, the “former division chief of pediatric psychology” at Helen DeVos Children’s Hospital in Grand Rapids, MI. That would him a kids’ therapist, but quite an enterprising and ambitious one to go into business peddling expensive treatments and unnecessary scans to concerned parents who want to do right by their children.

“Timothy G. Royer, Psy.D. is an expert in the use of biofeedback and autonomic nervous system (ANS) regulation to treat ADHD, sleep loss, anxiety and other stress disorders in children and adults.” – via the Neurocore site

“Using data from quantitative electroencephalography (qEEG), along with other physiological measures, Neurocore gains a deep understanding of why a person’s health and well-being may be compromised. Neurocore has provided diagnostic qEEG to over 10,000 children and adults.”

“Once the diagnosis is completed, Neurocore uses proven neurofeedback therapy to train the brain to operate more efficiently. Nearly 75% of individuals who have completed the neurofeedback program have eliminated or greatly reduced their medications for ADHD and experience improved performance and behavior. Adults report better sleep, renewed focus, and significant relief from stress.”  – via Neurocore

Does Timothy Royer (who calls himself Dr. Royer on Twitter) pass the credibility test?

Well, his PsyD degree is in psychology, not medicine, and he has no published research to be found. His site does not provide links to research studies that would indicate how he might substantiate his claims about raising the IQ with his technique or to back up his claims of so many patients who no longer require medication after his treatment.  Finally, he most definitely has a vested interest in selling his product: a very expensive treatment program which may not be covered by insurance, leading one to question the efficacy of the treatment.

While Royer’s claims are interesting and appealing to anyone who depends on medication to function normally, without conducting any research studies or publishing his findings for peer review, his claims remain unsubstantiated and anecdotal at best.

Joseph Mercola of mercola.com is another prime example of a dubious source. His articles often cite news articles or quotes from other “experts” with an agenda instead of peer-reviewed research and rely on circular reasoning and the logical fallacy of implied authority because he’s a “doctor” (so he must be correct). In ADHD Experts Re-evaluate Their Zeal for Drugs, he says:

“These are hard-core, “class 2″ narcotics, regulated by the Drug Enforcement Agency as a controlled substance because they can lead to dependence. The majority of kids diagnosed with ADHD will be prescribed these potentially dangerous drugs, the most common being Ritalin. By definition, Ritalin stimulates your central nervous system and may certainly interfere with the delicate and complex workings of your brain and personality.”

The problem with this passage is that “by definition,” many useful drugs are central nervous system stimulants. This does not make them harmful when used as prescribed. And by using the term “hard-core,” he attempts to confuse the issue with the fallacy of relevance. Yes, stimulants like methylphenidate are classified as Class 2 and carry a risk of dependence (like many other medications), but these medications are not normally abused by the children and adults to whom they are prescribed.

Joseph Mercola has been recognized as a quack doctor for awhile. He has been criticized by regulatory, medical, and scientific communities. A 2006 BusinessWeek editorial called his marketing practices as “relying on slick promotion, clever use of information, and scare tactics.” The U.S. Food and Drug Administration has even warned Mercola and his company to stop making illegal claims regarding his products’ ability to detect, prevent, and treat disease.

“Many of Mercola’s articles make unsubstantiated claims and clash with those of leading medical and public health organizations. For example, he opposes immunization, fluoridation, mammography, and the routine administration of vitamin K shots to the newborn; claims that amalgam fillings are toxic; and makes many unsubstantiated recommendations for dietary supplements. Mercola’s reach has been greatly boosted by repeated promotion on the “Dr. Oz Show.””  – via QuackWatch

How to Recognize a Credible Source

 

If a source is credible, the headline will be reasonable, rather than sensational, and the information will be presented factually, acknowledging the weaknesses in the study or the opposing viewpoint. The information should be well organized and with logical arguments, and claims will be backed up with research from other credible sources from research conducted in the last few years. Consider the source of the article, but also the source of the citations (The New York Times or similar sites/papers are not research journals and cannot be cited as references).

Finally, even if the source appears to come from a journal, not all journals are reliable. Some are “standalone journals” that allow submission from almost anyone or  have less than rigorous review methods. When checking the validity of a published journal article, see if it’s a standalone journal here.

Let’s consider the following, which poses a common question:

Are Stimulants Overprescribed? Treatment of ADHD in Four U.S. Communities

 

This was found in the prestigious Journal of the American Academy of Child and Adolescent Psychiatry.  JAACAP is the #1 journal in the category of pediatrics, as reported in the 2013 Journal Citation Reports (the recognized authority for evaluating journals).  JAACAP also ranks 11th of 135 psychiatry journals.

The lead researcher/author of cited is Dr. Peter S. Jensen,  a psychiatrist (a Medical Doctor specializing in psychiatric illnesses) and the Associate Director for Child and Adolescent Research, at the National Institute for Mental Health.   He has 233 publications to date. See the full list here.

His professional highlights include

  • Member, Scientific Council, NAMI, 2007-present
  • President, The REACH Institute (Resource for Advancing Children’s Health), 2007-present
  • Irving Philips Prevention Award, American Academy of Child & Adolescent Psychiatry, 2011
  • Alumni Inductee, Alpha Omega Alpha, 2008
  • Editorial Board Memberships: Journal of Child & Adolescent Psychopharmacology, 1990-present; Development and Psychopathology, 1993-present; Journal of Attention Disorders, 1995-present; Biologic Psychiatry, 1997-present; J. Abnormal Child Psychology, 1998- present; World Journal of Clinical Pediatrics, 2011-present; World Journal of Translational Medicine, 2011-present

Among the many others who participated in the research are Dr. Dulcan, the Head of the Division of Child and Adolescent Psychiatry, Northwestern University School of Medicine; Drs. Hoven and Bird with the Department of Child Psychiatry, New York State Psychiatric Institute; and Dr. Bauermeister with the Department of Psychiatry, University of Puerto Rico.

In this study, the authors examined epidemiological survey data from 1,285 children and their parents across 4 U.S. communities. They reported objective and verifiable facts gathered from their study, and shared their methods, results, and conclusions:

“Findings indicated that 5.1% of children met full DSM-III-R ADHD criteria across the pooled sample. Only 12.5% of children meeting ADHD criteria had been treated with stimulants during the previous 12 months. Some children who had been prescribed stimulants did not meet full ADHD diagnostic criteria, but these children manifested high levels of ADHD symptoms, suggesting that the medication had been appropriately prescribed. Children with ADHD were generally more likely to receive mental health counseling and/or school-based interventions than medication.”   – from JAACAP

“On the basis of these data it cannot be concluded that substantial “overtreatment” with stimulants is occurring across communities in general.”

This passes the quack test. The source is highly credible, the affiliated instituation and publishing journal are reputable and trustworthy, the findings are not sensationalized, the data are  objective,  and the data are from a large enough sample size to be worth considering (though a larger survey will always provide even better results).

Not all research studies are created equal. Depending on the sample size and the methods used, the significance of the results can vary greatly.

I plan to get into that later this summer with my next post in this series:  How to Analyze a Research Study

Did you find this post helpful? Please SHARE it!

 

 


About Carolyn

I'm Carolyn Mallon, RN, and I have ADHD. I'm also parenting at least one ADHD child, so it makes for quite an adventure! I don't have all the answers, but I certainly share the challenges of many ADHD parents! I started this blog as an exercise to help us improve our game at home and at school. Join us!

6 comments on “How to Recognize Credible Sources

  1. YES, YES, YES!!! I spend a good portion of my FB time trying to explain this to people, but man does it get tiring. I’m glad to hear that you really take this stuff seriously. I have been helping my friends understand how to navigate this as well. Here are a couple more of my hints
    1. If you click on the link and there is NO date to the article, or the date is today’s date you have to be just a tad suspicious, a lot of time the sites just fill in today’s date anytime you go there, so the article always looks current.
    2. If you click on the evidence links , they will often take you to no where, to places that no longer exist OR to articles that have nothing to do with the thing that you were looking for evidence on.
    3. Anything with conspiracy theory or the words ‘Big’ followed by something ‘BigPharma’ for example. Plays on peoples fears.
    4. And lastly articles that don’t really say antyhing, just discuss anecdotal evidence, for example their evidence that X treatment works is Sally got better, Joe got better, and Jane got better. Anecdotal evidence. No controlled trials.

    🙂 Love you even more. ~Stacy, RN.

    • Oops, i got so excited I put my last name in the comment before. Can you take it out? Thanks 😉

      • Fixed it! Just saw that! 🙂

        Thanks for the comments. I’m glad you appreciated the post here. I had another great comment… from a librarian. I think those of us who have had to do research in our educations have a health appreciation for this kind of thing!

    • Yes! It’s so important in the age of Internet! Anyone can make a website, but it doesn’t mean they have credible information! Even books have to be looked at more critically these days because of the rise of self publishing…

  2. This is such great advice! I participate in some an online support group for ADHD and I get overwhelmed with the quack science and misinformation that gets spread. I really like a blog called “Science Based Medicine” which exposed some of these things. One other red flag that I look out for is if all the analysis is based on one person’s experience as a clinical practitioner, but is presented as if it is a fact rather than an opinion. The Washington Post just had an article like this that was written by an occupational therapist about what is wrong with our schools. All her solutions were based solely on her opinion after years of practice.

Comments are closed.