Skip to main content

Tag: Systematic Research

Evidence-Based Practices: Quick Overview

Written by Stephanie M. Nixon, PhD, CCC-SLP on . Posted in Childhood Apraxia of Speech, Research. 1 Comment on Evidence-Based Practices: Quick Overview

Note. This is an updated version of an article I wrote in 2021 for Apraxia Kids. The main reference was to childhood apraxia of speech research, but this applies to research across the sciences.

Some current (2025) actions by the executive branch make information of utmost importance. Some changes threaten sources of independent research funding.

What is evidence-based practice?

You may have heard speech-language pathologists (SLPs) saying “evidence-based practices” when describing a specific treatment approach. Such practices are based on scientific evidence and professional experience while considering your child’s specific needs.

Scientific evidence is used by SLPs, educators, and medical professionals (among others) to guide interventions and strategies could help a person based on their specific needs. 

There is not one approach that works for all people, especially not for all children with childhood apraxia of speech (CAS).  There are some approaches that have not been studied at all or have been studied with research methods that don’t allow for very strong conclusions.

What is scientific evidence?

Scientific evidence is empirical data that is collected and interpreted using the scientific method.  This is evidence that serves to support or counter a scientific theory or hypothesis and is derived from controlled studies.

Is all scientific evidence high quality?

Simply put, no.

Well-designed research studies are more valuable tools than an anecdotal opinion.

What are the levels of evidence?

Additional information about evidence-based interventions as defined in the Every Student Succeeds Act (ESSA) from the California Department of Education.

Let’s summarize…

  • The lowest form of evidence = opinions based on “experience”
  • Best form of evidence = pulling multiple randomized controlled trials into guidelines or a meta-analysis

Anecdotes are not evidence especially when these anecdotes are offered by (or on websites of) people/companies with a commercial interest.

Note. Researchers/organizations obtain grant funding by applying. Even then, there is a review process to determine whether they receive the funding.

Applications are peer reviewed by field experts who determine whether the application meets the criteria among other standards.
Why does commercial interest matter in speech-language pathology or education?

What is a commercial interest? Earning money by selling a product OR saving money by not providing a service. (The latter is more relevant to health insurance – at least for now.)

Example: Dr. Blah at company XYZ creates and markets a program to teach math. If it is sold to school districts, then they will make money. Now if XYZ and Dr. Blah are the only ones who research the program they are more likely to determine this “works”.

What can decrease the quality of evidence?

Specific to CAS treatment research studies, look for study limitations (e.g., small sample size, inconsistent implementation of treatment approach, lack of a control group or condition, pre-treatment differences between groups, analysis performed by non-blinded outcome assessors, etc.).

Do we have much of the Level 1 standard research for Childhood Apraxia of Speech treatment?

No. We have a few meta-analysis studies and systematic reviews available.

Here are some systematic research reviews (most summaries by the American Speech Language Hearing Association in their Evidence Maps):

SLPs can look at the “evidence” to determine which approaches are most likely to work. The evidence includes the research available and the patient’s unique needs.

What characterizes a strong treatment research study?

The gold standard for a therapy approach study is a randomized controlled trial and/or meta-analysis. 

To evaluate a study, we look at the sample size, randomization, consistency of analysis, and fidelity.

  • Sample size:
    • How many participants/observations?
    • Diversity in the population? (i.e., Gender? Race? Ethnicity? Primary language? Age – depending? Diagnoses? Socioeconomic status (SES)? Geographic location? Gender identify and expression? Disability status?)
Why do we care about diversity in the sample ?

Simply put: Inclusivity matters. The linked 2023 article by Alice McCarthy details the need for inclusive clinical trials.

Yes, we’re dealing with speech-language therapy trials vs. drug trials; but, that doesn’t change the need to ensure a representative population. This might advise a research of possible barriers to care as well as how to address the barriers.

I will address more about the need to ensure inclusivity in research and health disparities. For now, please review this article by Megan A. Morris (2022), Striving Toward Equity in Health Care for People with Communication Disabilities. You can also review disparities associated with health insurance on my blog.

  • Randomization: There are many types of randomization as described in the article linked (Lim & In, 2019). Randomization helps researchers eliminate/minimize accidental bias (including selection bias).
    • e.g., We want to ensure that patients weren’t chosen to receive the “treatment” group merely because they seemed to have more or less speech sound errors.
Bias in research

Bias in research is beyond the scope of this article. If you’d like to learn more, here is one journal article by Pannucci and Wilkins (2010).

  • Consistency: This is often used interchangeably with reliability.
    • Internal consistency indicates that several items measuring the same idea generate similar responses.
    • External consistency is the degree to which results are consistent with prior research.
    • Specific to consistency of analysis, we want to know how reliable were measures of patient performance. What was the level of agreement between/among therapists? How about a novel therapist?
  • Fidelity:
    • Were the control and treatment interventions provided as intended? For example, did the SLPs administering the interventions stay within the guidelines provided by the research study? Did they introduce any concepts not included in the study?
    • Was the same modality of treatment used across patients in each condition? (in-person, telehealth, etc.)
Example.

(1) SLP working with a child in the “control” group includes some of the techniques from the “treatment” being tested and child shows progress in the “control” intervention.

Does this mean the “control” intervention works? No. It means the SLP did not follow the research protocol in the “control” intervention.

(2) SLP working with a child in the “treatment” group includes techniques beyond those described by the researchers as part of the treatment program (e.g., visual aids, cues, etc.) and the child shows progress with the “treatment” intervention.

Does this mean the “treatment” intervention works? No. It means the SLP did not follow the research protocol in the “treatment” intervention.

Other important factors

We also need to consider the presence of other factors that could explain improvement or regression (including whether these were controlled in the study). and whether these are controlled (e.g., if a child also received treatment elsewhere, or if a child started school, or had a medical procedure).

  • Was the child receiving treatment beyond that supplied in the study?
  • Did the child start school while in the study?
  • Did the child experience any medical changes during the study? (e.g., new medication, surgical procedure, new diagnoses, accident, etc.)

Maturation (i.e., getting older) is another factor to consider when reviewing treatment studies. That is, did the treatment group improve beyond what would be expected based with aging.

What if one approach with research support isn’t working for my child?

Well, if you have that concern, ask the practitioner. Practitioners usually track progress so they know when they need to adjust intervention (i.e., when the approach is not effective for the patient or when the patient masters an objective, etc.).

As a clinician, I ask myself these questions when an approach doesn’t seem to be “working”?

  • How long have we tried the approach?
  • Am I implementing the approach correctly? (It helps to review the approach.)
  • Is the child engaged (i.e., attending) with us during our sessions?
  • How consistent is the child’s attendance?
  • Have we provided appropriate family education (if included) with the approach? (This includes home practice)

Sometimes an approach just isn’t the best option for a child (and their family) and we need to adapt. This could include modifying the approach slightly to adapt or even changing approaches to one more suited for the child.

Regardless, an approach should have an evidence base, even if it was only demonstrated in a “single-subject design case study”. This allows SLPs to generate ideas/hypotheses (sophisticated educated guesses) about what might work for a given child.

Disclaimer. Information provided in this post by, Stephanie M. Nixon, Ph.D., CCC-SLP, is provided in good faith. Nixon Speech and Language, LLC makes no representation or warranty of any kind, express or implied regarding the accuracy, adequacy, validity, reliability, availability, or completeness of any information.

Information provided here is for informational purposes only.

This material is not substitute for professional medical advice or treatment. Always seek the advice of a qualified professional with any questions you may have regarding conditions and treatments/therapies. Do not delay seeking professional medical or educational advice/evaluation or delay seeking it because of something you read on this website.

See these websites for more information about evidence-based practices and research:

Leave a Reply

Your email address will not be published. Required fields are marked *

One response to “Evidence-Based Practices: Quick Overview”

  1. Call to action: Institute of Education Sciences: What is it? Why should I care? – Nixon Speech and Language

    […] posted this in a previous post, and I’m posting it […]

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *