On the Need for Skepticism

No, your experience as a parent is not more important than the opinion of the ‘experts’. Skepticism is important for all parents.

Many people have asked me why I am so obsessed with evidence. They say that the views and experience of parents/teachers etc are more important than the view of experts who always insist on checking everything. And they often say ‘what’s the harm, just try it and see’.

The ‘it’ they’re talking about changes every year or so. It might be fidget spinners, weighted blankets, special diets, supplements (the precise supplement has changed over the decades with fashion but some sort or other has been around for a long time). Or it might be their experience about how vaccines harmed their child and caused their autism (yes that theory is very much alive and kicking). Because if something happens at the same time as something else it must have been caused by it right? Eye roll.

This is of course rubbish. Why is it rubbish?

Let me tell you about one such story.

Not too long ago a new ‘theory’ hit the autism world by storm. A new technique. It was called Facilitated Communication. It had a laudable background. The idea was that all autistic people should be presumed competent. That even though they may not be able to speak (please note this was before iPads and proloquo2go) or react to simple instructions, even very severely affected children understood everything that was said to them. They just had no way of controlling their body enough to respond.

It was a lovely idea. Everyone that is autistic has a normal IQ, or a high IQ, they are just locked into their body and unable to show us how smart they are. How many parents wouldn’t want to believe something like that? it is an idea still often spread by memes (because if it’s written in a meme it must be true right?).

Facilitated Communication (FC)

So they came up with a technique. An autistic child (usually) would sit down next to a facilitator. A keyboard was placed in front of them. The facilitator would lightly touch the child on the shoulder/arm and ask them questions. Over time, developing their relationship (it was all about the relationship you see), and trusting that the child always understood everything they said, the facilitator would help the child ‘speak’ an answer on the keyboard.

Does this sound a bit like a Ouija board to anyone else? And yet people truly believed this worked. Not just some of the time. 100% of the time with all children. As long as the facilitator was a ‘good’ facilitator and as long as you had ‘faith’ that it would work. Strangely children would have completely inconsistent results across different facilitators, often weren’t looking at the keyboards as they ‘wrote’ on them, and commonly couldn’t get much of a result with their own parents.

It became so popular within just a couple of years there were university courses in it. The experts kept saying something about lack of evidence, but people ‘saw’ their kids communicating with them and ‘experienced’ how well it worked. It was rolled out all over America. Lots of special ed departments and schools used it. Lots of autistic people and autism advocates praised it. Plus, it wasn’t doing any harm right?

The harm begins

Wrong. Because that’s when the sexual abuse trials started. Over and over the children would start to accuse their own parents and even siblings of serious sexual abuse. And parents who loved this new way of communicating with their kids would get very confused as to why they were saying these things about them. Even the facilitators were confused. They were special education teachers who loved these children and wanted to help them. They were sometimes heartbroken at the stories coming out of them, so similar to the main news stories of the day about abusive childcare workers.

But of course when you have a trial in a court of law you have to suddenly prove things. And so an expert finally looked, publicly, at this method. He would show a flashcard with an object to a facilitator and a flashcard with a different object to the child. Over and over. Or the facilitator would leave the room, and the expert would show something to the child. And see what the child ‘wrote’ on the keyboard. Every time it was the object that the facilitator had seen, not the child. Over and over the experts showed that the whole program was nothing more than wishful thinking.

Parents often prefer a feel good lie to a harsh truth

One of the parents that was on trial for sexual abuse of their child spent three months in prison during the trial, some of the time in total isolation (to protect him from the other prisoners). After the trial he said that even though the method was rubbish, it was worth going through the ordeal of the trial and incarceration because he spent several precious months thinking that his daughter was really ‘speaking’ to him. They had tried PECS, they had tried every evidence based method, it hadn’t worked. And he was grateful for this short time feeling like he had found a connection. Even though of course it was total rubbish.

This all highlights the problems with autism therapies. Many, many of them are feel good nonsense. Often they just take an activity or animal people like (horses, dolphins) and attach the word therapy after it. And parents bite. They see an improvement, because of how much they want to. If their child has a high IQ, the child will be told ‘try this weighted blanket, it will help you sleep at night’. They will then try it and say ‘yes it was great mum it worked I slept better’. The placebo effect can be extremely high with ‘high functioning’ children, anything up to 70%. When a study compares their actual sleep with vs without a weighted blanket, it is no better. But who wants facts when they can have a comforting, warm, feel-good fiction?

Real therapy is time consuming and expensive. But it works.

The problem is that the real therapies, the ones that have been shown to work and improve skills, work so slowly. They take so long. They are so repetitive, so incremental, and so expensive.

But it’s important to block them out. Partially, because they are a drain on resources. You can’t afford everything! Partially, because they can do real harm. And mostly because it means you aren’t doing something else, that could actually help. Something that could improve IQ and adaptive skills three or four times more than a placebo, as tested and replicated many times over.

It doesn’t give you that warm glow you would get from your child suddenly writing advanced poetry at the age of eight. It isn’t as pleasant as just going swimming with dolphins a few times. But it is real, and it is lasting, and will not involve you having to go to hospital or prison as a direct result of your ‘therapy’. Or even just having four times less adaptive skills than they might have.

Experience is still important

Does this mean experiences of parents or autistic people are irrelevant and should be ignored? No way! They provide the most interesting avenues to be explored. They provide topics that should definitely be researched. You think that going on a particular swing or spending time around music makes you calmer and better able to concentrate? Let’s test it. Let’s measure it. Let’s see if it can be replicated with other people. And if it works, if it isn’t just bias or a placebo, let’s share it with the world.

And remember the people that spread facilitated communication were not villains. They were really nice people totally committed to the children under their care. Just not overly skeptical people. And sometimes, especially as parents, a healthy dose of skepticism can be our most important weapon.

Facebook Comments

2 comments

  1. Thanks for this post. The problem I am having is that even therapies promoted as evidence-based are not as rigorously evaluated as they should be or assumed to be. When my little one was first diagnosed, I consulted the evidence regarding interventions for ASD as I have been trained to do in my work as a research academic reviewing other people’s research and designing, writing and publishing my own research. When therapists in the world of ASD describe their interventions as “evidence-based”, it unfortunately bares very little resemblance to what medical/public health scientists would regard as “evidence-based”. My assessment of the evidence supporting interventions in ASD was sobering to say the least. My first port of call was the Cochrane Data Base of Systematic Reviews- a collection of reviews about almost every evaluated intervention in the world of medical practice, including psychological, educational and community-based interventions, carried out by independent researchers using standardised methods for reviewing research. It is regarded as the best resource for determining the quality of scientific literature. In short, we just don’t have the same quality of evidence to draw upon that underpins many other interventions. I disagree that there is evidence that “Something that could improve IQ and adaptive skills three or four times more than a placebo, as tested and replicated many times over.” There just isn’t that when we assess that evidence against gold-standard methods that is demanded of other therapies in the medical world. What I was hoping to see was evidence from large-scale randomised controlled conducted with adequate controls in place to guard against various biases. Trials that involve a large number of children, for example, with a good mix of therapists (not just one or two) across different centres to ensure generalisability of findings. Trials that, if described as randomised, provide enough information to assess against gold-standard accepted methods for being a well-conducted study. Trials with results that could be described as robust. Trials with almost 100% follow-up so you aren’t left wondering if results would be less convincing or nullfied if all children completed outcome assessment. Trials that have provided head-to-head comparisons of interventions, controlling for non-specific characteristics. Many of the studies are evaluated using assessments that are specific to the targeted behaviours of therapy and not to academic or social skills or other general skills. Will my child be able to generalise the skills painstakingly developed with his therapists and us to a school setting? Will he be able to pay attention in class, concentrate for sufficiently long enough to complete a work-sheet without being rewarded for functional behaviour by trained therapists (which his teachers aren’t going to be)? Will he become dependent on those rewards? Is 40/30/25 hours versus 10 hours really better? Could we draw upon research in training of athletes, for example. Is it possible that little spurts of 10 minutes or so peppered throughout the day of simple therapy “trials” are better than blocks of two hours or so? I don’t believe we have those answers. Is starting therapy at 18 months really better than starting at 36 months? If we build neural pathways to normalise autistic behaviours are we doing this at the expense of children developing special talents that many children with Autism have? Should we be targetting those special talents rather than their deficiencies for high functioning children? While I love our ABA program, we have seen detrimental effects of it as well- resistance to activities and rewards because it just means having to do something he doesn’t want to do etc and we have probably built up some negative neural pathways as well as functional ones… It really does depend on having skilled therapists that can avert or undo these effects. On the other hand, I have also had professionals advise us that ABA is not as great as it is touted to be because there isn’t the evidence to support its superiority. Insufficient evidence shouldn’t be confused with evidence against a specific therapy’s use. What ABA has going for it is a sound basis in the principles of behaviour modification/skills development. And, as you have identified in a previous post, it’s optimism…it is therapy for us as well! It’s an approach to learning, it provides us with valuable strategies to ensure our little one attends to the tasks that are designed to encourage his skills. But I am not sure that the evidence supporting it’s use can be described as “gold standard”. In fact, if I ever do get my “impossible moment” of regaining my research career I have already dreamed up a comprehensive research program to get the evidence we need to guide parents and clinicians to make evidence-based choices.

    Your post also refers to now discredited interventions as preying on the idea that “Everyone that is autistic has a normal IQ, or a high IQ, they are just locked into their body and unable to show us how smart they are.” I am in no way suggesting that mainstream therapists are exploiting this view, but I am constantly beleaguered by the view that we are not doing enough, that anytime our son manifests autistic behaviour that we just haven’t been working hard enough to modify that and turn it into functional behaviour. It isn’t helped by the fact that early enthusiasts of ABA described some children who underwent this therapy as “indistinguishable from their peers” (not sure of the direct quote, think it comes from Lomas’ seminal paper). Many respectable therapies also engender the view that Autism is something that can be overcome. That said, I haven’t encountered a therapist that has suggested or promised that. But the very idea you are advised to immerse yourself in this therapy up to 40 hours a week comes close to suggesting we can’t let children with autism get away with doing too much on their own.

    So for what it’s worth, I just don’t think there is any therapy for Autism that can be described as “evidence-based” in the way that phrase was meant to be used. We proceed with skepticism about even the more accepted therapies, but also realising that in the world of medicine, people have practiced therapies that have eventually been proven effective. The problem is that there are many that have also eventually proven ineffective or worse, harmful. Or not as effective as believed. We sorely need more research on what works and why.

    Reply
    1. Such a good, well thought out comment Mel! Thank you for this! You’re right, it is so hard to do proper studies with autism therapies. Research is sorely needed and especially into follow-up, as you say – did these skills stay over time, did they translate into a different classroom (or more usefully, did they translate into the workforce/general everyday living). It is very hard to do this to a gold standard because participants usually self select which group they want to go into. Often the children with higher support needs end up in the 35 hours of ABA group and the others elsewhere. Which should theoretically introduce a bias against ABA since a higher IQ at the start usually equals better results no matter what you do. And you’re right, the biggest sample size i have seen for autism therapies was about 500, which in scientific medical terms at least is nothing. That one was a 2010 review of published early intensive behaviour intervention studies. It’s available on PubMed https://www.ncbi.nlm.nih.gov/pubmed/20687823. ASAT (Association for Science in Autism Treatment) have a good website with summaries of studies and lots of good links. But is more research needed? Definitely!!!!!

      Reply

Add yours

*

*