How to create your very own evidence base … in a post truth World

"At one time we had truth and lies. Now we have truth, lies, and statements that may not be true but we consider too benign to call false." Ralph Keys

 

 

 

Hurrah! I hear you say (unless you are a member of the AACP that is … acupuncture is not recommended as a treatment for back pain) the new NICE Guideline - Low back pain and sciatica in over 16s: assessment and management has just been published. That means a group of experts have successfully completed months/years of hard labour, poring through evidence to produce a thorough carefully worded (italics for emphasis) analysis and clear guidance.

Of course, it doesn’t stop there! Then we get commentary and analysis via bloggers. Here’s an example of a nice balanced piece by Neil O’ Connell who was a member of the guideline committee. KarenMiddleton added her view on how the guideline marks an important moment for the physiotherapy profession, emphasising the ‘opportunity to evolve and look again at practice and re-evaluate what is best for patients.’ … the guideline gives clear wording (italics for emphasis) on that, and naturally is disappointing for some, yet welcome to others.

Naturally, as is the way of the World, we get the bloggers, sceptics and Twitterati who also helpfully summarise the guideline, and it is here where things begin to get a little messy. What is clear, to us all in the ‘modern World’ is that there is little time to read long documents, and it is human nature to look for short cuts (so far so good). So in many fields such as medicine, pharmacy and physiotherapy we have the rise of the ‘infographic’ … 

Now, don't get me wrong here, infographics are brilliant, because they are short snappy and summarise long papers in one interesting and attractive figure. I you’ve never seen the work of Yann Le Meur … then you should. The artistry, accuracy and attention to detail, is both impressive and incredibly useful to visual learners. 

HOWEVER

not all infographics … how shall I say … meet this exacting standard.

It became apparently obvious to me, that writing an infographic allows the writer to interpret something like a guideline, pretty much any way they want (just as Bloggers and internet commentators do), Herein, lies the rub …

IF the infographic writer, for one reason or another, chooses to alter the emphasis or wording, miss bit out, or get bits wrong … then the reader or recipient can be easily mislead. 

So, you end up with a snap-shot that can misrepresent the original document. Which of course, means that ANYONE can create their own version of the evidence base via the medium of the infographic.

HERE the fun begins!

Below I’ve written a short infographic (well ... some text and a picture) to help anyone create their own evidence base. If you follow the simple instructions, you can make any, paper review, guideline or article say exactly what YOU LIKE … it is a miracle! 

Try it yourself, adjust your favourite intervention, be it CBT, exercise, massage into a slightly bigger font or place on the positive side of the table/graph/pie chart in a slightly bolder colour etc. alter a word here and there (for emphasis/de-emphasis) and before you know it, you’ve changed the message of the original. You can even miss stuff out if you like, say radio frequency denervation or manual therapy (it is entirely your choice), if that’s is something you don’t particularly subscribe to.

Indeed ... if someone from the AACP had thought of it they could have re-inserted acupuncture into the LBP guidelines and lots of folk would never have noticed (cos’ they don’t have time to read the source document). That would of course, be disingenuous, but hey ... politicians do it daily, and we wouldn’t have had Iraq, Brexit or Donald Trump without a little (or more) distortion of the truth.

Where does this leave us? And why did I even bother to put pen to paper?

Well it requires us to have a realisation that we are in the ‘post-truth era’ as detailed so well by Ralph Keyes in his book The Post-Truth Era:Dishonesty and Deception in Contemporary Life. Keyes states:

‘At one time we had truth and lies. Now we have truth, lies, and statements that may not be true but we consider too benign to call false. Euphemisms abound. We’re “economical with the truth,” we “sweeten it,” or tell “the truth improved.” The term deceive gives way to spin.  At worst we admit to “misspeaking,” or “exercising poor judgment.”  Nor do we want to accuse others of lying.  We say they’re in denial.  A liar is “ethically challenged,” someone for whom “the truth is temporarily unavailable.”’ Ralph Keyes

Furthermore he states …

‘We can only understand the motives of such dissemblers by examining the sea in which they swim. Trends ranging from the postmodern disdain for “truth” to therapeutic non-judgment encourage deception. There is much incentive and little penalty for improving the “narrative” of one’s life. The increasing influence of therapists, entertainers, politicians, academics, and lawyers, with their flexible code of ethics, contribute to the post-truth era. So do ethical relativism, Boomer narcissism, the decline of community, and rise of the Internet.’

So there you have it folks … we have to adjust our radars, improve our awareness, be on our guard, even occasionally take the time to read a source document. Because, sometimes those nice folks who wrote that blog may just have written it to fit their own biases, or particular agendas, and in the same way, the handy visual snapshot of reality ... the infographic ... has sadly fallen foul of exactly the same concept. 

The dissemblers are amongst us, and I truly don't know why they behave that way.

(Source: http://www.skepticalaboutskeptics.org/)

Now  ... just a brief word on sceptics ... AND pseudo sceptics

The medical World has its fair share of sceptics or skeptics (as they are known in the USA), Ben Goldacre of Bad Science, is perhaps one of the most well known examples, Edzard Ernst is another. These brave souls, speak out on any issues from, basic bad science, global warming, Government policy, Prince Charles and his promotion of homeopathy, through to the risks of spinal manipulation etc. 

Good sceptics adopt an open minded approach and use science, debate, exposure (of bad practice) and apply critical examination and inquiry to all sides (including their own). 

Here's an example; Ernst has been a long time critic of alternative medicine (AM), and frequently adverse events (e.g. death after SMT) and general poor scientific practice relating to all sorts of disciplines within AM. These range from the more unconventional stuff like 'energy healing', and 'spiritual healing', through to say 'slapping therapy', and perhaps the more conventional, such as acupuncture. As such, he is commonly aggresively vilified and challenged by proponents of such therapies. His response is kindly, gentlemanly debate (often in the face of virtual abuse), use of science, critique, statistics etc. BUT when evidence comes along to challenge his World view on a topic, he has the good grace and conduct, to modify his view according to the developing and changing evidence base, or at the very least, air that evidence. 

Of interest to some physiotherapists (either historically or practically), MASSAGE had been a target of his for a number of years, until a new paper caught his eye and he wrote this and then later, this. Ernst is an example of how a GOOD honest sceptic is able to adapt their paradigm to new evidence and update their hypotheses to fit the data. In physiotherapy, SOME of our emerging breed of bloggers, podcasters and self-proclaimed sceptics are able to do that, and do it very well. SOME appear to default to the habits of pseudo-sceptics.

Pseudo-sceptics ... tend to:

• Persistantly judge as false, and debunk anything that contradicts their paradigm.
• Are partially interested in truth, evidence and facts, but MORE interested in defending their own views/stance.
• May fail to update their paradigm to incorporate new evidence, and deny or bury data which doesn't fit their view.

(Source http://www.debunkingskeptics.com/characteristics.php)

Sceptic or denier?

'A sceptic will question claims, then embrace the evidence. 

A denier will question claims, then reject (or bury) the evidence....'

and for a brilliant commentary on pseudo-skepticism by Marcello Truzzi go here.

So be on your guard people … it's a tough environment out there, and distinguishing truth from fiction in a post-truth World ain't easy ... false news, pervades every aspect of the internet, even those sources, you thought you could trust, AND relies entirely on a passive (non-thinking) click of a button. Those who use it to their advantage know that, and you should too ... if you don't already! 

After all, that is what brought us Donald Trump. Good luck folks.

I’ll leave you with the wise words of Ralph Keyes who vocalises this much better than I ever could …

‘Post-truthfulness builds a fragile social edifice based on wariness. It erodes the foundation of trust that underlies any healthy civilization. When enough of us peddle fantasy as fact, society loses its grounding in reality.  Society would crumble altogether if we assumed others were as likely to dissemble as tell the truth. We are perilously close to that point.’ Ralph Keyes

Be careful out there folks … words matter, and semantics are important AND that is the TRUTH

HT to Woody  Guthrie for being an inspiration -  "It's a folk singers critical thinkers job to comfort disturbed people and to disturb comfortable people" ... see what I did?
 

Author: Alan J Taylor is a writer and critic who thinks about stuff and works as a Physiotherapist, University Assistant Professor and Medico-Legal expert witness ... The views contained in this blog are his own and are not linked to any organisation or institution. Like Bukowski, he 'writes to stay sane'.

Confirmation bias, physiotherapy and the highlighter pen ...

The recent Body in Mind blog post ‘Exercise for chronic whiplash: does it matter how we do it?’ by Zoe Michaleff, was an unexpected delight and breath of fresh air. Here you can see a top researcher reporting on a study that found the virtual OPPOSITE of what her team found, in an open objective account, without bias and without trying to tear that study apart from a methodological (or any other) perspective. She even went on to offer a plausible scientific explanation for the differences between the studies.

Why, you may ask is this so refreshing? Well, because we so seldom see it!

Whilst some commentators have gamely suggested we think carefully before alighting the latest 'bandwagon' (whatever that may be this week) and others have exhorted us to simply think. More commonly, we see a plethora of blogs/articles/tweets which support various theories or schools of thought, which are driven consistently by confirmation bias. In fact, it could be argued, some have even gone so far as to patrol the internet, helpfully fulfilling the role of rapid response 'Cyber Evidence Police'... and that's OK ('it's for your own security'), providing they are always right, BUT sadly, no-one is right 100% of the time.

Let's first consider the definition of confirmation bias or 'my side' bias ... 'the tendency for people to favour information that confirms their beliefs'. 

This YouTube clip is an excellent short resource.

Have you ever picked up a newspaper or magazine article and been drawn to a particular article and read it voraciously, lapping up the prose and rhetoric as you go? Conversely, have you picked up an article and discarded it after the first paragraph or less? Most people have, and the one of the factors that dictates whether you persist or stop is confirmation bias. In other words we like and enjoy reading stuff that floats our boat, or that confirms our long held/new found ideas and beliefs, and tend to avoid or ignore the converse.

The result is that we may become biased towards articles, research literature, books blogs, tweets, people etc. that confirm our thinking … It is human nature. I often see students working on critique of research papers for example, using a single marker pen. I can’t resist asking what they are highlighting … “the interesting stuff”, “the relevant stuff”, “the bits that support the theory I’m working on” etc. 

Only occasionally does anyone have two different coloured marker pens … one for the supporting evidence and one for the opposite point of view. Try it yourself, concentrate on what doesn’t support your theory for a moment, concentrate in the holes in the theory, the method, look at an argument from the opposite side. It is both illuminating and revealing and naturally, you’ll probably find the second marker pen is used a lot less than less than the first. Of course, that may be because of what you chose to critique in the first place.

Believe it or not … Human nature and confirmation bias may lead you to overlook the methodological, scientific, statistical or philosophical holes in one paper, whilst using all of your might to apply those things to another. In other words, there may be a tendency to ignore or under weigh evidence that could dis-confirm an opposing hypothesis. We see examples of this time and time again in the interpretation of physiotherapy evidence.

It is always worth asking yourself ... “When is the advocated theory ever wrong?” Nothing works all the time. An astute person is aware of the limitations of any idea or practice. Challenge any 'authority' to explain the alternative of their position, and when they might use another option ... You can see more on that approach here.
 

What is even more revealing is when we see ‘conversions’ from one school of thought to another. Take a moment to look at this blog article on the 'conversion' of a staunchly religious person to an atheist and what happened thereafter. Not untypically, the post conversion individual went on the attack and " … dismissed all others’ beliefs and assumed his was correct". That frankly, is not a brilliant scientific or philosophical stance to take regardless of the 'evidence', there is always a contrary view and another side to any coin ... and the evidence base, as we know is an ever changing landscape. 

So that is why Michaleff's article was a breath of fresh air … I think, as a profession we can all take something from that ... and perhaps 'work together and learn together'.

If you want to read a little more on the various types of bias that may permeate our World ... this article is not a bad place to start. 

Personally, I’m just off to the 24h multicoloured highlighter pen shop.

Author: Alan J Taylor is a writer and critic who thinks about stuff and works as a Physiotherapist and University lecturer ... The views contained in this blog are his own and are not linked to any organisation or institution.

You'll find him mostly on Twitter https://twitter.com/TaylorAlanJ