Showing posts with label evidence based medicine. Show all posts
Showing posts with label evidence based medicine. Show all posts

Saturday, 3 December 2016

How to create your very own evidence base … in a post truth World


"At one time we had truth and lies. Now we have truth, lies, and statements that may not be true but we consider too benign to call false." Ralph Keys

 

 
 
Hurrah! I hear you say (unless you are a member of the AACP that is … acupuncture is not recommended as a treatment for back pain) the new NICE Guideline - Low back pain and sciatica in over 16s: assessment and management has just been published. That means a group of experts have successfully completed months/years of hard labour, poring through evidence to produce a thorough carefully worded (italics for emphasis) analysis and clear guidance.



Of course, it doesn’t stop there! Then we get commentary and analysis via bloggers. Here’s an example of a nice balanced piece by Neil O’ Connell who was a member of the guideline committee. KarenMiddleton added her view on how the guideline marks an important moment for the physiotherapy profession, emphasising the ‘opportunity to evolve and look again at practice and re-evaluate what is best for patients.’ … the guideline gives clear wording (italics for emphasis) on that, and naturally is disappointing for some, yet welcome to others.

Naturally, as is the way of the World, we get the bloggers, sceptics and Twitterati who also helpfully summarise the guideline, and it is here where things begin to get a little messy. What is clear, to us all in the ‘modern World’ is that there is little time to read long documents, and it is human nature to look for short cuts (so far so good). So in many fields such as medicine, pharmacy and physiotherapy we have the rise of the ‘infographic’ … 

Now, don't get me wrong here, infographics are brilliant, because they are short snappy and summarise long papers in one interesting and attractive figure. I you’ve never seen the work of Yann Le Meur … then you should. The artistry, accuracy and attention to detail, is both impressive and incredibly useful to visual learners. 

HOWEVER

not all infographics … how shall I say … meet this exacting standard.


It became apparently obvious to me, that writing an infographic allows the writer to interpret something like a guideline, pretty much any way they want (just as Bloggers and internet commentators do), Herein, lies the rub …

IF the infographic writer, for one reason or another, chooses to alter the emphasis or wording, miss bit out, or get bits wrong … then the reader or recipient can be easily mislead. 

So, you end up with a snap-shot that can misrepresent the original document. Which of course, means that ANYONE can create their own version of the evidence base via the medium of the infographic.

HERE the fun begins!

Below I’ve written a short infographic (well ... some text and a picture) to help anyone create their own evidence base. If you follow the simple instructions, you can make any, paper review, guideline or article say exactly what YOU LIKE … it is a miracle! 


Try it yourself, adjust your favourite intervention, be it CBT, exercise, massage into a slightly bigger font or place on the positive side of the table/graph/pie chart in a slightly bolder colour etc. alter a word here and there (for emphasis/de-emphasis) and before you know it, you’ve changed the message of the original. You can even miss stuff out if you like, say radio frequency denervation or manual therapy (it is entirely your choice), if that’s is something you don’t particularly subscribe to.

Indeed ... if someone from the AACP had thought of it they could have re-inserted acupuncture into the LBP guidelines and lots of folk would never have noticed (cos’ they don’t have time to read the source document). That would of course, be disingenuous, but hey ... politicians do it daily, and we wouldn’t have had Iraq, Brexit or Donald Trump without a little (or more) distortion of the truth.

Where does this leave us? And why did I even bother to put pen to paper?

Well it requires us to have a realisation that we are in the ‘post-truth era’ as detailed so well by Ralph Keyes in his book The Post-Truth Era:Dishonesty and Deception in Contemporary Life. Keyes states:

‘At one time we had truth and lies. Now we have truth, lies, and statements that may not be true but we consider too benign to call false. Euphemisms abound. We’re “economical with the truth,” we “sweeten it,” or tell “the truth improved.” The term deceive gives way to spin.  At worst we admit to “misspeaking,” or “exercising poor judgment.”  Nor do we want to accuse others of lying.  We say they’re in denial.  A liar is “ethically challenged,” someone for whom “the truth is temporarily unavailable.”’ Ralph Keyes

Furthermore he states …

‘We can only understand the motives of such dissemblers by examining the sea in which they swim. Trends ranging from the postmodern disdain for “truth” to therapeutic non-judgment encourage deception. There is much incentive and little penalty for improving the “narrative” of one’s life. The increasing influence of therapists, entertainers, politicians, academics, and lawyers, with their flexible code of ethics, contribute to the post-truth era. So do ethical relativism, Boomer narcissism, the decline of community, and rise of the Internet.’

So there you have it folks … we have to adjust our radars, improve our awareness, be on our guard, even occasionally take the time to read a source document. Because, sometimes those nice folks who wrote that blog may just have written it to fit their own biases, or particular agendas, and in the same way, the handy visual snapshot of reality ... the infographic ... has sadly fallen foul of exactly the same concept. 

The dissemblers are amongst us, and I truly don't know why they behave that way.

(Source: http://www.skepticalaboutskeptics.org/)

Now  ... just a brief word on sceptics ... AND pseudo sceptics

The medical World has its fair share of sceptics or skeptics (as they are known in the USA), Ben Goldacre of Bad Science, is perhaps one of the most well known examples, Edzard Ernst is another. These brave souls, speak out on any issues from, basic bad science, global warming, Government policy, Prince Charles and his promotion of homeopathy, through to the risks of spinal manipulation etc. 

Good sceptics adopt an open minded approach and use science, debate, exposure (of bad practice) and apply critical examination and inquiry to all sides (including their own). 

Here's an example; Ernst has been a long time critic of alternative medicine (AM), and frequently adverse events (e.g. death after SMT) and general poor scientific practice relating to all sorts of disciplines within AM. These range from the more unconventional stuff like 'energy healing', and 'spiritual healing', through to say 'slapping therapy', and perhaps the more conventional, such as acupuncture. As such, he is commonly aggresively vilified and challenged by proponents of such therapies. His response is kindly, gentlemanly debate (often in the face of virtual abuse), use of science, critique, statistics etc. BUT when evidence comes along to challenge his World view on a topic, he has the good grace and conduct, to modify his view according to the developing and changing evidence base, or at the very least, air that evidence. 

Of interest to some physiotherapists (either historically or practically), MASSAGE had been a target of his for a number of years, until a new paper caught his eye and he wrote this and then later, this. Ernst is an example of how a GOOD honest sceptic is able to adapt their paradigm to new evidence and update their hypotheses to fit the data. In physiotherapy, SOME of our emerging breed of bloggers, podcasters and self-proclaimed sceptics are able to do that, and do it very well. SOME appear to default to the habits of pseudo-sceptics.

Pseudo-sceptics ... tend to:
  • Persistantly judge as false, and debunk anything that contradicts their paradigm.
  • Are partially interested in truth, evidence and facts, but MORE interested in defending their own views/stance.
  • May fail to update their paradigm to incorporate new evidence, and deny or bury data which doesn't fit their view.
(Source http://www.debunkingskeptics.com/characteristics.php)

Sceptic or denier?

'A sceptic will question claims, then embrace the evidence. 

A denier will question claims, then reject (or bury) the evidence....'

and for a brilliant commentary on pseudo-skepticism by Marcello Truzzi go here.

So be on your guard people … it's a tough environment out there, and distinguishing truth from fiction in a post-truth World ain't easy ... false news, pervades every aspect of the internet, even those sources, you thought you could trust, AND relies entirely on a passive (non-thinking) click of a button. Those who use it to their advantage know that, and you should too ... if you don't already! 

After all, that is what brought us Donald Trump. Good luck folks.

I’ll leave you with the wise words of Ralph Keyes who vocalises this much better than I ever could …

‘Post-truthfulness builds a fragile social edifice based on wariness. It erodes the foundation of trust that underlies any healthy civilization. When enough of us peddle fantasy as fact, society loses its grounding in reality.  Society would crumble altogether if we assumed others were as likely to dissemble as tell the truth. We are perilously close to that point.’ Ralph Keyes

Be careful out there folks … words matter, and semantics are important AND that is the TRUTH

HT to Woody  Guthrie for being an inspiration -  "It's a folk singers critical thinkers job to comfort disturbed people and to disturb comfortable people" ... see what I did?
 





Author: Alan J Taylor is a writer and critic who thinks about stuff and works as a Physiotherapist, University Assistant Professor and Medico-Legal expert witness ... The views contained in this blog are his own and are not linked to any organisation or institution. Like Bukowski, he 'writes to stay sane'.

Monday, 23 February 2015

Forget gurus, the cult of the evidence-based blogger has taken over ... 'Biased BLOG Bingo'


WARNING! This blog may contain traces of humour ... 

If you suffer a sense of humour deficit ... DO NOT READ ON!

Some time ago now, I had the pleasure of reading an excellent article by Will Self called ‘The awful cult of the talentless hipster has taken over’ … 

Whilst I didn’t necessarily agree with his diatribe entirely, I lapped up the trademark dour humour, empathised with his view and thoroughly enjoyed the read. Then in a bizarre moment of thought association, my mind turned to my own area of interest, Health Sciences and evidence based practice (Physiotherapy in particular) … though this undoubtedly pertains to medicine and all other areas of health care ... I began to ponder ‘the cult of the evidenced based blogger’, which now appears to pervade the zeitgeist of our increasingly confused World




Don’t get me wrong, our World is important (to us … and the people we care for, hopefully) as is the evidence. I like and respect (most) bloggers (cos’ they’re out there), I blog myself … But sometimes I begin to wonder about the whole process, or perhaps question the motives of the bloggers (myself included).

Bloggers and Twitterati, these days are ubiquitous; everyone seems to be having a go and some appear to be very authoritative. Yet blogging is a strange and precarious pastime/hobby/profession, which is both time and thought consuming. So, unless they are getting paid for it (some are … Will Self falls into that category), one would have to debate what motivates the ardent blogger. Shouldn’t they have just gone out for a run or cycle ride or something? 

What would actually drive someone to spend valuable time writing and airing their thoughts on any topic? What drives them to risk an avalanche of comment/critique if their particular diatribe hits the wrong button, or perhaps, a rising tide of gushing agreement from the ‘Bloggioso’ or the ‘Twitterati’ for their latest fashionable and populist masterpiece?

Some blogs are really helpful (or are they?) because they interpret and decipher some of those peer reviewed papers (which to be fair, may be a bit complex/wordy). So blogs may appear really helpful for those busy clinicians who only get limited time for reading/analysing the latest news on a topic.  

One clear attraction of blogging (for bloggers), unlike the restricted writing of peer review for instance, which requires writers to declare conflicts of interest ... is that you can say what you like … and it is clear that some bloggers ‘like what they say’.

But … is what they/we say, prone to BIAS or subject to ‘conflicts of interest’ as a result? 

To help my own decision making, I decided to gather a concoction of my own observations on some of the blog sites that may be influencing how we think and view evidence. Some bloggers may see parts themselves or their writing in one or all of the types (I did). That's not really the intention. Rather, it is for readers to see how a topic may, or may not, be spun.

I'll leave you to decide what you think ... Perhaps after a lighthearted game of 'biased BLOG bingo'.

I wrote about this in my last blog, so if you haven't already, take some time to consider the definition of confirmation bias ... 'the tendency for people to favour information that confirms their beliefs'. 

A quick look at this short YouTube clip may help.



Then consider the concept of conflict of interest which may be easy to identify in medicine for example, with concepts like 'Big Pharma' and 'Bad Pharma', which even have their own Wiki pages, but perhaps less overt in Health Sciences and related subjects. 

With those things in mind, here is my tongue in cheek personal take (from the experience of both reading and writing blogs) on a few of the types of blogger you may encounter out there … and some of the factors that may influence them ... Enjoy! 



Meet some of the ‘blogger types’

1.     The altruistic/educational/hobby blogger – Blogs about a variety of topics of interest to potential readership. Evidence based, educational conduit, who likes to hear the sound of his/her voice … Keeps up to date and an open mind, avoids extremism and generally goes out of his/her way to avoid bias, May throw in some controversy for interest, but sticks to honest appraisals of the evidence. Likes a little devilish humour and for folks to read his/her blog. No commercial interests, no adverts, no shop. Altruistic, ego driven, no nonsense profile builder. Moderate use of social media (SoMe) to promote blogs. Checks blog metrics occasionally. Likes to be asked to ‘guest blog’. Secretly hopes for a trip to Hawaii to speak on his/her latest blog topic. Conflict of interest - Nil of note. Has had a book 'in the pipeline' for 15 years. 

2.     Student blogger – Students who (led by their University Professor) have entered the World of blogging, without actually knowing what they have let themselves in for. Main qualities are passion for the topic and enthusiasm. Downfall may be inexperience (blogging), naivety and failure to critique, or cover the topic from a wide and unbiased perspective. May find themselves, unwittingly at the sharp end of criticism from outraged readers. Not always prepared for this. Variable use of SoMe use, that is until they realise blog metrics contribute to their overall assessment mark. Comments are enabled … until they get thoroughly blasted by someone. Not entirely sure ‘exactly’ where Hawaii is, but would love to go. Conflict of interest? ... Actually, the main thing is ... to pass the assessment!

3.     The Snake oil seller – So enthusiastic about their particular brand of ‘snake oil’. They forgot, or chose not … to support any of their claims with even a shred of evidence. They tend to rely entirely on anecdote and personal recommendation from users of the product/idea/treatment technique. Tales of miracle cures are commonplace. Comments are moderated to include more anecdotal claims or gushing personal endorsements. Commercial interests are generally utmost on their minds and they will ALWAYS have an advert for their particular type of ‘snake oil’ on their blog. This blogger, will be targeted mercilessly by bloggers number 5 and 7, generally to no effect (because they live in their own World, or maybe Hawaii). Heavy use of SoMe. The course for this is product /idea/service is ‘brilliant’, said a delegate. Conflict of interest – Zero (there is no conflict, as their sole interest is profit). A modern day ‘medicine show’. Link directly to the 'shop' here. Entirely and unashamedly biased.

4.     Society or organisation ‘news’ blogs – Generally low-key interest/news articles for members. Designed to update folks on the latest development in the field of interest/profession. Tend to report and stick to facts, seldom court controversy and may tend to be a little bland. Media spokesperson quotes some 'evidence', but may not always be in context or entirely up to date (this will be spotted and hastily dealt with by blogger number 7). Low to moderate use of SoMe for promotion of blogs/articles. Comments commonly disabled. Bland content, seldom gives opportunity for bias. Metrics? Hawaii? Humour? Shop? ... Pardon me!?

5.     EBM proponent/Targeted attacker – Wily operator, who picks a specific (often universally disliked target) and exposes it/them and provides reams of evidence to back up his/her claims. The best of these will end up on a TV show deliver a TED talk or get a column in a broadsheet. Heavy use of SoMe to promote blogs. Commercial interests may include books, newspaper articles, TV show appearances, talks etc. Sycophants and wannabees will include them in a Tweet in the hope of a rebound (seldom works). Comments ARE enabled, and this blogger loves to argue the toss with anyone who cares to have a go and often does so with incisive humour or complex statistics. Perhaps a little obsessed and in possession of a large ego. Naturally biased towards own (often populist views) but generally and genuinely supported by the evidence. This blogger loves notoriety. Metrics are through the roof (seldom needs to check) Goes to Hawaii regularly via private jet. Conflict of interest? ‘Pah … call my booking agent, I'm busy working on my next book/TV show’.

       The rest … (Type 5b), may end up looking looking like aspiring wannabees with a particular axe to grind. These yet to be so ‘successful’ wannabee type 5's, are feverishly typing whilst waiting for ‘the phone call’ and busily fending off ad hominem attacks from ‘outraged of Tunbridge Wells’ or assorted trolls. 

6.     The evangelical blogger – Combines the friendly bonhomie of blogger type 1 with the spin of blogger 3 but has a clear underlying mission of promoting a particular product/method or school of thought. May have a track record in peer review publication, suggestive of authority, yet routinely cherry pick evidence, to support a particular view.  Entirely convinced by the sanctity of their chosen path/product. Extremist disciples, lambast non-believers or other 'churches' as unseeing heretics.

Blog comments are ALWAYS moderated and predominantly populated by devout and enthusiastic followers. Humour is not a common feature of their writing. Evidence based, but a stoic adherence to one doctrine/product/method, leads to blogs that are littered with confirmation bias and supported by cute anecdotal stories. Heavy use of SoMe for promotion. Strongly motivated by metrics and sales. Conflict of interest - You can join the latest crusade (in Hawaii) next week, which ironically coincides with their latest blog (submit HERE to apply). Merchandise shop? Click here.

7.     Frustrated, change agent blogger – This passionate and profuse blogger is entirely frustrated by the speed at which his/her profession effects change. Routinely supplies or demands ‘the evidence’, which calls for the immediate abolition of out-dated ideas, methods and products, which have been ‘shown to be ineffective’. Such vigour and attention to detail, means they may occasionally therefore, resemble the internet 'evidence police' or media watchdog. 

    Fierce belief that the only really valid evidence is the ‘gold-standard’ RCT. Particularly adept at highlighting what doesn't work ... but may omit to offer alternatives to the discarded idea/method/product ... therefore run the risk of leading his/her colleagues into an ‘evidence based’, but tool less cul-de-sac. Energetic, challenging and authoritative. Couldn’t make the conference in Hawaii, but hopes to be there next year (recently spoke in Milton Keynes/Basildon). Busy writing another blog in the mean time. Loves Twitter. Metrics matter. Big fan of type 5a.  Ubiquitous. No time for shops, but currently working on a few other conflicts of interest.


NOW it’s time brighten up the academic tedium with a game of ‘Biased BLOG Bingo’ with the blog YOU last read … errr NO, not this one, it is ENTIRELY biased to my point of view!



DISCLAIMER: Bloggers take part in ‘Biased BLOG Bingo’ entirely at their own risk …
This presentation does not pertain to any bloggers called ‘Hamlin’ …or anyone else, it is merely a parody conglomeration of stereotypes. Anyway ... lighten up! 

Grid design MattLowPT
 
10 points = BINGO (start at zero) 

The higher the score the MORE biased the blog … GOOD LUCK!

1.     Is this blogger a 'snake oil seller'? (Score 6 immediately)

2.     Does the blog contain ANY credible evidence? (Deduct 1)

3.     Is the blog full of anecdote, personal experience and endorsements from users of said product/service/doctrine? (Score 3)   

4.     Is the blog balanced and offers more than one school of thought? (Deduct 2)

5.     Does the blog direct you to ALL of its sources? (Deduct 1)

6.     Does the blog direct you to just the sources it wants you to read? (Score 2)

7.     Does the blog promote ONE specific idea, method product/service? (Score 3)

8.     Does the blog recognise and report opposing views objectively? (Deduct 2)

9.     Does the blog denigrate/mock the ideas of others? (Score 2)

10.  Does the blog cherry pick evidence to support an idea, method product/service (Score 2)

11.  Are comments allowed? (Deduct 2)

12.  Are comments moderated? (Score 1)

13.  Are comments disallowed? (Score 2)

14.  Are the moderated comments predominantly congratulations from ardent ‘followers’? (Score 3)

15.  Is there a SHOP? (Score 3)

Note to BLOG readers: If your favourite blog scored 10 or more (arbitrary unscientific score) ... Just take a moment to reflect on that. There is no suggestion that blogs should not be biased, bloggers write (and sell) what they like ... BUT having a shop for instance, starts them nicely on the road to a pretty impressive BINGO score. However, it is entirely up to the reader to interpret what they see, or to recognise and identify sources of potential bias (if that is, they want to..?) just as they would, if perhaps they chanced upon a copy of the Daily Mail.

As one blogger recently said, "We are all biased" ... it is just a case of how much? So, whilst you may heartily agree with what your favourite blogger says, does or sells, it may be worth reflecting upon their potential for bias/conflict of interest AND how that affects your decision making ... and subsequent actions.


Yes, yes … I know! … I’ll be scoring my own blog later (BINGO!)

Big thanks to Will Self for his inspiration … I may include him in a Tweet (secretly hoping for a rebound ‘re-Tweet’) just before I delete my own Twitter account or become blogger number 4

Author: Alan J Taylor is a writer and critic who thinks about stuff and works as a Physiotherapist and University lecturer ... The views contained in this blog are his own and are not linked to any organisation or institution. Like Bukowski, he 'writes to stay sane'.

You'll find him mostly on Twitter https://twitter.com/TaylorAlanJ
... that is until, he finally deletes his account, or is 'evidence based blogged' to oblivion. 

Biased blog bingo grid design .... via @MattLowPT 

Wednesday, 29 October 2014

Confirmation bias, physiotherapy and the highlighter pen ...



The recent Body in Mind blog post ‘Exercise for chronic whiplash: does it matter how we do it?’ by Zoe Michaleff, was an unexpected delight and breath of fresh air. Here you can see a top researcher reporting on a study that found the virtual OPPOSITE of what her team found, in an open objective account, without bias and without trying to tear that study apart from a methodological (or any other) perspective. She even went on to offer a plausible scientific explanation for the differences between the studies.

Why, you may ask is this so refreshing? Well, because we so seldom see it!

Whilst some commentators have gamely suggested we think carefully before alighting the latest 'bandwagon' (whatever that may be this week) and others have exhorted us to simply think. More commonly, we see a plethora of blogs/articles/tweets which support various theories or schools of thought, which are driven consistently by confirmation bias. In fact, it could be argued, some have even gone so far as to patrol the internet, helpfully fulfilling the role of rapid response 'Cyber Evidence Police'... and that's OK ('it's for your own security'), providing they are always right, BUT sadly, no-one is right 100% of the time.



Let's first consider the definition of confirmation bias or 'my side' bias ... 'the tendency for people to favour information that confirms their beliefs'. 

This YouTube clip is an excellent short resource.



Have you ever picked up a newspaper or magazine article and been drawn to a particular article and read it voraciously, lapping up the prose and rhetoric as you go? Conversely, have you picked up an article and discarded it after the first paragraph or less? Most people have, and the one of the factors that dictates whether you persist or stop is confirmation bias. In other words we like and enjoy reading stuff that floats our boat, or that confirms our long held/new found ideas and beliefs, and tend to avoid or ignore the converse.

The result is that we may become biased towards articles, research literature, books blogs, tweets, people etc. that confirm our thinking … It is human nature. I often see students working on critique of research papers for example, using a single marker pen. I can’t resist asking what they are highlighting … “the interesting stuff”, “the relevant stuff”, “the bits that support the theory I’m working on” etc. 


Only occasionally does anyone have two different coloured marker pens … one for the supporting evidence and one for the opposite point of view. Try it yourself, concentrate on what doesn’t support your theory for a moment, concentrate in the holes in the theory, the method, look at an argument from the opposite side. It is both illuminating and revealing and naturally, you’ll probably find the second marker pen is used a lot less than less than the first. Of course, that may be because of what you chose to critique in the first place.

Believe it or not … Human nature and confirmation bias may lead you to overlook the methodological, scientific, statistical or philosophical holes in one paper, whilst using all of your might to apply those things to another. In other words, there may be a tendency to ignore or under weigh evidence that could dis-confirm an opposing hypothesis. We see examples of this time and time again in the interpretation of physiotherapy evidence.

It is always worth asking yourself ... “When is the advocated theory ever wrong?” Nothing works all the time. An astute person is aware of the limitations of any idea or practice. Challenge any 'authority' to explain the alternative of their position, and when they might use another option ... You can see more on that approach here.
 
What is even more revealing is when we see ‘conversions’ from one school of thought to another. Take a moment to look at this blog article on the 'conversion' of a staunchly religious person to an atheist and what happened thereafter. Not untypically, the post conversion individual went on the attack and " … dismissed all others’ beliefs and assumed his was correct". That frankly, is not a brilliant scientific or philosophical stance to take regardless of the 'evidence', there is always a contrary view and another side to any coin ... and the evidence base, as we know is an ever changing landscape. 

So that is why Michaleff's article was a breath of fresh air … I think, as a profession we can all take something from that ... and perhaps 'work together and learn together'.

If you want to read a little more on the various types of bias that may permeate our World ... this article is not a bad place to start. 

Personally, I’m just off to the 24h multicoloured highlighter pen shop.


Author: Alan J Taylor is a writer and critic who thinks about stuff and works as a Physiotherapist and University lecturer ... The views contained in this blog are his own and are not linked to any organisation or institution.

You'll find him mostly on Twitter https://twitter.com/TaylorAlanJ