Lets have a discussion

Other Discussions Pain Science & Rehab Discussion
1

So something that I’ve been hearing since I’ve been exposed to the research field, so about 8 months now is that “the research says/suggests”. Even my professors use this phrase, two of which have ph.ds from big name universities and one that is currently working on her dissertation.( Sociology, Anatomy and Physiology and Psychology for the later). Sociology professor regularly shows us articles in her powerpoints and teachings so i dont doubt that she’s up to date with current evidence and somewhat the same with my A&P professor. Even still i would find it hard for them to evaluate the entire scope of evidence and find it hard to believe that they dont have biases that they would like to pass off.

Even now the more that “evidence based” (insert product/field/idea/topic here) becomes more popular, I feel that this phrase should not be used so casually or sparingly.
Here’s my rational, I understand that i am pretty much a nobody because i dont have a graduate degree( dont even have an bachelors) so naturally i dont have any proper experience when it comes to evaluating research and implementing it. But even for those that evaluate and implement research into their practice, i believe that whenever asked a question about what the “evidence says” about a certain topic, they should answer by saying “Well, the evidence I’VE READ says/suggests X”.

I think that this revised statement would be a better reply because to my naive mind its impossible to know what the entire field of research has to say about a specific topic without having a biased opinion and theres no realistic way to read every relevant study about the topic because I think it would be extremely difficult to evaluate every article ever published to a specific topic and then tell someone “well the research suggests” when you clearly dont know the entire gamut.

Sorry if that turned into a rant, and this is in no way targeted to anyone just a perspective that i have.

2

This is a good topic. I’ll keep my perspective related to the health field, given my background. While it is difficult to remain abreast of all research developments in real time, we have the ability to relay what the current research evidence suggests by reviewing high quality studies pertaining to the discussion of interest. In science fields (I can’t speak for sociology or other similar fields) meta-analyses and systematic reviews reflect the highest quality of research evidence, whereas expert opinion reflects the lowest form of evidence. Hierarchy of Evidence - Evidence-Based Practice in Health - UC Library Guides at University of Canberra

It is true that personal biases may cloud one’s judgment, hence why expert opinion does not carry the same credibility of a high quality SR. This also comes down to one’s guiding principles when reviewing new information that may contradict one’s preconceived notions and force one to exercise introspection when dealing with conflicting information (i.e. willingness to align oneself with the information synthesized and presented in high quality studies VS negating or ignoring high quality evidence in favour of personal beliefs, a penchant for ignorance, secondary gain, etc).

At the same time, as you stated, simply claiming to be evidence based is not enough, as appraisal of the evidence to ensure that a study’s findings are legitimate and reliable is of paramount importance, especially when offering an informed opinion or recommendation.

I am confident that your frustration will continue to grow the more you study as you will realize how much junk “evidence” exists in the literature and will develop a healthy level of skepticism as you calibrate your BS meter.

Being a part of this community as well as others, like Clinical Athlete, will curb some of your frustration, as it is refreshing to be able to discuss ideas and concepts with like-minded individuals who are motivated to grow their knowledge base.

1 Like
3

Also, even the highest hierarchy of evidence is only worth as much as you can explain the facts. Many things remain unexplained, because people think stating facts is enough to understand and use the information. Good Evidence is the basis but “What is it?” and “How does it work?” should be the questions in focus.

1 Like
4

Good topic. Something that I was actually having a discussion along those lines yesterday, though not necessarily about research specifically, but more along the lines of general knowledge. I’ve noticed the more that I learn (not just about health, fitness and nutrition, but also in my own career field of software engineering and any other topic of interest really) the more I realize that I don’t know. Even though I’m a lot smarter now than I used to be, I don’t feel as smart as I used to feel. It’s really easy to feel like you know everything when you’ve only read a book or two on a subject. It’s just deceptively enough info to feel that you have a firm conceptual grasp on a subject. It can deceive you into thinking that you know more than you actually do. Usually I’ve found that years later after I have a lot more experience, research, and time to question and dig through the layers of nuance on a particular subject that my views are very different from where they were at that initial stage when I thought I knew it all. I also find myself less sure on things. Where at the early stages of learning a topic I would find myself speaking more in absolutes on the subject, I found that the closer I came to mastering a subject matter that my phrasing in discussion would change to be a bit more unsure. Where before I would start a sentence off something like “I know” or “I read” as I progressed I found I changed my wording to be more along the lines of “I think” or “I suspect”. I used to think I was very smart, now I’m not so sure. I think a good part of that is also growing and maturing in general, as at 36 I’m a lot different than I was at 26. I approach life and research with a lot more skepticism than I once did. Gone are the days where I read something in a book and assume that because it was published that it must be true. Gone are the days when I feel that something must be true because some expert said it is true. The fitness and diet topics have especially shown me just how bad general information on the internet and popular culture are. I think that we all can benefit from a healthy dose of skepticism, even in regards to scientific research. Especially since most people just read the abstract, take it as the gospel, and call it a day. They don’t take the time to actually read the study, think long and hard about the methods involved, and try to find holes in the research. Scientific research is also changing all the time. It’s not uncommon for science to disprove itself over time. People also love to latch onto extremes, and usually the more extreme “science” is the more widely it spreads and ironically the more likely it is to be disproven in time. When I was a kid eggs were considered to be healthy for you. Then it was widely accepted that eggs were basically the devil for about 15-20 years there. Now they are considered healthy again, provided you eat them “in moderation”. “The literature” and “experts” can’t seem to make up their minds half of the time, and that makes it almost impossible for me to really feel sure about anything.

2 Likes
5

A few things on the phrase “evidence based.” As a PT, I hear it everywhere, and yes, it is overused.

If you want to understand what the evidence says talk with statisticians, biostatisticians, etc, not those doing the research or anyone that may have some disclosure, agenda, etc.

A statistician will look DEEEEEEP at the original research and give an objective critical appraisal of the methods and results. If the methods are flawed, they’ll know. If the methods are sound, but the results are “meh” they’ll tell you. They don’t have skin in most games, just a level of statistical understanding to give you the reality of the “evidence.”

As a clinician that actually appraises articles and hears “evidence based” daily my interpretation is typically the opposite of most.

Here is one way I see people “spin” results.

“The evidence shows treatment A is equally effective as treatment B for knee pain, therefore I use treatment A. I am evidence based”

My interpretation

“Treatment B is INEFFECTIVE for knee pain, so Treatment A is equally INEFFECTIVE as treatment B for knee pain.”

6

I would 100% agree with you that the more I learn the more I realize the more I do not know. Great perspectives everyone.

1 Like
7

You may want to look into the Dunning-Kruger effect

https://www.theifod.com/ignorance-and-the-dunning-kruger-effect/

Some in healthcare will use the saying “knows enough to be dangerous…” Typically means, they stand on Mount Stupid.

8

THAT was a great read. Thanks for the link.

9

I can see this as it may relate to, for example, a individual who cant sing a note and exists the stage genuinely flabbergasted that the American Idol judges chose not to send them through to the next round, but framing it in the context of healthcare professionals seems a little ironic, don’t you think? I would imagine (hope) such a person would only be confident issuing treatment / diagnosis within a very narrow band of their expertise.
Presumably you’re not talking about nephropathy practitioners here.

10

‘I think that this revised statement would be a better reply because to my naive mind its impossible to know what the entire field of research has to say about a specific topic without having a biased opinion and theres no realistic way to read every relevant study about the topic because I think it would be extremely difficult to evaluate every article ever published to a specific topic and then tell someone “well the research suggests” when you clearly dont know the entire gamut.”

Welcome to,the world of academia where you know everything about one thing. You can know and evaluate every study on a specific topic but that focus is super narrow so it’s probably not as much research as one would expect. Usually most people use systematic reviews as their go to as mentioned above. It’s an efficient way to have the poor studies weeded out (usually, there’s been recent talk about how many SR are crappy too). There is no best way, just doing the best you can. I do like your reframing to the research i’ve read says . . .

The other annoying piece piece is using might, maybe, possibly etc since most research is not a definitive answer. It’s hard to sound confident when you say this might help according to the research but it’s accurate.

1 Like
11

kelleyhenry,

Here is something I think describes what you’re talking about:

12

Awesome discussion everyone. Lots of great input already.

Dhruv, as some have already stated, yes we all have biases that influence our assessment and interpretation of data (this is the discussion and conclusion section of research articles). With that said, when recommending clinicians be evidence based the instruction is to assess the totality of evidence; which can be done by analyzing systematic reviews and meta analysis. We can have a discussion on the efficacy and utility of systematic reviews and meta-analysis as these were recently shared with me by @Derek_Miles (I’ve not read these yet):

Are systematic reviews and meta-analyses still useful research? No - https://link.springer.com/article/10…134-018-5066-3
Are systematic reviews and meta-analyses still useful research? We are not sure - https://link.springer.com/article/10…134-017-5039-y
Are systematic reviews and meta-analyses still useful research? Yes - https://link.springer.com/article/10…134-018-5102-3

We also have peer review in place to help mitigate bias - BUT we can also have a discussion on the flaws of this system.

In the end, as the psychologist Brain Nosek once said, “Science is wrong about everything, but you can trust it more than anything.” (on this podcast - https://youarenotsosmart.com/2017/07…cation-crisis/). Science, when done appropriately, is a self-correcting system. Despite our biases and the flaws of the system, the scientific method remains our best bet at gaining new perspective and minimizing our own shortcomings as humans. I also prefer the premise of not out to be right but rather to be less wrong each day. I know I’m already going to be wrong about the majority of things, the question becomes can I be less wrong. In my biased opinion this is the point of science, can we continue to gain perspective in our understanding of the world around us and the reality we find ourselves in an effort to be less wrong. Happy to discuss.

1 Like
13

I’ll add a few more things to this thread. (I’m a fan of reference people smarter than I am…)

Remember science is effective to “falsify” not “prove.”

From Nick Tuminello:
Responses to common arguments against taking a more science-based approach to training and nutrition: Post #2

Common argument: “It’s unrealistic to limit my programming to only published evidence”

RESPONSE:

Taking a more science-based approach to training does not mean that you won’t let your clients or athletes perform anything without a PubMed reference in hand – that’s a ridiculous straw man argument. As I said in my book Strength Training For Fat Loss:

“It’s great to use work-outs that have been evaluated in a study, but it’s unrealistic to ask that of every workout, especially when we’re changing workouts every few weeks to keep things fresh and interesting. Specific workouts don’t have to be scientifically studied as long as they are scientifically founded, meaning they are founded on the general principles that have been repeatedly shown to elicit the results you’re after.”

I think Dr. Jason Silvernail said it best:

“No one says you have to limit yourself to using just published evidence. You have to limit your claims to what you can defend with evidence, though.”

You see professionals like myself who promote a more science-based approach to training and nutrition are not discounting anecdotal claims as a whole, but discounting them in the context in which they are being provided. In that, what we take issue with is when people make objective (truth) claims based solely upon their subjective experiences. This is because the type of evidence they have provided (subjective) doesn’t warrant to the type of claim(s) they’ve made (objective).

Since ideas are only as good as the evidence that supports them, we should all want people to understand the difference between objective (i.e., matters of fact) and subjective reality (i.e., matters of personal taste), and seek to create a professional environment that demands intellectual honesty. There is no issue with people sharing their subjective experiences. Heck, I provide lots of anecdotal information in my articles, videos and workshops. Yet, none of my fellow promoters of a more science-based approach feel the need to “call me out” because I don’t give them a reason to do so.

Is it really too much to ask just be more humble and intellectually honest about the claims you make (and don’t make) by choosing your words carefully when communicating about your beliefs and practices, and avoid claiming to know things you don’t know?! This is all professionals who promote taking a more science-based approach to training, rehabilitation and nutrition keep trying to point out.

1 Like
14

As always, xkcd put it best.

purity.png
purity.png740×308 30.3 KB

2 Likes