Registered: 1189031616 Posts: 2,020
Reply with quote #1
Sometimes I feel like a lot of us feel overwhelmed, bewildered, or just plain turned off by the debates around EDs, Evidence Based Medicine, different therapies & treatments, and all the research and data we throw around this joint. In a good way! Even if sometimes our passions make the discussions contentious, *that* the discussions are happening is very valuable, IMHO. But I think they could be even *more* valuable.
What I'd like to do on this thread is give readers a practical tutorial with the following goals: On FEAST-ED, and here on the Forum, there is a lot of focus on Evidence Based Medicine and "The Research". Some might have a vision of a world where all ED research was well designed with clear outcomes that are plainly explained, and these would form a pool of data - evidence - that would lead to uniformly effective treatment. And that would be swell, but that is not the world we live in. Explain the basics of medical research; Explain the relationship between research and the practice of medicine; Give readers the tools to critique Mainstream Media (MSM) reporting of primary research; Give readers the tools to evaluate a primary research paper for their own understanding. The truth is that no research is perfect, some research is poorly designed or executed, and no treatment for any disease or condition is uniformly effective. We think of Medicine as a Science. It is NOT, it is an Art that is *informed* by science. And actually, it's a GOOD thing that Medicine does not rely 100% on science - it would be bad if a discipline as complicated and encompassing as Medicine had to wait for and rely on only one source of information. That said, the science behind Medicine is REALLY IMPORTANT! *************************** Step 1: Understanding Medical Research -Different Kinds of Research Give You Different Kinds of Information. The first thing is to understand that the "knowledge base" of medicine comes from many sources. Each of these has strengths and weaknesses. The FIRST thing to do when you are looking at a piece of research is to identify which category it falls under. Then you have an idea of what kind of information the study design is good at getting, and what the limitations are. This is by no means exhaustive, but it's a good starting point : Primary bench science: this would be the kind of experiments NOT run on people at all, but more like the stuff most of us did in biology or chemistry in school. This would include experiments on animals. Pros: Valuable for understanding basic mechanisms, lots of control over conditions, you can easily repeat what was done. Cons: hard to know how well what happens in a lab will match what happens in a real live human being, lab conditions do not replicate real world conditions very well. Case reports details a single patient or small series of patients. These describe what the patient(s) showed up complaining of, their medical history, physical exam, symptoms, the treatment(s) administered and the outcome of the patient. Prior to formally applying the Scientific Method to medical research, Case reports comprised the bulk of medical knowledge. They are still extremely valuable for detailing rare diseases and conditions,novel treatments, unusual outcomes, or other anomolous or singular patient care events. Pros: Pretty much the only way to study rare or singular events, good for descriptive purposes. Cons: very small numbers, not repeatable, hard to know how generalizable (how applicable to another person or situation) the results or findings are. Observational studies which involve no "experimental" manipulation of patients in any way, but rely on identifying a set of patients with some shared quality - a diagnosis, an exposure, a treatment, an outcome - and either looking back in time to see what happened to them before the diagnosis/treatment/outcome of interested (in a retrospective study design) or forward to see what happens AFTER your event of interest (a prospective design). The important part here is that you aren't changing anything that happened or will happen to the people you are studying. Pros: Inexpensive to do, easier to get large numbers, the study is already looking at "real life" situations, low risks to study participants. Cons: Can't control all of the conditions, can only show relationships but CANNOT show which thing causes what, can be hard to replicate. Randomized, controlled, clinical trials which involve carefully selecting people to get a treatment of your specific design. RCTs involve active participation from the subjects because you are deliberately manipulating what happens to them. Also very important: The treatments administered in RCTs have the PRIMARY GOAL of benefiting OTHER people! Now, the risks to the patient have to be acceptable - you can't do something worse to a person than what would happen if they *weren't* in your study. But since you're conducting an experiment, you don't KNOW if you're doing anything better, so really, the people who get the MOST benefit from an RCT are the people who get treatments that come AFTER the results of the study are known. RCTs are called the "Gold Standard of Proof". What that means is that RCTs are the only experimental design that gives you live people getting real treatments or exposures in such a way that you can *show* that what you did CAUSED the findings you observed. Pros: You can control conditions and participants so you don't have contamination from unknown sources. You can "prove" causality. Easy to duplicate Cons: Expensive, study participation limits can limit generalizability. BUT! Not every problem can be investigated with an RCT. And you DON"T need an RCT if the preponderance of evidence reaches a certain tipping point. Real life example: - Smoking has never been "proven" to cause lung cancer in humans. This is because it would be unethical to design a study where you make one group smoke. The preponderance of the evidence from primary bench science, animal studies, and observational human studies is that smoking *does* cause lung cancer. This did not stop scientists in the employ of tobacco companies from loudly proclaiming that the connection wasn't "proven" for decades. They were technically right, but ethically reprehensible. The other issue is that RCTs participants are often very unlike the larger group of people who would get the treatment, and that the treatment is administered under ideal, and not real world conditions. Because RCTs can be very small in terms of numbers of participants, it can be hard to get detailed data about subgroups of patients - men vs. women, responders vs. not responders, that kind of thing. So, take home of part 1: When reading a report about research, look for what kind of study it was. Think about the plusses and minuses of that kind of study. Then look at the results being reported and see what kind of "yeah but" (or caveat) might apply. __________________ IrishUp
Registered: 1235803307 Posts: 2,852
Reply with quote #2
Thank you, Irish. Great post! This series of yours will provide very valuable information for those of us who read ATDT.
__________________ Daughter age 28, restrictive anorexia (RAN) age 11-18, then alternating RAN with binge eating disorder and bulimia with laxatives, is in remission from EDs for 3 years after finally finding effective individual therapy. Treatment continues for comorbid disorders of anxiety, ADD and depression. "Perseverance, secret of all triumphs." Victor Hugo
Registered: 1284535839 Posts: 3,152
Reply with quote #3
Irish great information but I would also add about blinding, because Double blind randomised controlled trials are the gold standard, ordinary RCT without blinding are considered substandard. Depending on what is being studied blinding can be incredibly important. The use of placebos and blinding have changed the interpretations of many RCTs where they were not used. Perhaps you were leaving this for another time. Not wanting to whinge because I think you have really expressed this information pretty well.
__________________ D diagnosed restrictive AN June 2010 age 13.5. Weight restored July 2012. Relapse and now clawing our way back. Treatment: multiple hospitalisations and individual and family therapy.
Registered: 1267616328 Posts: 2,682
Reply with quote #4
Irish, are studies carried out with special interest in outcome skewed either by underlying beliefs of those administering or designing the study or even those funding the study.
I've seen studies done that so state the obvious in there hypothesis that I wonder why it was done in the first place. I really value the science behind research but I do believe it has its limits. It's seems like applying information to our existing belief system can be a huge variable. Sometimes most effective and appropriate and sometimes grossly distorted to fit what we already belief. Does that make sense?
Thanks kiddo for breaking down how research works. __________________ "Sometimes you just have to be your own hero"
Registered: 1185895491 Posts: 3,340
Reply with quote #5
Thank you Irish - I love this post.
Maybe we can add more specific information relating to the science of eating disorders as we go on and as anyone has it. It's interesting about blinding. As you say Foodsupport blinding may be extremely relevant in some studies, but is surely impossible in psychological treatments? My particular interest (apart from that of being a parent and wanting to know the best way to help my daughter which is the primary reason why I'm here) is in the history of medicine. Until recently case reports and observational studies were the gold standard in medicine (not just psychiatry, all medicine). Some people, convinced by decades of experience and being held up as the experts in the field, just don't understand why their ideas, underpinned as they are by all that observation, are being challenged. If their response is hostility and bluster then that's sad, and not going to further anyone's cause. On the other hand if the response of their critics is to dismiss all that evidence out of hand then the whole field will have lost out enormously and the whole field includes your child and mine. __________________ Fiona Marcella UK
Registered: 1306246748 Posts: 634
Reply with quote #6
Would you be interested in also doing these as guest posts on ED Bites? __________________ Carrie Arnold Blogger and author at http://www.edbites.com
Registered: 1331162725 Posts: 523
Reply with quote #7
Piling on- great idea and great post!
Irish, I think you need to stress the importance of RANDOMIZATION. What randomization does is control for variables you haven't thought of or don't even know about. Randomly assigning subjects to treatment groups means that important variables that could impact the study will end up in all comparison groups equally. Effectiveness of randomization needs to be checked, and it is highly dependent on sample size. In a small study you can easily find things that pop out as different in the groups despite the randomization. For example, if a study has an age range for inclusion, you can check and make sure that the age distribution in the 2 groups is the same after the study has been recruited. This is the hardest part about recruiting a trial for medical care. Most people want to have the medical treatment that their Dr. thinks is best for them individually. In an RCT you will be randomized to treatment. Randomizing to a placebo group is even harder, and here's where blinding is essential because otherwise subjects who end up in the placebo group tend to drop out. Blinding is also important, but as someone mentioned, can be difficult. Not impossible, though. In some cases, subjects can be blinded to the type of therapy they are receiving if they don't know much about it (don't know the difference between CBT and ACT). Also, the experimenters should make sure whoever is measuring the outcome (for example, administering the psychological questionaire) is different from the person who supplied the therapy, and is not told which arm the subject was on. Double blinding (means Drs are also blinded)- impossible for therapy, standard of practice (or should be) for RCT drug trials. __________________ Mom to recovered RAN daughter, now age 18
Registered: 1189031616 Posts: 2,020
Reply with quote #8
Firstly, let me apologize for the delay. My internet has been down for most of the day.
Second, I'm very glad that people are finding this helpful! I had organized this in my mind as a series. The 101 level posts first, followed by deeper posts on each topic. For sure, I have in my head further posts on each kind of research, using specific landmark papers from the ED literature, and it looks like those would be welcome. Third, specifically for Marcella, Strongmom, Ern & Foodsupport; absolutely important stuff, and I'm glad you each got into more detail! I was hoping that other knowledgeable people (including other pros!) would add. No one post can cover everything, and I am grateful for your input. I look forward to delving deeper into these important issues, but I believe an overview of the basics was needed first. Interested readers can look forward to posts with deeper discussions. Carrie - absolutely; do you want to link here, or should we cross-post? The same goes for anyone else in the blogosphere. __________________ IrishUp
Registered: 1276303355 Posts: 1,395
Reply with quote #9
Thank you so much for posting this. I am going to read and reread. I have forgotten much of what I learned years ago.
Registered: 1189031616 Posts: 2,020
Reply with quote #10
Carrie has cross-posted the initial post, and some of the contributions from this thread over at ED Bites (which is now located at http://edbites.com, if people need to update their bookmarks).
A commenter there noted that (and I'll just excerpt) " It seems that the main stream media has a habit of always sensationalising outcomes of any research or trial involving unusual outcomes or which diapproves traditional thinking leading to un-ncessary confusion in minds of uninformed readers of the publication." Which is a valid point; part of the problem we are dealing with is a main-stream media (MSM) that has a set of priorities which may NOT include "scientific accuracy". With Carrie's permission, here is my reply:
The misrepresentation and sensasionalistic reporting approach in MSM is certainly a problem. While I wouldn’t deny that certain media outlets have an obvious bias or agenda, when it comes down to it, even the science journalists are often not very science literate. In the modern era, the science reporter is not necessarily chosen for their science “chops” or journalistic craft, and the expected turn-around time from story idea to airing/publication is a fraction of what it once was – leaving little-to-no-time for the reporter to investigate their topic. Reporters themselves are often writing in ignorance of these same issues, at the same time they have a short deadline, and an Editor or Producer who is shaping the “angle” of the story.
So I agree that it is a situation conducive to creating confusion that needn’t be.
That said, there is more primary source data available to the general public than ever before. In the US, research with federal funds MUST be publicly available (see
here *) and the rules of citation and attribution are still in place. Many people won’t look at these primary sources for fear that it will be too confusing and they won’t understand it.
This series is intended to teach people about research and how research is presented, so that they WON’T be confused and WILL understand enough of it to make up their own minds, without having a staff of people in the hire of an MSM company do that for them. Underneath all the jargon and Latin words, a research publication is a very basic story of who? why? when, where & how? and what?
Who was studied? Who did the study?
Why is this study important? Why do we care about the research question and any answers this study found? When & where was the study done? How was the study conducted? What were the results of the study? What do those results mean to ME, who is interested in this topic? This is information that a regular person can get, once they know how to do it. *The hyperlink goes to http://nihpublications.od.nih.gov/ . __________________ IrishUp
Registered: 1210867727 Posts: 5,573
Reply with quote #11
No pressure, Irish, but sometime I hope you'll continue this thread. I LOVED this. I think it's so important for us to be able to look critically at the studies that have been done. I want to hear more about sample size, statistics, etc. Whatever you want to tell us. I'm sitting at your feet waiting for crumbs of knowledge to fall.
__________________ Colleen in the great Pacific Northwest, USA "What some call health, if purchased by perpetual anxiety about diet, isn't much better than tedious disease." Alexander Pope, 1688-1744
Registered: 1196266882 Posts: 1,267
Reply with quote #12
Nice post Irish Up! Miss seeing all of you.
One of the previous posters asked if special interests, monies, political clout mattered in the execution of clinical trials and it most certainly does. Also, in the ED field there seems to be a tendency to stop looking once a certain clinical trail has been done...this is not what real science does...it keeps on going. My PhD was in biochemistry and I saw all the time that scientists would discover something about a molecule and then all those around them would then postulate another function, configuration or association for that molecule. In the the ED field...just because someone does one or two clinical trials and then develops a protocol and manual from that for treatment should not mean exploration of the original research ought to stop. Not at all! It means we need to keep going and find out the details of what worked in that original research and continue to apply the essential elements to other protocols.