Evidence-Based Practice in Children's Mental Health
 Within the field of children’s mental health, there is growing attention to the concept of evidence-based practices or EBPs. Increasingly, policy and legislation at the local, state, and national levels call for increasing the proportion of funds for mental health care that are expended on EBPs.
While the definition of what constitutes an EBP may vary from one context to another, the movement toward EBPs is based on the shared idea that decisions about behavioral healthcare should reflect scientifically obtained knowledge about the effectiveness of a treatment or service. This seems to be a reasonable proposition, especially given that many popular treatment and practice approaches currently used in children’s mental health have little or no evidence in support of their effectiveness. What is more, many of these approaches are very expensive. The turn towards EBPs is thus often portrayed as an opportunity to improve the quality and decrease the costs of treatment and services. Such an opportunity is particularly attractive in a context of growing demand and shrinking resources for care for children with emotional and behavioral difficulties and their families.
Few would argue with the goal of providing more effective, and cost effective, services to children and families. One of the key challenges in defining EBP, however, revolves around the issue of how “evidence” is defined. Beyond this, there are further reasons to be cautious in moving towards evidence-based mental healthcare. These reasons include the following:
- The number of practices for which there is a substantial evidence base is relatively small, and EBPs are not available for all problems and needs.
- The research that forms the evidence base for a given practice may well have occurred in contexts that do not reflect the diversity and complexity of real-life clinic and community settings.
- The focus on EBPs could undermine innovation in treatment and service approaches, particularly with regard to approaches that are culturally driven and/or developed in community settings.
- The focus on EBPs may also conflict with family and consumer choice about which treatment or service approaches to pursue.
The National Technical Assistance Center for Children’s Mental Health recently produced an issue of their newsletter, Data Matters, focusing on evidence-based practices. This issue explores the issues bulleted above, as well as other perspectives that reflect optimism as well as concern about the momentum of the movement toward EBPs.
What do you think about the evidence-based practice “revolution” in mental health care? We invite you to post your opinions, observations, and thoughts on this topic.
We look forward to hearing from you.
Janet S. Walker,
Web editor
Your thoughts…
Comments:
 I think this may be a down side to the whole EBP issue, or maybe it's just a big, underlying irony: we pride ourselves on the accumulation of evidence that our practices to "fix" people are improving (EBPs are now all the rage across human services), yet at the same time, our society by many measures is becoming more unequal, more unjust, and more oppressive to those at the bottom of the economic heap. We multiply the risk and then congratulate ourselves for being able to "fix" some of the people who act in ways that are bothersome. Reducing income inequality and providing opportunities for people to make a living wage are the ultimate evidence-based practices. Posted Wednesday, September 22, 2004 at 07:51 AM
The last posting describes my concerns quite well. The problem with EBP is that psychotherapy is not an experimental variable. It cannot be such because of its, contingent, unstandardized, idiosyncratic nature. One of the truths that most psychotherapists subscribe to is that the relationship between the therapist and the patient is crucial. This can't be standardized or manualized. How do you subject play therapy to the kind of study required to endorse it as EBP? Yet, my experience tells me that play therapy is very valuable as a therapeutic intervention with certain children and adolescents. I, too am concerned about charlatans and inflated testimonials that aren't supported by scientific rigor. But an excessive dependence on EBP will hurt both therapists and patients alike. Posted Wednesday, March 3, 2004 by Al Galves, Las Cruces, NM at 02:24 PM
Now that is a scary idea (for-profit takeover of MH research and development), and an aspect of EBPs that I have not seen mentioned before. Why not? Is there some reason why this sort of thing is unlikely? Posted Thursday, February 5, 2004 at 08:53 AM
I am not so sure how directly the situation here parallels that in education, but certain questions are the same-- in particular I wonder about how economic incentives to develop evidence will figure into all fo this, i.e. just as educational research on curriculum is increasingly being done by the companies that manufacture textbooks and other media, will there be incentive in mental health for for-profit development of proprietary interventions with associated ongoing fixed costs? WIll future research be funded by the developers of new EBPs? We have seen within medical research that the privatization of evidence gathering has led to questionable practices (see the earlier discussion on medications) like repressing research unfavorable to the company's products, inappropriate reporting of results (ie inflating what the evidence states), basing research on phony comparisons, etc. Posted Wednesday, February 4, 2004 at 09:56 AM
I share the concern mentioned in a previous post regarding how mandating that intreventions be based in "scientific evidence" can contribute to a distortion of practice. I too have been watching what has happened with a parallel process that has been going on in education at the federal level, with the No Child Left Behind act requiring schools to adopt reading programs that are "scientifically supported." While the original legislation seemed to make it clear that any approach with requisite evidence could be used, an appendix to the legislation contained a list of approved curricula, almost all of which focus narrowly on phonics (and which, coincidentally, are often the property of well-heeled corporations) and approaches that are highly teacher-driven (as opposed to student-driven, constructivist approaches). Now, a number of states are even considering rejecting the federal money associated with NCLB so that they can build reading programs that include vocabulary building, reading comprehension and other approaches. Posted Tuesday, February 3, 2004 at 09:13 AM
This is a topic that will be explored at the 3rd annual 2004 CHARPP Real Data in Real Time conference in Portland Oregon on September 13 & 14. The conference, entitled "Evidence Based Practice: Bringing the Beauty Out of the Beast" seeks to identify and develop requisite skills for developing EBPs for children's behavioral healthcare; promote multi-faceted discussions of EBPs and their current and future status in the field; present current applications and findings regarding EBPs; and examine various measurement tools and their applicability to EBP. If you would like to download a Call for Papers to submit a presentation for this conference, or view more information about this conference or review the proceedings from last year's conference ("Best In Show: Evidence Based Practice in Services to Children and Families"--keynote by Kimberly Hoagwood, PhD and welcome by former Oregon Governor, the Hon. John A Kitzhaber, MD) please go to www.CHARPP.org. Posted Friday, January 30, 2004 at 10:19 AM
Given the current state of the field of behavioral healthcare for children and adolescents, I think EBP is really in the early development state. I also think that rigid State imposed mandates requiring the use of EBP at this time may simply result in fewer services for children because there are very few established EBPs identified and even those are "proven" for a very specific population. One concern I have is that EBPs get overly promoted as appropriate for populations that haven't been evaluated for that specific EBP. Additionally, some EBPs have been developed in a "sterile" environment that excludes factors that quite simply are reality in the everyday delivery of services to children and their families. At the same time there are behavioral healthcare practices that appear to be effective, though there isn't yet any formal evidence that clearly identifies exactly what it is about that practice that yields the positive outcome. I think that the children's behavioral healthcare field needs to step up and develop practice based evidence so that evidence based practices can be identified and shared. I think it is much too early to strictly mandate the usage of evidence based practice, but I think the timing is excellent to mandate the development of practice based evidence as the necessary catalyst for expanding the body of evidence based practice. Once the body of EBP is sufficiently developed to meet the range of children that need behavioral healthcare services, then mandate the use of EBP along with the development of EBP via practice based evidence. Posted Friday, January 30, 2004 by Rich Blum CIO Trillium Family at 09:58 AM
I have to say thanks to all the people who have raised such important issues here. It has contributed a lot to my education about this topic. Keep it up! Posted Thursday, January 29, 2004 at 01:14 PM
I appreciate this thoughtful discussion. I have been troubled by the dilemmas involved for some time. On the one hand who can be against a progressive move to more and more scientific validation and guidance to what we do in clinical practice. But the cautions in the set up to this discussion are all very valid. I would like to add some more.
Often, on top of some good science in experimental design, there is very poor science in interpretation. All the cautions and disclaimers don't undo the clear implications that a statistical trend is taken as "THE TRUTH" when it isn't. Many of us are not statisticians with the dry and steely cognitive skills to interpret with caution. Many of us simply can't understand the arcane world of statisical analysis or even evaluate an experimental design. There is a trend to climb on board by many adminstrators, clinicians and even researchers and trumpet "evidence" that could and should be challenged. To challenge is unsafe and can make one very unpopular.
The question on what is relevant "evidence" for a particular clinician with a particular kid in a particularly dicey situation is the real question. For me the interesting question is to look at the research and see what is factored in or what is factored out of the study design. Very few studies factor in to the research design the nature of the treatment relationship between the subjects and the researcher; is it only a one time encounter with a research assistant doint a structured interview who is a total stranger carrying a clip board? Or is it a long term emotionally close and very positive mentor like relationship? Marsha Linehan with DBT and Howard Liddle of MDFT fame are amoung the few researchers I know who factor this important issue into their research as a critical variable.
Another factor: Even in multisite blind controlled drug studies which should be the easiest, most straight forward experiments to do the subjects are mostly from academic communities, or areas accessable to major universities where academics regularly go: already a bias is set up.
And they come up with typical results: 30% placebo respond, 70% investigated medication respond. This is a statistical significant response we are told. When we read the results we act as though the statistical norm is like a person. We humanize the results and presume that most folks walking around are reducable to the variables controled in the study and that they will behave like the mean! I am more interested in the 60% outliers then the 40% we presume are actually helped by the new medication or the new technique. My particular patient is likely to be in that 60% group. So I don't really think I have evidence that proves much of anything.
I also have questions like what are the exclusion criteria and how does this apply to my patient. How long were the cases followed. A two year follow up helps me a lot but many are only 3 or 6 months and are not very helpful to me as I see kids over many years.
Statistics, which I am amittedly a novice at - being a non-researcher, seem to be subject to great manipulation. But at its best it identifies curious trends, not facts. Those trends are subject to interpretation. The more one knows about the non-controlled variables in the lives of actual people, even those who volunteer as research subjects, the more those non-controlled variables seem to weigh in as critical. Yet researchers often are dismissed those variables as if they wash out of importance if the "n" is big enough. I doubt that this is true.
When I am sitting with a teenager I am treating and I look at them closely with weeks of experience listening to them, knowing the particulars of their and their families lives, and I am awash with all the complexities of their lives, the less my comfort is with a simple diagnosis. They are complex. Their diagnosis doesn't seem to be the key to understanding them, even if it s a clear one like OCD or PTSD with dramatic and signficant symptoms. Some of us clinicians have a joke that if you ever find an individual with a pure, uncomplicated depression you MUST send them quick to a university because that is the only place they are found.
My patients are so much more then their diagnosis. Yet most of the research I read is based on diagnosis, or, if not, on vague and meaningless terms such as Seriously Emotionally Disturbed. When I look over my case load an try to formulate the problems before me, I am impressed that the heterogeneity of etiology is astounding and baffling. The elegant notion of the Stress/Diathesis hypothesis is in my mind constantly. Yet we don't think etiologically in our mental health world. We are off on a wild goose chase of phenomenology just like Ptolemy! This perspective assures for me that "evidence" that I am supposed to use to decide on a treatment seems grossly inadequate when I try to apply it.
This in no way means that I am against having "evidenced base" options. I am greatful for them. I am trained as a DBT practioner. I have become familiar with CBT and Exposure therapy techniques. They add to my tool box. I am greatful for the pharmaceuticals that are available to me. I strongly support the wise application of all researched treatment. I can use them carefully and to good effect. But this not because my patients are like those imaginary statistical creations. They are most definitely not.
Marcia Goin, current president of the American Psychiatric Association in a newsletter article, and an elegant clinician, talks about being "guided by" EBP's and the research behind them. That seems OK to me. I do find the evidence useful sometimes as a guide. But heaven forbid we get locked into a system that demands "evidence" for funding. Nothing about EBP's seems to me to be really proved due to supposed evidence. We have some guidance from current research. Some specific techniques and medications are validated by SOME evidence and these are proving useful if applied with art and skill. But most clinicians know that 90% of what we do has little evidence to back it up. There IS NO evidence that applies most of the time. Just because there is no evidence does NOT mean a therapeutic effort is not worthwhile, just as the lack of evidence means we cannot know if it is worthwhile. Lack of evidence proves nothing. Many act as if it does. Administrators trying to justify cut backs in MH care can take that leap and refuse to fund work we do due to lack of evidence. This is as illogical and destructive as clinicians who promote silly and grossly ineffective treatment that defies common sense. Clinicians are left to do the best we can, which includes tapping into the "interesting" results of current research and taking it with a grain of salt. Posted Thursday, January 29, 2004 by Charley Huffine, MD at 10:22 AM
One of the previous comments says that "the work is as much art as science." On the one hand, this is undeniably true. On the other hand, I don't think this precludes holding some evidence based (or at least evidence-inspired) standard to the practice. COnsider the example of psychotherapy for children. There is a good case to be made that, while psychotherapy works in university settings where it is developed, it is much less effective, possibly ineffective, in "normal" community settings (where it forms the basis of the preponderance of plans of care!). At the same time, there is some evidence that psychotherapy with children is more likely to be effective when the therapist successfully establishes a therapeutic alliance with the caregiver(s). SUppose therapists' contracts were predicated on their ability to consistently develop such alliance. Such a stipulation has the potential to bridge the gap between excessive rigidity (ie attempts to over-specify practice and thereby eliminating the "art") and excessive flexibility (i.e. therapists' rationalizing whatever treatment practices they feel most comfortable with or like best as "art") by using standards of evidence. Posted Wednesday, January 28, 2004 at 07:32 PM
Another issue that seems, in keeping with the posts made so far, to show up the general theme that there's potentially an upside and/or a downside to the movement toward EBPs: As mentioned in the original piece, there is not a clear definition of what "evidence" is, and I can see how this can lead to some difficulties. For example, in our state, there is legislation that specifies that in 2 years, 50% of MH money will be expended on evidence based practices, and in 4 years, 75%. One concern is the potential problem of there not being enough EBPs to meet half the demand for services/supports; but another issue is what that pressure to be at least called an EBP will lead to. For example, there is some evidence for the effectiveness of mentoring; however in my work I have encountered many communities who use mentoring programs that do not appear to have any of the characteristics that are thought to lead to beneficial outcomes. Posted Wednesday, January 28, 2004 at 07:22 PM
I prefer to think about this "revolution" as a "paradigm shift" that puts "evidence" on the table and into the hands of families and the service professionals and other who are their advisors. Discussion about the treatements, services, and supports to be provided for a child, must inclue data about these interventions. That data could come from many sources starting with the results of "scientific" studies. It could include baseline and follow-up data for children who received this treatment. It could also include the views of other families and youth who experienced the intervention. Families have a critical role in making decisions about the mental, physical, and behaivoral health care their children receive. The better the information families are given about the potential impact and effects (good and bad) of all the options, the better their choices will be. Posted Wednesday, January 28, 2004 by Trina W. Osher, Federation of at 10:37 AM
Clearly some behavioral health practices and techniques are more effective than others. I think, for example, of Motivational Interviewing as a demonstrably successful wayto address ambivilence and effect change. But I am skeptical of the bandwagon movement toward EBP's. There is no magic wand, the holy grail remains hidden, the work will always be as much art as science. Posted Wednesday, January 28, 2004 by fkennedy@soastc.org at 10:24 AM
I am curious about the individualized approaches that Pam Kay mentions in her comment. Can you give some more information about what they are and how their effectiveness has been measured (and how they were determined to be more individualized that the treatment or services received by the matched pairs)? Thanks!! Posted Wednesday, January 28, 2004 at 08:21 AM
I share your concerns about the potential for unintended outcomes in the scramble to adopt EBPs, especially where this practice becomes heavy-handed, or limiting to family choice. However, I disagree with the premise behind your last statement, that flexible and individualized approaches are difficult to administer and evaluate quantitatively. A good research or evaluation design can easily demonstrate and measure the effects of individualized practices with a matched pairs strategy. As a matter of fact, I take great pleasure in proving their effectiveness! Posted Tuesday, January 27, 2004 by Pam Kay, University of Vermont at 04:27 PM
Personally I am quite split about whether this is a good or a scary phenomenon. Certainly I am in favor of abandoning approaches that are ineffective and even harmful, not to mention expensive; and substituting for those services, supports, and service delivery methods that seem to produce outcomes. I do have numerous fears as well, however, that focus on evidence will mean a sort of unfair advantage for approaches that are easy to measure (Think of phonics as the equivalent in reading. Easy to "administer" and check, yet impoverished in terms of helping with deep understanding of written material and antagonistic to development of internalized interest in reading) and worry that the field will be distorted in favor of approaches that are more rigid (yet therefore easier to define and easier to check adherence) over those that are more flexible and individualized. Posted Monday, January 26, 2004 at 11:48 AM
Post a Comment:
Return to
Current Featured Discussions Page
Top |
