 Family
Participation
in Evaluation:
Training Needs
In recent years there has been a growing movement to involve family
members in research and in the evaluation of programs designed to
serve their families. The National Institute of Disability and Rehabilitation
Research encourages consumer and family participation in research
and evaluation. A requirement for systems of care funded by the
Community Mental Health Services for Children and their Families
Program is that family members be involved in all aspects of the
planning, delivery, and evaluation of services. Family and consumer
participation in research and evaluation are also promoted by The
Surgeon General’s Report on Mental Health and the report of
The President’s New Freedom Commission on Mental Health. These
initiatives are part of a general trend toward increased participation
of consumers and other stakeholders in evaluation. Consequently,
family members are now working on evaluation teams in all regions
of the nation.
Several researchers and writers have pointed to the benefits of
community member involvement in research and evaluation. Potential
benefits include selection of more relevant research questions,
increased cultural competence, better retention of study participants,
improved interpretation of findings, and wider dissemination and
utilization of findings. Increased family participation has also
uncovered the need to train family members to participate in evaluation
teams and to train evaluators to collaborate effectively with family
members.
To meet families’ needs for information about research and
evaluation, the Federation of Families for Children’s Mental
Health developed a three-course training program. This training
program teaches family members the basics of evaluation, prepares
family members to participate in evaluation teams, and trains them
to conduct their own evaluations. (The RTC has worked with the Federation
to evaluate this training program, and the report, Evaluation of
Course One, How to Understand Evaluation, can be downloaded from
our website, http://www.rtc.pdx.edu/pgProjEvaluators.php)
While there is a growing literature on the benefits of participatory
approaches to evaluation, there is little information available
about training to prepare evaluators to work with family members,
consumers, or other community members. An ongoing project at the
Research and Training Center is examining evaluators’ and
family evaluators’ perspectives on their collaborative work
and developing training materials for evaluators. Few of the evaluators
who were interviewed for the study reported that they had received
training on how to collaborate with family members and other stakeholders.
Most evaluators in the study indicated that they had learned on
the job to work with family members, a process which involved learning
by doing and learning from their mistakes. Participants in the study
described challenges they had faced in their work and strategies
they had used to improve their collaboration. The findings of the
study are guiding the development of training materials for evaluators
and family members. The training will contain guidelines and tips
for evaluators and family members working together on evaluation
teams.
We are eager to learn from researchers, evaluators, and family
members who have worked on research or evaluation teams about your
experiences. Specifically, we would like to hear your perspectives
on:
- The benefits and any disadvantages of family participation in
evaluation
- Challenging situations that may arise in a collaborative evaluation
team and how these challenges may be resolved
- Training opportunities that can help prepare families and evaluators
for this type of collaborative work, and
- College or university evaluation courses that teach skills
that are important for collaboration, especially collaboration
between evaluators and family members, community members, and/or
consumers.
In the training materials we are developing, we plan to use case
studies to illustrate the challenges facing collaborative evaluation
teams and effective strategies for resolving them. If you are willing
to contribute your story for use in the training, please contact
Kathryn Schutte at kmschutt@pdx.edu.
We look forward to hearing from you.
Janet S. Walker,
Web editor
Your thoughts…
Comments:
 "Doing justice" to a perspective doesn't mean falsifying results or bending interpretations, but it does mean allowing diverse perspectives to shape research questions, methods, and strategies-- all of these shape findings in ways that often have nothing to do with objectivity. Posted Tuesday, February 24, 2004 at 12:07 PM
I'm not sure how we can 'do justice' to perspectives that reject data that don't support their position. I've had people tell me that I should get different statistical tests so that the data will be significant- as if there were dozens of correct ways to analyze data, all yielding different results, to be chosen from depending on how you want the analysis to turn out. How can we 'partnership' research integrity, if one party simply wants the data to prove his/her point? And can we do justice to the position that the data mean whatever it is we want it to mean?
Not everyone joins into a research process for the purpose of finding something out. Some folks just want proof for positions they already hold. Posted Monday, February 23, 2004 at 12:08 PM
I have sometimes heard the criticism that family members may not be objective or scientific enough as they participate in data gathering, analysis, and interpretation. The idea is that family members will have an agenda that undermines their ability to be dispassionate, or that even trumps their desire to be so. On the other hand, we have researchers whom we might perceive to be more objective, yet in meta-analytical studies of interventions, it turns out that a major predictor of treatment effectiveness is often whether the study was done by the person who created the treatment approach. Now maybe this has to do with the fact that replications are not as good as the original-- on the other hand, who is more motivated than the designer of treatment approach to see that approach be successful? We all need to pay attention to the possibility of bias. Posted Thursday, February 19, 2004 at 09:12 AM
Partnerships in research and evaluation--regardless of who the partners are--can be associated with a whole set of issues particular to the sharing of responsibility for and products from the research. For example, who has ultimate responsibility for research integrity? Who has final decisionmaking power over how the results will be reported and the products that come out of the process? While these issues could easily come up between any co-investigators, they may be more likely to emerge or to be problematic when the partners come from different backgrounds and see the research as a tool to serve different sorts of goals. I think we need models for how to talk about these sorts of issues in ways which do justice to each perspective, and also examples of successful partnerships and strategies for achieving that. Posted Wednesday, February 18, 2004 at 01:08 PM
Post a Comment:
Return
to Current Featured Discussions Page
Top |
