social work practice is usually embedded in programs
DQ
Dudley (2009) points out that social work practice is usually embedded in programs. While you looked at practice evaluation using single-subject design in Week 3, this week, you shift focus to program evaluation. Program evaluation serves many purposes, including accountability to funders and to the public. Often, funding sources such as government agencies or private foundations requires periodic program evaluations. These evaluations can help provide answers to many different questions and can contribute to improvement of services. There are a variety of program evaluation models that are appropriate for addressing different questions as well as facilitating the collection and analysis of many different types of data.
Required Reading
1. Document: Logan, T. K., & Royse, D. (2010). Chapter 13 Program evaluation studies. In B. Thyer (Ed.), The handbook of social work research methods (2nd ed., pp. 221–240). Thousand Oaks, CA: Sage. (PDF)
2. Dudley, J. R. (2014). Social work evaluation: Enhancing what we do. (2nd ed.) Chicago, IL: Lyceum Books.
· Chapter 1, “Evaluation and Social Work: Making the Connection” (pp. 1–26)
· Chapter 4, “Common Types of Evaluations” (pp. 71-89)
· Chapter 5, “Focusing an Evaluation” (pp. 90-105)
NOTE: You are going to have to pick a program within an agency that you are familiar with or locate one if you are not familiar with.
To prepare for this Discussion, identify a program within an agency with which you are familiar, which could benefit from process evaluation and outcome evaluation. You do not need to identify the agency in your post. Also, review the different evaluation models highlighted in this week’s resources (needs assessment, program monitoring, client satisfaction study, outcome evaluation, or cost benefit study).
Post a brief summary 1 page of the program that you selected. Recommend a program evaluation model that would answer a question relevant to the program. Explain the potential benefits of the program evaluation that you proposed (both process and outcome). Identify 2–3 concerns that stakeholders might have about your proposed evaluation and how you would address those concerns. Then explain 2–3 concerns that stakeholders may have about your proposed program evaluation and how you would address those concerns.
Use APA Format and intext citations and references from both required reading and also creditable websites outside required reading.
Program Evaluation Studies
TK Logan and David Royse
Pg 221
A variety of programs have been developed to address social problems such as drug addiction, homelessness, child abuse, domestic violence, illiteracy, and poverty. The goals of these programs may include directly addressing the problem origin or moderating the effects of these problems on individuals, families, and communities. Sometimes programs are developed to prevent something from happening such as drug use, sexual assault, or crime. These kinds of problems and programs to help people are often what all acts many social workers to the profession; we want to be part of the mechanism through which society provides assistance to those most in need. Despite low wages, bureaucratic red tape, and routinely uncooperative clients, we tirelessly provide services that are invaluable but also at various Limes may be or become insufficient or inappropriate. But without conducting evaluation, we do not know whether our programs are helping or hurting, that is, whether they only postpone the hunt for real solutions or truly construct new futures for our clients. This chapter provides an overview of program evaluation in general and outlines the primary considerations in designing program evaluations. Evaluation can be done informally or formally. We are constantly, as consumers, informally evaluating products, services, and information. For example, we may choose not to return to a store or an agency again if we did not evaluate the experience as pleasant.
Similarly, we may mentally take note of unsolicited comments or anecdotes from clients and draw conclusions about a program. Anecdotal and informal approaches such as these generally are not regarded as carrying scientific credibility. One reason is that decision biases play a role in our “informal” evaluation. Specifically, vivid memories or strongly negative or positive anecdotes will be overrepresented in our summaries of how things are evaluated. This is why objective data are necessary to truly understand what is or is not working. By contrast, formal evaluations systematically examine data from and about programs and their outcomes so that better decisions can be made about the interventions designed to address the related social problem. Thus, program evaluation involves the use of social research methodologies to appraise and improve the ways in which human services, policies, and programs are conducted. Formal evaluation, by its very nature, is applied research. Formal program evaluations attempt to answer the following general question: Does the program work? Program evaluation may also address questions such as the following: Do our clients get better? How does our success rate compare to those of other programs or agencies? Can the same level of success be obtained through less expensive means?
Pg 222
What is the experience of the typical client? Should this program be terminated and its funds applied elsewhere? Ideally, a thorough program evaluation would address more complex questions in three main areas: (1) Does the program produce the intended outcomes and avoid unintended negative outcomes? (2) For whom does the program work best and under what conditions? and (3) How well was a program model developed in one setting adapted to another setting? Evaluation has taken an especially prominent role in practice today because of the focus~ on evidence-based practice in social programs. Social work, as a profession, has been asked to use evidence-based practice as an ethical obligation (Kessler, Gira, & Poertner, 2005). Evidence-based practice is defined differently, but most definitions include using program evaluation data to help determine best practices in whatever area of social programming is being considered. In other words, evidence-based practice includes using objective indicators of success in addition to practice or more subjective indicators of success. Formal program evaluations can be found on just about every topic. For instance, Fraser, Nelson, and Rivnrd (1997) have examined the effectiveness of family preservation services; Kirby, Korpi, Adivi, and Weissman (1997) have evaluated an AIDS and pregnancy prevention middle school program. Morrow-Howell, Beeker-Kemppainen, and Judy (1998) evaluated an intervention designed to reduce the risk of suicide in elderly adult clients of a crisis hotline. Richter, Snider, and Gorey ( 1997) used a quasi-experimental design to study the effects of a group work intervention on female survivors of childhood sexual abuse. Leukefeld and colleagues ( 1998) examined the effects of an HIV prevention intervention with injecting drug and crack users. Logan and colleagues (2004) examined the effects of a drug court intervention as well as the costs of drug court compared with the economic benefits of the drug court program.
Basic Evaluation Considerations
Before beginning a program evaluation, several issues must be initially considered. These issues are decisions 1 hat are critical in determining the evaluation methodology and goals. Although you may not have complete answers to these questions when beginning to plan an evaluation, these questions help in developing the plan and must be answered before an evaluation can be carried out. We can 1.um up these considerations with the following questions: who, what, where, when, and why. First, who will do the evaluation? This seems like a simple question at first glance. However, this particular consideration has major implications for the evaluation results. Program evaluators can be categorized as being either internal or external. An internal evaluator is someone who is a program staff member or regular agency employee, whereas an external evaluator is a professional, on contract, hired for the specific purpose of evaluation. There are advantages and disadvantages to using either type of evaluator. For example, the internal evaluator probably will be very familiar with the staff and the program. This may save a lot of planning time. The disadvantage is that evaluations completed by an internal evaluator may be considered less valid by outside agencies, including the funding source. The external evaluator generally is thought to be less biased in terms of evaluation outcomes because he or she has no personal investment in the program. One disadvantage is that an external evaluator frequently is viewed as an “outsider” by the staff within an agency. This may affect the amount of time necessary)’ to conduct the evaluation or cause problems in the overall evaluation if agency staff are reluctant to cooperate.
Page 223
Second, what resources are available to conduct the evaluation? Hiring an outside evaluator can be expensive, while having a staff person conduct the evaluation may be less expensive. So, in a sense, you may be trading credibility for less cost. In fact, each methodological decision will have a trade-off in credibility, level of information, and resources (including time and money). Also, the amount and level of information as well as the research design will be determined, to some e11.”1ent, by what resources are available. A comprehensive and rigorous evaluation does take significant resources. Third, where will the information come from? If an evaluation can be done using existing data, the cost will be lower than if data must be collected from numerous people such as clients and/or staff across multiple sites. So having some sense of where the data will come from is…
