A Worksheet has to be completed and separate from the paper: This worksheet is intended to help you prepare for your Agency-Based Evaluation Proposal. The answers to the questions (below) will serve as the foundation for the assignment sections on the description of your agency, context, and stakeholders. This worksheet will also help you hone in on a needed and feasible evaluation focus and question. Proper citation rules apply. That is any information that is not original needs to be cited and put in direct quotes (if applicable).

Individual Paper:

The purpose of this assignment is to prepare a proposal for a process or outcome evaluation that could be implemented at your field placement. Please structure your paper as outlined below and include the headers in your paper. Each section contains questions for you to consider as you develop your proposal. Please use these questions as guidelines to ensure that you cover all aspects of an effective evaluation plan.

Please see the uploaded file.

Name:                                                 

Evaluation Context Worksheet-2020 COVID-19 Version*

This worksheet is intended to help you prepare for your Agency-Based Evaluation Proposal. The answers to the questions (below) will serve as the foundation for the assignment sections on the description of your agency, context, and stakeholders. This worksheet will also help you hone in on a needed and feasible evaluation focus and question. Proper citations rules apply. That is, any information that is not original needs to be cited and put in direct quotes (if applicable).

 

*Please note that this worksheet has been amended in response to students not being in field placement due to the COVID-19 pandemic. Thus, it will look slightly different than the one given as an example on the course site, so please use this version. The fundamental difference is that questions requiring you to ask supervisors and other co-workers questions have been removed. Instead, you will be asked to draw from your memory or working knowledge of the agency.

 

1. Agency Description (If you are not comfortable using the name of your agency, please develop a descriptive pseudonym. Please note: you will describe the program in the next section.)

Agency Name:

 Type of Agency (e.g., state, protective, residential, community-based, medical [inpatient, outpatient, both]):

 

Agency Context: What are the contextual factors influencing the agency? Please address all of the following:  neighborhood characteristics, important historical factors, accrediting bodies, and agency partnerships.

 

Policy Context: Provide a substantial and thoughtful description of relevant local and national policies (including temporary mandates, such as response to COVID-19) that influence your agency and its programs and what the agency response to these policies looks like. Be sure to discuss relevant informal or formal internal agency policies, as well.

 Your role at the agency: Describe where you fit (or did fit) in the organization, including your role and responsibilities. Are you the only intern or one of several?

Population the Agency Generally Serves:

Problems the Agency Generally Addresses:

 Agency Mission Statement:

Agency Stakeholders:

 2. Program within the agency that you are considering evaluating

 Program Name and Description:

Program Mission Statement (if applicable):

Specific Population(s) the Program Serves:

Specific Problems the Program Addresses:

Program Stakeholders (if different from agency stakeholders):

Program Goals, Activities, and Outcomes (Note: You are required to fill this section out, regardless of whether or not your program has a formal articulation of its goals, objectives, and activities):

 Goals (i.e., broadly speaking, what does the program want clients to achieve in the future?)

 Program Activities (i.e., actions the program takes with clients to help them achieve outcomes)

 Client Outcomes (i.e., the changes in knowledge, behaviors, attitudes, and/or states of being that will occur for clients as a result of participation in program)

Short term	Intermediate	Long-term (if applicable)

 

 

1. Potential Evaluation Focus
2. Typically, we ask students to talk with their supervisors and co-workers to get feedback on a potential evaluation focus. Given the pandemic, we do not expect that you will be able to do so at the moment. Even if you have current access, it is advised to keep conversations minimal to avoid adding to the burden on agencies during the crisis. Therefore, instead, please share what the ideal process would look like by responding the following questions.
   • If you were able to have conversations with the staff at your agency, who would you choose to talk with and why? What do you think you would encounter during those conversations: resistance, interest, hesitation, fear, something else? What do you think they might be curious to know more about based on your time with them?
   • Based on your working knowledge, have there been any evaluations conducted in the past that would influence future evaluations? What current data are currently collected (e.g., via intake, assessments, other forms)?
3. What aspects of your agency or program do you think need to be evaluated? In other words, what has stood out to you as potentially problematic — or simply not attended to — and in need of evaluation?
4. Of those ideas, what is your initial evaluation question or questions? You must indicate whether the question reflects a process evaluation or an outcome evaluation.

 

Feasibility Considerations:

• For the final project (and any evaluation), you will need to think through issues of feasibility (i.e., barriers and facilitators)

 

1. What are the potential barriers to this proposed evaluation?

 

1. What are the potential facilitators to this proposed evaluation?

Individual Final Paper: Agency-Based Evaluation Proposal. Specifications:

• Page-length requirement: No fewer than nine pages and absolutely no more than 11 pages
• Include a cover page; also append a logic model and an informed consent (or assent) form.

(Note: The cover page, logic model, consent form, and appendices do not count toward the

page-length requirement.)

• Times New Roman, 12-point font, one-inch margins, double-spaced, and APA format

The Agency-Based Evaluation Proposal

• The purpose of this assignment is to prepare a proposal for a process or outcome

evaluation that could be implemented at your field placement. Please structure your paper

as outlined below and include the headers in your paper. Each section contains questions

for you to consider as you develop your proposal. Please use these questions as

guidelines to ensure that you cover all aspects of an effective evaluation plan.

• Please make sure all of the required elements for this assignment are compiled

into ONE file.

1. Introduction and Background (approximate length: one page)
2. Briefly describe the organizational context or setting of your agency; make sure to

identify key stakeholders. Include in your discussion any diversity that is present in terms

of perspectives or client groups (e.g., racial, ethnic, economic, or other oppressed

groups).

2. Include a description of the program you are proposing to evaluate and how the program

works or is supposed to work (i.e., program theory). Make sure the program’s activities,

outputs, and outcomes are clearly articulated in a logic model that you include as an

appendix.

1. Evaluation Purpose and Significance (approximate length: two to three pages)

Describe the identified evaluation need, overall purpose and significance, and key

literature that will inform the evaluation design. Consider the following questions:

3. Why is this evaluation important? What are the issues that this evaluation would

address? How would findings from this evaluation improve services? Is this an

issue for all clients or is it an issue more relevant for a particular client population?

Are there clients from marginalized groups? If so, what particular evaluation issues

need to be considered? Is this an issue concerning how a program is being offered

(process evaluation) or an examination of the impact of a program on client

outcomes (outcome evaluation)?

4. Link to agency. What impact does this issue have on the agency and worker’s

ability to effectively serve clients? Alternatively, what impact does this issue have

on the client’s ability (or interest) in attending, participating, or engaging in

services? Make sure that there is a link between the issue that you have identified

and the agency mission.

5. What do we know from the academic literature and websites about this issue? How

can information on best practices inform your evaluation focus? How has the

literature on best practices addressed issues of race, ethnicity, class, or other

oppressed groups? [Find at least 6 empirical references to support your ideas.]

6. End with a clear, precise statement of the evaluation question(s).
7. Evaluation Design and Methods (3-4 pages)
8. Evaluation Approach: Discuss your approach. This includes the type of evaluation, (i.e., process or outcome evaluation), goal of the research (e.g.,

exploratory, descriptive, explanatory); type of research design (e.g., experimental,

quasi-experimental, pre-experimental, qualitative); and type of information needs

(e.g., qualitative, quantitative, mixed).

8. Sampling Plan:

• Describe your sampling strategy (i.e., probability/non-probability and type of

sampling). If relevant, in what ways does the sampling strategy address issues

of race, ethnicity, class, or other oppressed groups present in your client

population? If you are not drawing a sample because you plan to recruit the

entire population, please include rationale.

• Describe your intended sample or population. This includes the target,

inclusion criteria that determine eligibility (e.g., age, type of care, length of

program involvement, caregiver, etc.), anticipated sample size with rationale,

and relation to population of interest.

• Describe how you will recruit participants and/or select documents to review.

9. Measurement: Discuss your measurement (i.e., content areas that you are

collecting, major concepts). For quantitative information, include the

operationalization of these major concepts. Ensure the discussion of major

concepts flows from your evaluation question and the literature.

10. Measurement continued:

• If you are designing your own questionnaire or other data collection

instrument (or altering an existing one), attach a document with the questions

and response options. Describe the format and your rationale for those

questions in the text.

• If you plan to use standardized instruments or questionnaires “as is” (if you

can access them at no charge to you) attach them and briefly describe the

instrument and the psychometric properties in the text.

• Regardless, the content and format of your questions should align with the

rest of your study methods.

11. Data Collection Strategies: Describe your data collection methods (e.g., focus

groups, face-to-face interviews, mailed questionnaire, etc.) and describe in detail

your proposed procedures.

12. Data Collection Feasibility: Discuss the feasibility of your proposed methods,

including the strategies you favor and your rationale. Briefly discuss the benefits

and tradeoffs of the proposed methods.

13. Data Analysis Plan: Briefly describe how you will analyze the data you collect

and how your analysis will answer the key evaluation questions.

 

1. Ethics: (~1 page)
2. Ethical Issues:

• Identify and discuss the ethical issues that you have considered in the design

and implementation of the evaluation. Include risks and benefits for

participants at each stage of the evaluation (e.g., sampling, data collection,

analysis) and discuss planned protections against potential risks (e.g.,

voluntary participation, confidentiality).

• Be sure to address any issues that may need consideration due to study

participants’ race, ethnicity, social class, or membership in other oppressed

groups.

• Describe how you will be securing informed consent (or assent). (Note: this

process is different for anonymous questionnaires & other forms of collecting

information.)

13. Consent Form: Attach a detailed and appropriate consent or assent form as an

appendix.

1. Feasibility and Translation (1.5 pages)
2. Feasibility: Discuss feasibility considerations regarding organizational facilitators

and obstacles. These include, but are not limited to, existing policies at the

program and agency level; extent of organizational buy-in; and potential risks,

challenges, and benefits to staff and the organization.

18 Translation of findings to practice: How and to whom will findings be presented?

Who may gain or lose as a consequence of the findings? How will conducting the

evaluation facilitate the development of research and practices within the

organization? How might it inform (and be informed by) policies at all levels,

specifically for particular ethnic, racial, lower income, or other oppressed groups?

Make sure to consider what your findings can actually inform given the

limitations of your design and methods.