1	The Policy Context for Tracking and Reporting on Family Outcomes
2	Participant Outcomes: Why Bother with Accountability? Judith K. Chynoweth June 17, 2005 Monterey, CA
3	The “Other Directed” answer: Funders say we have to Funders won’t fund us unless we do
4	The Policy Context: CA’s budget and finance environment Harsh for most community-based organizations Long term trend – no turn around in sight Initiative governance: what does the public want? Good schools Health care access (for me) Child-care: before and after school Safe communities – safe children Jobs, jobs, jobs
5	The Policy Context (cont’d) The Recall: what the public doesn’t want! More government More taxes (though the polls say different) To give “handouts”
6	The Policy Context (cont’d) Is comprehensiveness alive and well? At state level: maybe “yes” – maybe “no” First 5 State Early Childhood Comprehensive Systems Project After School – 21^st Century Child Welfare Systems Reform Mental Health – Prop 63 Education Reform State Interagency Team Deep-seated ambivalence: “comprehensiveness” means more money
7	The Policy Context (cont’d) The Bottom Line: The public wants results Government has to get results YOU have to produce YOU have to “Make the Case”
8	The Policy Context (cont’d) Make what case? Produce what? Pre-School for all vs. prevention The Working Poor vs. reducing poverty Families that function (under stress) vs. bad parents Safe, strong communities vs. soft on immigrants
9	We want to be Accountable! Which Outcomes? Which Indicators? Evaluation vs. accountability: We don’t live by the “gold standard” Evaluation tells us what works and why Accountability tells us if we did what we said we would and if it made a difference
10	We want to be Accountable! Which Outcomes? Which Indicators? Community/population results vs. participant results Examples: All the families in our neighborhood earn the right amount of money for our local cost of living (as measured by …) VS. Families who participate in our family economic success program have developed and are working on an economic plan to increase their financial resources for the local cost of living OR Families who participate in our family asset development program are saving to buy a home based on their personal asset development plan.
11	We want to be Accountable! Which Outcomes? Which Indicators? Whose priorities? The families’ The funders – public or private The community’s It’s a balancing act!
12	We want to be Accountable! Which Outcomes? Which Indicators? Outcomes for whom? Does it measure: Participant outcome: Family feels confident in dealing with child’s behavior (Matrix call this an indicator) Worker outcome: % of families seen by worker A that improve in relation to their baseline Program/Agency outcome: % of families in program A that improve in relation to their baseline (Friedman calls this a performance measure)
13	We want to be Accountable! Which Outcomes? Which Indicators? Criteria for choosing measures: Data power Proxy power Communication power
14	We want to be Accountable! Which Outcomes? Which Indicators? Metrics and Methodology: Rely on the experts and their tools for: Reliability Validity Rely on yourself for meaning If it doesn’t produce meaning for you, Don’t do it!
15	Your Policy on Accountability What are the principles by which you will build an accountability system? Five key principles (grounded in the work of Mark Friedman) Focus on Results: On ends – not means Get your language straight What happens to participants? For individual families For all the families in a program
16	Your Policy on Accountability (cont’d) Be Inclusive Involve people who are affected by accountability decisions in accountability design: Families Workers Managers Funders Differences in perspective/opinion are good “Meaning” is the key to compromise
17	Your Policy on Accountability (cont’d) Share Accountability All stakeholders share accountability for outcomes: Families Workers Program managers and directors Funders When accountability is shared power becomes more equalized Be clear about who is accountable for what when accountability issues occur (and they will)
18	Your Policy on Accountability (cont’d) Use Measurement as a Learning Tool Measurement focus attention on: How families are doing How workers are doing How programs/agencies are doing Measurement is a more accurate indicator of reality than a rough guess, a story, past experience or a feeling Measurement is used to learn and improve: Everyone explores and communicates how s/he contributes to the outcome/indicator/performance measure Everyone takes responsibility for learning about what’s working, not working and why Data belongs to the people who generate it – NOT those who collect it.
19	Your Policy on Accountability (cont’d) Use Data and information to do/fund what works Most funders want to fund what works but they don’t for many reasons Most program directors and managers want to do what works but they don’t for many reasons One is they don’t know what works Most families want to do what works but they don’t for many reasons Sometimes cultural change is required for families, workers or organizations to make decisions on the basis of what is being learned. This can take time.
20	But I can’t afford it You cannot “not afford it!” The Consortium is conducting an assessment on available client outcome tracking and follow-up systems. A Report will be available in late Fall 2005 and will include ideas on how to find resources to support the development of accountability systems. See www.foundationconsortium.org