Home

USER'S GUIDE - Success By 6

1. 7 ECD Evaluation User Guide 35 7 5 Appendix E Interpreting Data The Community Action Report As a participant in the ECD Funders Integrated Evaluation project you will be sent Outcomes and Outputs reports that demonstrate the results of your evaluation work and provide province wide comparisons The next and final steps in the process are for you to consider the results draw some conclusions and submit a brief final report This document is intended to help you get there Part 1 makes suggestions for how your ECD table might workshop the data Part 2 is a template for your final report Background When you have the Outcomes and Outputs reports that demonstrate the results of your community capacity building work and provide province wide comparisons the next and final steps in the process are for you to consider the results draw some conclusions and submit a brief final report This document is intended to help you get there Part 1 makes suggestions for how your ECD table might workshop the data Part 2 is a template for your final report Part 1 Outline for a Data Interpretation Workshop 1 Invite ECD table participants and others as appropriate to take part in a workshop Three hours or more is recommended Let them know that you will be reviewing your evaluation Outcomes Report with a view to interpreting the data and deciding what steps you might take to address challenges and build upon successes 2 Pr
2. si 29 Please tell us if there are any MOUs Memorandum of Understanding in place that impact your local ECD planning and coordination check all that apply O None There is a Memorandum of Understanding between funders local or regional There is a Memorandum of Understanding between community agencies There is a Memorandum of Understanding involving Aboriginal bands or communities O Other please specify 30 What has your ECD community planning table done to address access barriers to service for young children and families in the last 12 months check all that apply LINo activity in the last 12 months L Developed an accessibility plan O Conducted accessibility training L Conducted needs assessment on accessibility L Addressed barriers related to families accessing programs Q Addressed barriers related to participation at our ECD table O Other please specify 31 If you have made ECD services more accessible e g programs delivered in various languages satellite or mobile programs bus taxi vouchers extended days and hours of service changed location of services to ECD Evaluation User Guide 22 families with young children please tell us which groups you have put accessibility initiatives in place for check all that apply O Low income families O Aboriginal families L Immigrants and refugees O Children with special needs O Families living in rural and remote communities O Other please specify
3. collection by May 30th our workshop Review Section 4 5 and Appendix D A ofthe guide Complete the Community Action Report Submit your Report _ Appendix D Part 2 by November 15h ECD Evaluation User Guide Project Contacts For more information contact Evaluation Coordinator Vera Radyo for any questions related to conducting the evaluation veradyo shaw ca or 604 904 3772 Due Dates e May 30 o Completion of all ECD Managers Surveys through the online format o Completion of all Stakeholder Surveys through the online format e Early fall holding a community table meeting to discuss the evaluation data e November 15 o Submission of Community Action Reports your community table s interpretation of your evaluation data and what you plan to do about it to Vera Radyo at veradyo shaw ca e Subsequently submission of your evaluation results and your Community Action Report to your funders ECD Evaluation Project User Guide Activities Barriers Benchmarks Community Activities are what a program does with its inputs the services it provides to fulfill its mission Examples are sheltering homeless families parent education programs providing adult mentors for youth Barriers that families encounter when trying to access services Program or service not there Costs too much Transportation Time offered Language Not culturally appropriate Fragmentation Lack of I
4. 32 Please provide details of any policy changes that have occurred in the past 12 months as a result of your ECD community planning table work e g Taxes no longer required for non profit child care in city buildings Chamber of Commerce adopted family friendly business policy E Ej 33 Please list any new or improved ECD or family support programs that have been developed in the last 12 months as a result of your ECD community planning table activity E 34 Please list the training educational and or mentorship activities your ECD community planning table has undertaken in the last 12 months j E 35 Please tell us about any major social economic or political changes that have occurred in your community or BC overall in the last 12 months that have impacted the ability of your ECD community planning table to make a difference for children and their families e g forestry industry in recession funding for new programs etc zi ECD Evaluation User Guide 23 This is the Long Form survey as it will appear to your stakeholders in the online format 7 3 APPENDIX C STAKEHOLDERS SURVEY Long Form Thank you for choosing to complete the Stakeholder Survey The data from your community and other communities in BC will assist us in creating a comprehensive overview of the powerful community building work that is taking place around the province It will also provide your ECD community planning table with infor
5. OStrongly Agree OAgree ONeither Agree or Disagree ODisagree OStrongly Disagree O Don t know ECD Evaluation User Guide 34 16 Please provide an example of how families in your community have benefited over the past 12 months as a result of improved ECD services and supports E A 17 Please tell us what sector you represent at your community ECD table If you are bringing the perspective of more than one sector to the ECD table check all that apply e g community non profit and CAPC LIMCFD Ministry of Children and Family Development LICAPC Community Action Program for Children O Children First O Success By 6 and or Council of Partners O Understanding the Early Years O Aboriginal Band O Aboriginal community members O Aboriginal ECD planning table LIAHS Aboriginal Head Start O Business private sector LICPNP Canada Prenatal and Nutrition Program O College University OCommunity non profit agencies ODay care and pre school providers O Children and youth with special needs O Faith based groups O Federal government O Health Authority Public Health O Library Literacy groups O Ministry of Human Resources O Multicultural immigrant groups O Municipal L Parents grandparents O Politicians O School districts O Volunteers O Other please specify Thank you for your participation Please use this space to tell us about how easy or difficult this questionnaire was to complete and to add any additional comments you wish to make
6. Other please specify E E 25 How many volunteers have participated in the community ECD community planning table and all its events in the last 12 months e g helped to organize or run table initiated events or research projects participated in planning sessions etc Only count people who do not receive any financial compensation for the time they spend participating in your table s activities Number of volunteers 26 How many community leaders participated in the activities of your ECD community planning table over the past 12 months Every person holds influence based on their connections and skill sets In this case we are looking for influential community members who are connected with infrastructure and relationship bridging influence such as business leaders chief of a First Nation municipal council member Number of community leaders ECD Evaluation User Guide 21 27 Which community leaders participated gt r 28 Did your ECD community planning table partners engage in any activities that aimed to improve coordination integration and partnerships among service providers in the last 12 months check all that apply C None were undertaken in the last 12 months L Coordinated service provision L J oint intake process L Cross referral joint assessment and or case management O Co location of services O Development of integrated neighbourhood services Hub model L Other please specify
7. Managers Survey allows you to document the direct products of your activities It is anticipated that this information can be extracted from your meeting minutes ECD plans marketing plans etc Enter your data in the online Survey Monkey sometime between the beginning of April and the end of May Appendix B to this guide is a copy of the online survey and will walk you through the questions The green callouts should clarify the questions where necessary Use this form to gather the data you require prior to completing the online version Be sure to have this Appendix in hand as you enter your data into the Survey Monkey The Stakeholders Survey should be completed by people who are involved in your initiative You can solicit the participation of members of your steering committee sub committee members and others in the community who are involved in helping your ECD community table to reach its goals You will be provided with a link to the survey and an ECD community table Code The Participant Code allows us to filter results by initiative You can simply email the link and code to prospective respondents along with some explanation of what you want them to do For example Dear Maryn Colleagues ete if you are sending a group email As you know Success By 6 and Children First ECD tables in communities around the province including our own are participating in the 2009 ECD Community Capacity Building evaluation As a l
8. OCommunity non profit agencies ODay care and pre school providers O Children and youth with special needs O Faith based groups O Federal government O Health Authority Public Health LILibrary Literacy groups O Ministry of Human Resources O Multicultural immigrant groups O Municipal L Parents grandparents O Politicians O School districts O Volunteers O Other please specify Thank you for your participation Please use this space to tell us about how easy or difficult this questionnaire was to complete and to add any additional comments you wish to make ECD Evaluation User Guide 30 This is the Short Form survey as it will appear to your stakeholders in the online format 7 4 APPENDIX D STAKEHOLDERS SURVEY Short Form Thank you for choosing to complete the Stakeholder Survey The data from your community and other communities in BC will assist us in creating a comprehensive overview of the powerful community building work that is taking place around the province It will also provide your ECD community planning table with information that will assist you in your ongoing planning for children and families The purpose of this survey is to gather information about the outcomes of community capacity building work from stakeholders at local ECD community planning tables province wide The survey looks at three aspects of community capacity building work Community Planning and Coordination Community Awareness a
9. are in place planning Local evaluation is initiated ni and type of processes used to create Increased use of a local ECD plan plans to guide initiatives 3d and type of strategies that are developed 3e and description of local priorities Increased use of evaluation for 4a and type of table led evaluation initiatives improvement Active cross sectoral table More inclusive active community coalition Local awareness initiatives 1a and type of awareness messages Increased community awareness of 1b of awareness activities the importance of the early years Local resource development 2a and type of local allocations decision allocation making processes Increased community resource development amp allocation Increased community engagement 3a of volunteers of community leaders engaged Community Engagement Service coordination 1a and type of coordination activities Increased service coordination 1b and type of MOU s in place and integration Accessibility initiatives 1c and type of new or improved programs ai y 1d and type of changed or new policies Increased accessibility 2a and type of accessibility activities 3a and type of training mentorship l a knowsdga or bast practicos activities Training mentorship opportunities ECD Evaluation Project User Guide April 2009 OUTCOMES OUTCO
10. from external agencies If you have asked the right questions an evaluation will tell you not only what you have achieved but also how you did it and what was most effective For a good evaluation data must be collected and analyzed systematically and interpreted carefully 3 3 Evaluation Terms Commonly used evaluation terms include goals objectives benchmarks outcomes outcome indicators outputs inputs and activities You will find definitions of these terms in the Glossary of this guide ECD Evaluation User Guide 5 3 4 The Program Logic Model A logic model is a diagrammatic representation of a program showing what it is supposed to do with whom and why Logic models show the link between what we do and the impact we are having provide a common vocabulary focus on quality and continuous improvement Logic models bring detail to broad goals and show the chain of events that link inputs to results i e the relationship between what we put in inputs what we do outputs and results outcomes The direct products What our of the activities The resources program does sag ae atch ae How we know if expended to with the Inputs _ 9 we have if ii D affected chang achieve our A y A x goals Short Term Intermediat Long Term Outcomes Outcomes Outcomes The ultimate change expected Links the short term often a change in outcomes to the long term i condition or status outcomes often a o lt change
11. many people participate at the ECD community planning table you are reporting for of participants 10 1oo do 1 Please indicate which of the following ar scented at the community table you are reporting for Children First People who participate at your community Success By 6 on a more or less reaular basis CAPC Community Action Program for Children Credit Union Understanding the Early Years Aboriginal Band Aboriginal community members Aboriginal ECD planning table AHS Aboriginal Head Start Aboriginal service organizations Aboriginal Transition Authority Business private sector Child care and preschool providers Child Care Resource and Referral Children and Youth with special needs College university Community non profit agencies CPNP POPS Canada Prenatal Nutrition Program Pregnancy Outreach Program Faith based groups Federal government Health Authority including Public Health Prevention amp Intervention Health Services etc Library literacy groups MCFD Ministry of Children and Family Development Ministry of Employment and Income Assistance Multicultural immigrant groups Municipal politicians Municipal staff including Parks and Leisure Services Parents ECD Evaluation User Guide 16 Physicians optometrists dentists r School districts Other please er 11 Over the past 12 months how often did your ECD community planning table meet Regular table
12. or re affirmed re vamped our Vision statement Owe created or re affirmed re vamped our values and or guiding principl Owe assessed our internal environment strengths and weaknesses Owe assessed our external environment threats and opportunities Owe reviewed the results of our research activities Owe held a public meeting Owe conducted a visioning process Owe identified critical or strategic issues Owe set priorities Owe designed broad strategies to address the critical issu Owe identified specific activities to address the broad strategies L we developed timelines Owe identified areas of responsibit Lwe created a budget for our plan Owe created an evaluation plan with which to Owe used the Children First Strategic Planning Gui O Other nee aa itor progress on our strategic 18 If you created or updated your ECD plan in the last 12 months does your ECD plan include strategies or activities related to the following check all that apply OQ Building collaboration amongst community partners O Public awareness LIExpanded community engagement in local ECD planning service implementation O Further research ECD Evaluation User Guide O Reaching out to immigrant and refugee populations O Reaching out to Aboriginal families O Reaching out to families who are isolated O Reaching out to families who are living in poverty O Child care and preschool services O Reaching out to children and youth with special needs
13. over the past 12 months e g community socio economic data information on current services service gaps use of data from the Early Development I nstrument EDI information from parents OStrongly Agree OAgree ONeither Agree or Disagree ODisagree OStrongly Disagree O Don t know 4 Our ECD community planning table used our ECD strategic community plan to guide or influence activities in our community over the past 12 months if you don t yet have an ECD plan do not answer this question OStrongly Agree OAgree ONeither Agree or Disagree ODisagree OStrongly Disagree O Don t know 5 We used ECD community planning table led evaluation findings for improvement over the past 12 months e g formal evaluation of the ECD community table evaluation of specific ECD table sponsored programs OStrongly Agree OAgree ONeither Agree or Disagree ODisagree OStrongly Disagree O Don t know ECD Evaluation User Guide 32 6 What is the most significant change improvement your ECD community planning table has made influenced over the past year as a result of your joint coordination and planning a E This year compared to last year would you say that 7 Our community has an increased awareness of the importance of early childhood development as a result of the work of our ECD community planning table OStrongly Agree OAgree ONeither Agree or Disagree ODisagree OStrongly Disagree O Don t know 8 Our locally identified ECD priori
14. past 12 months while 10 disagreed and 10 didn t know Numbers do not speak for themselves They need to be interpreted based on careful and fair judgments Similarly narrative statements need interpretation The same information can be interpreted in various ways As Coordinator you will have your own perspective Others will look at your information through different eyes Greater understanding usually results when you take the time to hear how different people interpret the same information so it will be important to involve your ECD table in the interpretation process It is recommended that you hold a data interpretation workshop with your ECD table early in the fall We would like your table to review your successes challenges and what you plan to do regarding the issues you have identified This meeting would provide an opportunity for the group to engage reflect upon and interpret data and to develop implications from the data about your initiative An outline for such a workshop and the reporting template are attached to the Guide as Appendix E 4 5 Reporting your Evaluation Outcomes At the end of the evaluation process it is anticipated that you will provide your respective funders with copies of 1 The Data Analysis Reports provided to you by central data processing 2 Your Community Action Report the results of your interpretation workshop See Appendix E for the Report format ECD Evaluation User Guide 8 The
15. us b What are our successes c What are our challenges d How do our outcomes compare to the provincial outcomes e If there are areas where a significant number of respondents checked off don t know or disagree What might we do about that f What recommendations do we want to make relative to building on our successes and or addressing our challenges Allocate an appropriate amount of time for discussion and wrap up and make sure to give the group a 10 minute wrap up warning Ask the recorder to document successes challenges and make any recommendations for action to build on successes and or address challenges 5 When each Outcome area has been examined bring the group together to look at the results of your work Discuss and prioritize a few key strengths and challenges in each of the three Intermediate outcome areas as well as any recommendations for change Don t forget to develop timelines and assign responsibility either at this meeting if you have time or plan to do so at an upcoming meeting perhaps when you are updating your strategic plan 6 Complete and submit your Community Action Report to Vera Radyo Evaluation Project Manager veradyo shaw ca The Intermediate and Short Term Outcomes Both your Outcome and Outputs reports are in this format Intermediate outcome Improved community driven planning and coordination Short Term Outcomes 1 More inclusive active community coalition 2 Impr
16. ECD Evaluation Community Capacity Building USER GUIDE Getting Read Gathering Data Reporting Out Using Your Findings ECD Evaluation Project User Guide April 2009 Table of Contents 1 Abo ou t the Gd ipsa oases eee eee ei eee 3 2 The Purpose of this Project 22 ccssccccssecsnesseesseecennenessneeesseeensnneseenenensnnesessnenesnnaeenene 3 2 1 Background erronee egies cede ceed canada net deed dua chee dace EEES 4 2 2 How the Evaluation Results will be Used cece cece eee eee tetera teeta een eenaenaes 4 2 3 THM GTAIMNG a seceaedc hve Maasai a tained a E EE E AE EE E tem aa AEE Eae a AE ANA 4 3 About Evaluation in Gemerrall n ecccescccceeeeceeseeeeeeeeeeeeeeeseeeeeeeneeeseeaeeeseeeseenaeeeseeeeeseeeeeseenees 5 3 1 What is EVAlUAtION cies storie ceden niinn pre gence deed en sd eene aden soln a eta A TAa a 5 3 2 Why Evaluate tinon oiiaee ein tel te ddeteneeide Meira ble EE a Ea A eagia Perce tladew es 5 3 3 EValUAtION TERMS sicc eedcctn seen Ceaein de aai aada dave nle bed ew ead agua iaaa Of ap idedees 5 3 4 The Program Logic Model ccecce cece cece eee e eee eee eens eee tees tent rtrt rnrn rnrn rran rrrn rnnr annat 6 4 Capacity Building Evaluation Overview ccccccccceseeeeseeeeeeeeeeeeeeeeeeeeeeeseseeeeseeeeeeeines 7 4 1 What and How We Will Measure 0 cece cece erent nett ne nett anata 7 4 2 When To Measure 2 cece cee REE ene enna ae eae 7 4 3 What To Do w
17. Information from childcare preschool providers on satisfaction Neighbourhood specific information olol Ol Of DQ Q O Q Q C O No OJO OVO of of OQ C O O O J OVO Of OF OOOO OIOI O O Early childhood development best practices overview Other please specify 15 Has your community table developed or updated a community ECD plan in the last 12 months Check one Lwe have no community ECD plan go to question 19 We have a community ECD plan that was developed more than 12 months ago go to question 19 Owe are in the process of developing an ECD plan go to question 19 ECD Evaluation User Guide 17 Owe completed or updated a community ECD plan in the last 12 months 16 If you completed or updated a community ECD plan in the past 12 months please indicate the NUMBER of people from the following groups who were are actively engaged at the table in the planning process Parents _____ Community child and family serving agencies ___ Child care daycare ___ MCFD Aboriginal people representatives of Aboriginal groups ___ Faith based Public Health Health Authority School district Business people Municipal staff Library literacy Other 17 If you completed or updated your ECD plan in the last 12 months what processes were undertaken and documented in the development of that plan check all that apply Owe created or re affirmed re vamped our Mission statement Owe created
18. ME INDICATORS Improved Community Planning and Coordination and of community table participants who agree that 1a The table reflects the diversity of the community 1b They are working well together to strengthen ECD 2a Research has been used to guide the work of the initiative 3a The ECD plan has guided the initiative 4a Evaluation is being used to foster improvement More inclusive active ECD community table Improved knowledge and ability to use research and evidence to guide initiatives Increased use of local ECD plan to guide initiatives and of community table participants who can provide examples related to the Increased use of evaluation for improvement above indicators Improved Community Awareness and Mobilization and of community table participants who agree that 1 Increased community awareness of the 1a Community awareness of the importance of ECD has increased importance of the early years 2a The ECD table has increased its ability to make joint decisions related to ECD resources 3a Increased volunteerism is helping to mobilize the community 3b Increased participation of community leaders is helping to mobilize the community 2 Increased local resource development and decision making 3 Increased community engagement and of community table participants who can provide examples related to the above indicators Improved Service Delivery and of community table participa
19. O Involving child care preschool providers at the planning table O Service integration E O Financial sustainability O Including more parents in planning decision making L Advocacy L Training O Evaluation O Screening O Succession planning O Other please specify 19 Please list the local priorities for young children and their families as defined in your community ECD plan vr 20 Please tell us about evaluation activity your ECD community planning table has undertaken over the past 12 months check all that apply O None prior to this evaluation We evaluated some specific programs and or projects undertaken by our ECD table L We participated in this evaluation process in 2008 O we participated in the pilot of this evaluation process in 2007 O Other please specify ECD Evaluation User Guide 19 21 Which of the following messages has your ECD community planning table promoted through public education awareness activities in the last 12 months check all that apply O None in the last 12 months Go to Question 23 O Awareness of children s rights O Awareness of the research on early brain development O Awareness of the research on the importance of ECD O Awareness of the importance of quality child care O Awareness of the factors that contribute to maternal and or children s well being O Awareness of the community as an ECD child friendly environment O Awareness of th
20. Some example of community leaders would be Mayor Chamber of Commerce Chair Parent Advisory Committee chair local bank manager Aboriginal chief The process of organizing and classifying the information you have collected tabulating it summarizing it comparing the results with other appropriate information and presenting the results Deciding what the result of your data analysis means putting the results into context and drawing conclusions The plans that are currently guiding ECD work in communities It called also be called by another name such as strategic plan or action plan A goal is the end towards which funded activities are directed Goals are ambitious statements of a desired end and lead efforts towards achieving a local vision What big global results or changes do you want to see for children families teachers communities or systems Example Every child in BC has access to a high quality early childhood program Inputs are resources a program uses to achieve program objectives Examples are staff volunteers facilities equipment curricula and money A program uses inputs to support activities The community group that has been selected to participate in the evaluation could also be called ECD table Success By 6 or Children First Decisions made by the table or group chosen for the evaluation Also called the average is equal to the sum of all measurements divided by the number of measurement
21. ake this happen are Ministry of Children and Family Development Public Health Agency of Canada United Way HELP Community Action Program for Children Children First Success By 6 Max Bell Foundation and local ECD coalitions In 2008 60 communities throughout BC participated in the evaluation and as a result the tools have been refined and simplified This common evaluation process will reduce duplication and multiple reports to different funders especially for those of you who have both Success By 6 and Children First support to your coalition In addition the evaluation process will provide useful information for strategic planning and an ability to demonstrate provincial results for ECD community capacity building Provincial Roll Out Spring 2009 The best thing about the evaluation is that it is simple You only need to complete two online surveys one by the ECD Coordinator Manager and another by ECD Table participants A User Guide and support is provided for all coordinators who participate Each community will receive their own local results along with provincial comparisons The time frame in 2008 is e April May 31 Community coordinators choose a 2 week period during these two months for online survey completion e June analysis of data by the provincial Evaluation Team e August Analysis provided to local communities along with provincial comparisons e Early fall Evaluation project participants hold community meetings to d
22. an that has been achieved over the past 12 months f7 5 We used ECD community planning table led evaluation findings for improvement over the past 12 months e g formal evaluation of the initiative evaluation of specific ECD table sponsored programs Agree Q Agree QODisagree QO Don t know ECD Evaluation User Guide 26 5a If you selected Agree please provide an example of one significant change your ECD table has made over the past 12 months as a result of evaluation activity B a COMMUNITY AWARENESS AND MOBI LIZATION gt This year compared to last year would you say that 6 Our community has an increased awareness of the importance of early childhood development as a result of the work of the ECD community planning table Q Agree O Disagree OQDon t know 6a If you selected Agree Can you tell if your ECD community planning table s work to raise awareness is having an effect Please provide an example of a change that has taken place in your community that in your opinion could be attributed to your ECD community planning table s efforts to raise awareness of the importance of ECD ry B 7 Our locally identified ECD priorities are guiding the allocation of new funds becoming available for ECD programs and services in our community e g funds from municipal provincial federal or other sources O Agree O Disagree OQDon t know 8 Our locally identified priorities are guiding the re allocat
23. d our MCFD Region is Thank ne for a E 5 A day or two after your deadline you may contact the Survey Administrator veradyo shaw ca to determine how many people have responded to your survey we won t be able to tell you who has responded though just the number of responses If you feel that you need more responses you can send out a final email For example Dear Colleagues or whatever form of address you usually use The deadline for responses to the Stakeholder survey is now past Thanks to those of you who have responded The Survey Administrator tells me that the response from participants in our initiative is a bit low so am extending the deadline for another three days until the end of the day on in hopes that those of you who have not responded will do so Please take this last chance to provide your input Here again is the link to the survey Our ECD Community Table Code is and our MCFD Region is Thanks eis ECD Evaluation User Guide 11 OUTPUTS OUTPUT INDICATORS 1a and type of community partners 1b of table subcommittee meetings 1c of funded hours of coordination time 2a f ECD rel h Usea research and a and type of ECD related researc l evidence information mproved knowledge and ability to ice based informatio 3a of ECD plans that are developed seresoarch and evidence to Inclusive local ECD plans 3b and type of partners participating in guide initiatives
24. demonstrating province wide outcomes Phase 1 involved designing and pilot testing the evaluation system A number of communities were invited to take part in a pilot The communities were chosen with a view to reflecting the diversity of the population around the province geographic urban rural Aboriginal and culturally diverse populations Sixteen communities agreed to participate An evaluation framework logic model and two data collection tools questionnaires were developed with input from the partners The first of these questionnaires The Managers Survey was designed to gather output information from managers of the ECD initiatives The second questionnaire The Stakeholders Survey was designed to gather outcomes information from local ECD table participants and other stakeholders ECD Evaluation User Guide 3 The pilot communities collected data in April and May of 2007 The data was eithe entered online or sent to a central data processing resource Over the summer the data was collated and reports were sent back to the pilot communities Subsequently locas initiatives met to discuss the evaluation findings and write brief final reports Lessons learned in the pilot were used to refine the evaluation system and it was administered in 60 communities in 2008 In an effort to refine the system further 2 versions of the Stakeholders survey are being administered in 2009 the original version with some chan
25. e and find yourself at the Success By 6 website The survey allows only one response per computer You may return to your survey to update change your responses but you must do so on the same computer you used to start with Please complete your survey by the requested date Before entering the survey please enter some information that will help us to sort out the data we are gathering in different ways The answers to the next two questions will be in the email sent to you with the survey link Please check to make ECD Evaluation User Guide 31 sure the answers are identical to those provided to you 1 Please enter the initiative code here 2 Please identify the MCFD Region in which your initiative resides O North O Interior O Fraser OVancouver Coastal O Vancouver Island Would you Say that 1 Our ECD community planning table reflects the diversity of our community e g parents service providers Aboriginal business faith groups municipal education health etc OStrongly Agree OAgree ONeither Agree or Disagree ODisagree OStrongly Disagree O Don t know 2 We are working well together at the ECD community planning table to strengthen ECD services and supports e g progressing towards better coordination collaboration OStrongly Agree OAgree ONeither Agree or Disagree ODisagree OStrongly Disagree O Don t know 3 Our ECD community planning table used research to inform planning and to guide our work
26. e range of ECD programs and services O Awareness of other community assets O Awareness of service gaps and resource needs O Awareness of how local children are faring in terms of ECD O Awareness of the existence and expertise of the local ECD table O Awareness related to the role of fathers and or other caregivers O Awareness of the primary role of parents as the principle caregivers for their children O Awareness of ECD as prevention rather than intervention O Awareness of the link between early learning ECD and schools O Awareness of the Children First initiative O Awareness of Success By 6 O Other please specify 22 Please identify type of awareness activities your ECD community planning table undertaken in the last 12 months OECD Initiative sponsored community events e g resource fairs workshops Ready Set Learn conferences and forums on ECD etc L Information booths at community events or celebrations e g National Family Week National Child Day etc OECD brochures O Calendars O Posters O Resource directories L Newspaper magazine articles L Magazine developed your own O Newsletters electronic L Newsletter hard copy O List serve L Radio interview or advertisements LITV interview or advertisements LIECD related website LIECD public speaking presentations to various groups e g municipal councils child care providers service clubs politicians chambers of commerce multicultural gr
27. ey please contact the ECD coordinator who sent you the survey link At the end of each section click on Next When you reach the end click on Done You will know that your survey has been uploaded to the Survey Monkey site when you click Done and find yourself at the Success By 6 website The survey allows only one response per computer You may return to your survey to update change your responses but you must do so on the same computer you used to start with Please complete your survey by the requested date ECD Evaluation User Guide 24 Before entering the survey please enter some information that will help us to sort the data we are gathering in different ways The answers to the next two questions will be in the email sent to you with the survey link The Survey tool will not allow you to proceed until you have entered the answers Please check to make sure the answers are identical to those provided to you 1 Please enter the Initiative Code here 3 Please identify the MCFD Region in which your initiative resides O North O Interior O Fraser OVancouver Coastal O Vancouver Island Would you say that 1 Our ECD community planning table reflects the diversity of our community e g parents service providers Aboriginal business faith groups municipal education health etc OAgree O Disagree O Don t know la Please comment on your successes and challenges in creating and maintaining an ECD com
28. ges to the open ended survey questions designed to gather data that is more indicative of impact and a shortened version Some agencies will be requested to complete the longer version and some the short Should the short version produce enough data to demonstrate impact this will be used in an ongoing way so as to reduce the reporting burden in subsequent administrations This revised version of the Guide is intended to assist coordinators to participate in the provincial 2009 roll out 2 2 How the Evaluation Results will be Used It is expected that evaluation results will be useful to the local communities to assess the progress of their ECD initiatives make improvements where indicated and as a reporting mechanism to funders The results from all the evaluations will be rolled up to provide a province wide overview of ECD capacity building work 2 3 Timeframe Here is the timeframe April May Choose a 2 week period for the collection of information in participating communities in online format Mid August Reports downloaded from online survey tool and sent back to communities by a central data processing resource September November Data interpretation workshops in local communities discussing successes challenges and plans for any action that will be taken as a result of the evaluation November 15 Submission of Community Action Report based upon the above noted ECD community table meeting Subsequent
29. hed to this Guide Tips for Getting Good Response Rates 1 Let prospective participants know about the survey ahead of time if possible in person at a meeting of your initiative Distribute the one page project information sheet which provides background information in the project Appendix F Tell them that their participation is valuable that it will be appreciated and that it will help to move the work of your initiative to a new level Provide them with your timelines and ask for their commitment 2 If possible send out individual emails with the survey link to potential respondents rather than a group message 3 Ask people to email you back when they have completed the survey Then you can cross them off your list and send reminders to those who have not completed the survey 4 Give a maximum of two weeks for responses Typically people will respond in the first few days Then two or three days before your deadline send out a reminder letting them know that the deadline is approaching thanking those who have responded and encouraging anyone who has not responded to do so by your deadline For example Dear The deadline for responses to the Stakeholder survey is ae ECD Evaluation User Guide 10 Thanks to those of you who have responded If you have not yet completed a survey please try to do so by the deadline Your participation is important Here again is the link to the survey Our ECD Community Table Code is an
30. ide 46
31. in behaviour resulting from a change in knowledge attitude or skil The initial benefits of a program often a change in knowledge attitude or skill The logic model for this evaluation is Appendix A to this Guide ECD Evaluation User Guide 6 4 1 What and How We Will Measure For the purpose of this project we will be measuring the Outputs the direct products of your activities and the short intermediate term Outcomes results The questions we will be asking related to the Outputs and Outcomes have been drawn from the list of Indicators in the logic model see Appendix A Outputs The Outputs will be measured using the ECD Managers Survey which captures data on a number of Output Indicators The Coordinator of each ECD initiative will complete one of these forms in an online format using the Survey Monkey tool A link to the tool will be provided to participating Coordinators Managers by email Outcomes The short term intermediate Outcomes will be measured using the ECD Stakeholders Survey Coordinators in each community will invite participants stakeholders involved with ECD initiatives to complete these surveys in an online format again using the Survey Monkey 4 2 When to Measure Outputs ECD Initiative Managers information should be entered into the Survey Monkey by the end of May It is suggested that you begin to gather the data in April Outcomes ECD Stakeholders Surveys should be completed onl
32. ine during a two week period of your choice during April May with all surveys completed before the end of May 4 3 What to do with the Data It is expected that the data entry phase will be completed by May 31st Following that our central data processing will compile the data create a roll up of province wide information and provide you with individual and roll up reports This phase should be completed by mid August In the fall you can review and interpret the results with your community tables make decisions about how the evaluation results will be used and report back to your funders ECD Evaluation User Guide 7 4 4 Interpreting the Results of the Data Gathering Interpreting your data involves putting it into perspective for example comparing results to what you expected or to initiative goals or to the results of similar initiatives then drawing conclusions In this instance you will be able to compare your results to your initiative goals where appropriate and to the results of the province wide roll up Data from this year may serve as a benchmark for future evaluations that ask similar questions The data reports will quantify your data in some way for example 75 of respondents agreed that the community table is reflective of community diversity while 25 of respondents disagreed or 80 of respondents agreed that community awareness of the importance early childhood development has increased over the
33. ion of existing ECD funds i e we are looking at all the funding that comes to our community for ECD and re allocating where necessary to achieve our shared objectives OAgree O Disagree ODon t know 9 Please comment on your ECD community planning table s ability to allocate re allocate funds in ways that address locally identified priorities E a ECD Evaluation User Guide 27 10 There has been an increase in the number of volunteers who participate in community ECD activities over the past 12 months e g helped to organize or run ECD table led community events or research projects participated in planning sessions etc Volunteers are people who do not receive any financial compensation for the time they spend participating in your ECD activities OAgree O Disagree OQDon t know 10a If you selected Agree please provide an example of how an increase in volunteerism is helping your ECD community planning table to achieve its goals E g 11 In the past 12 months we have attracted influential community members to our community events and initiatives Every person holds influence based on their connections and skill sets In this case we are looking for influential community members who are connected with infrastructure and relationship bridging influence such as business leaders chief of a First Nation municipal council member OAgree O Disagree ODon t know lla If you selected Agree please provide a
34. iscuss their evaluation results e November 15 Evaluation participants submit a brief final report detailing how they will use their evaluation results ECD Evaluation User Guide 40 Appendix G Quick Guide to the ECD Funders Integrated Evaluation Project Use this map for quick reference Refer to the User Guide for more information Review the Form and explanations of questions Section 5 and Appendix B Review instructions in A in the Guide A the User Guide Section 6 Clarify anything you are not sure about Provide one page information sheet Inform potential respondents 3 EE about the upcoming surve NOO Gather your data Assure confidentiality ahead of pass g y y Best done in person e g at table meetin Call for help if you are unsure about anything Enter the data in Survey Monkey by May 31st Send out link to survey see email example in Give a maximum of Section 6 2 weeks for response Review Section 4 4 and Appendix D of the User Guide 2 3 days before the deadline send a reminder see email example in 6 A day or two after your deadline contact the Survey Administrator to find out how many respondents you have Review the data analysis provided by central data processing If your response rate Hold a data is low send out a final interpretation reminder see email example workshop in Section 6 Record the Complete your data outcomes of
35. ith the Data cece cece eee tne ene eee eaten 7 4 4 Interpreting the Results of the Data Gathering 0c cececeee cee eee eee eae teeta nae naes 8 4 5 Reporting your Evaluation OUtCOMES cece eee eee e teeta ease a ete een eens 8 5 Completing the Coordinator Managers Output Reporting Survey 00 9 6 Completing the ECD Stakeholders Outcome Reporting Survey 2 esse 9 Tips for Getting Good Response Rates cc ccecce cece eee eee eee e eee ee nae ea seat eaten 10 7 APPENDIXES iach eases esctecteacttscide ectesie decd acess AAAA NAANA AARRE VANE adaa 12 7 1 APPENDIX A Logic Model 0 cece een nn eed 12 7 2 APPENDIX B ECD Coordinator Managers Reporting Tool Guide sessessseeseereererre 14 7 3 APPENDIX C Stakeholders Outcome Reporting Survey Long Form n ssseesecceeeerren 24 7 4 APPENDIX D Stakeholders Outcome Reporting Survey Short FOrM s s m ssmememrmrmen OL 7 5 APPENDIX E Interpreting Data The Community Action Report ssesseeseseseereererreen 36 7 6 APPENDIX F One Page Project Information Handout casein eee eee AO 7 7 APPENDIX G Quick Reference Guide Np 0 corte ae whe sade ade cee tone see nates 41 7 8 APPENDIX H Project Contacts and Timelines 0 cece einen AD 7 9 APPENDIX GIOSSAPY ainkaan een Aa ECD Evaluation User Guide 2 This guide is intended for the use of ECD Coordinators participating in the ECD Eva
36. luation Project The project is a collaborative effort of representatives from Success By 6 Children First the Community Action Program for Children CAPC United Way of the Lower Mainland BC Ministry of Children and Family Development MCFD Public Health Agency of Canada PHAC Human Early Learning Partnership HELP and local BC communities The partners thank the Max Bell Foundation for their generous financial support of this project 2 1 Background In the spring of 2006 a group of funders of early childhood development initiatives in BC the BC ECD Funders Table agreed upon four shared long term goals for early childhood programs and services 1 Mothers are healthy and give birth to healthy infants who remain healthy 2 Children experience healthy early child development including optimal early learning and care 3 Parents and families have the knowledge resources and support they need to help their children develop to their full potential 4 Communities support the development of all children and families Subsequently three initiatives CAPC Community Action Program for Children Children First and Success By 6 agreed to work together to design an integrated evaluation system related to goal 4 Phase 1 Their goals were to reduce the reporting burden on ECD capacity building initiatives that were having to report in different formats to different funders and develop a reporting system that had the capability of
37. ly Submission of evaluation data and Community Action reports to funders ECD Evaluation User Guide 4 3 1 What is Evaluation A straightforward dictionary definition of evaluation is to put a value on something Fundamentally evaluation asks the following questions What Did we do what we said we would do Why What did we learn about what worked and what didn t work So what What difference did it make that we did this work Now what What could we do differently Then what How do we plan to use evaluation findings for continuous learning alr gad tae 3 2 Why Evaluate Evaluation has two main purposes for learning and development and for accountability i e to show others that your program is effective For learning and development Evaluating your program will help you assess how well you are doing in order to help you do it better It is about asking what has happened and why what is and what is not working It is about learning more about your activities and then using what you have learned for improvement For accountability to show others that you are effective Funders and other stakeholders want to know whether a project has spent its money appropriately Evaluation allows you to provide evidence of your success and strengthen the case for continuing your program Evaluation should not only answer questions It should also prompt fresh thinking within your organization and with your contacts
38. mation that will assist you in your ongoing planning for children and families The purpose of this survey is to gather information about the outcomes of community capacity building work from stakeholders at local ECD tables province wide The survey looks at three aspects of community capacity building work Community Planning and Coordination Community Awareness and Mobilization and Service Delivery Your responses are confidential in that you will not be identified as a respondent They will simply be grouped with other responses to provide an overall look at the outcomes of the ECD work in your community The results of surveys in your area will be collated and sent back to your community coordinator It is expected that your ECD table will then discuss the findings and decide what if any changes your table wants to make as a result of this discussion A brief report on how your table intends to use the findings along with the details from your coordinator s Outputs report will constitute an annual report to your ECD funders How to Complete the Survey The survey consists of a number of statements related to building community capacity for ECD You can choose to answer Agree Disagree or Don t Know Please choose the answer that most closely matches your opinion If you don t know the answer to some questions that s OK it is not expected that you will know the answer to every question If you have any questions about the surv
39. meetings Special table meetings e g strategic planning session Table subcommittees working groups meetings action teams etc 12 Please tell us the number of coordinator manager hours per week that are funded for your ECD planning table Number if hours per week 13 Approximately how many months has the current coordinator manager been in the position Approximate NUMBER of months 14 Please list the ECD research related activities that your ECD community planning table has undertaken or updated to guide the work of the local initiative Please check all that apply Research Activity Not done Planning to do Done in the Done in the this in the last 12 last 3 years coming year months Local inventory of ECD programs and services Q Q Q Mapping of ECD assets Q Q Q An analysis of ECD funding in your community Q Q Q Analysis of community socio demographic characteristics Q Q Q Early Development Instrument EDI data review 5 Q Information gathered from parents on needs and gaps Information from parents on barriers Information from parents on satisfaction Information from service providers on needs and gaps Information from service providers on barriers Information from service providers on satisfaction Information from child care preschool providers on needs and gaps Information from childcare preschool providers on barriers
40. member OStrongly Agree OAgree ONeither Agree or Disagree ODisagree OStrongly Disagree O Don t know ECD Evaluation User Guide 33 12 How do you know that your ECD community planning table s work to increase community awareness and mobilization has had an effect Please provide an example of a change that has taken place in your community that could be attributed to your awareness raising activities E a Would you Say that 13 Services and supports have improved in our community over the past 12 months as a result of our ECD community planning table activities OStrongly Agree OAgree ONeither Agree or Disagree ODisagree OStrongly Disagree O Don t know 14 As a result of out ECD community table work we have made EC services more accessible to families with young children in the following groups over the past 12 months e g programs delivered in various languages satellite or mobile programs bus taxi vouchers extended days and hours of service changed location of services Agree Disagree Don t know Low income families Q Q Q Aboriginal Families Q Q Q Immigrants and refugees Q Q Q Children with special needs Q Q Q Families in rural and remote Q Q Q communities Other Q Q Q Please specity the other 15 Over the past 12 months the ECD community planning table has supported training opportunities that are leading to an increase in knowledge of best practices in early childhood development in our community
41. munity planning table that is representative of your community over the past 12 months l E 2 We are working well together at the ECD community planning table to strengthen ECD services and supports i e progressing towards better coordination collaboration Q Agree QODisagree OQDon t know ECD Evaluation User Guide 25 2a If you selected Agree please provide an example of one thing your ECD community planning table has accomplished in the past 12 months because you are working well together to plan for ECD services and supports E 3 Our ECD community planning table used research to inform planning and to guide the work of the initiative over the past 12 months e g community socio economic data information on current services service gaps use of data from the Early Development Instrument EDI information from parents OAgree O Disagree OQO Don t know 3a If you selected Agree please provide an example of an improvement change your ECD community table has made or influenced as a result of research findings during the past 12 months E 4 Our ECD community planning table used our strategic community plan to guide or influence initiatives in our community over the past 12 months if you don t yet have an ECD plan do not answer this question Q Agree QODisagree ODon t know 4a If you selected Agree please provide an example of a goal or objective in your ECD table s strategic community pl
42. n example of how an influential person in your community has helped your ECD community planning table to achieve one of your goals in the past 12 months E f7 Would you say that 12 ECD services and supports have improved in our community over the past 12 months as a result of our ECD community planning table activities OAgree O Disagree OQDon t know ECD Evaluation User Guide 28 12a If you selected Agree please provide an example of a change improvement to service delivery in your community in the past 12 months as a result of your ECD community table s work a 13 As a result of out ECD community table work we have made EC services more accessible to families with young children in the following groups over the past 12 months e g programs delivered in various languages satellite or mobile programs bus taxi vouchers extended days and hours of service changed location of services Agree Disagree Don t know Low income families Q Q Q Aboriginal Families Q Q O Immigrants and refugees Q Q O Children with special needs Q Q Q Families in rural and remote Q Q Q communities Other Q Q Q Please specify the other 13a If you selected Agree for any of the above please provide one or more examples of how you have increased accessibility for any of these groups over the past 12 months E B 14 Over the past 12 months the ECD community planning table has supported training opportunities tha
43. nd Mobilization and Service Delivery Your responses are confidential in that you will not be identified as a respondent They will simply be grouped with other responses to provide an overall look at the outcomes of the ECD community planning table work in your community The results of surveys in your area will be collated and sent back to your community coordinator It is expected that your ECD community planning table will then discuss the findings and decide what if any changes your table wants to make as a result of this discussion A brief report on how your table intends to use the findings along with the details from your coordinator s Outputs report will constitute an annual report to your ECD funders How to Complete the Survey The survey consists of a number of statements related to building community capacity for ECD You can choose to answer Strongly Agree Agree Neither Agree or Disagree Disagree Strongly Disagree or Don t Know Please choose the answer that most closely matches your opinion If you don t know the answer to some questions that s OK we don t expect that every respondent will know the answer to every question If you have any questions about the survey please contact the ECD coordinator who sent you the survey link At the end of each section click on Next When you reach the end click on Done You will know that your survey has been uploaded to the Survey Monkey site when you click Don
44. nformation Conflicting Expectations Social Distance Parental Consciousness Barriers that agencies encounter when trying to provide services e Agency mandate too narrow Agency does not have the skill base Agency does not have the resources No inter sectoral group to work with Planning cycles longer than the cycles of mobility amp economic security e Federal and provincial agencies provide funding incentives to entice local government and other stakeholders to assume responsibility for providing ECD services without transferring sufficient taxation authority Source BC Atlas of Child Development pp 141 142 Benchmarks are performance data that are used for comparative purposes A program can use its own data as a baseline benchmark against which to compare future performance It can also use data from another program or industry standard as a benchmark For the purpose of the evaluation Community will be negotiated with each site It could mean a municipality the Aboriginal people within a certain geographic area etc In ECD Evaluation User Guide 43 Community leader Data Analysis Data Interpretation ECD Plan Goal Inputs Local initiative Local decision making Mean Median this context Community could also be described as community table ECD table or local initiative People in the community who have a high profile and some influence as opinion leaders
45. nts who agree that 1a Services are improved as a result of better coordination 2a Services are more accessible to families and young children 3a Staff and volunteer training mentorship has led to improved service delivery 1 Increased service coordination and integration 2 Increased accessibility and of community table participants who can provide examples related to the 3 Improved knowledge of best practices in ECD above indicators ECD Evaluation User Guide 13 This is the survey as it will appear on the Survey Monkey 7 2 APPENDIX B ECD Managers Survey Thank you for choosing to complete the ECD Manager s Report The data from your community and other communities in BC will assist us in creating a comprehensive overview of the powerful community building work that is taking place around the province It will also provide your ECD community planning table with information that will assist you in your ongoing planning for children and families There are a total of 35 questions in this Report form Many of these can be answered quickly There is a copy of this form in the ECD Evaluation User Guide It is suggested that you use the form in the Guide to assemble information prior to completing this online version There are some questions that require you to enter actual numbers rather than check marks or X s If you enter an X or a check mark where a number is required the Survey Monkey will take you back
46. ocal partner in our ECD community table s work please take the time participate in the evaluation by completing an online Stakeholder Survey by ate To access the Stakeholder Survey simply click on this link SURVEY At the beginning of the survey you will be asked to enter a code for our ECD community table and to identify our ECD table s MCFD Region Our ECD Community Table Code is and our MCFD Region is Please note that you will not be able to proceed through the survey until you enter the code and region ECD Evaluation User Guide 9 Your participation is very important The data from our community will assist us with local planning and province wide data will be used to create an overview of the powerful community building work that is taking place around the province If you have any questions about the survey please contact me PS would appreciate if you would send me a quick note when you have completed your survey so that can cross you off my list Individual respondents to the Stakeholders Survey will be asked which sector they belong to e g Health MCFD Community Non Profit Agency but will not be identifiable by name The data entered will be automatically uploaded to the Survey Monkey site where central data processing can access it for the purpose of creating reports which will be sent to you by the end of August Copies of both the online Stakeholder surveys are attac
47. oups etc O Other please list ECD Evaluation User Guide 20 ba 23 Please tell us about the local funding allocation decision making processes that have been used by your ECD community planning table check all that apply Our initiative allocates any new ECD funding to an appropriate service provider L Our initiative makes recommendations related to the dispersal of any new ECD funding O Individual agencies apply for any new ECD funding independently O Individual agencies apply for any new ECD funding with the knowledge of our ECD initiative O Our initiative looks at all ECD funding old and new in our community and makes decisions related to allocation based on our research and community plan Our initiative has formal policies and procedures in place to oversee funding allocations O Other please specify E E 24 Please indicate if your ECD table had local decision making and or influence related to funding from the following sources in the past 12 months check all that apply Funding Source We had We made influence funding decision s CAPC CPNP AHS Children First Corporations business Local fundraising MCFD other than Children First Municipal Grants Health Authority School Districts Success By 6 O O O O O O O O O O O O Ol O O O O O O O O O Understanding the Early Years UEY United Way other than Success By 6
48. oved knowledge and ability to use research and evidence to guide initiatives 3 Increased use of a local ECD plan to guide initiatives 4 Increased use of evaluation for improvement Intermediate outcome Improved local awareness and mobilization Short Term Outcomes 5 Increased community awareness of the importance of the early years 6 Increased ability to resource locally identified priorities 7 Increased community engagement ECD Evaluation User Guide 37 Intermediate outcome Improved service delivery Short Term Outcomes 8 Increased service coordination and integration 9 Increased accessibility 10 Improved knowledge of best practices in ECD ECD Evaluation User Guide 38 Part 2 Community Action Report Date submitted Name of Initiative Our Initiative is __ Children First _ Success By 6 ___ Both Children First amp SB6 Area Region Coordinator Name Address Phone Email 1 Process Please describe what process you used to share and discuss your evaluation results with the community a ____ ECD table coalition met to review and discuss b Other please describe 2 Improved Community Driven Planning and Coordination a What are your successes b What are your challenges c What action do you plan to take to build on successes and or address challenges 3 Improved Local Awareness and Mobilization a What are your successes b What are your challenges c What action do you
49. ovide participants with copies of the outcomes report in advance of the workshop It will also be helpful for them to have copies of the one page outcome reports for your initiative and the provincial 3 Because of the length of the Outputs report and the time it would take for your group to consider all the outputs it is suggested that you focus your workshop on the Outcomes Report However the Outputs are important too and it is suggested that you prepare a brief presentation to the group for the start of the workshop providing a synopsis of the Output information along with any recommendations you have related to the outputs Alternatively you could form an Ad Hoc committee to consider the Outputs data and report back to your table with any recommendations In either case it will likely be helpful to have a few Outputs reports on hand at the workshop for people to refer to if necessary ECD Evaluation User Guide 36 4 The workshop itself if you have a large enough group you could divide the participants into three sub groups with each one looking at your outcomes under each of the Intermediate Outcomes headings If you have a small group you will need to work through each set of outcomes with the full group Plan your time accordingly Ask the group s to choose a facilitator and recorder Then engage in discussion of each of the short term outcomes a What do the examples provided by stakeholders who responded to the survey tell
50. plan to take to build on successes and or address challenges 4 Improved Service Delivery a What are your successes b What are your challenges c What action do you plan to take to build on successes and or address challenges 5 Comments anything you want to say about the evaluation process or other comments Please submit your report in electronic format to veradyo shaw ca by November 15 2008 ECD Evaluation User Guide 39 7 6 APPENDIX F One Page Project Information Handout ECD Evaluation Building Community Capacity April 2009 Be part of a historic first As far as we know this is the first time in Canada that a number of funders and local groups are working together to develop and implement a common evaluation process It arises from the vision and ultimate outcomes adopted by ECD Funders in 2006 These are Vision Children are healthy and develop to their full potential Ultimate Outcomes 1 Mothers are healthy and give birth to healthy infants who remain healthy 2 Children experience healthy early child development including optimal early learning and care 3 Parents and families have the knowledge resources and support they need to help their children develop to their full potential 4 Communities support the development of all children and families We are now rolling out the evaluation process province wide for long term outcome 4 building community capacity for ECD The partners who worked together to m
51. rm may apply to government private sector organizations and groups and individuals The policy process includes the identification of different alternatives such as programs or spending priorities and choosing among them on the basis of the impact they will have Policies can be understood as political management financial and administrative mechanisms arranged to reach explicit goals A customary way of operation or behaviour A practice may not be enshrined in policy but a change in practice indicates a change in the way things are done or a changed pattern of behaviour Rates indicate how common an event is in the population being studied For example consider a birth rate If there were 203 births last year in your community to females aged 15 44 and 5 775 females ages 15 44 then you would calculate the birth rate as follows 203 births to females ages 15 44 5 775 females ages 15 44 0351515 x 1 000 35 births per 1 000 females 15 44 People or groups having an interest or stake in an issue Data that is expressed in numbers Quantitative data looks at the incidence and quantity of events Data gathered through quantitative methods is numerical and may be analyzed by calculating averages ranges percentages and proportions Data that is expressed in narratives or words Qualitative data is information that is primarily expressed in terms of themes ideas events personalities histories etc ECD Evaluation User Gu
52. s contained in a data set For example to calculate the mean of 50 35 41 and 68 first determine the sum of 50 35 41 68 which is 194 Then divide by 4 The mean is 48 5 The middle of a data set It is determined by arranging the data set in numeric order With an odd number of data measurements the median is the middle number For ECD Evaluation User Guide 44 Objective Outcomes Outcome Indicators Outputs Partnership example the median of 6 6 7 8 and 100 is 7 the middle data point For example assume the wages per hour of several workers were 6 6 7 8 100 The median income of this group would be 7 00 The mean income of this group is 25 40 and is heavily influenced by the presence of the one highly paid worker An objective is a statement of aim or a strategic position to be obtained An objective should be measurable In other words objectives are steps that tell in detail how goals will be accomplished Example All teachers working in early childhood programs in BC will have a bachelor s degree in Early Childhood or a related field Outcomes are benefits for participants during or after their involvement with a program Outcomes may relate to knowledge skills attitudes values behaviour condition or status Examples of outcomes include greater knowledge of nutritional needs improved reading skills more effective responses to conflict getting a job and having greater financial stabili
53. t are leading to an increase in knowledge of best practices in early child development in our community Q Agree QODisagree OQO Don t know 14a If you selected Agree please provide an example of how an increase of knowledge of best practices has improved service delivery for children and families in your community over the past 12 months E B ECD Evaluation User Guide 29 15 Over the past 12 months the ECD community planning table has supported training opportunities that are leading to an increase in knowledge of best practices in early child development in our community OAgree O Disagree ODon t know 15a If you selected Agree please provide an example of how an increase of knowledge of best practices has improved service delivery for children and families in your community over the past 12 months B E 16 Please tell us what sector you represent at your community ECD table If you are bringing the perspective of more than one sector to the ECD table check all that apply e g community non profit and CAPC LIMCFD Ministry of Children and Family Development LICAPC Community Action Program for Children O Children First L1 Success By 6 and or Council of Partners O Understanding the Early Years O Aboriginal Band O Aboriginal community members O Aboriginal ECD planning table LIAHS Aboriginal Head Start O Business private sector LICPNP Canada Prenatal and Nutrition Program O College University
54. ties are guiding the allocation of new funds becoming available for ECD programs and services in our community e g funds from municipal provincial federal or other sources OStrongly Agree OAgree ONeither Agree or Disagree ODisagree OStrongly Disagree O Don t know 9 Our locally identified priorities are guiding the re allocation of existing funds i e we are looking at all the funding that comes to our community for ECD and re allocating where necessary to achieve our shared objectives OStrongly Agree OAgree ONeither Agree or Disagree ODisagree OStrongly Disagree O Don t know 10 There has been an increase in the number of volunteers who participate in community ECD activities over the past 12 months e g helped to organize or run ECD table led community events or research projects participated in planning sessions etc Volunteers are people who do not receive any financial compensation for the time they spend participating in your ECD activities OStrongly Agree OAgree ONeither Agree or Disagree ODisagree OStrongly Disagree O Don t know 11 in the past 12 months we have attracted influential community members to our community events and activities Every person holds influence based on their connections and skill sets In this case we are looking for influential community members who are connected with infrastructure and relationship bridging influence such as business leaders chief of a First Nation municipal council
55. to the place s where you made an error and ask you to enter a number before allowing you to advance to the next page The survey allows only one response per computer You may return to your survey to update change your responses but you must do so on the same computer you used to start with 1 Before proceeding please enter the ECD Community Planning Table Code that was sent to you with the survey link here You must answer the first 2 questions before proceeding Your Code and Region are in the email you will receive with the survey link 2 Which MCFD Region of the province is your initiative in O North O Fraser O Interior O Vancouver Coastal O Vancouver Island ECD Evaluation User Guide 3 Please identify the initiative you are reporting for O Children First O Success By 6 L Children first and Success By 6 Other please specify 4 Please tell us the name of your ECD planning table NS 5 What is your catchment area geographical coverage The communities for which your initiative kK a 6 Please tell us your name and title plans makes decisions 7 Please provide your phone number and email address f 8 Please tell us how many individuals you have invited to complete the Participants Stakeholders Outcome Reporting survey this allows us to compare to your response rates h H ECD Evaluation User Guide 15 9 How
56. ty There can be various levels of outcomes with short term outcomes leading to longer term outcomes For example a child in a pre school program may receive assistance to improve social and literacy skills which can lead to success in school and eventual high school graduation Outcome Indicators are the specific items of information that track a program s success on outcomes They describe the observable measurable characteristics or changes that represent achievement of an outcome For example a program whose desired outcome is that participants pursue a healthy lifestyle could define lifestyle as not smoking maintaining a health weight blood pressure cholesterol level etc The number and percentage of program participants who demonstrate these behaviours is an indicator of how the program is doing with respect to the outcome Outputs are the products of a program s activities such as the number of meals provided classes taught brochures distributed or participants served Another term for outcomes is units of service A program s outputs should produce desired outcomes for participants A group of agencies and citizens working together to improve policies practices and services for children 0 6 and their families ECD Evaluation User Guide 45 Policy Practice Rates Stakeholders Quantitative Data Qualitative Data A policy is a plan of action to guide decisions and actions The te

USER'S GUIDE - Success By 6

Contents

Download Pdf Manuals

Related Search

Related Contents