|  
  |  

Evaluation Manual

How do we use our evaluation findings? 

Accountability

Meeting financial education needs of diverse societal groups is a challenging task because of limited resources. Therefore, financial educators must be prepared to distinguish effective financial education programs from ineffective ones. This determination can be achieved only if the educator uses program evaluation as a measure to discern effective programs. If the financial education program is effective in terms of bringing desired outcomes to the target population, the educators should continue to implement the program. Additionally, evaluation is useful as a public accountability measure to secure stakeholder support for effective financial education programs. Communication of evaluation findings with stakeholders such as funding agencies, decision makers and policymakers is essential to convince them about the importance of investing in financial education. Financial education evaluation provides a means to receive due attention and garner support from the public and policymakers.

Funding

Funding is essential to continue financial education programs. Funding agencies look for successful programs and are sensitive to cost effectiveness. Therefore, it is important to ensure the cost effectiveness of financial education programs. Cost effectiveness can be illustrated using evaluation data to distinguish effective programs from ineffective programs. Competition for limited funding is very high. Therefore, the educator must be prepared to present strong financial education programs as examples when applying for funding. Follow these important steps when presenting evaluation data to support continuous funding.

  • Make evaluation an integral part of the program.
  • Document participant reactions (e.g., satisfaction, relevance to their lives).
  • Document evaluation findings about program quality to demonstrate strong implementation. o Clearly communicate program outcomes with the funding agency.
  • Communicate outcomes with potential funding agencies to seek future funding.

Communication with Stakeholders

Partnerships involve two or more individuals, groups or organizations working together for a common goal. When two or more partners work together, it is important that they mutually understand the program development and delivery process. If changes are made to program delivery, all the partners should understand the rationale for those changes. Formative evaluation data can be utilized to justify the changes needed without being biased to one partner. This is an essential element to managing partnerships.

At the end of the program, partners and stakeholders normally like to share credit for the program and be informed about the program’s successes. The following tips are helpful when using evaluation results for building strong partnerships:

  • Keep partners informed about the evaluation plan.
  • Share ongoing evaluation data about program quality with partners on a regular basis.
  • Document and share outcomes with partners to highlight the worth of the partnership.
  • Acknowledge the contribution of each partner.

Marketing

Evaluation findings also can be useful for marketing the program to potential participants or the broader community. Documenting improvements in financial education made by participants or participants’ positive perceptions of the program can be persuasive for program marketing materials.

Program Improvement

Evaluation data can be used to inform program improvement efforts. Reviewing the evaluation findings helps the educator decide whether the program is achieving set objectives or not. If the program is not being implemented in a high-quality manner or the program outcomes are below expectation, the educator needs to identify alternatives to modify the program before presenting it next time. Program improvement can be done only if the educator has collected formative evaluation data to identify strengths and weaknesses. If the strengths and weaknesses of the program have been identified, the educator can plan alternatives to eliminate weaknesses and capitalize on strengths to make the program stronger next time (or to conduct mid-course corrections).

Use these helpful tips to use evaluation data for the improvement of educational programs:

  • Compare outcomes with the program objectives and goals.
  • If outcomes are below expectations, find alternatives to improve the program.
  • Review strengths and participants' comments to identify alternative approaches to address weaknesses.
  • Identify or develop alternatives before the next program cycle.
  • Review process evaluation data such as participants' ratings of instructors and education materials.
  • When there are low ratings of educational materials or instructors, modify that item or find alternatives before presenting the next program.
  • Perform continuous evaluation to further improve the program. This process maximizes the cost-effectiveness of financial education.

Continuous Quality Improvement

continuous quality improvement: plan, do, study and act

Evaluation processes are an important component of continuous quality improvement. Continuous quality improvement (CQI) is a systematic and ongoing process of improving programs and services. The CQI process is an ongoing cycle of planning for an intervention, implementing the intervention (do), evaluating the implementation and effectiveness of the intervention (study), and acting to make improvements based on the evaluation findings (act).

Financial educators should be engaging in systematic evaluations of the financial education services provided and using these evaluation findings to inform future implementation of program services. This process ensures that decisions are made based on the evaluation findings (data-driven decision making) and that the program or intervention quality is improved in a proactive way, as opposed to being reactive to issues and challenges