What is evaluation in financial education?
Evaluation is defined as “the systematic assessment of the operation and/or outcomes of a program, compared to a set of explicit or implicit standards, as a means of contributing to the improvement of the program” (Weiss, 1998). Financial education program evaluation is the process of systematically assessing the implementation of a financial education intervention and comparing learner achievements with program goals and objectives to determine the success or failure of the educational program.
These definitions point to two primary objectives of evaluation:
- exploring the operations or process of a financial education intervention, and
- exploring the learner outcomes or achievements as a result of participation in the intervention.
These two primary objectives, while not mutually exclusive, correspond the two broad types of evaluation: (1) formative evaluation, and (2) summative evaluation.
| Formative Evaluation |
Summative Evaluation |
|
Formative evaluation helps educators decide whether the program is meeting the needs of program recipients,whether the activities implemented are of high quality, and whether any improvements are required. By engaging in formative evaluation, the educator is able to identify whether the desired program activities and processes are being implemented with fidelity and quality. The educator can identify and capitalize on the program's strengths and identify and rectify the program’s barriers and weaknesses. A good formative evaluation provides data to support these areas, ultimately helping educators make program improvements.
|
Summative evaluation helps educators document participant outcomes associated with or attributed to financial education programs. Summative evaluation provides data to show whether or not a program is effective in promoting learning about financial education concepts and behavior change, including the actual and perceived benefits associated with services. These findings also can provide data to justify the continuation of the program or to request additional funding by illustrating the relative cost-benefit ratio. Documented program outcomes help funding agencies measure the worth of programs and allocate more funds to stronger educational programs.
|
|
Example: A formative evaluation of a financial education program might explore the participants needs and components of implementation quality, including:
- Does the financial education program or curriculum address participant needs?
- Is the information presented relevant to the participants?
- Are the materials being implemented as intended?
- Is the material presented in an engaging manner?
- Are there positive relationships between the educator and the participants?
- Do participants regularly use financial education services?
|
Example: A summative evaluation of a financial education program might explore the outcomes associated with participation in the program, and participant experiences, including:
- Do participants demonstrate improvements in intended outcomes after participating in the services?
- Is there a significant change in participants’ outcomes before and after participation?
- Do participants demonstrate better outcomes than nonparticipants?
- Are participants satisfied with the services provided?
- Do participants believe the services benefitted them?
- Do participants experience long-term benefits?
|
It is not uncommon for an evaluation to encompass both formative and summative priorities. This is particularly true for summative evaluation because it is important to examine the consistency and quality of implementation (features of a formative evaluation) when exploring participant outcomes. How the intervention was delivered will impact whether, or how much, the participants benefit from the intervention. In this way, information about how the intervention was implemented and delivered can help explain (or even be used to statistically predict) changes in participant outcomes and impact in summative evaluation processes.