What type of evaluation should be conducted? 

The evaluation questions and priorities should inform the type of evaluation conducted. In a simplified sense, there are two primary types of evaluation: formative evaluation and summative evaluation.

As the table below demonstrates, if you are using the evaluation to examine the needs of participants, the adequacy of resources, or the quality of implementation to aid learning and program improvement, it is recommended that you conduct a formative evaluation. If you are interested in documenting the benefits of the program to determine the effectiveness and impact, it is recommended that you conduct a summative evaluation process. Evaluations can, and often do, include components of both types of evaluations depending on the evaluation questions and priorities. In fact, it is useful to explore implementation (as common in formative evaluations) when you are interested in program effectiveness (summative evaluation) because information about the quality of services is useful for explaining why a particular project or intervention was successful or not. When evaluations include both formative and summative priorities, best practices in implementation can be identified to promote participant outcomes most effectively. The table below summarizes these types of evaluations, and the designs that are most commonly employed.

A good evaluation design addresses the evaluation questions, is appropriate for the evaluation context (e.g., time, resources), and provides sufficient and critical data. The table above provides a brief summary of the designs and the common methods that are typically involved in formative and summative evaluation. This list of methods and designs is not exhaustive, but provides a starting place to select the appropriate design and methods aligned with the evaluation questions and priorities. As with the other decisions made during the evaluation, the educator should consider several factors before selecting the data collection methods, including resources available, potential data sources that are already available or being collected, practicality, feasibility (particularly given the expertise of the evaluation team), and funding.

As the table below demonstrates, there are three primary evaluation designs that are suited for distinct evaluation goals and contexts. Evaluations conducted by financial educators likely will employ descriptive or correlational designs, given that more sophisticated evaluation designs require funding, resources and experimental control that is commonly is lacking at this level. Unless the educator possesses high levels of knowledge about experimental and quasi-experimental designs, it is recommended that educators seek external evaluation assistance from a professional evaluator to conduct these types of evaluations.

Another consideration is when and how often to collect data. Evaluation designs that collect data at only one time point are called a cross-sectional design. The most common cross-sectional design is to collect data only at the end of the program or intervention (post-test only). These designs limit your ability to examine changes over time in participant outcomes, however a comparison sample still can be used to explore differences between participants and nonparticipants after the program or intervention. The alternative is using a longitudinal design, meaning that evaluation data is collected at least twice or more during the evaluation. The benefit of longitudinal data collection is the ability to explore how outcomes change over time, such as comparing outcomes before the intervention (pre-test or baseline) to outcomes after the intervention (post-test).