What questions do we want to answer? 

Evaluation questions provide the foundation upon which the evaluation is built. These questions should outline the issues and concerns of the educator and other program stakeholders, as well as addressing the presence and quality of the logic model. It should be noted that evaluation questions are separate and distinct from survey questions or items. Evaluation questions lay out the priorities of the evaluation and outline the information that is sought during the evaluation in a global sense, whereas survey questions are used to examine changes in participant outcomes and explore their experiences (when these topics are included in the evaluation priorities). The questions below provide an example of this distinction.

Evaluation Question:	Survey Item:
Do participants demonstrate an improvement in their confidence about financial behaviors after participating in financial education services?	I feel more confident about my ability to save money regularly (response options: strongly disagree to strongly agree

 

Evaluation priorities commonly are determined by the program’s stage of maturity. This means that new programs or those in early phases of implementation should focus on the early components of the logic model including stakeholder needs, resources and the implementation of key activities. Common evaluation questions for new programs seek to understand if there are enough high-quality resources to provide educational opportunities for the target participants and to determine the quality of activities provided. These types of questions are common in formative evaluations.

 

Programs that have a longer history of implementation may consider exploring questions related to short- and long-term outcomes and impact — common themes of summative evaluation processes. As evidence is established to support the existence of a strong, consistent and high-quality program or intervention, evidence can be collected to explore more rigorous evaluation questions related to impact, such as those employing an experimental design (randomized control trial) or quasi-experimental design. Programs or interventions interested in summative impact evaluation should first establish several prerequisites to ensure the success of this type of evaluation, including a sustainable, stable intervention with demonstrated fidelity to intervention models and sufficient exposure or dosage for program participants. These prerequisites can be established via formative evaluation processes.

Writing Evaluation Questions

Program History	Common Evaluation Questions	Type of Evaluation
New programs or early stages of program	What are the critical needs of the program participants? Is the program targeting the appropriate recipients?	Needs assessment, Formative evaluation
	Is the program or intervention being delivered in a consistent manner? Are the program services being implemented in a high-quality manner?	Formative evaluation
Mature, stable, late-stage programs	Are the desired outcomes being obtained? Do participant outcomes improve over time? Do participant outcomes differ across characteristics of the programs or the participants?	Summative outcome evaluation
	After participating in the program, do participants possess greater outcomes than nonparticipants? Did the program cause the desired impact?	Summative impact evaluation

 

There are many different types of evaluation questions that can be useful for financial educators. The main goal is to determine what information would be useful for you and your program’s stakeholders (e.g., participants, parents, school administrators, funders) and write questions to address these needs. Below are some example evaluation questions that address several components of the logic model and are relevant to financial education programs.

 

Needs Assessment

• What financial education topics are most relevant to the participants?
• In which topic areas are youth lacking essential financial knowledge?
• Which financial education topics are participants most interested in learning?

Resources

• Is the financial education curriculum relevant to the participants’ characteristics, needs and interests?
• Are there adequate materials for participants and an appropriate space for program implementation?

Activities

• Are participants getting sufficient exposure to the program or intervention?
• Are the services offered being implemented as intended and of sufficient quality?

Outcomes

• Is the program achieving the goals and objectives it intends to accomplish?
• Does the value or benefit of the program exceed the cost of producing it?
• Can the intended outcomes be linked back to the program, as opposed to other influences?

As mentioned above, formative evaluations commonly examine the needs, resources and activities implemented, and summative evaluations are more focused on outcomes and impact. Again, it can be crucial to include evaluation questions related to the resources, and quality of activities for a summative evaluation because these evaluation questions will lead to data that can be useful in explaining trends in participant outcomes. This is particularly true when outcomes are not as expected or particular groups of participants experience greater changes in outcomes than other participants. Trends in needs, resources and implementation may explain surprising findings.

This section provided instructions on how to prepare for the evaluation by setting up the evaluation team, developing a logic model, and writing the evaluation questions. Proper planning for the evaluation will improve the likelihood of conducting a high-quality evaluation and producing useful information. The following section presents information on how to conduct the evaluation of interest.