There are four primary facets of efficacy. In this illustration, a result of the Global Victoria EdTech Innovation Alliance — Innovation Workshop, discover the key learnings from the provocation on efficacy. 

Efficacy: Key Takeaways

Research evaluation is a key part of evaluating the efficacy of a product or solution. Research evaluation describes how the project has been implemented, usually in relation to the intended process and expected milestones, effects or activities. This is what you deal with along the way. Outcome evaluation tells us what kind of change has occurred, typically in the target population in reference to the stated objectives. Impact evaluation paints a picture as to how a program might have affected participants’ lives on a broader scale.

There are four primary facets of efficacy:

Process
Outcomes are not the only measure of success. Rapid prototyping, for example, is a process-based evaluation that informs design. It not only informs what you did, but is a marker of what you will be doing. Consider how your product draws upon learning theory, aligns with good educational practices, and ensures it is ethical to use for both learners and teachers.

Intended outcome
Qualitative data can reveal contradictions. Intended outcomes are easy to write, but can be difficult to achieve in regards to measurement.

Acceptability
Whether users enjoy engaging with a product is not the same as if the product is effective for learning and teaching outcomes. However, how much a student likes a solution is important. It can be a gatekeeper for if it can be used in other departments or at other schools. For example, if a service nudged a learner each time an assessment came up, they might find this irritating. Despite the nudges potentially having a positive impact on whether they turned in the assignment in on time, it does not gain the desired acceptability.

Feasibility
This takes into account the cost, resources and timing of an intervention. Expensive is also educator cost or student cost.

Efficacy = P(rocess) x I(ntended outcome) x A(cceptability) x F(easability)

This workshop was led by two researchers, Professor Margaret Bearman, Deakin University, and Professor Michael Henderson, Monash University. 

The Global Victoria EdTech Innovation Alliance program is funded by the Victorian Government as part of the $3.6 million International Research Partnerships program under the International Education Short-Term Sector Recovery Plan.

For more information about the initiative visit Global Victoria EdTech Innovation Alliance.