Tools & Templates
Level 1
LLM evaluations
Hierarchical AI Evaluation by Gamma
LLM evaluation in the social sector
Level 2
The tech industry has published numerous guidebooks and tools to help you define, collect, and analyze user funnel metrics. For details on how to construct common metrics, consider reviewing The Agency Fund’s User Funnel Playbook.
In addition, you can leverage these reference materials:
For more details on A/B testing, please review these resources:
Level 3
Case Study: ChatSEL is a GenAI coach developed at the Agency Fund that provides teachers with evidence-based and context-sensitive guidance on understanding and implementing SEL programs in a low-resource classroom. Please see the following document for how we might measure Level 3 outcomes in the context of ChatSEL.
User Evaluation Workshop - ChatSEL
Process Evaluations
IDinsight. “Process Evaluation.” IDinsight Impact Measurement Guide, https://guide.idinsight.org/process-evaluation/
World Health Organization. Monitoring and Evaluating Digital Health Interventions: A Practical Guide to Conducting Research and Assessment. World Health Organization, 2016. https://saluddigital.com/wp-content/uploads/2019/06/WHO.-Monitoring-and-Evaluating-Digital-Health-Interventions.pdf
Implementation Monitoring and Process Evaluation (Practical Guidebook) Bliss, M. J., & Emshoff, J. G. (2018). Implementation Monitoring and Process Evaluation. SAGE Publications.
Last updated
Was this helpful?