Learning from Evaluation Mistakes with Kylie Hutchinson

Kylie Hutchinson Learning from Evaluation Mistakes episode on EvalNetwork Podcast

Kylie is the 2020 recipient of the Canadian Evaluation Society’s Award for Contribution to Evaluation in Canada. She has spent the past thirty years working in the not-for-profit sector as a consultant, trainer, program manager, board member, and volunteer. 

There are many valuable nuggets from this video. We discuss the following during the interview:

– Reasons why Kylie loves evaluation.

– Why evaluation is relevant for health and human services students who are not necessarily interested in research or evaluation.

– Why she wrote Evaluation Failures: 22 Tales of Mistakes Made and Lessons Learned.  She highlights some of the most important  lessons from the book.

– Kylie talks about how everyone makes mistakes and the key is learning from them.

– She reviews specific strategies for engaging project stakeholders in an evaluation.

– Kylie provides her favorite resources regarding program design.

– How mindfulness factors into her work and how it can inform the work of evaluators.

– Books she recommends.

You can connect with Kylie through https://communitysolutions.ca or @EvaluationMaven at Twitter.

Watch the video recording of our interview here:


James Pann smiling at the camera, sitting in front of green trees

James Pann, Ph.D. is a Professor at Nova Southeast University and a highly experienced psychologist and evaluator with nearly 25 years of experience. He conducts research and evaluation projects with non-profit organizations in the fields of health, human services, and education, and has received funding from multiple government agencies.

James holds multiple degrees including a Ph.D. in Counseling Psychology, an M.S. Ed. from the University of Miami, and a BBA in Accounting from the University of Texas at Austin. He is also the host of the EvalNetwork podcast, a frequent conference presenter, and has published several peer-reviewed research articles and co-authored a book. James currently resides in Miami, Florida with his family and enjoys backpacking trips. Find out more about his work here.


Retrospective pretest designs

The Power of Retrospective Pretests to Address Common Survey Research Challenges

James Pann interviews Melanie Hwalek, Ph.D., a program evaluation consultant, to discuss the retrospective pretest (RPT) design, focusing on its practical applications and the findings from her recent research detailed in the paper, “Designing a Questionnaire with Retrospective PrePost Items: Format Matters.” RPT is particularly useful for evaluating changes in participants’ perceptions or self-assessments following

Read More »
Empowerment Evaluation

Empowering Change: David Fetterman on Using Evaluation to Build a Better World

David Fetterman is a leading expert in empowerment evaluation, an approach that emphasizes collaboration, participation, and capacity building. He has written extensively on the topic, and his work has been used in a wide range of settings, including government agencies, non-profit organizations, and businesses. David focuses on helping people evaluate their programs and initiatives. Moreover,

Read More »