Book Review: The ABCs of Evaluation - Boulmetis & Dutwin

I've rediscovered another older book (2005) on the bookshelf. I was browsing round the home library searching for inspiration in developing an evaluation for a leadership development program. I remembered what I liked about this book the first time round.


The authors have produced a book which is set out logically. It covers the choices you have and steps to take to carry out an evaluation. I found the section describing different evaluation models and which one to choose was just what I was looking for:
1. Discrepancy Model: use when the program is so interwoven within the organisational context it is difficult to discover freestanding changes
2. The Goal Free model focuses on the participants needs and is independent of the organisation's needs.
3. The transactional model is similar to the goal free model with the exception that the evaluator plays a more active role in the evaluation process.
4. The decision makingg model focuses less on the success of the program and more on the longer term sustainability
5. The most popular model is the goal-based where the outcomes are assessed against the organisation's objectives (or program objectives)
The authors do also list other evaluation models and I would be interested in using systems analysis with my healthcare programs. The version using art criticism where you bring in an expert in the field to review progress is an interesting one I had never thought of. The adversary model where participants present their views and a "jury" hears their evidence might be useful for the rapid assessments we sometimes need to make.

The evaluator's role is covered as a continuum from the start of the program through to the end as well as the level of active participation in the evaluation process.

I was interested in what an evaluation is supposed to measure. Having got used to most healthcare evaluations answering "has the program delivered its objectives" it was good to see that "was the program efficient in terms of cost and staff time - was the expense worthwhile?". What we often mean when we say sustainability, the authors use impact. I like this as it covers not just the sustainability of the results but also the magnitude of the results as contributing to the organisation's long term objectives.

The book also includes the types of data, how to present it and the process of writing up your report.

I'd recommend this book if you're starting out in evaluation or need a refresher. If you're a seasoned program evaluator I suspect you will find other books go to more depth.