What questions would we like evaluation to answer? A logical first step in evaluation is to achieve a clear description of the program as it was actually implemented “on the ground.” What happened at each site as a result of our funding? That question argues for an in-depth, qualitative assessment that captures not only the basic numbers of clients served, but more importantly, the “texture” of the program. Such an evaluation requires skilled observers and interviewers who are attuned to the cultural context of the organizations and the participants. That kind of analysis is also most likely to surface the participants’ true feelings about a program and how it might be improved in a way that typical questionnaires might not. It can also help document changes in the program itself over time, in response to corrective feedback.

As we are moving to grant programs that focus on the sustainability of grantee organizations, another important focus for inquiry is the measurement of institutional capacity. Have we strengthened the participating organizations? Too many evaluations focus on the “theory” of the intervention, ignoring basic dimensions of organizational functioning that can be absolutely key to the quality of the programs, and thus the ultimate success or failure of the project. Although there are not many examples in our field, we have begun to look across other sectors of philanthropy (e.g., youth development, the arts) for examples of grant programs whose primary purpose has been to build organizational capacity. There is much to be learned from their experience to inform our next steps. 

Much of the recent interest in evaluation among foundations has been in response to critiques raised both by board members and foundation observers that we have paid insufficient attention to the measurement of the outcomes of our grants. What was actually achieved? To be fair, the state of the art of assessment in health and human services, whether funded by government or private sources, has concentrated on process variables such as the number of patients treated, the number of classes presented, and short-term results such as scores on “post-tests.” Given the nature of much of our work¾primary and secondary prevention¾many of the ultimate outcomes we seek are not readily measurable. Is it sufficient to settle for intermediate measures? Which raises an additional question: