Program evaluators typically have to pay attention to two opposing sides of interest and try to balance them out carefully. On the one hand, they have a stake of their own in the evaluation. In the interest of being recognized professionally, they have to please the study sponsor. If the evaluator is a staff person within a non-profit, she would be concerned about pleasing her boss and colleagues and thus advancing her career. If she is from outside the non-profit and a funder has sponsored the study, she is concerned about pleasing the funder so that they send more work her way.
On the other hand, she has an obligation to the honesty and integrity of her work. This conflict will come to light if you consider a simple example: Many studies involve using a random sampling technique in order to determine the participants of the study. If the evaluator is asked to exclude a site X from the sample of the participants because it may bring to light some of the drawbacks in the program delivery, the evaluator has to consider her obligation to the integrity of the study very seriously and then make a decision.
When organizations are considering hiring a program evaluator as an employee or as a consultant, it is always better to have her report to or work within a team other than the program delivery team itself. This will help the program evaluator tremendously to maintain their integrity .
I would also like to make an analogy of this scenario to the corporate world. In the corporate world, I have hardly ever seen a quality assurance team belong to a product-making or a service-delivery division. The quality assurance is typically a parallel team created to be a neutral space between the end-customers and the products/services team. In the non-profit world, the clients are the funders and constituents, and the program evaluation team could be considered as the neutral space between the program delivery team and the funders and constituents.
Like the comparison to quality assurance
Gayathri,
I think this is great comparison. Unfortunately for many small to mid-sized nonprofits, there aren't the resources avaible for external evaluation unless there is a big federal or foundation grant that funds an independent evaluator. The same people running the programs are often then performing the evaluation.
With Sarbanes Oxley putting more emphasis accountability and the role of the Board of Directors in corporations, I wonder if we ought to have program evaluation committees as standard if not required part of Board composition. In this way, even small organizations could have some independent oversight of evaluation.
What do you think?