Evaluators Role and Purposes

{{active_language_id_uc}} {{settings.doi}}

Evaluator Titles and Jobs

While it is true that everyone conducts informal evaluations all the time, it is also true that those who conduct formal evaluations do not always identify themselves as evaluators. 

Most jobs require evaluation skills and abilities; not all job descriptions specifically articulate this as a requirement. People conducting evaluations as a part of their job go by various names and titles.  Some of these include – auditor, assessor, analyst, judge, compliance officer, arbitrator, counselor, consultant, director, specialist, manager, supervisor, and advisor. Specific jobs for evaluators in educational settings include – evaluator, teacher, the office of teaching effectiveness and innovation, instructional designer, training and support coordinator, evaluation and research assistant/associate/specialist, director of evaluation and assessment, curriculum specialist/director, program manager/coordinator, assessment/data analyst, and institutional researcher.

Internal and External Evaluators

We should also make a distinction between internal and external evaluators. Internal evaluators, those working within an organization as evaluators, work as full- or part-time employees. When evaluators work as consultants, completing as-needed contract work, they are referred to as external evaluators. There are benefits and disadvantages to both.

When an organization is committed to evaluation and the benefits of continuous improvement, and the organization is large enough to warrant hiring full-time evaluators, it can be cost-effective and efficient to have internal evaluators on staff. Internal evaluators can develop an in-depth understanding of the organization, its purpose, goals, politics, structure, and personnel. They have access to the information and informants they need to do their job and can work on several interrelated evaluation projects (of various sizes and scopes) within the organization. This can be extremely beneficial to an organization. However, internal evaluators can become biased or jaded. They may begin to advocate for specific solutions for political reasons or base findings on unimportant criteria. If internal evaluators do not have some degree of autonomy or do not develop and maintain solid evaluative thinking abilities, professional ethics, appropriate soft skills, and healthy relationships with those they serve, their evaluation efforts can be ineffective and impeded by others in the organization. Above all, evaluators need to be trusted.

External evaluators are sometimes needed for various reasons. At times the evaluation needs of an organization exceed the capacity of internal evaluators. Those in the organization may not have time or expertise to complete a required evaluation. It can be cost-effective and prudent to hire an external evaluator to provide professional services on an as-needed basis. Sometimes, for political reasons, an evaluation needs to be perceived as unbiased and objective. An external evaluator’s reputation as a competent ethical evaluator can provide the organization with results other stakeholders and the general public can trust. There are, however, challenges an external evaluator may face. An external evaluator needs to gain an understanding of the organization’s goals and structure. They (or the evaluation team) need to develop relationships, get access to information and informants, understand how an individual project fits into the overall picture and the reasons the evaluation is being commissioned (i.e., needed).  

Working with Clients

You will recall that one of the main differences between research and evaluation has to do with the role the principal investigator plays in terms of who is responsible for making final decisions regarding the inquiry’s design (i.e., purpose, questions, and methods). Researchers typically work for themselves (i.e., they are in charge) while evaluators work for clients. There are exceptions, for example when a researcher works on an institutional research team or when an evaluator is conducting evaluation research. Still, evaluators typically work for a client as a service provider, on an evaluation team, or as an independent consultant. We use the term client as a broad description referring to the person commissioning an evaluation project. The client may be a supervisor or manager who assigns the project to an internal evaluator or a team of evaluators. The client may also be an employer hiring an external evaluator to conduct a specific evaluation. In either case, the role of the evaluator can be characterized by the involvement they have in the decision-making process and their responsibilities in implementing the evaluation. The relationship between the evaluator’s involvement in the project and their role is depicted in Figure 1.

Service Providers. When the evaluator is hired as an external consultant or is the director of a department, and they have an extensive say in determining the purpose, goals, and questions for an evaluation, as well as, the methods utilized to complete the evaluation, they might best be described as a service provider. In these cases, the evaluator (or their team) is typically responsible for the implementation of the evaluation. They will then report back to the client once the evaluation is completed.

Evaluation Partner. An evaluator may work as a partner with a client. This is what Patton (2010) calls developmental evaluation. The evaluator may be working as an internal or external evaluator, but when the evaluator’s role is one of partner, the control and the implementation of the evaluation responsibilities are shared between the client and evaluator.

Evaluation Consultants. An evaluator may also be hired solely as a consultant. In these situations, the evaluator lends their expertise to recommend the best way for the client to complete the evaluation. The evaluator may supervise and train individuals as they complete various evaluation activities. The purpose of an evaluator serving in this role is to build capacity and put in place needed evaluation processes so the organization can conduct its own evaluations without the evaluator in the future. Stufflebean (2014) warns however that this type of evaluation may become a pseudo-evaluation if the evaluator is expected to simply sign off on the evaluation as if the evaluation was completed by the evaluator and not the client.

Figure 1: Evaluator Role by Involvement.

References

Cronbach, L. J., & Shapiro, K. (1982). Designing evaluations of educational and social programs. Jossey-Bass,.

Patton, M. Q. (2010). Developmental evaluation: Applying complexity concepts to enhance innovation and use. Guilford press.

Stufflebeam, D. L., & Coryn, C. L. (2014). Evaluation theory, models, and applications (Vol. 50). John Wiley & Sons.