A distinction has to be made between the dimensions of an evaluation and the criteria used to express such dimensions (Beghin 2009).
Dimensions
The dimensions are characteristic aspects to be assessed by the evaluation, such as in the expression "the dimensions of a problem". They are the standards on which the evaluation is based.
In the case of development research, five dimensions are most commonly considered :
- relevance for development
- scientific quality
- valorisation (of the research)
- performance (particularly when evaluating researchers and research teams)
- impact
Further dimensions relevant to development research have been considered in the annexes of the present Guidelines.
The object being evaluated can be considered from many angles: evaluation is necessarily multidimensional, in other words a variety of dimensions are needed in each case.
Each specific evaluation guide will privilege the dimensions best adapted to the local situation, the evaluation objectives, the discipline(s) involved, the degree of development of research in the country etc... A motivated selection of dimensions is an important component of the preliminary stages of writing a specific guide.
Criteria
A dimension can be expressed by using one or more criteria. An example of how a variety of criteria can be used to express a single dimension is the dimension scientific quality and its criteria : bibliometric indices, peer reviews, panels, etc.
Criteria vary widely according to the circumstances and have to be selected on the basis of each situation and need. In addition a certain number of criteria can be used to reflect different dimensions.
From a practical point of view, in the specific guide itself each criterion must be concisely and clearly explained to the final user, viz the person who will conduct an evaluation. Such explanation should cover:
- a precise definition
- instructions on how to interpret the criteria
- instructions on the weight of each criterion in the evaluation process.
In quite a few cases it will be more practical to formulate questions rather than to define criteria. This will depend on the situation and circumstances.
References
|
A reader of these Guidelines did mention the « terms of reference » (ToRs) as being part of the evaluation process. Actually, in our view, ToRs do not pertain to the present Guidelines : they belong to another conceptual category.
ToRs are mainly used in rather large development projects when a development agency wishes an external consultant or team to assist at various stages of the project cycle: project identification, project preparation, implementation, advising, or evaluation.
They are a document through which the project sponsor instructs an invited expert or team of experts about what is expected from them and how they should proceed. They possess a contractual nature, whether formalised or not, and are not necessarily legally binding.
For further information on ToRs, one may go to :
|
