Definition: implementation of the results. Totality of the measures taken in order to have the results of the research widely spread and realized, so that they get a real impact. This impact may be economic, scientific, pedagogical,....
To be used for: evaluation of research projects or of researchers
Criteria
- Increase of knowledge and skills.
- Scientific impact: publications, communications for the international community; do the conclusions constitute a valuable step forward in our knowledge?
- Impact on the ground: publication and local dissemination of the results, back information to the concerned persons, structures and organizations.
- Pedagogical impact: PhD thesis, formation of new researchers, training in new (laboratory…) technologies, know-how transfer, appropriation of methods and knowledge by the local researchers or teachers.
- Sustainable impact on the target population.
- Technological impact: possible extension, commercialization, patents...
- Social impact: possible effects for the population. What are the benefits of the research in terms of improving quality of life, standard of living?
- Economic impact: involvement of local industries (e.g., introduction of a new method of manufacturing or a local high yield food cultivation).
- Sustainability: long term effect of the trainings, multiplying effects, use of the new technologies or working of the new implemented local industries after the end of the research.
Mechanism(s) for choosing the criteria and the precautions to be taken at this stage, including appropriate weighing
The choice of the criteria depends on whether they will be applied for researchers or research projects and of course on the kind of research and its objectives. In some researches economic or technological impact will be of concern whilst in other ones, publications, communications or PhD theses will be the key elements to be evaluated. Before the beginning of the evaluation process, it may be decided to attribute different weights to each selected criterion.
Interpretation of the criteria, and potential effects of changes in the weights on conclusions
Criteria have to be explicit and clearly defined. If it was decided to weight the criteria, it must be clearly understood that different weights may lead to different conclusions.
Example: publications. Often, indirect evaluations are performed on the basis of external indicators that try to measure the impact that a paper has (or potentially may have) on the further development of science or on interesting applications. The indicators may be the number of citations that the paper receives in the subsequent literature or a surrogate indicator often used, the so-called impact factor (IF). However, both criteria have disadvantages. The number of citations may be abused (friends citing each other), a citation may contain very negative criticism, the number of citations measures more the popularity of a subject or the size of the specific scientific community rather than the intrinsic value of a paper and it can take several years before the value of a paper can be rightfully assessed on the basis of the number of citations. The “impact factor” (IF) measures the average number of citations received during two years by all papers in the journal; a high impact factor guarantees somehow a positive intrinsic evaluation. But publications in international journals should be judged on the quality of the research, which is not always measurable in a quantitative way. To have impact on society in the South, one should not be cited from international journals, but one should be read or heard by those who could possibly use the knowledge and apply it. Giving more weight to the criterion “impact factor” will often lead to a conclusion different from that reached by the evaluators who choose the number of citations as main criterion and probably very different from that reached by those who choose to give more weight to the impact on society in the South.
