Since the number of excellent evaluators is limited, the number of evaluation processes
should be reduced in order to avoid over-use of first-class evaluators. There is a concern that
different agencies and institutions have carried out an excessive number of routine evaluations
over the last decades, putting too much pressure on the best evaluators. First-rate evaluators
are increasingly reluctant to commit to time-consuming and unproductive evaluation
exercises. It is of great importance to reduce the number of evaluations and to confine them to
the core issues of research that only peers are able to judge. Evaluators provide a “free
resource” as part of their academic duty and this resource is over-exploited. Evaluating bodies
must recognise that good evaluation is a limited and precious resource.
A page limit for submissions to all evaluation processes is needed. Excessively long
submissions are counter-productive: evaluators need to be able to concentrate on the
essentials, which is problematic with very lengthy submissions.
Rotation of evaluators is essential to avoid excessive or repeated influence from the same
opinion leaders. The panel of experts should be adapted to reflect the diversity of disciplines
or scientific domains. Although gender and geographical distribution will be factors in the
selection of evaluating groups, excellence must remain the primary criterion.
2.2. Ethical guidelines and duties of evaluators
Evaluators should clearly declare possible conflicts of interest before the evaluation
process. The confidentiality of expert reviews and of the discussions in the evaluation panel
must be strictly respected to protect both the evaluators and the evaluated persons.
While reviewers have often learned the practice of evaluation by experience and self-
teaching, this competence cannot be taken as given. Methods and approaches to evaluating
and reviewing should become part of all researchers’ competence as should the ethical
principles involved. Evaluators should be made aware of the dangers of “unconscious bias”.
There should, as far as possible, be equivalent standards and procedures for all research
The evaluation procedures must also include mechanisms to identify the cases of biased
or otherwise inappropriate reviews and exclude them from consideration.
2.3. Evaluation criteria
Evaluations must be based under all circumstances on expert assessment of scientific
content, quality and excellence. Publications that are identified by the authors as their most
important work, including major articles and books, should receive particular attention in the
evaluation. The simple number of publications should not be a dominant criterion.
Impact factors of journals should not be considered in evaluating research outputs.
Bibliometric indicators such as the widely used H index or numbers of citations (per article