Critical Appraisal

The meta-rating serves as a data basis for the WK-specific meta-ratings. WKs that consider this international meta-rating helpful for orientation can place the rating positions of the journals relevant to them on their respective VHB-WK page. The methodology used results in an overall list across the areas of business administration. It is the consequence of the fact that international ratings also contain all business sub-disciplines. We explicitly emphasize the importance of a responsible and thoughtful use of this list. Even the present meta-rating cannot replace reading the respective publications – especially when evaluating the research achievements of individuals. Any evaluation of research performance based on standardized procedures and average values involves simplifications and thus a narrow perspective.

The meta-rating is primarily intended as an orientation for own article submissions. It is particularly important for young scholars to know what reputation is associated with a publication in a particular journal from the perspective of the international scientific community.

Any evaluation of research performance that is limited to a simple addition of articles weighted by rating values is problematic. Among the most important problems are:

  • The variance problem: The reputation of a journal is formed in the long term by the quality of the articles published in it. However, this does not at all mean that all articles published in it are equally good. Rather, there are both ground-breaking articles in journals with low reputation and weak articles without any influence in top journals. On average, of course, it is the other way around: The reputation of a journal is a (weak) indicator of the quality of the paper – no more and no less.
  • The incompleteness problem: The meta-rating refers exclusively to publications in journals. Business research is also published in other media, such as monographs, conference proceedings, commentaries, or is reflected in the development of prototypes. These research achievements are not covered by the meta-rating.
  • The opportunism problem: Whenever performance appraisals are made on the basis of a KPIs, the problem arises that this is anticipated. In such a situation, it is rational to optimize only the KPI. If this is only correlated to the actually intended performance objective, massive problems can be the result. The academic quest for relevant and innovative insights is complex, and the evaluation of research performance – as important as it is – is always error-prone and provisional. Scholars who focus only on publication success rather than the objectives of true and meaningful insights may be tempted to achieve one at the expense of the other. Salami publications, “deals” and other forms of legal and illegal attempts to “cut corners” can be the result.

Thus, this meta-rating is in no way a substitute for the differentiated picture that every colleague should form when assessing publication achievements. In order to assess the originality and relevance of the research question, the soundness and argumentative stringency of the theoretical contribution, and the methodological rigor in the design and execution of model building and/or the empirical study, one must actually read an article – this has been and continues to be the case.

Finally, it must be pointed out that academic research is always only one among many tasks of scholars. Teaching, services in self-administration and for the community of researchers, and the transfer of academic knowledge into practice are of course also important, and outstanding achievements in these areas also deserve appreciation and recognition.