University Performance Indicators in Use: Administrative Data on State Control of Russian Universities

Leiden Repository

University Performance Indicators in Use: Administrative Data on State Control of Russian Universities

Type: Article in monograph or in proceedings
Title: University Performance Indicators in Use: Administrative Data on State Control of Russian Universities
Author: Guba K.
Journal Title: STI 2018 Conference Proceedings
Start Page: 1292
End Page: 1297
Publisher: Centre for Science and Technology Studies (CWTS)
Issue Date: 2018-09-11
Keywords: Scientometrics
Abstract: Evaluation of university performance is one of the most controversial topics in Russian higher education. During the last five years, the government has produced a vibrant discourse on the necessity to reduce the number of low-quality and weak institutions. As part of the task to ‘clean the system of higher education,’ the state agency developed a methodology to evaluate the performance of educational organizations. All Russian universities are obliged to participate in the annual evaluation and provide reliable information about more than one hundred statistical indicators. The state agency claims that metrics are used for making decisions about enforcement actions like onsite inspections, pending accreditation and even shut down a university. The literature on evaluation systems and practices has covered different topics from the consistency and reliability of indicators to effects of indicator uses (Rijcke et al., 2016), but the research less focuses on the question about decision-making based on quantitative indicators. The following question is rarely raised: Have the results of the evaluations lied on the basis of decisions made by the state agency? Administrative dataset gathered by the Russian state agency has the potential to contribute to the analysis of the consequences of evaluations. This study uses data on the application of inspections in relation to universities. The data about inspections are merged with results of annual evaluations of universities conducted by the state agency in 2015 and 2016. Within the classical decision-theory point of view, information is gathered because of the desire to improve decisions. Empirical research on using evaluations in the case of state decisions about universities demonstrates a pattern that can’t be explained in such terms. Particularly the state systematically penalizes the universities which are simultaneously evaluated as effective organizations. Instead of using different kinds of metrics about research and educational activity the state bases their decisions on the university type. Inspections as well as the severe sanctions including the shutdown the university depend on whether the university is a private organization. We suggest that this finding is a consequence of bureaucratic control of academic organizations which operates while being distanced both physically and symbolically. Fine-graded metrics concedes simple categorization because their application makes the decision-making more complicated and questionable.

Files in this item

Description Size View
application/pdf STI2018_paper_212.pdf 920.5Kb View/Open

This item appears in the following Collection(s)