Effects of Indicators on Knowledge Production in Astronomy

Leiden Repository

Effects of Indicators on Knowledge Production in Astronomy

Type: Article in monograph or in proceedings
Title: Effects of Indicators on Knowledge Production in Astronomy
Author: Heuritsch J.
Journal Title: STI 2018 Conference Proceedings
Start Page: 101
End Page: 106
Publisher: Centre for Science and Technology Studies (CWTS)
Issue Date: 2018-09-11
Keywords: Scientometrics
Abstract: Nine interviews were conducted with astronomers from Leiden University, and a document analysis was performed on relevant institutional (self-) evaluation documents, annual reports, and CVs of the interviewees. The aim was to perform a qualitative study about the relationship between the research behaviour of astronomers and how their science is being evaluated. This study encompassed the funding and publication system as well as the indicators used to measure the scientific output, its quality and the research performance. It sheds light on how astronomers define high-quality research and how they think that creating knowledge of value is encouraged or hampered by the evaluation processes. The research shows that astronomers are realists who define scientific quality on the basis of “truth” and are driven by curiosity. These two factors make up their intrinsic values and motivation to perform Astronomy. Publication pressure, arising from the requirements of “the system”, creates an extrinsic motivation to perform. This results in premature publications, low readability and replicability, risk aversion and a focus on quantity rather than quality. Hence, indicators do not merely represent quality, but also co-constitute what counts as good research. While such constitutive effects of indicator use on research behaviour and content are observed, there is no indication that the astronomer's intrinsic values are co-constituted. This gives rise to a discrepancy between what is being measured by indicators and what astronomers define as scientific quality; the so-called ""evaluation gap"". Findings on constitutive effects and the evaluation gap in Astronomy lays out the conceptual groundwork for further empirical research and for policy advice on alternative evaluation practices and innovative indicators with the aim of bridging the ""evaluation gap"".
Handle: http://hdl.handle.net/1887/65237
 

Files in this item

Description Size View
application/pdf STI2018_paper_24.pdf 950.3Kb View/Open

This item appears in the following Collection(s)