Applying Machine Learning to Compare Research Grant Programs

Leiden Repository

Applying Machine Learning to Compare Research Grant Programs

Type: Article in monograph or in proceedings
Title: Applying Machine Learning to Compare Research Grant Programs
Author: Khor K. A.Ko G.Walter T.
Journal Title: STI 2018 Conference Proceedings
Start Page: 816
End Page: 824
Publisher: Centre for Science and Technology Studies (CWTS)
Issue Date: 2018-09-11
Keywords: Scientometrics
Abstract: Evaluating and comparing the performance of research funding programmes is challenging because programmes differ in how they (a) select grants and (b) select research areas for funding. Do programmes perform well because they are good at selecting research projects, or because they concentrate funding in high-output research areas? These mechanisms can be distinguished if we can identify and control for a set of research projects in common scientific areas. Previous approaches have used costly and arbitrary manual classification of research projects, or relied on coarse research groupings defined by bibliometric services. We propose a new solution: Apply machine learning to map research projects funded by one agency into the funding structure of a different agency. We identify and control for common areas of research, and separately identify the effects of grant selection from research area composition. We apply our method to compare three high-impact high-risk research programmes funding early-career life scientists: The U.S. National Institutes of Health’s New Innovators Award (NIH-NIA), the European Research Council Starting Grant (ERC-StG), and the Singapore National Research Foundation Fellowship (NRFF). We show that the NIH-NIA and NRFF concentrate research funding in selective portions of the life sciences, compared to the ERC-StG which by design evenly distributes research funding. Within common research areas, NIH-NIA and NRFF researchers exhibit faster growth in citations, and to a lesser extent publications, than equivalent ERC-StG researchers. This suggests the NIH-NIA and NRFF are able to select researchers who deliver superior research outcomes.

Files in this item

Description Size View
application/pdf STI2018_paper_141.pdf 1.294Mb View/Open

This item appears in the following Collection(s)