Journal Article10.1177/016224398501000304
Peer Review of Interdisciplinary Research Proposals
TL;DR: This paper found that ratings of National Institutes of Health (NIH) initial grant applications were correlated with independent ratings of their later reapplications by a thin 0.4 (i.e., only 16% of the variance was accounted for by the other rating).
read more
Abstract: The peer review of research results submitted for journal publication raises elementary issues of fairness and reliability.' Peer review of proposals to perform research in the future, however, is even more problematic. For many reasons, judging untested ideas is inherently more uncertain than evaluating completed work. Research has a way of evolving in directions that are unchartable in advance-certain data may prove unattainable, new discoveries by the researchers or by others may point to reorientation, or personnel may change. In a 1974 study, Grace Carter found that ratings of National Institutes of Health (NIH) initial grant applications were correlated with independent ratings of their later reapplications by a thin 0.4 (i.e., only 16% of the variance was accounted for by the other rating).2 While this figure includes unreliability due to an independent rerating, it also reflects changes in the perception of the value of specific projects as research progresses. One National Science Foundation (NSF) proposal reviewer made special note of "exemplary staffing plans" of a proposal he evaluated.3 Ironically, that same project changed staff repeatedly to the extent that it was not possible even to identify a project leader. In essence, then, peer review of proposals is a difficult business. Peer review as a process has engendered strong
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Bias in peer review
TL;DR: This review provides a brief description of the function, history, and scope of peer review, and characterizes and examines the empirical, methodological, and normative claims of bias in peer review research; and assesses possible alternatives to the status quo.
959
How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management
TL;DR: In this article, the authors provide quantitative evidence on how the use of journal rankings can disadvantage interdisciplinary research in research evaluations using publication and citation data, and compare the degree of interdisciplinarity and the research performance of a number of Innovation Studies units with that of leading Business & Management Schools (BMS) in the UK.
549
The Art of Getting Funded: How Scientists Adapt to their Funding Conditions
TL;DR: A comparative interview-based study of experimental physicists working at Australian and German universities found that strategies differ between scientists in the two countries because of different funding conditions; and they differ between top scientists and others.
New light on old boys: Cognitive and institutional particularism in the peer review system
TL;DR: In this article, the authors argue that the existing debate emphasizes the more uninteresting aspects of the peer review system and that the operation of old boyism, as currently understood would have little effect on the overall direction of science.
259
Tracking the evolution of waste recycling research using overlay maps of science
TL;DR: How ostensibly 'excellence-based' journal rankings exhibit a systematic bias in favour of mono-disciplinary research is illustrated, which is likely to affect negatively the evaluation and associated financial resourcing of interdisciplinary research organisations, and may result in researchers becoming more compliant with disciplinary authority over time.
233
References
Chance and consensus in peer review
TL;DR: An experiment in which 150 proposals submitted to the National Science Foundation were evaluated independently by a new set of reviewers indicates that getting a research grant depends to a significant extent on chance.
634
Experience with NIH Peer Review: Researchers' Cynicism and Desire for Change:
TL;DR: In the United States, peer review is central to the process by which many government agencies select research proposals for funding as mentioned in this paper, and although several different agency versions of peer review are practiced, they share one characteristic: scientists judge both the potential value of proposed research projects and the ability of proposers to perform the studies.
49
Interdisciplinary Integration Within Technology A ssessments
TL;DR: In this article, the authors present the complete research findings of a two-year study of the process of integrating the disciplinary components of TAs (Rossini et al., 1978).
12