Policy variation in the external evaluation of research for tenure at U.S. universities

There is a robust literature documenting differences in peer review processes for scholarly outlets. Knowledge of this variability has provoked thoughtful debate about the best approach for promoting rigor and innovation in scientific research (e.g. single-blind vs. double-blind review, or more recently, double-blind vs. open review). We aim to expand this conversation to external peer review processes specified in rank and tenure guidelines. We qualitatively analyze a corpus of publicly available rank and tenure procedures at research-intensive universities in the United States. Results indicate significant variation in (1) the required minimum and maximum number of external reviews, (2) the candidate’s role in the reviewer selection process, (3) the level of ensured anonymity for reviewers, and (4) attention to potential conflict of interest scenarios. We argue that many of the debates about best practices in research evaluation for journals and funding agencies are also relevant for rank and tenure procedures. Moreover, rank and tenure policies may be subject to unique tensions, such as cases where external reviews are formally cast as disinterested assessments by referees but informally understood as letters of recommendation by sponsors. We discuss the relevance of our findings for existing work on the perceived clarity of evaluation criteria—especially how guideline ambiguity can be linked to inequality and how labor issues can conflict with idealized assessment principles.