© 2000 by European Society of Cardiology
Copyright © 2000, European Society of Cardiology
Reviewer's recommendations predict impact: why you have to send us your very best work
aDepartment of Medical Physiology, University Medical Center Utrecht, The Netherlands
bEditorial Office of Cardiovascular Research, Academic Medical Center, Room J-127, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands
* Corresponding author. Tel.: +31-30-253-8923; fax: +31-30-253-9036 t.opthof{at}med.uu.nl
Received 31 July 2000; accepted 31 July 2000
Six months ago we acknowledged the help of many thousands of our reviewers who helped us with the editorial decision process over the last 5 years [1]. The aim of editors is two-fold: they wish to make fair decisions, but they also want to select the best work from the submitted material in order to increase the esteem of their journal which is to a large extent reflected by its impact factor. Moreover, editors have to deal with page limitations. The task of editors is therefore not easy, also because reviewers in fact relatively seldomly recommend to reject a paper [2]. Reviewer's reports are in general constructive and aim at improvement of a manuscript and not at its rejection. Therefore the high rejection rates of leading journals are primarily based on priority considerations [2–4].
 |
1 Peer review |
|---|
 Top 1 Peer review 2 Geographical aspects of...3 Do high reviewer's...References |
|---|
Research of peer review is a young science with its own world congress [5–7] linked to the World Association of Medical Editors (WAME). The review process is in general anonymous and the pros and cons of this have been debated in the past in this journal [8,9] with comments from several experts [10]. The outcome of a survey amongst both authors and reviewers was that the majority of both groups voted for continuation of anonymity. One important other incentive for editors to follow such advice is that about half of the reviewers stated that disclosure of their names would cause them to refuse to review or at least to be more reluctant to do so in the future [8]. Obviously, editors cannot afford such a loss of expert knowledge, if they aim at peer review of the large majority of the submitted material. Although the quality of the review process would probably benefit from disclosure of the identity of the reviewers [9,11], logistic reasons prevent such a policy.
|
2 Geographical aspects of submissions, publications, reviewers and reviewer's reports |
|---|
|
Top1 Peer review2 Geographical aspects of...3 Do high reviewer's...References |
|---|
With the above information in hand one may wonder who should be selected as a reviewer. In the April 2000 issue we already noted [1] that the general idea that older more experienced individuals might be better in putting research in perspective [12,13], although younger reviewers might be better in more detailed technical knowledge, is not supported by facts [11]. Younger reviewers performed better in both aspects. Therefore we adopted as a policy to ask individuals to act as a reviewer as soon as they have a paper accepted in the journal [1]. Fig. 1 shows the geographical distribution of original manuscripts submitted and published between January 1997 and December 1999, as well as that of available reviewers at the end of 1999 and that of the frequency by which the advice of those reviewers for all submitted original manuscripts was sought between October 1997 and December 1999. Nine countries (the G7 countries plus Australia and the Netherlands) made up together 79.3% of all submitted manuscripts and 82.6% of all published manuscripts. The percentage of reviewers from those countries was by and large the same as the percentage of publications, the exception being the USA. We emphasized in previous editorials that the geographical distribution of accepted manuscripts is not equal to that of submitted manuscripts [14–18].
|
|
3 Do high reviewer's ratings predict future citations? |
|---|
|
Top1 Peer review2 Geographical aspects of...3 Do high reviewer's...References |
|---|
Fig. 2 shows the effect of reducing the contents of Cardiovascular Research in an artificial away. The contents of 1997 and 1998 were relevant for the impact factor of the journal in 1999. All peer reviewed papers (review papers and original manuscripts) were ranked according to the reviewer's priority. In the case that three reviewers assigned a high priority to a particular manuscript, its reviewer's priority score was 100%. If two reviewers assigned a high priority and one a low priority or an intermediate priority this score would be 66.7 or 83.3% respectively. In the same way the reviewer's priority was scored for manuscripts with only two reviewers. We determined the number of citations of all individual manuscripts. Fig. 2 shows that removal of papers with very low reviewer's priority hardly affects content or impact factor, because it seldomly occurs that such papers are accepted for publication. Still, subsequent removal of higher ranked papers increases the impact factor. Thus, if only manuscripts with 100% priority scores would have been published the journal would have been reduced to 24% of its original contents (Fig. 2, left ordinate), whereas the impact factor would have increased by 40% (Fig. 2, right ordinate). Apart from the fact that this would have led to an unacceptable commercial product for the publisher, two conclusions can be drawn:
|
- The priority assigned by reviewers to a manuscript has a positive predictive power in terms of future citation power, but only if the reviewers agree.
- Another editorial policy with no other decision criterion than reviewer's priority potentially may increase the impact factor, but by no more than 40%. The last official impact factor of 1998 was 2.99 and after draconic interventions as described above it could not have increased above 4.20.
The first observation is of interest for all future authors, reviewers, editors as well as publishers, because it demonstrates that there might still be a role for peer review — despite its shortcomings [19–24] — when in the future, paper journals have ceased to exist and when electronic publishing has obliterated the problem of page limitations.
The second observation is of interest for our readers and prospective authors. It shows that we can only improve the position of our journal during the next years if they send us their very best work.
|
Notes |
|---|
On behalf of the Editorial Team
|
References |
|---|
|
Top1 Peer review2 Geographical aspects of...3 Do high reviewer's...References |
|---|
- Opthof T., Coronel R., Janse M.J. On behalf of the Editorial Team. Where do our reviewers come from? Cardiovasc Res (2000) 46:1–13.
[Free Full Text] - Coronel R., Opthof T. The role of the reviewer in editorial decision-making. Cardiovasc Res (1999) 43:261–264.
[Abstract/Free Full Text] - Opthof T. On behalf of the Editorial Team of Cardiovascular Research. Submissions, acceptance rate, rapid review system and impact factor. Cardiovasc Res (1999) 41:1–4.
[Free Full Text] - Opthof T., Furstner F., Van Geer M., Coronel R. Regrets or no regrets? No regrets! The fate of rejected manuscripts. Cardiovasc Res (2000) 45:255–258.
[Free Full Text] - Guarding the guardians: research on editorial peer review: selected proceedings from The First International Congress on Peer Review in Biomedical Publication. J Am Med Assoc 1990;263:1317–1441 (theme issue).
- The Second International Congress on Peer Review in Biomedical Publication. J Am Med Assoc 1994;272:91–173 (theme issue).
- The Third International Congress on Peer Review in Biomedical Publication. J Am Med Assoc 1998;280:213–302.
- Hearse D.J. Anonymity of reviewers — editorial comment. Cardiovasc Res (1994) 28:11–33.
- Fabiato A. Anonymity of reviewers. Cardiovasc Res (1994) 28:1134–1139.
[Free Full Text] - Fletcher R.H., Fletcher S., et al. Anonymity of reviewers. Commentaries. Cardiovasc Res (1994) 28:1140–1145.
[Free Full Text] - Evans A.T., McNutt R.A., Fletcher S.W., Fletcher R.H. The characteristics of peer reviewers who produce good-quality reviews. J Gen Intern Med (1993) 8:422–428.[Web of Science][Medline]
- Lock S. A difficult balance: editorial peer review in medicine. Philadelphia: ISI Press (1986) 1–38:121–132.
- Honig W.M. Peer review in the physical sciences: an editor's view. Behav Brain Sci (1982) 5:207–208.[Web of Science]
- Opthof T. On behalf of the Editorial Team of Cardiovascular Research. Geographical shifts in submissions. Cardiovasc Res (1998) 38:1–2.
[Free Full Text] - Opthof T. On behalf of the Editorial Team of Cardiovascular Research. Geographical shifts in publications. Cardiovasc Res (1999) 42:1–2.
[Free Full Text] - Opthof T. On behalf of the Editorial Team of Cardiovascular Research. Increase in submissions from Europe — from where and since when? Cardiovasc Res (1998) 39:261–262.
[Free Full Text] - Opthof T. On behalf of the Editorial Team of Cardiovascular Research. Submissions, publications and reviewers from Europe: focus on Spain. Cardiovasc Res (1999) 43:265–267.
[Free Full Text] - Opthof T, Coronel R, Janse MJ, on behalf of the Editorial Team. Submissions, Editorial Process and Impact Factor 1992–2000: Focus on Europe. Cardiovasc Res 2000;47:203–206.
- Siegelman S.S. Assassins and zealots: variations in peer review. Radiology (1991) 178:637–642.
[Free Full Text] - Scharschmidt B.F., DeAmicis A., Bacchetti P., Held M.J. Chance, concurrence, and clustering. Analysis of reviewer's recommendations on 1000 submissions to The Journal of Clinical Investigation. J Clin Invest (1994) 93:1877–1880.[Web of Science][Medline]
- Ernst E., Resch K.-L. Reviewer bias: a blinded experimental study. J Lab Clin Med (1994) 124:178–182.[Web of Science][Medline]
- Peters D.P., Ceci S.J. Peer-review practices of psychological journals: the fate of submitted articles, submitted again. Behav Brain Sci (1982) 5:187–195.[Web of Science]
- Kemper K.J. Pride and prejudice in peer review. J Clin Epidemiol (1991) 44:343–345.[CrossRef][Web of Science][Medline]
- Moed H.F., Burger W.J.M., Frankfort J.G., Van Raan A.F.J. A comparative study of bibliometric past performance analysis and peer judgement. Scientometrics (1985) 8:149–159.[CrossRef][Web of Science]
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  T. Opthof, R. Coronel, and M. J Janse Submissions and impact factor 1997-2001: Focus on Sweden Cardiovasc Res, August 1, 2001; 51(2): 202 - 204. [Full Text] [PDF]
|
|
|
|
|
|
T. Opthof, R. Coronel, and M. J Janse Impact factor of Cardiovascular Research in 2000: all time high! Cardiovasc Res, April 1, 2001; 50(1): 1 - 2. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||