bannière skepti

SKEPTISCIENCE
Open Peer Review
Post-Publication Peer Review

Social Studies of Open Peer Review & Post Publication Peer Review

  • Bordignon Frédérique, 2020, “Self-correction of science: a comparative study of negative citations and post-publication peer review”, Scientometrics, SpringerVerlag, 124 (2), 1225-1239, https://doi.org/10.1007/s11192-020-03536-z

    Abstract

    This study investigates whether negative citations in articles and comments posted on post-publication peer review platforms are both equally contributing to the correction of science. These 2 types of written evidence of disputes are compared by analyzing their occurrence in relation to articles that have already been retracted or corrected. We identified retracted or corrected articles in a corpus of 72,069 articles coming from the Engineering field, from 3 journals (Science, Tumor Biology, Cancer Research) and from 3 authors with many retractions to their credit (Sarkar, Schön, Voinnet). We used Scite to retrieve contradicting citations and PubPeer to retrieve the number of comments for each article, and then we considered them as traces left by scientists to contest published results. Our study shows that contradicting citations are very uncommon and that retracted or corrected articles are not more contradicted in scholarly articles than those that are neither retracted nor corrected but they do generate more comments on Pubpeer, presumably because of the possibility for contributors to remain anonymous. Moreover, post-publication peer review platforms, although external to the scientific publication process contribute more to the correction of science than negative citations. Consequently, post-publication peer review venues, and more specifically the comments found on it, although not contributing to the scientific literature, are a mechanism for correcting science. Lastly, we introduced the idea of strengthening the role of contradicting citations to rehabilitate the clear expression of judgment in scientific papers.

  • Dubois Michel, Frenod-Dumand Alexandra, Gargiulo Floriana, Guaspare-Cartron Catherine, 2021, “Le contrôle par les pairs au temps du coronavirus”, in Chauvin Pierre-Marie, Clement Annick (dir.), Sorbonnavirus, Regards sur la crise du coronavirus, Paris, Sorbonne Université Presses

  • Malički Mario, Costello Joseph, Alperin Juan Pablo, Maggio Lauren A., 2021, “Analysis of single comments left for bioRxiv preprints till September 2019”, Boichem Med, 31 (2), https://doi.org/10.11613/BM.2021.020201

    Abstract

    Introduction: While early commenting on studies is seen as one of the advantages of preprints, the type of such comments, and the people who post them, have not been systematically explored. Materials and methods: We analysed comments posted between 21 May 2015 and 9 September 2019 for 1983 bioRxiv preprints that received only one comment on the bioRxiv website. The comment types were classified by three coders independently, with all differences resolved by consensus. Results: Our analysis showed that 69% of comments were posted by non-authors (N = 1366), and 31% by the preprints’ authors themselves (N = 617). Twelve percent of non-author comments (N = 168) were full review reports traditionally found during journal review, while the rest most commonly contained praises (N = 577, 42%), suggestions (N = 399, 29%), or criticisms (N = 226, 17%). Authors’ comments most commonly contained publication status updates (N = 354, 57%), additional study information (N = 158, 26%), or solicited feedback for the preprints (N = 65, 11%). Conclusions: Our results indicate that comments posted for bioRxiv preprints may have potential benefits for both the public and the scholarly community. Further research is needed to measure the direct impact of these comments on comments made by journal peer reviewers, subsequent preprint versions or journal publications.

  • Ortega José Luis, 2021, “Classification and analysis of PubPeer comments: How a web journal club is used”, Journal of the Association for Information Science and Technology, https://doi.org/10.1002/asi.24568

    Abstract

    This study explores the use of PubPeer by the scholarly community, to understand the issues discussed in an online journal club, the disciplines most commented on, and the characteristics of the most prolific users. A sample of 39,985 posts about 24,779 publications were extracted from PubPeer in 2019 and 2020. These comments were divided into seven categories according to their degree of seriousness (Positive review, Critical review, Lack of information, Honest errors, Methodological flaws, Publishing fraud, and Manipulation). The results show that more than two-thirds of comments are posted to report some type of misconduct, mainly about image manipulation. These comments generate most discussion and take longer to be posted. By discipline, Health Sciences and Life Sciences are the most discussed research areas. The results also reveal “super commenters,” users who access the platform to systematically review publications. The study ends by discussing how various disciplines use the site for different purposes.