Sebastian's Thoughts on Open Review
My name is Sebastian Bobadilla-Suarez and I am an early career researcher (ECR — postdoc’ing in the Love Lab). I did my PhD with Brad Love at UCL as well. This post is about recent events regarding the review process of our manuscript titled Measures of neural similarity. Our manuscript was submitted to a prestigious journal and went through a formal review process. It was rejected by the reviewers, which is fine, but one of the reviewers decided to post his review on his own blog. This was problematic for several reasons, see here for Brad’s response. Sam Schwarzkopf also shared his take too.
Before I go on, I want to say that I am entirely in favor of open debate of ideas, open science and fully deconstructing manuscripts. I fully encourage this. However, open science should not be used to maintain the status quo but to challenge it. Also, I really appreciate the time and effort that goes into providing feedback on manuscripts, whether in a formal review process or not. Although we may not always agree with reviews on a manuscript, they are always welcomed as useful in one way or another to improve the work.
After reading some of the threads on this, I’d like to give my two cents as first author. I was surprised to see how polarized the subject of open science can become. A lot of the discourse from certain individuals seems hopelessly Manichaeistic (e.g., “I’m for open science, you’re not”). I am for open science, as I said above, but I am also for understanding how new and open science systems impact those lower in the scientific ranks. I would assume we are all pro open science as a default but still working on best practices, including practices pertaining to open review. To be clear, the motivation for this blog post is not to sidestep the points made in the original review. The goal here is to share my experience and perspective.
I feel the review process has been tainted for this project; a project that I hold close to my heart as one of my favorite initiated during my PhD. This obviously makes me biased, but then again who is going to stand up for my work if not me? I understand that uploading your manuscript on a preprint server invites informal comment and feedback, which is one of the reasons to do it in the first place, and as I said I fully welcome and appreciate any and all comments on my work. However, posting a formal review as a blog post necessarily carries more weight than any feedback provided outside the formal review process, especially when posted by one of the leaders in the field. I did feel wronged by how one of our reviewers has handled the process. The junior authors were not given the professional courtesy of notification and none of the authors opted into this way of handling reviews — we had no opportunity to reply before rejection. Ultimately, is accepting to review a manuscript with the goal of eventually blogging about it (as opposed to improving it) a conflict of interest? I am still developing my opinions with respect to best practices in open science.
I see that transparency can have pitfalls when the line between formal review and public debate has been blurred (especially at early stages on the rocky road to getting published). The fact that this line was blurred in a non-consensual fashion when best practices have not been cast into convention yet is unacceptable. Nonetheless, the fact that we in the open science community can discuss these cases, push back against dominant power structures, and set precedents will be beneficial moving forward.
I hope there are lessons to be learned here in general and that in my specific case future reviewers may place appropriate weight on the posted review in question — avoiding the use of such a post as a heuristic to base their own opinions on, since I personally think the review misses the point of my preprint. Is it now impossible to claim that future reviews are somehow independent of each other given that the blog post in question was the outcome of a formal review process? I think it is now a case of how to appropriately contextualize them. Maybe proponents of open science, and open review specifically, have thought of such situations and how privileged voices can drown out the voices of more junior people, like me. My biggest hope for open science is that it will create a fairer and more accessible system for the new generation of scientists. This is only possible by having these debates on new ways of doing things.