Samenvatting

Background: Peer review is at the heart of the scientific process. With the advent of digitisation, journals started to
offer electronic articles or publishing online only. A new philosophy regarding the peer review process found its
way into academia: the open peer review. Open peer review as practiced by BioMed Central (BMC) is a type of peer
review where the names of authors and reviewers are disclosed and reviewer comments are published alongside
the article. A number of articles have been published to assess peer reviews using quantitative research. However,
no studies exist that used qualitative methods to analyse the content of reviewers’ comments.
Methods: A focused mapping review and synthesis (FMRS) was undertaken of manuscripts reporting qualitative
research submitted to BMC open access journals from 1 January – 31 March 2018. Free-text reviewer comments
were extracted from peer review reports using a 77-item classification system organised according to three key
dimensions that represented common themes and sub-themes. A two stage analysis process was employed. First,
frequency counts were undertaken that allowed revealing patterns across themes/sub-themes. Second, thematic
analysis was conducted on selected themes of the narrative portion of reviewer reports.
Results: A total of 107 manuscripts submitted to nine open-access journals were included in the FMRS. The
frequency analysis revealed that among the 30 most frequently employed themes “writing criteria” (dimension II) is
the top ranking theme, followed by comments in relation to the “methods” (dimension I). Besides that, some results
suggest an underlying quantitative mindset of reviewers. Results are compared and contrasted in relation to
established reporting guidelines for qualitative research to inform reviewers and authors of frequent feedback
offered to enhance the quality of manuscripts.
Conclusions: This FMRS has highlighted some important issues that hold lessons for authors, reviewers and editors.
We suggest modifying the current reporting guidelines by including a further item called “Degree of data
transformation” to prompt authors and reviewers to make a judgment about the appropriateness of the degree of
data transformation in relation to the chosen analysis method. Besides, we suggest that completion of a reporting
checklist on submission becomes a requirement.