To many, this paragraph may sound familiar: “As you may know, we screen manuscripts against our editorial criteria, and decline a substantial proportion of them without input from external referees (pertinent journal-level statistics are available). In such cases, even if reviewers were to certify the manuscript as technically correct, we feel that it would not be of outstanding interest to merit publication in Nature Biomedical Engineering. These editorial judgements are based on considerations of the degree of advance, broad implications, and breadth and depth of the work reported in the manuscript.”

Old male boilerplate.

The paragraph is included in our e-mails informing authors of a new manuscript that we “regret that we are unable to consider it for peer review”. We then write a bespoke paragraph, conveying the main reasons for the decision. Judging by some of the replies to these e-mails, it is clear that, in many instances, we could have better explained the rationale that the editor handling the manuscript had followed, sometimes in consultation with another editor in the editorial team.

We aim for balance. On the one hand, we dislike boilerplate text (pictured); it doesn’t give authors any useful feedback, and we wish to give them an opportunity to challenge our decisions when it is clear that we have misinterpreted or misjudged their work. On the other hand, we do not feel it would be useful to convey in detail our assessments of manuscripts or to pass on any of our written comments or discussion points; the specific thought process that we follow for each manuscript cannot be jotted down as a checklist of purely analytical arguments, and we would not wish that every decision be perceived as an invitation to further discussion (our manuscript workload would make this impossible). Yet, at a minimum, we owe it to authors to show them that we have understood the main points of their work, and to give them a sense of why it doesn’t meet our editorial criteria.

Therefore, what does it actually mean to assess “the degree of advance, broad implications, and breadth and depth of the work reported in the manuscript”? We appreciate that any detailed explanation will seem unsatisfactory to many authors; persistent agreement across judgements and parties, even within a close-knit editorial team, is a mirage. What’s more, as with most judgement calls, our assessments involve a mix of factual scientific information and contextual understanding, and their subjective weighing. And no editor — nor any person making a complex judgement — can free themselves from all biases and from all subjective effects of deliberation, made individually or as a team (M. Usher et al. Front. Psychol. https://doi.org/10.3389/fpsyg.2011.00037; 2011). However, we can certainly convey our general framework for the assessment of first submissions.

First, our assessments are primarily editorial; yet technical considerations, for example regarding the degree of support for the claims, or how the data have been obtained, can also weigh in the decision. In fact, we rarely reject manuscripts on purely technical grounds (that is, when key data seem to be missing, or when we notice shortcomings in the reporting of the methodology); if a manuscript meets our editorial criteria, we give authors the opportunity to improve their work, sometimes before peer review. Second, we do not outsource editorial-suitability assessments to peer reviewers (or to an external editorial board, which Nature Biomedical Engineering does not have); that is, we avoid the shortcut ‘let’s see what reviewers think of this work’. For every manuscript that we select for external peer review, we write down why we are considering it, and then check this reasoning against the feedback from external peer reviewers. And when we have doubts about particular aspects that may influence the editorial suitability of a manuscript, we may ask experts for specific feedback about these. Third, when assessing topics that we are insufficiently familiar with, we are committed to learning the necessary background; an editor cannot fairly assess a manuscript if they don’t sufficiently understand the main findings and the scientific context of the work. Fourth, we do not contemplate predictions of a manuscript’s citation impact (Nat. Biomed. Eng. 2, 1; 2018), nor do we weigh the prominence of subject areas. And fifth, we are aware that ‘gut feeling’ is unreliable, and that the more experience an editor has in assessing manuscripts, the easier it is to rely solely on intuition; but this can lead to hidden biases, to inconsistency in decisions and to unfairness to authors. Instead, we aim to subdue the influence of instinct on assessments — and, importantly, of any influence of the perceived stature of the authors or their institutions — by purposely focusing our evaluation on the actual findings reported in the manuscript in the context of the relevant literature.

Context is central to our assessments. We evaluate each research manuscript against a scientific background that is wider than that typically considered by specialists in the topic, yet much narrower than the journal’s scope. If we set the context too broad, hardly anything seems to be really new; if we set it too narrow, just about everything appears to be novel. Placing a manuscript in its appropriate context is the most difficult aspect of our editorial assessments, and that which benefits the most from editorial experience. Bearing the context in mind, we then assess the advances that the manuscript’s main claims represent from several perspectives, with the relevance of each perspective depending on the type of research reported. These editorial criteria are: technological (for example, does the manuscript report a new proof of concept, extensive technology development, or an enabling optimization?); methodological (does it report a new method, or a particularly useful optimization or combination of existing methods?); applicational (are new applications reported, or does the technology offer wider applicability?); with regard to the claimed functionality or performance (for instance, do the authors claim to make an application perform much better, easier, faster or cheaper?); and translational, clinical, or in fundamental or mechanistic understanding of human disease or health. We also assess the biomedical or clinical need, the implications of the findings (and whether these are broad or narrow), and the breadth and depth of the evidence (such as multiple applications or performance comparisons, or the extent of the support for the claimed mechanism of action) against our appreciation of the standards of recently published papers in the same area of research. And for some areas or topics — such as oncology, disease modelling, machine learning and device engineering — we have specific requirements (such as the numbers of cell lines, tissue types or animal models, the extent of the validations, or the need for performance comparisons) that evolve with relevant scientific progress and research practices.

For the vast majority of new manuscripts that we decline, we return the work for reasons of insufficient advance rather than of narrow implications or biomedical need. If a manuscript meets our editorial threshold for scientific advance, yet we deem it to be too preliminary in breadth or depth of the evidence, we typically engage with the authors to gauge what improvements would be possible. And, on occasion, the utility of the datasets or the present noteworthiness of the problem being addressed (such as the need for tests for a current viral outbreak, a notable controversy, or a substantial lack of reproducibility in a particular subject area) may be more editorially weighty than the perceived advance.

Overall, we strive for consistent decision-making across the research areas that we publish in. Naturally, we can only assess what is claimed; if the story is unclear or incomplete, then so may be our assessment of it. Storytelling in research is underappreciated (Nat. Biomed. Eng. 2, 53; 2018), and may impact our evaluation of the relatively small fraction of manuscripts (arguably, less than 10%) that fall near our threshold for the degree of advance for the research area. A clear and engaging narrative of incremental work doesn’t change its nature. But a confusing story of great work may undermine it; that’s one we wouldn’t want to decline.