Protocol paper: From Chaos to Order. Augmenting Manual Article Screening with Sentence Transformers in Management Systematic Reviews

Abstract

[EN] A spanish version of the article is provided (see section before Acknowledgements) As scientific output grows, systematic reviews have become essential yet increasingly challenging. Our approach to this protocol aims to make this process more effective, efficient and accessible to researchers worldwide, including those in developing countries. We developed a tool to complement human judgment in the screening phase using pre-trained language models and natural language processing techniques. This tool generates text embeddings and calculates semantic similarities, prioritizing potentially relevant articles. The goal is to utilise the similarity ranking instead of reviewing articles randomly or following the relevance sort option of search engines like WOS or Scopus. Coders can start with those closest to the category/categories of interest and progressively move towards the more distant ones. This approach would save time and effort while reducing the fatigue and biases of the coders. The models we have tested in this research are all-MiniLM-L6-v2, all-distilroberta-v1, all-mpnet-base-v2, paraphrase-multilingual-mpnet-base-v2, distiluse-base-multilingual-cased-v1, all-MiniLM-L12-v2, allenai-specter, allenai/scibert_scivocab_uncased, distilbert-base-nli-mean-tokens, roberta-base-nli-stsb-mean-tokens, distiluse-base-multilingual-cased-v2, paraphrase-multilingual-MiniLM-L12-v2, stsb-roberta-large, bert-base-nli-mean-tokens. The method was implemented using limited computational resources and open-source software, ensuring accessibility for research teams with restricted economic resources. Results indicate a possible reduction in screening time and improved consistency in article selection. The tool demonstrated utility in classifying relevant studies and would facilitate more comprehensive reviews. By providing a low-cost solution, we aim to level the playing field in global research, enabling researchers from diverse economic backgrounds to participate more fully in producing scientific knowledge.