Tried paper mill detector to weed out pseudoscience

Close-up of a hand holding a red pen and marking a document

An online tool can help publishers spot fake research papers.Credit: Getty

Publishers are testing a prototype of an automated system to flag submitted manuscripts with signs of paper mills — businesses that produce bogus research papers.

tool, which will eventually form a part online integrity center, is the result of a year-long collaboration between 24 publishers and academic analytics providers. Together, companies are trying to combat the growing scourge of science fiction publications.

Jana Christopher, an image integrity analyst at FEBS Press in Heidelberg, Germany, said the magazine’s submissions to paper mills have grown dramatically over the past few years. “If we don’t do anything about it, the literature is really hopeless,” he said. “It’s something we can’t afford.”

Fighting pseudoscience

In recent years, journals retracted hundreds of articles over concerns that the work contained fabricated data and images. As a result, publishers are trying to improve their defenses to prevent such documents from passing through their submission systems. Several private companies offer it Software to detect potentially processed images or a flag feature that suggests paper mill operations.

As of 2020, the International Federation of Scientific, Technical and Medical Presses (STM) is a global trade association based in The Hague, Netherlands. and working with publishing companies to develop common software standards This can detect potential problems with images during peer review. In December 2021, it announced the development of the STM Integrity Hub, which will provide a tool that allows editors from any publisher to review submitted articles for research integrity issues.

Companies participating in the initiative include BMJ, Elsevier, Frontiers, IOP Publishing, JAMA Network, Sage Publishing, Taylor & Francis, Wiley, Springer Nature (Nature)’s news team is independent of its publisher). A small group is currently testing prototypes of two of the three proposed tools. STM declined to name the publishers involved in the trial, saying it was too early to have any meaningful data on the tool’s effectiveness.

“Tip of the Iceberg”

Nature Elsevier, Taylor & Francis and Frontiers are understood to be the ones testing the prototypes.

Sabina Alam, director of publishing ethics and integrity at Taylor & Francis, says suspicions about paper mills account for about half of all ethics cases involving publishers. “The problem is not just in terms of volume, it’s important because there are different types of paper mills and they all have the ability to adapt. Therefore, studying the problem in sand migration environments poses a great challenge,” he said.

“I suspect that what we’ve found so far is only the tip of the iceberg,” said David Knutson, spokesman for the open-access PLOS Press in San Francisco, California. “A single paper mill can affect dozens, hundreds, or even thousands of articles from multiple publishers.”

The Integrity Center’s first tool works by scanning paper looking for more than 70 signals that indicate the manuscript was created by a paper mill. Concerned people still don’t know what these signals are so that they don’t alert the fraudsters. But previous public work may include article titles, layouts, bar charts with the same profile that represent different experimental data, suspicious-looking author email addresses, or strange phrases that may indicate the use of automatic translation software.

Duplicate filings

The second tool is designed to notify editors when someone submits an article to several journals at the same time. Paper mills use this tactic to try to get papers accepted faster (submitting full manuscripts to multiple journals at once is considered inappropriate).

Christopher said releasing these duplicate materials would be a “really big and important step” in the fight against the paper industry. Previously, it was impossible to know whether an article intended for publication in a journal was being reviewed elsewhere.

Due to data processing and antitrust laws, the secure sharing of data between publishers is legally problematic. Manuscripts submitted by researchers to journals are confidential and cannot be easily shared between journals and publishers. But the center is taking a number of technical measures to collect the minimum amount of information from publishers. “The center works by connecting and comparing information,” said Heilke Koers, head of STM Solutions, the STM subsidiary that is developing the center. Also, data is encrypted for security.

Such cooperation between publishers is crucial, says Elena Vika, head of research integrity at Frontiers, an open publishing house in Lausanne, Switzerland. “If we don’t work together, the problem will just move from one magazine or one publisher to another,” he said.

The final element of the center’s technique will be software analysis that can detect altered images from handwriting.

Help for editors

Joris van Rossum, Director of Research Integrity at STM, said the organization hopes to have paper mill detection and duplicate signal options widely available by early next year.

The programs will support editors and reviewers rather than replace them, says Nicola Nugent, publishing manager for quality and ethics at the Royal Society of Chemistry in London, who was involved in developing the centre. Alerts will still need to be handled by humans, but some degree of automation is important because “publishers evaluate and process large volumes of submissions.”

In addition to the online tools, STM is working with the UK-based Press Ethics Committee to provide guidance and policy to publishers on how to deal with integrity issues. It is organizing a series of workshops for publishers to share information and learn from each other.

So far, the center has been funded by STM’s initial, undisclosed investment. Next year, the company will seek voluntary funding from members of the center and explore how to make the project financially sustainable. “It’s too early to predict, but we expect to cover at least part of our operating costs with fees for integrating the hub into publishers’ editorial systems,” van Rossum said.

Source link

Leave a Comment