Scientists Increasingly Depend on ‘Black-Box’ Tools They Cannot Control or Fully Understand

hubie writes:

Scientists are increasingly relying on powerful data sources and tools that they often cannot fully understand, inspect or verify, according to a new study:

State-of-the-art tools and data like artificial intelligence (AI), satellite imagery, online data and digital sensors are revolutionising the way scientists study the natural world.

But such systems effectively operate as scientific "black boxes" that can increasingly challenge the trust in science.

The new study, by an international team of scientists and available here, addresses the problems of reproducibility, trust and the future of scientific research in an era when critical technologies can shape science and influence knowledge without being fully open to scrutiny.

These technologies can process enormous amounts of information, monitor biodiversity and threats across continents, and reveal patterns that would once have been out of reach.

"However, many of these tools represent true black boxes, by keeping the processes behind those results largely hidden," said Ivan Jari, researcher from the University of Paris-Saclay, and lead author of the study.

"They are often owned by private companies that intentionally limit access to information about how their systems operate or process data, guided by proprietary constraints and commercial aims".

The paper identifies several types of black boxes that are becoming widely used in ecology and conservation.

One of the most prominent examples are large language models and other AI technologies, increasingly used to analyse massive datasets, interpret satellite imagery, and model ecosystems.

However, researchers often have little or no access to the data used to train these systems, the underlying algorithms, direct system testing, or understanding how and why they generate particular outputs.

Read more of this story at SoylentNews.