Scientists are increasingly concerned that the lack of reproducibility in research may lead to, among other things, inaccuracies that slow scientific output and diminished public trust in science. Now, a team of researchers reports that creating a prediction market, where artificially intelligent—AI—agents make predictions—or bet—on hypothetical replication studies, could lead to an explainable, scalable approach to estimate confidence in published scholarly work.
Replication of experiments and studies, a critical step in the scientific process, helps provide confidence in the results and indicates whether they can generalize across contexts, according to Sarah Rajtmajer, assistant professor in information sciences and technology, Penn State. As experiments have become more complex, costly and time consuming, scientists increasingly lack the resources for robust replication efforts—what is often referred to now by them as the “replication crisis.”
“As scientists, we want to do work, and we want to know that our work is good,” said Rajtmajer. “Our approach to help address the replication crisis is to use AI to help predict whether a finding would replicate if repeated and why.”
