Skip to main content

Posts

Showing posts with the label Reproducibility

A Prototype Only Its Creator Can Run Is Still a Private Miracle

Scientists love the sentence: “It works.” The more important sentence is: “Someone else made it work.” A prototype can perform beautifully on its creator’s computer and still be unusable everywhere else. Perhaps it depends on an undocumented library, a particular driver, a forgotten environment variable and one sacred folder that must never be renamed. At that point, the invention is not entirely a tool. Part of it is folklore. I encountered this while building the Virtual BioSSD. Making the system run was one achievement. Making it possible for another researcher to install, test and reproduce without reenacting every setup struggle became a different engineering problem. The second problem is less glamorous. It may also determine whether the first achievement matters. The US National Academies distinguishes reproducibility from replicability. Reproducibility asks whether another researcher can obtain consistent results using the same data, code and methods. Replicability asks whet...