Department of Computer Science
Florida State University
"The 'White Rat' of Numerical Reproducibility"
, 499 DSL
Abstract:
We explore an application from the author’s work in neuroscience. A code used to investigate neural development modeled 100 neurons with all-to-all excitatory connectivity. We used a simple ordinary differential equation system to model each neuron, and this model was used to produce a paper published in the Journal of Neurophysiology. Later a colleague used our code to continue this work, and found he could not reproduce our results. This lead us to thoroughly investigate this code and we discovered that it offered many different ways to thwart reproducibility.
Numerical reproducibility is considered a task that directly follows from the determinism in computations. However, reproducibility has become an intense concern and and issue for research. In fact, the author developed an international workshop of numerical reproducibility that is now a regular offering at the annual Supercomputing XX conference. We will show how this particular code provides a lack of reproducibility from the following sources:
The non-associativity of floating-point operations in two ways Differences in library mathematical functions whose reliability and correctness we take for granted
This code’s sensitivity makes it a very powerful tool to explore many different manifestations of numerical reproducibility. However, this code is by no means exceptional, as in neuroscience these types of models are used extensively to gain insights on the functioning of the nervous system. In addition, these types of models are widely used in many other fields of study.
This is joint work with Prof. Wilfredo Blanco in CS at Universidade do Estado do Rio Grande do Norte in Brazil, and Woohyeong Kim, who is currently my graduate student.
