In Vivo Software Testing for Python

Project Description

In vivo testing is a powerful method for identifying bugs in software for which the creation of a comprehensive test suite that addresses all possible software configurations is not possible. In contrast to software testing that takes place before an application is deployed, in vivo tests use a deployed application's run-time state and inputs to assess the application's behavior in unforeseen situations [Murphy2010]. In vivo testing has been implemented for languages such as Java [Murphy2009] and C whose strong typing facilitates the implementation of tests, but has yet to be implemented for dynamically typed languages such as Python and Ruby.

Validating the correctness of neural circuit models entails examination of the model's responses to various inputs. Since the range of possible inputs typically cannot be a priori described, in vivo testing may be a valuable tool for identifying unanticipated scenarios when a model behaves in an unexpected manner. To use in vivo testing for this purpose, we need to develop a testing mechanism that can be used with Python code and not require modification of the actual model implementation being tested. We would also eventually like to support in vivo testing for Python code that makes use of GPUs.

Possible Project Goals

  • An in vivo testing mechanism for Python that can be used with Python features or packages extensively used in neural circuit emulations, .e.g, Numerical Python.
  • Generation of LLVM bitcode by a JIT compiler to accelerate test execution. Existing packages that perform JIT compilation such as Parakeet, Numba, or NumbaPro might be useful for this.
  • Use of code analysis techniques to automatically determine which parts of tests can be accelerated without annotation of the code being tested.
  • Use of polymorphism to support dynamic types.
  • Acceleration of large/computationally intensive suites of tests by parallel execution on GPUs.
  • Extension of implementation to support Python code that uses the GPU, e.g., via PyCUDA.

Skills Gained

  • Exposure to powerful open-source scientific software packages such as Scientific Python
  • Experience implementing and using novel software testing methodologies for novel applications.