1. Development and deployment of
a cross-platform
open-source software package:
the Infobiotics Workbench
Jonathan BlakesJonathan Blakes
26/10/201026/10/2010
Supervised by Professor Natalio KrasnogorSupervised by Professor Natalio Krasnogor
2. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 2
Outline
Introduction to Infobiotics
Organisation of software
Supporting technologies
Tour of the Infobiotics Dashboard UI
Interface considerations
Deployment on Windows, Mac and Linux
Finishing touches and availability
Conclusions
3. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 3
Infobiotics Workbench
4. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 4
Infobiotics Models (syntax)
5. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 5
Infobiotics Models (semantics)
6. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 6
Infobiotics Experiments
Command line tools with a common parameter
file format:
simulation (mcss)
model checking (pmodelchecker)
model structure and parameter optimisation
(poptimizer)
Desktop applications (not cloud-based)
Written in C++ for speed
7. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 7
Infobiotics Dashboard
User friendly graphical user interface for
perform Infobiotics experiments
setting, loading and saving of parameters
running experiments and reporting progress
plotting output
Present options in a clear and understandable
manner, setting limits on parameter values
Give feedback when things go wrong!
Written in Python
8. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 8
Why Python?
Dynamic language: rapid prototyping
Large stdlib: tempfile, XML parsers, itertools
3rd
party Libraries:
NumPy – n-dimensional arrays and ufuncs
pytables – HDF5 (simulator output)
PyQt – Qt GUI bindings (GPL), QScintilla text editor
Matplotlib – 2D plotting
Mayavi – 3D VTK-based visualisation (TVTK)
Traits...
9. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 9
Traits = Reactive Programming
10. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 10
Simulation Parameters (file)
11. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 11
Simulation Parameters (GUI)
12. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 12
Model checker parameters (1)
MC2
13. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 13
Model checker parameters (2)
14. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 14
Simulator Results
Simulation output – amounts of each species in
each compartment at each timepoint for each
run - quickly become very large (gigabytes), but
only some data is of use to the modeller.
selecting important information
scaling amounts (molecules) to biologist-friendly
units (e.g. nanomolar)
calculating statistics across runs (and other
dimensions)
plotting timeseries, exporting data, etc.
15. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 15
SimulatorResultsDialog
16. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 16
Selecting data (filtering)
Filter compartments by name using wildcards
Using set class difference method
Custom ListWidget is a factory for filter LineEdit
17. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 17
Selecting data (sorting)
By providing custom QListWidget with regular
expression and function map
Sort species by name
Sort compartments by name and/or positions
18. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 18
Selecting data (sampling)
Reducing data size by
sampling reduces
memory requirements,
speeds up
calculations and
unclutters plots
100 of 10,000 runs
Every 100th
recorded
timepoint
19. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 19
Transforming data (scaling)
Converts time, substance quantities and
volumes from simulator data units into user-
friendly display units, using python-quantities
For calculating concentrations, use compartment
volumes (at each timepoint) if recorded, otherwise use
one volume for all compartments at all timepoints
20. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 20
Exporting data
Selected data can be exported in several
common formats for use with other tools:
text (.csv)
Excel (.xls)
NumPy (.npz)
21. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 21
Plotting timeseries (old)
22. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 22
Plotting timeseries (improved)
23. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 23
Exporting figures
Added ability to save figures resized to specific
screen dimensions (pixels), as surprisingly,
matplotlib only supports physical dimensions
(inches)
24. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 24
Model checker results
25. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 25
Visualising simulations (Mayavi)
26. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 26
Cross-platform issues
A lot of communication in Workbench uses
temporary files – these behave differently on
Windows to Linux
must close temp files before use, which can delete
them!
Pseudo-terminal emulation library pexpect used
to gather progress information not for Windows
winpexpect port not functioning at time of release
fall back to 'working' progress indicator
not good for long experiments
27. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 27
Deploying Python apps
A lot of dependencies inc. Python
Python comes with most Linux distributions, but not
with Windows (unlike JVM) and Mac system Python is
crippled
Therefore we needed to bundle Python interpreter and
all libraries for Windows/Mac
Possible solution: 'freezing' Python applications,
creates a single executable (~100 megabytes!)
Need a solid base to start from and add missing
libraries also python(x,y)
for Windows
28. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 28
Freezing Python applications
Freezing works well for small scripts, not so well for
applications with many libraries
uses static code analysis for determine libraries for
bundling – TraitsUI/Matplotlib use dynamic imports
Freezers: cx_freeze, bbfreeze, PyInstaller,
Wrap python setup.py py2exe/py2app with bash scripts
copy missed files, unzip zipped Python classes
patch Matplotlib
fix Mac library paths using install_name_tool
29. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 29
Windows and Mac non-Python
mcss, pmodelchecker and poptimizer need cross-
compiling for 32/64-bit Linux (chroot), Windows (32-
bit) and Mac (OSX Snow Leopard only)
pmodelchecker requires PRISM and MC2 which are
both Java applications
InstallJammer builds cross-platform installer:
30. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 30
Linux (Debian)
Not trivial either – no authorative packaging
guide
Make packages for mcss, ..., PRISM, MC2, libhdf5-
1.8
Used python-support to install Dashboard for
system Python(s)
(1) add apt repository to sources.list
(2) sudo apt-get install infobiotics-workbench
Automatic updates!
Fedora/Suse packages made using Alien
31. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 31
Documentation (User Guide)
32. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 32
Documentation (User Guide)
Produced using Sphinx:
input is reStructuredText
output is high-quality formatted HTML, PDF, …
Used Enthought reST editor for writing
33. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 33
Documentation (API)
HTML API
generated from
docstrings
using Endo
Alternatives:
Epydoc
Doxygen
34. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 34
Code availability
35. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 35
Soliciting user feedback
getsatisfaction.com Javascript added to Sphinx template
36. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 36
Conclusions
Cross-platform application development in
Python is easy, deployment is hard
Need to become an expert in knowing the
differences between platforms
50 / 50 development vs. the rest
Good to be part of a team :)
37. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 37
Publications
38. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 38
Acknowledgements
Jamie Twycross (mcss, packaging)
Fran Romero Campero (IML, pmodelchecker, docs)
Claudio Lima (poptimizer, docs)
Pawel (for turning me on to Python)
Natalio Krasnogor
39. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 39
Questions?
40. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 40
Slides available on web
41. Jonathan Blakes - Development and deployment of a cross-platform open-source software package 41
Infobiotics