Presentation by Prof. Ola Spjuth at the ELRIG conference 'Robotics and Automation 2022' at FESTO, Esslingen, DE on 2022-12-01. ABSTRACT: We have set up an open source robotized lab for image-based drug screening and cell profiling. The lab can operate on multiple simultaneous 384-well microplates, using high-content microscopy imaging as the primary readout. Our main protocol is morphological profiling using multiplexed fluorescent dyes (Cell Painting) after cells being exposed to treatments with individual or combinations of chemical compounds, but we are increasingly using live cell imaging in brightfield for temporal analyses. In this presentation I will describe our open source efforts in automation, covering how we apply artificial intelligence / machine learning to design efficient multi-well and multi-plate experiments, and execute the generated protocols using robotics in our cell-based lab. I will also describe our approach to preprocess, filter, store and analyze the large amounts of images produced. An important application we are targeting is exploration of combination effects of drugs and environmental chemicals.

1. Automating cell-based screening
with open source, robotics and AI
Ola Spjuth, Professor
Department of Pharmaceutical Biosciences and Science for Life Laboratory
Uppsala University
https://www.linkedin.com/in/olaspjuth/

2. l Processing Stream Processing
Dat a
osit ory
Query
request
response
Real- T ime
Analyt ics
Dat a Result s
Current fact finding
Analyze data in motion – before it is stored
ormation stored on disk
Our mission: Accelerating drug discovery

3. Applications
in
drug
discovery
Cell profiling
Automation
AI/ML
Accelerate drug discovery using AI,
automation and intelligent design
of experiments
• Predict safety concerns
• Predict drug mechanisms
• Screen for new drugs
• Precision medicine
Research objective:

4. Applications
in
drug
discovery
Cell profiling
Automation
AI/ML
Open source fully automated cell profiling lab
Cell Painting – high-content imaging Computational and Data Management Infrastructure
AI and Robotics
Morphological profiling experiments
Intelligent
design of
experiments
Vision: Autonomous
experimentation

5. Cell
profiling
AI/ML
Morphological
Profile
Cell segmentation
& Image analysis
Microscopy
imaging
Chemical perturbation
of cells
Experiments in
multiwell plates
Cell Painting Bray et al. Nat Protoc 2016
DNA Damage
Prediction
Deep Neural Network
Drug-treated cells
Kinase inhibitor
Kensert A, Harrison PJ, Spjuth O.
Transfer learning with deep convolutional neural network for classifying cellular
morphological changes. SLAS DISCOVERY: Advancing Life Sciences R&D. 24, 4 (2019)

6. Our objectives with automation
• Fully automated Cell Painting
• Robotized experiments
• Configurable, serviceable by us
• Cost-efficient, use off-the-shelf instruments
• Full control of all steps
• Automated analytics
• Standardized data capturing, preprocessing,
and AI modeling
• Closed-loop science
• Full control of all steps open up for online
decision making
AI and Robotics
Morphological profiling experiments
Intelligent
design of
experiments
Vision: Autonomous
experimentation

7. Automating cell profiling:
Building an open source robotized lab
Plate incubator
Washer
Dispenser
Robot arm
https://github.com/pharmbio/aros
Washer

8. incubation for 24-48h
at 37℃ and 5% CO2
Washer
(2X PBS)
Mitotracker staining
(live cells staining)
[dispenser peripump 1]
incubation for 20 min
at 37℃ and 5% CO2
Washer
(3X PBS)
Fixation
(4% PFA)
[dispenser Syringe A]
Seeded plates with compound
treatment
Washer
(3X PBS)
incubation for 20 min
in room temperature
Permeabilization
(0.1% Triton X-100)
[dispenser Syringe B]
Washer
(3X PBS)
incubation for 20 min
in room temperature
Post-fixation staining
(staining mixture of 5 dyes)
[dispenser peripump 2]
Washer
(5X PBS)
incubation for 20 min
in room temperature
Imaging store in 4°C
Optimizing and executing experiments
Detailed logs
Checkpoint('batch_1')
WaitForCheckpoint('batch_1') + 'plate_1_get_delay'
EnqueueOnMachine(incubator, RunMachine(incubator, 'get L1'))
RunMachine(robotarm, 'goto incubator')
WaitForMachineReady(incubator)
RunMachine(robotarm, 'get from incubator')
RunMachine(robotarm, 'lid off')
RunMachine(robotarm, 'to washer')
EnqueueOnMachine(washer, [
WaitForCheckpoint('batch_1') + 'plate_1_wash_delay',
RunMachine(washer, 'wash_2X_before_mito.LHC')])
EnqueueOnMachine(dispenser, [
Idle('mito_prime_delay', optimize='maximize'),
RunMachine(dispenser, 'mito_prime.LHC')])
WaitForMachineReady(washer)
RunMachine(robotarm, 'wash to disp')
WaitForMachineReady(disp)
EnqueueOnMachine(dispenser, RunMachine(dispenser, 'mito_40ul.LHC'))
WaitForMachineReady(disp)
Checkpoint('plate_1_incubation_1')
...
Protocols expressed in code.
Generates a protcol in a DSL.
Optimized using constraint programming.
https://github.com/pharmbio/aros

9. AI for experimental design
AI for optimizing
experiments
AI and robotics to automate cell profiling
AI for data analysis
AI to guide image
acquisition
AI for cell segmentaion

10. Microplate
Wells
1
384
Sites
3 456
Images
>15-25 K
(200 GB)
Cells
>300 K
Measurements
>400 M
x9/ 2h

11. Dealing with continuous large
scale data
GPU cluster
CPU server
Storage
Cloud
Online processing
Automating analysis
pipelines
Images AI modeling
QC
Morphology profiles
Computational
infrastructure

12. • Image and metadata storage in database
• Access through browser
• Quick image visualisation including:
o Timepoint
o Site
o Channel
• Direct link to analysis pipeline
Image Database + web viewer
https://imagedb.k8s-prod.pharmb.io/

13. Automating Analysis pipelines

14. Jupyter Notebooks for data analysis
• QC pipeline
• Data analysis
• Data visualisation
• Publish via github

15. How to scale up imaging?
• Minimize images captured and stored, also go towards live-cell imaging
• Get more and faster microscopes
Hongquan L. et al. “Squid: Simplifying Quantitative
Imaging Platform Development and Deployment.”
bioRxiv 2020.12.28.424613
https://squid-imaging.org

16. Data-driven image prioritization
Microscope farm Image streams
…
…
…
Feature
Extraction
Interestingness
function
Policy
Prioritization
How can we deal with large, continuous streams of images?

17. Designing cell profiling experiments
• Many options to consider:
• Which cell line(s)
• How many and which compounds
• Concentrations
• Replicates
• Exposure time(s)
• Total number of plates
• Positive controls, negative controls
• Can AI be used to optimize design of
experiments?
• Minimize experiment time and cost
• Maximize usefulness of resulting data

18. -Manuscript Draft- 13th
Dec, 2021, 15:32
Plate effect Border layout Random layout Effective layout
Figure 1: Examples of the distribution of 20 negative control in layouts for
384-well microplates. The colors indicate the intensity measured at each well.
Approach: Use constraint programming to design
effective plate layouts
• Optimal under set of constraints
• Declarative / flexible to adapt
Designing microplate experiments with AI
Rodríguez MAF, Carreras-Puigvert J, and Spjuth O. “Designing microplate layouts
using artificial intelligence.” bioRxiv. 2022.03.31.486595 (2022).
-Manuscri
(b) Relative IC50/ EC50
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
(d) Residuals
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
⇤
Screening
Dose-response

19. https://github.com/pharmbio/plaid
PLAID: Plate Layouts
using Artificial
Intelligence Design

20. From prediction to decision making:
What is the next, best experiment?
Automated lab AI models Data
Decision making

21. Screening for antiviral drugs by phenomics
Reversal of the SARS-CoV-2-induced phenotype in Vero E6 cells
Unpublished data
Rietdijk J, Tampere M, Pettke A, Georgieva P, Lapins M, Warpman Berglund U, Spjuth O, Puumalainen MR,
Carreras-Puigvert J. “A phenomics approach for antiviral drug discovery.” BMC Biology 19 (2021).
Can we improve using
drug combinations?

22. • Safety of chemicals and drugs assessed individually
• Combination therapy is common for many diseases
• How can we research combinations/mixtures of chemicals?
• Our approach:
• Cell Painting: Ideal tradeoff between speed/cost/information content
• AI/ML: Select and design next batch of combinations to test
• Automation: Iteratively carry out experiments and repeat
Ongoing research: Study combinations
of compounds

23. Mixtures and Cell Painting

24. AI and Robotics
Morphological profiling experiments
Cell lines /
Primary cells
Profiling of individual drugs with
known mechanisms
….
Drugs
Profiling and screening for
drug combinations
….
….
A discovery engine!
Iterative exploration of
individualized drug combinations
Primary
cells and
organoids
Individualized drug
combination
Patient

25. • Manage life cycle of AI
models, improve FAIR1
• Deploy and use apps and AI
models in production
• Effort at SciLifeLab Data
Center, Sweden
Working with ML models
1 Spjuth O, Frid J, and Hellander A.
The Machine Learning Life Cycle and the Cloud: Implications for Drug Discovery
Expert Opinion On Drug Discovery. 16, 9, 1071-1079. (2021).
DOI: 10.1080/17460441.2021.1932812 https://serve.scilifelab.se

26. Teaching next-generation scientists
Introduction to Lab Automation, 7.5 c
Learn to use automated pipetting, plate
handling, microscopy.
Big Data in Life Science, 5c
Compute infrastructure and analysis
methods to analyze large data.
Other courses (italic=online):
Pharmaceutical Bioinformatics, 7.5 c
Pharmaceutical Bioinformatics with Sequence Analysis, 7.5c
Applied Pharmaceutical Bioinformatics, 5c
Applied Pharmaceutical Structural Bioinformatics, 5c
Artificial Intelligence in drug
discovery, 7.5c
Online introductory course.

27. Acknowledgements
Funding:
Pharmb.io research group
Jordi Carreras-Puigvert
Wesley Schaal
Jonathan Alvarsson
Maris Lapins
Polina Georgieva
Malin Jarvius
Anders Larsson
Dan Rosén
Andreina Rodrigues
Christa Ringers
Staffan McShane
Amelie Wenz
Ebba Bergman
Phil Harrison
Jonne Rietdijk
David Holmberg
Akshai Sreenivasan
Martin Johansson
HASTE project
Carolina Wählby, UU
Andreas Hellander, UU
Salman Toor, UU
Håkan Wieslander, UU
Ankit Gupta, UU
Tianru Zhang, UU
Xiaobo Zhao, UU
Ben Blamey, UU
Alan Sabirsh, AstraZeneca
Ida-Maria Sintorn, Vironova
Karolinska Institutet
Päivi Östling
Brinton Seashore-Ludlow
Marianna Tampere
SciLifeLab Data Center
The Serve team
NTNU/Trondheim
Åsmund Flobak
Astrid Laegrid
Ulf Norinder
Ernst Ahlberg
Lars Carlsson
Fredrik Svensson
CBCS

28. Research group website: http://pharmb.io
Ola Spjuth
ola.spjuth@farmbio.uu.se
https://www.linkedin.com/in/olaspjuth/
Thank you