SlideShare ist ein Scribd-Unternehmen logo
1 von 15
www.ccdc.cam.ac.uk
1
CCDC’s (ongoing) Journey to Continuous Delivery
Clare Macrae
Cambridge Crystallographic Data Centre
1 December 2015
@ClareMacraeUK
www.ccdc.cam.ac.uk
2
The Cambridge Crystallographic Data Centre
Established in 1965.
- UK Registered Charity.
- Financially self-supporting, not-for-profit.
- University Partner Institute.
International Data Repository
Archive of crystal structure data
High quality scientific database
Scientific Software Provider
Search/analysis/visualisation tools
Scientific applications
Collaborative Research Organisation
New methodologies
Fundamental research
Employer of around 60
permanent staff
- Scientific editors
- Software developers
- Applications scientists
- Cambridge, UK
- Rutgers University, US
www.ccdc.cam.ac.uk
3
Experimental Data
Structural Knowledge
Jeff Dahl, CC-BY-SA
CC-BY-SA
Radspunk, CC-BY-SA
C10H16N+,Cl-
Scientific Context
www.ccdc.cam.ac.uk
4
Development Context
• C++, Python, Fortran (decades old) and more
• No high-level support for CI, due to 2- to 3-day builds!
Several million lines of
code
• We had build and release automation in the ‘80s
• Many more manual layers added since then
• Couldn’t remember the last time we had a green build
Legacy build &
release tools & scripts
• Around four months elapsed time in preparation
• Too much manual effort, for release and testingOne release per year!
• Due to effort currently required in each release
Belief: we cannot
release more often
www.ccdc.cam.ac.uk
5
Initial Concerns – why did we wait for CI?
“Our builds
are too slow”
“Our builds
never
succeed”
“Too many
warnings from
our code”
www.ccdc.cam.ac.uk
6
C++ CI Journey…
C# team
started using
TeamCity
Jan 2014:
started
background-
project for
our TeamCity
C++ builds
Sep 2014:
Big push to
move C++
builds to
TeamCity
Oct 2014:
C++
TeamCity
builds turned
on
www.ccdc.cam.ac.uk
7
Getting to Green
• Several people in concerted effort to get any green builds
• Before TeamCity, we hadn’t had any green builds in years!
www.ccdc.cam.ac.uk
8
Speeding up tests
• Incredible value of powerful tools for seeing where time goes
www.ccdc.cam.ac.uk
9
• TeamCity showed evidence of need for new build machines…
• … and the benefit of the new machines
Speeding up build machines
www.ccdc.cam.ac.uk
10
Patterns become visible
www.ccdc.cam.ac.uk
11
Teams auto-deploying (internally)
www.ccdc.cam.ac.uk
12
Teams (and individuals) taking initiative
• This is right outside the Director’s office!
www.ccdc.cam.ac.uk
13
Finally treating warnings as errors
www.ccdc.cam.ac.uk
14
Where are we now?
Much happier
developers!
Team
agreements
needed for
handling
build errors…
More reliable
infrastructure
required
Discussions
have started
on releasing
more often!
www.ccdc.cam.ac.uk
15
Initial Concerns –what really happened?
“Our builds are
too slow”
CI system
identified
bottlenecks, and
speeded up builds,
tests
Also got us new
build hardware
Easy to identify the
slow tests – either
speed them up, or
run less often
“Our builds
never
succeed”
CI system
massively
simplified fixing
build errors
Seeing test history
across all builds
exposes flicker
tests
Per-push feedback
within 15-30
minutes during the
day, instead of the
next morning
“Too many
warnings from
our code”
Fast feedback
made it viable to
treat warnings as
errors

Weitere ähnliche Inhalte

Ähnlich wie CCDC’s (ongoing) Journey to Continuous Delivery - London Continuous Delivery - Dec 2015

Quarterly Technology Briefing, Manchester, UK September 2013
Quarterly Technology Briefing, Manchester, UK September 2013Quarterly Technology Briefing, Manchester, UK September 2013
Quarterly Technology Briefing, Manchester, UK September 2013
Thoughtworks
 
SDCSB CYTOSCAPE AND NETWORK ANALYSIS WORKSHOP at Sanford Consortium
SDCSB CYTOSCAPE AND NETWORK ANALYSIS WORKSHOP at Sanford ConsortiumSDCSB CYTOSCAPE AND NETWORK ANALYSIS WORKSHOP at Sanford Consortium
SDCSB CYTOSCAPE AND NETWORK ANALYSIS WORKSHOP at Sanford Consortium
Keiichiro Ono
 

Ähnlich wie CCDC’s (ongoing) Journey to Continuous Delivery - London Continuous Delivery - Dec 2015 (20)

Quarterly Technology Briefing, Manchester, UK September 2013
Quarterly Technology Briefing, Manchester, UK September 2013Quarterly Technology Briefing, Manchester, UK September 2013
Quarterly Technology Briefing, Manchester, UK September 2013
 
RDM initiatives
RDM initiativesRDM initiatives
RDM initiatives
 
2015 Broken Hill Resources Investment Symposium - Unearthed - Zane Prickett
2015 Broken Hill Resources Investment Symposium - Unearthed - Zane Prickett2015 Broken Hill Resources Investment Symposium - Unearthed - Zane Prickett
2015 Broken Hill Resources Investment Symposium - Unearthed - Zane Prickett
 
PuppetConf 2017: Cloud, Containers, Puppet and You- Carl Caum, Puppet
PuppetConf 2017: Cloud, Containers, Puppet and You- Carl Caum, PuppetPuppetConf 2017: Cloud, Containers, Puppet and You- Carl Caum, Puppet
PuppetConf 2017: Cloud, Containers, Puppet and You- Carl Caum, Puppet
 
AWS re:Invent 2016: Hardware-Accelerating Graphics Desktop Workloads with Ama...
AWS re:Invent 2016: Hardware-Accelerating Graphics Desktop Workloads with Ama...AWS re:Invent 2016: Hardware-Accelerating Graphics Desktop Workloads with Ama...
AWS re:Invent 2016: Hardware-Accelerating Graphics Desktop Workloads with Ama...
 
Clipper jisc rdn cambridge 2016
Clipper jisc rdn cambridge 2016Clipper jisc rdn cambridge 2016
Clipper jisc rdn cambridge 2016
 
Clipper, research data network
Clipper, research data networkClipper, research data network
Clipper, research data network
 
SDCSB CYTOSCAPE AND NETWORK ANALYSIS WORKSHOP at Sanford Consortium
SDCSB CYTOSCAPE AND NETWORK ANALYSIS WORKSHOP at Sanford ConsortiumSDCSB CYTOSCAPE AND NETWORK ANALYSIS WORKSHOP at Sanford Consortium
SDCSB CYTOSCAPE AND NETWORK ANALYSIS WORKSHOP at Sanford Consortium
 
The SKA Project - The World's Largest Streaming Data Processor
The SKA Project - The World's Largest Streaming Data ProcessorThe SKA Project - The World's Largest Streaming Data Processor
The SKA Project - The World's Largest Streaming Data Processor
 
WAPWG Clark defining capturing_web-based_if
WAPWG Clark defining capturing_web-based_ifWAPWG Clark defining capturing_web-based_if
WAPWG Clark defining capturing_web-based_if
 
Acumen global-solutions
Acumen global-solutionsAcumen global-solutions
Acumen global-solutions
 
Ds for finance day 4
Ds for finance day 4Ds for finance day 4
Ds for finance day 4
 
Accelerating Data Lakes and Streams with Real-time Analytics
Accelerating Data Lakes and Streams with Real-time AnalyticsAccelerating Data Lakes and Streams with Real-time Analytics
Accelerating Data Lakes and Streams with Real-time Analytics
 
Demonstrating 100 Gbps in and out of the Clouds
Demonstrating 100 Gbps in and out of the CloudsDemonstrating 100 Gbps in and out of the Clouds
Demonstrating 100 Gbps in and out of the Clouds
 
What's next? Emerging trends in cloud computing
What's next? Emerging trends in cloud computingWhat's next? Emerging trends in cloud computing
What's next? Emerging trends in cloud computing
 
The Catalan Research portal: collecting information from Catalan universities...
The Catalan Research portal: collecting information from Catalan universities...The Catalan Research portal: collecting information from Catalan universities...
The Catalan Research portal: collecting information from Catalan universities...
 
The Catalan Research portal: collecting information from Catalan universities...
The Catalan Research portal: collecting information from Catalan universities...The Catalan Research portal: collecting information from Catalan universities...
The Catalan Research portal: collecting information from Catalan universities...
 
Fiatech 2014 - Big BIM Implementation, Zuhair Haddad
Fiatech 2014 - Big BIM Implementation, Zuhair HaddadFiatech 2014 - Big BIM Implementation, Zuhair Haddad
Fiatech 2014 - Big BIM Implementation, Zuhair Haddad
 
QuSandbox+NVIDIA Rapids
QuSandbox+NVIDIA RapidsQuSandbox+NVIDIA Rapids
QuSandbox+NVIDIA Rapids
 
Real time data viz with Spark Streaming, Kafka and D3.js
Real time data viz with Spark Streaming, Kafka and D3.jsReal time data viz with Spark Streaming, Kafka and D3.js
Real time data viz with Spark Streaming, Kafka and D3.js
 

Mehr von Clare Macrae

Quickly and Effectively Testing Legacy c++ Code with Approval Tests mu cpp
Quickly and Effectively Testing Legacy c++ Code with Approval Tests   mu cppQuickly and Effectively Testing Legacy c++ Code with Approval Tests   mu cpp
Quickly and Effectively Testing Legacy c++ Code with Approval Tests mu cpp
Clare Macrae
 
Quickly and Effectively Testing Legacy C++ Code with Approval Tests
Quickly and Effectively Testing Legacy C++ Code with Approval TestsQuickly and Effectively Testing Legacy C++ Code with Approval Tests
Quickly and Effectively Testing Legacy C++ Code with Approval Tests
Clare Macrae
 
Quickly Testing Legacy C++ Code with Approval Tests
Quickly Testing Legacy C++ Code with Approval TestsQuickly Testing Legacy C++ Code with Approval Tests
Quickly Testing Legacy C++ Code with Approval Tests
Clare Macrae
 

Mehr von Clare Macrae (14)

Testing Superpowers: Using CLion to Add Tests Easily
Testing Superpowers: Using CLion to Add Tests EasilyTesting Superpowers: Using CLion to Add Tests Easily
Testing Superpowers: Using CLion to Add Tests Easily
 
Quickly and Effectively Testing Legacy c++ Code with Approval Tests mu cpp
Quickly and Effectively Testing Legacy c++ Code with Approval Tests   mu cppQuickly and Effectively Testing Legacy c++ Code with Approval Tests   mu cpp
Quickly and Effectively Testing Legacy c++ Code with Approval Tests mu cpp
 
Quickly and Effectively Testing Legacy C++ Code with Approval Tests
Quickly and Effectively Testing Legacy C++ Code with Approval TestsQuickly and Effectively Testing Legacy C++ Code with Approval Tests
Quickly and Effectively Testing Legacy C++ Code with Approval Tests
 
How to use Approval Tests for C++ Effectively
How to use Approval Tests for C++ EffectivelyHow to use Approval Tests for C++ Effectively
How to use Approval Tests for C++ Effectively
 
Cpp Testing Techniques Tips and Tricks - Cpp Europe
Cpp Testing Techniques Tips and Tricks - Cpp EuropeCpp Testing Techniques Tips and Tricks - Cpp Europe
Cpp Testing Techniques Tips and Tricks - Cpp Europe
 
C++ Testing Techniques Tips and Tricks - C++ London
C++ Testing Techniques Tips and Tricks - C++ LondonC++ Testing Techniques Tips and Tricks - C++ London
C++ Testing Techniques Tips and Tricks - C++ London
 
Quickly Testing Qt Desktop Applications
Quickly Testing Qt Desktop ApplicationsQuickly Testing Qt Desktop Applications
Quickly Testing Qt Desktop Applications
 
Code samples that actually compile - Clare Macrae
Code samples that actually compile - Clare MacraeCode samples that actually compile - Clare Macrae
Code samples that actually compile - Clare Macrae
 
Quickly Testing Legacy C++ Code with Approval Tests
Quickly Testing Legacy C++ Code with Approval TestsQuickly Testing Legacy C++ Code with Approval Tests
Quickly Testing Legacy C++ Code with Approval Tests
 
Quickly Testing Legacy Cpp Code - ACCU Cambridge 2019
Quickly Testing Legacy Cpp Code - ACCU Cambridge 2019Quickly Testing Legacy Cpp Code - ACCU Cambridge 2019
Quickly Testing Legacy Cpp Code - ACCU Cambridge 2019
 
Quickly testing legacy code cppp.fr 2019 - clare macrae
Quickly testing legacy code   cppp.fr 2019 - clare macraeQuickly testing legacy code   cppp.fr 2019 - clare macrae
Quickly testing legacy code cppp.fr 2019 - clare macrae
 
Quickly Testing Legacy Code - ACCU York April 2019
Quickly Testing Legacy Code - ACCU York April 2019Quickly Testing Legacy Code - ACCU York April 2019
Quickly Testing Legacy Code - ACCU York April 2019
 
Quickly testing legacy code
Quickly testing legacy codeQuickly testing legacy code
Quickly testing legacy code
 
Escaping 5 decades of monolithic annual releases
Escaping 5 decades of monolithic annual releasesEscaping 5 decades of monolithic annual releases
Escaping 5 decades of monolithic annual releases
 

Kürzlich hochgeladen

AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
AI Mastery 201: Elevating Your Workflow with Advanced LLM TechniquesAI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
VictorSzoltysek
 
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdfintroduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
VishalKumarJha10
 
CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICECHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
9953056974 Low Rate Call Girls In Saket, Delhi NCR
 

Kürzlich hochgeladen (20)

Architecture decision records - How not to get lost in the past
Architecture decision records - How not to get lost in the pastArchitecture decision records - How not to get lost in the past
Architecture decision records - How not to get lost in the past
 
VTU technical seminar 8Th Sem on Scikit-learn
VTU technical seminar 8Th Sem on Scikit-learnVTU technical seminar 8Th Sem on Scikit-learn
VTU technical seminar 8Th Sem on Scikit-learn
 
AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
AI Mastery 201: Elevating Your Workflow with Advanced LLM TechniquesAI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques
 
Announcing Codolex 2.0 from GDK Software
Announcing Codolex 2.0 from GDK SoftwareAnnouncing Codolex 2.0 from GDK Software
Announcing Codolex 2.0 from GDK Software
 
Chinsurah Escorts ☎️8617697112 Starting From 5K to 15K High Profile Escorts ...
Chinsurah Escorts ☎️8617697112  Starting From 5K to 15K High Profile Escorts ...Chinsurah Escorts ☎️8617697112  Starting From 5K to 15K High Profile Escorts ...
Chinsurah Escorts ☎️8617697112 Starting From 5K to 15K High Profile Escorts ...
 
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdfintroduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf
 
Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf
Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdfPayment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf
Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf
 
Right Money Management App For Your Financial Goals
Right Money Management App For Your Financial GoalsRight Money Management App For Your Financial Goals
Right Money Management App For Your Financial Goals
 
%+27788225528 love spells in Vancouver Psychic Readings, Attraction spells,Br...
%+27788225528 love spells in Vancouver Psychic Readings, Attraction spells,Br...%+27788225528 love spells in Vancouver Psychic Readings, Attraction spells,Br...
%+27788225528 love spells in Vancouver Psychic Readings, Attraction spells,Br...
 
Define the academic and professional writing..pdf
Define the academic and professional writing..pdfDefine the academic and professional writing..pdf
Define the academic and professional writing..pdf
 
OpenChain - The Ramifications of ISO/IEC 5230 and ISO/IEC 18974 for Legal Pro...
OpenChain - The Ramifications of ISO/IEC 5230 and ISO/IEC 18974 for Legal Pro...OpenChain - The Ramifications of ISO/IEC 5230 and ISO/IEC 18974 for Legal Pro...
OpenChain - The Ramifications of ISO/IEC 5230 and ISO/IEC 18974 for Legal Pro...
 
Microsoft AI Transformation Partner Playbook.pdf
Microsoft AI Transformation Partner Playbook.pdfMicrosoft AI Transformation Partner Playbook.pdf
Microsoft AI Transformation Partner Playbook.pdf
 
%in ivory park+277-882-255-28 abortion pills for sale in ivory park
%in ivory park+277-882-255-28 abortion pills for sale in ivory park %in ivory park+277-882-255-28 abortion pills for sale in ivory park
%in ivory park+277-882-255-28 abortion pills for sale in ivory park
 
%in Lydenburg+277-882-255-28 abortion pills for sale in Lydenburg
%in Lydenburg+277-882-255-28 abortion pills for sale in Lydenburg%in Lydenburg+277-882-255-28 abortion pills for sale in Lydenburg
%in Lydenburg+277-882-255-28 abortion pills for sale in Lydenburg
 
%in kempton park+277-882-255-28 abortion pills for sale in kempton park
%in kempton park+277-882-255-28 abortion pills for sale in kempton park %in kempton park+277-882-255-28 abortion pills for sale in kempton park
%in kempton park+277-882-255-28 abortion pills for sale in kempton park
 
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
 
AI & Machine Learning Presentation Template
AI & Machine Learning Presentation TemplateAI & Machine Learning Presentation Template
AI & Machine Learning Presentation Template
 
CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICECHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE
 
Introducing Microsoft’s new Enterprise Work Management (EWM) Solution
Introducing Microsoft’s new Enterprise Work Management (EWM) SolutionIntroducing Microsoft’s new Enterprise Work Management (EWM) Solution
Introducing Microsoft’s new Enterprise Work Management (EWM) Solution
 
SHRMPro HRMS Software Solutions Presentation
SHRMPro HRMS Software Solutions PresentationSHRMPro HRMS Software Solutions Presentation
SHRMPro HRMS Software Solutions Presentation
 

CCDC’s (ongoing) Journey to Continuous Delivery - London Continuous Delivery - Dec 2015

Hinweis der Redaktion

  1. Every drug discovered in the last 30 years has in part owed its discovery to our database and software!
  2. Inevitable mixture of new and legacy code, data, and build systems Many manual tasks for release (despite multiple automation efforts) Much testing by internal users
  3. C# team didn’t have the legacy code
  4. linux-32 release continuous
  5. https://code.google.com/p/wall-t/ https://github.com/crunchie84/teamcity-dashboard/
  6. Mention code coverage too
  7. Don’t underestimate the value CI for providing or enabling: Fast feedback to developers All kinds of data about your builds Integrations with other tools (for dashboards, code quality, deployment, etc.) Visibility to internal stakeholders I wish I'd known about the last three much earlier It can be hard to persuade stakeholders in advance of the value Just do it! Implementing CI has enabled much faster innovation than I dreamed possible Many code-quality things we’d talked about for years are now just implemented