A short talk given to the Drupal Show and Tell meetup, 24 October 2013. Based on their experience as developers in Capgemini UK Digital, Andy Read covers a high level overview of architectures for high performance websites and Temi Jegede looks at some of the team dynamics issues.
2. Who Are We?
• Andy Read (Andy.Read@Capgemini.com)
• Temi Jegede (Temi.Jegede@Capgemini.com)
(140,000 people)
(140 people)
Drupal
Sites
3. Big Sites: It’s Not Rocket Science
Rockets
101
1 billion users
Arc
3 day
round
trip
Thrust
Gravity
Just follow the
laws of physics
My Big
Site 101
3 sec
round
trip!!
Dozens of
Servers
Numerous 3rd-party services
Virtually infinite complexity!
It’s Much, Much Harder!
4. Goals
•
•
•
•
•
•
•
Scale (number of concurrent users)
Performance (fast response time)
Functional Scope
Integration with 3rd-party services
Team size and many roles (Dev, Test, Ops, BA...)
Testability / Reliability / Reproducibility
Security & Corporate governance (reputation)
5. Drupal / LAMP Architecture
Drupal
Single
[virtual]
server
Apache
(Web Server)
MySQL
PHP
File system
Linux
6. A Bit More Performance
Drupal
Varnish
cache
DB Server
Apache /
PHP
Nginx
APC
MemCache
SSD
RAM+++
File system
Linux
MySQL
7. Serious Performance!
Drupal
Apache /
Nginx
Master DB
PHP
MemCache
File system
Drupal
RAM+++
Apache /
Nginx
MySQL
SSD
PHP
Linux
Varnish
cache
MemCache
File system
RAM+++
SSD
Linux
Load
Balancer
SAN
Servers ++
Varnish
cache
Slave DB
Network
storage
Drupal
Apache /
Nginx
MySQL
Slave DB
PHP
MemCache
File system
Drupal
RAM+++
Apache /
Nginx
SSD
PHP
MySQL
Slave DB
Linux
MemCache
File system
RAM+++
SSD
Linux
MySQL
8. But... We’re Not Alone!
Product Catalogue
Drupal
ESB
HighPerformance
Mega
-Site
SalesForce
Postcode Address
Q
u
e
u
e
JSON / REST
XML/WSDL
SMTP
SOAP
Google Analytics
More Analytics
Yet More Analytics
Payment Gateway
Identity Check
Adobe PDF
Email
Graylog
9. Development Process
Test, Test, Test, Unit Test, BDD (Behat), Manual Test...
Dev
Dev
Dev
VM
VM
SIT
VM
Test Data
Code & Configuration
(Drupal 8 FTW!)
UAT
PreProd
Prod
User Data (!)
Content
(Editors &
Contributors)
Git / SVN
Automate everything! Review, Build, Test & Deploy
10. Team Dynamics
The way a team plays as a whole determines its
success. You may have the greatest bunch of
individual stars in the world, but if they don't
play together, the club won't be worth a dime.
Babe Ruth
11. Small Site -> Small Team
• A Developer
• Client
• Tools (A dev environment, test/production server)
13. Big Site -> Big Team
Multiple different roles within a team:
Dev-OPS
Developers
Project managers
Designers
Testers
Web authors
Business analyst
Clients
16. In a perfect situation each team member should have a solid
understanding of their job role and also a good grounding and
understanding of other team members’ job roles.
18. The Blind Men and The Elephant
Different people with different roles and perspectives will have
different ideas of what the “whole” is... Since each person can’t
see the whole, we have to have good communication with each
other to understand what the whole really is.