Video and slides synchronized, mp3 and slide download available at http://bit.ly/1cghaiX.
Mat Wall describes some of the tools & techniques that are used within the UK Government Digital Service to try and make the government behave less like an enterprise and more like a startup.Filmed at qconlondon.com.
Mat Wall is a technical architect who can still actually make things and now works for the Cabinet Office, a rare thing indeed. Mat works on elements of the GOV.UK website, as well as working within departments on transactional services. Twitter: @matwall
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Green Shoots in the Brownest Field: Being a Startup in Government
1. Green shoots in the brownest field
Mat Wall (@matwall)
Government Digital Service
https://www.gov.uk
Being a startup in government
Friday, 8 March 13
2. InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
http://www.infoq.com/presentations
/UK-gov-gds-tools-techniques
3. Presented at QCon London
www.qconlondon.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
4. GDS
Government development skills largely outsourced in 80s
Government IT vendor driven, procurement focussed
People who can write code (not COBOL) very rare
Friday, 8 March 13
5. GDS set up about 18 months ago
Bring digital capability & vision into government
Stimulate economy of smaller vendors
A startup within government
Friday, 8 March 13
6. Initially build alpha of GOV.UK
Now fully released into production
Migrating government department sites
Beginning to work on transactions
Friday, 8 March 13
7. Design Principles
1. Start with user needs
2. Do less
3. Design with data
4. Do the hard work to make it simple
5. Iterate wildly
6. Build for inclusion
7. Understand context
8. Build services, not websites
9. Be consistent, not uniform
10. Make things open: it makes things better
https://www.gov.uk/designprinciples
Friday, 8 March 13
35. ESB
ESB
OPAEDM
WEB WEB WEB WEB
Security stuff
XML translation layer
SAN SAP
Enterprise app servers
Friday, 8 March 13
36. ESB
ESB
OPAEDM
WEB WEB WEB WEB
Security stuff
XML translation layer
SAN SAP
Enterprise app servers
Now, what shall we make?
Friday, 8 March 13
37. ESB
ESB
OPAEDM
WEB WEB WEB WEB
Security stuff
XML translation layer
SAN SAP
Enterprise app servers
Imagine being a developer here
Most decisions made for you
Little room for innovation
Friday, 8 March 13
38. reduce risk = no innovation, create lock in
minimise cost = ha ha
predictable results = slow delivery
Friday, 8 March 13
47. Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
.gov.uk
frontend
inside gov
smart
answers
licensingsearch
Content API
Publisher
Panopticon MongoDB MySQL
CMS tools
Licensing
admin
Sign onneed-o-tron
Friday, 8 March 13
48. Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
.gov.uk
frontend search
Content API
Publisher
PanopticonMongoDB
need-o-tron
routing
Friday, 8 March 13
49. Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
.gov.uk
frontend search
Content API
Publisher
PanopticonMongoDB
need-o-tron
routing
Content model split across apps
Complex interaction between apps
Brittle, what if interaction fails?
Friday, 8 March 13
50. Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
.gov.uk
frontend search
Content API
Publisher
PanopticonMongoDB
need-o-tron
routing
Friday, 8 March 13
51. Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
.gov.uk
frontend search
Content API
Publisher
PanopticonMongoDB
need-o-tron
routing
Friday, 8 March 13
52. Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
.gov.uk
frontend search
Content API /
Panopticon
Publisher
MongoDB
need-o-tron
routing
Event broadcast
Friday, 8 March 13
53. Space
Help developers create a safe space
Create small, simple apps in that space
Provide lightweight components & processes, and ability to create more
Don’t worry about trying to solve all of the problems up front
Allow for innovation & learning
Friday, 8 March 13
55. Momentum
We far prefer momentum to strategy
Make decisions as late as possible
Short iterations mean you can’t go to far wrong
Easy to put things back on track
Allows for innovation, experiment & learning
Friday, 8 March 13
56. New system: apply to register to vote (ERTP)
Initially seemed quite simple
Surely just a form and a database?
Real world is more complex
Friday, 8 March 13
57. Local authority 1 Local authority 2 Local authority 387....
Citizens
DWP
Friday, 8 March 13
58. Citizen API
Local auth API
MatchingAPI
Local authority 1 Local authority 2 Local authority 387....
DWP
Web space
LA space
DWP space
Friday, 8 March 13
59. Citizen API
Local auth API
MatchingAPI
Local authority 1 Local authority 2 Local authority 387....
DWPMongoDB
Frontend web
Feeds Monitoring
Friday, 8 March 13
63. Local auth API
MatchingAPILocal authority 1 Local authority 2 Local authority 387....
DWP
Frontend web
Anything goes to keep momentum
Friday, 8 March 13
65. So far most of our systems have been very simple
Easy to be small, beautiful & agile
But that will change with transactions
Friday, 8 March 13
66. Government IT estate complex, old
Pre-web technologies
Oldest system I have found so far is 40 years!
How can we work in this environment?
How can we retain our momentum?
Friday, 8 March 13
67. ESB
ESB
OPAEDM
WEB WEB WEB WEB
Security stuff
XML translation layer
SAN SAP
Enterprise app servers
Friday, 8 March 13
68. Integration strategies:
Could rebuild old system.
Could export data from old system into new.
Use what we have learned to create our own space, with our own momentum
Friday, 8 March 13
69. Shiny application
Domain API Domain API
Create prototype application backed of simple REST domain APIS
Each API encapsulates a single domain concept
(ie: User profile, transaction history, application service)
Domain APIs filled with mock data
HTTP integration test suite produced
Domain APIDomain API
Friday, 8 March 13
70. Shiny application
Domain API Domain API
Integration tests form spec that can be shared between old & new world
Api built by new team, using old APIs / integrations
APIs can be built in existing system, by existing team
Or as a joint effort, using a facade
Domain APIDomain API
Friday, 8 March 13
72. Shiny application Domain API
New space Old space
http, JSON
GET /user/341353/transactions
select * from transactions where
userId=431353
Friday, 8 March 13