Bye Bye Legacy: Simplifying the Journey
•Als PPTX, PDF herunterladen•
2 gefällt mir•3,279 views
Speaker: Nick Maybin, Development Manager, HSBC Speaker: Andrew Matthews, Equities Architect, HSBC Level: 100 (Beginner) Track: RDBMS to MongoDB A presentation from HSBC's Investment Bank Technology division, about a journey of two asset classes (Equities and Fixed Income), who have jointly started using MongoDB as a core technology within an enterprise strategy for simpler, better, and faster organizational change. What You Will Learn: - Simpler: Applying NoSQL to migrate from aging relational models to schema-less designs. - Better: As part of a structured transition plan from legacy technology to MongoDB, building an ODS toolkit with configurability, flexibility and enough dynamism to support two highly complex trading areas. - Faster: How to keep pace, convince, and adopt while introducing new technologies to the organization.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie Bye Bye Legacy: Simplifying the Journey
Ähnlich wie Bye Bye Legacy: Simplifying the Journey (20)
Mehr von MongoDB
Mehr von MongoDB (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Bye Bye Legacy: Simplifying the Journey
- 1. PUBLIC
- 2. PUBLIC
- 3. PUBLIC
- 4. The World’s Largest International Bank 2014 2017 93PB of Data 56PB of Data DATA ASSETS 4,500 Branches 37m Customers Total Assets $2.375tn Reported Revenue $47.9bn 70 countries & territories Present in 2015 77PB of Data 45% of clients have international presence PUBLIC
- 5. “Adopt technology that makes HSBC simpler, better and faster” “Adopt technology that makes HSBC simpler, better and faster” Technology Strategy PUBLIC
- 6. { simpler }; { better }; { faster }; Toolkit configurability, flexibility & dynamism supporting complex trading migrating relational models to dynamic designs Keep pace, convince and adopt new technologies Topics for today PUBLIC NoSQL ODS Adopt
- 7. { simpler }; Topics for today PUBLIC NoSQL migrating relational models to dynamic designs
- 8. PUBLIC Dynamic designs • Implicit schema • Tolerant readers • Easier evolution • Denormalization E x p l i c i t ^ { simpler }; NoSQL
- 9. PUBLIC Flexibility • Polymorphic data objects • Complex products • Regulatory change • Business demand { simpler }; NoSQL Credit Default Swap Interest Rate Swap Bond > db.trade.find({ “tradeDateTime“ : { "$lte" : ISODate("2017-05-13T00:00:00Z") } } }).pretty()
- 10. PUBLIC Availability & Scalability • Old world • Always-on • Failover, Resiliency & Redundancy • Out of the box! { simpler }; NoSQL
- 11. { better }; Topics for today PUBLIC ODS Toolkit configurability, flexibility & dynamism supporting complex trading
- 12. PUBLIC
- 13. PUBLIC Operational Data Store Client • GUI View Server • Microservices • Persistence • Evolving Data Model } { better }; What do we mean by an ODS Toolkit?
- 14. ODS Toolkit PUBLIC Abc { better };
- 15. What is it? a data modelling technique designed to handle historical data along two different timelines. This makes it possible to rewind information “as it actually was” combined with “as it was recorded”. Why is that useful? Within trade reporting it’s often a requirement to be able to recreate an old report both as it actually looked at the time of creation and as it should have looked given corrections made to the data after its creation. { better }; Bitemporal PUBLIC
- 18. PUBLIC { better }; > db.trade.update({ "_id": ObjectId("75f19449c6ff43b9a716330a"), "validTo": ISODate("2017-04-03T15:36:157Z") }) > db.trade.insert({ "tradeToken": "ODS0000654487", "version": 1, "validFrom": ISODate("2017-04-03T15:35:47Z"), "validTo": “INDEFINITE”, "direction": "BUY", "size": 100, "ticker": "MONGO.DB", "price": 210, "action": "NEW" "transactionTime": ISODate("2017-04-03T15:35:47Z") }) > { "tradeToken": "ODS0000654487", "version": 1, "validFrom": ISODate("2017-04-03T15:35:47Z"), "validTo": ISODate("2017-04-03T15:36:157Z"), "direction": "BUY", "size": 100, "ticker": "MONGO.DB", "price": 210, "action": "NEW" "transactionTime": ISODate("2017-04-03T15:35:47Z") }) > db.trade.insert({ "tradeToken": "ODS0000654487", "version": 2, "validFrom": ISODate("2017-04-03T15:36:157Z"), "validTo": “INDEFINITE”, "direction":, "BUY", "size": 200 "ticker": "MONGO.DB", "price": 210, "action": “AMEND“, "transactionTime": ISODate("2017-04- 03T15:36:157Z") }) Microservice: TradeAmendProcessor
- 19. { faster }; Topics for today PUBLIC Adopt Keep pace, convince and adopt new technologies
- 20. Clear goal Timeboxed Throwaway The strategy PUBLIC { faster }; • Do something small • Keep it simple • Choose carefully Proof of Concept Proof of Value Cost Value Working Vision
- 21. The people { faster }; • You will need: Techies Influencers Communicators • Then Celebrate successes • Followed by Continued collaboration PUBLIC
- 22. • Trial and error (Doh!) • Automate • Streamline • Take the org with you Team Organisation Technology { faster }; PUBLIC The process DiminishingOps TraditionalOps . . . DevOps LessOps NoOps? Time Workload
- 23. It's about the journey… PUBLIC NoSQL •Dynamic •Flexible •Available ODS •Toolkit •Bitemporal •JSON Adoption •Strategy •People •Process simpler better faster
- 24. Thanks for listening PUBLIC Nick Maybin Andrew Matthews
Hinweis der Redaktion
- Play introductory video from off stage
- Good morning and thanks for coming to our talk! We’re very excited to be here. Andrew Matthews Solution Architect & Distinguished Engineer SLOW HSBC’s Investment Banking > Equities Technology New Zealand > darta > data! Nick Maybin Development Manager & Accredited Engineer Also HSBC’s Investment Bank > Fixed Income IT United Kingdom > JASON > JAYSON Global Banking and Markets 1 of 4 core businesses offered to customers Global Banking and Markets provides institutional investors and hedge funds access to trade equities, debt, money and foreign exchange markets, supporting thousands other thriving businesses worldwide. GBM IT – People in the US, Canada & Mexico
- 37 million customers in 70 countries & territories with 4,500 branches + digital channels Data assets rapidly growing. Now more than ever, data management is critical to all areas of the bank. Growing because our Customers are demanding more Growing because the Regulators we work with are demanding more And Growing because our business is creating more
- Our Executive board introduced new key strategies & transformation programmes SLOW 1 of those is to adopt technology that makes HSBC, simpler, better and faster Today’s focus 2 trading technology teams where we’ll detail a collaborative initiative we’ve recently commenced using MongoDB. Building an Operational Data Store (or ODS) to enable, equip and engage trading desks, salespeople and operations teams across the world, rapidly serving up data consistently and reliably. More about the Joint Venture
- So we are going to talk through 3 topics for today FIRST I will NoSQL – and how it makes data modelling simpler for our traded products SECOND Andrew will Operational Data Store (or ODS) to enable Front Office Trading teams to be better THIRD Our framework to introduce new technology helping change our organisation to make us faster
- Ok so let’s take a look at NoSQL… Cast mind back 18 months there were some obstacles to overcome moving to NoSQL Initially there was resistance & concerns in my department about the perceived lack of a schema. But soon after working MongoDB for a while we realised we had a schema…
- <CLICK> an implicit schema… our CODE a contract defined by our code. The developers in my team are able to work more effectively and concentrate in a single location with their code, where they can test, develop and solve business problems. Basic example – as you can see not real code. Now when you only have 1 App Team and 1 Database == fine BUT more than 1 team… starts to get more challenging Principle of <CLICK> Tolerant Readers (Martin Fowler) “take only what you need and you can be impervious to change” <CLICK> Orders with a nested instrument – basic example Next transition was to use <CLICK> REST APIs that provide explicit schemas and abstract complexity from consumers. <CLICK> Enabling rapid change data denormalization <CLICK>, Performance Reasons migrating from many tables with numerous inner joins to single documents.
- Big benefit … NoSQL <CLICK> Polymorphic data objects - flexibility in the data layer <CLICK> These can be complex products This HELPS us accommodate frequent changes required for instance <CLICK> when working on Regulatory requirements There are increasing levels of data transparency and real-time reporting required for authorities across many jurisdictions. <CLICK> Equally new business demand to implement features and concepts for trading desks.
- The final area for NoSQL that gave us a few challenges are around our Data Center topology Some of our trading hubs only have two data centers. <CLICK> Manual failovers? <CLICK> from primary to secondary data centres, even with sophisticated clustering services it’s an excruciating experience. <CLICK> NoSQL > database horizontal scaling and high availability <CLICK> MongoDB > replicasets, redundancy & fault tolerance, out-of-the-box. few lines of config. Take out expert advice reviewing access patterns & write concerns. Challenges with our data centre topology, where by redundant pairs (at least two servers in two data centres) were sufficient, but with automated consensus based voting we’ve been pushed to designs requiring tertiary data centres. on-prem, 3rd party hosting or cloud.
- We’re going to take you through our technology choices at HSBC using MongoDB
- GUI Client and Server toolkit <AUTO FLICK in 3s> Microservices Event sourcing patterns – CQRS
- GUI Client and Server toolkit Microservices Event sourcing patterns – CQRS Wrapper / Data Wall / REST APIs to Avoid… Schema leakage ACID Batch to Event JSON Front to back – native
- <AUTO FLICK in 3s>
- Bitemporal - What does it mean Example: Source Control systems are Bitemporal, you can work on one time axis, but if you need the truth as of last Wednesday you can branch from that version.
- Bitemporal - What does it mean Example: Source Control systems are Bitemporal, you can work on one time axis, but if you need the truth as of last Wednesday you can branch from that version.
- Bitemporal - What does it mean Example: Source Control systems are Bitemporal, you can work on one time axis, but if you need the truth as of last Wednesday you can branch from that version.
- Common thread, data format <AUTO FLICK in 3s>
- Common thread, data format
- You might be fortunate enough to work in small organisations – like startups, but when you work for an organisation that employees over 200,000 people it’s not always easy to introduce new technology, there’s a degree of agreement and approvals needed. Over our time at HSBC we’ve had our share of lively debates to advance technology and tools forward? So we have put this Strategy together
- Small – Timeboxed (depends on product, could be days, into weeks). Disposable or throwaway, emotionally detached. Goal orientated Keep it simple – there’s lots of technology in all quarters, many competitors and with those projects comes features, it is often easy to trial and use additional features, but this can lock you in. In the early days keep your use cases simple. <CLICK> Choose carefully – it’s likely it will take a while to get it approved, and you run with it for a good period of time. <CLICK> PoC vs <CLICK> PoV Not CV++
- Use your network <CLICK> Techies – like this little guy, eager, curious, innovators & early adopters <CLICK> Influential leaders – in large organisations you often need some backing <CLICK> Communicators – people who’ll share progress effectively – Emotional Attachment, Job Protection-ism <CLICK> Celebrate successes – celebrate actual successes, because of put in Subversion… we have now… improved automated CI tooling <CLICK> Collaboration – internals webcasts, face to face show & tells, external meet-ups to gain knowledge
- Trial & Adopt – do > doh > learn <CLICK> Automate <CLICK> Streamline <CLICK> Many organisations are adopting DevOps these days, where will it go, … even LessOps, is NoOps a possibility? Teams move at different paces > Technology drives you forward > Hearts and Minds <CLICK> Andrew – be open, tell people what you are doing