Seldon @ London.AI 2016-03-03
•
2 likes•985 views
Seldon is an open-source machine intelligence platform for enterprise. In this talk for London's AI meetup (http://london.ai/) on 03/03/2016 I cover: - Seldon's journey from closed source recommendation engine to open source machine learning platform. - the rapidly evolving open source machine learning landscape and the opportunities this presents. - overview of Seldon platform architecture and features. - our strategic direction and roadmap as we move towards our next major release. To find out more about Seldon: Github: https://github.com/SeldonIO/seldon-server/ Docs: http://docs.seldon.io/ VM/AMI download: http://www.seldon.io/get-started/ We're hiring: http://www.seldon.io/careers Big thanks to everyone at the event, the organisers John Henderson, Alexandre Flamant, Nathan Benaich and Danielle Chapman, and to our friends at Rainbird, Magic Pony and Signal - all building amazing AI startups out of the UK.
Recommended
Recommended
More Related Content
Recently uploaded
Recently uploaded (20)
Featured
Featured (20)
Seldon @ London.AI 2016-03-03
- 1. Machine Intelligence for Enterprise © 2016 Seldon Technologies Ltd. This work is licensed under a Creative Commons Attribution 4.0 International Public License
- 2. SELDON.IO ALEX HOUSLEY FOUNDER AND CEO @AHOUSLEY Open Source Machine Intelligence 03/03/2016
- 5. SELDON.IO Economic Social Technological 2016 the breakout year for open machine intelligence ● Lower computecosts ● Disruptive start-ups ● Data privacy and compliance ● Consumer expectations ● Workforce automation ● Data scientists and decision makers want control ● Commodificationof ML/AI technologies ● Growth in connected devices (mobile & IoT) ● Exponentialdata
- 6. SELDON.IO Our Journey 2011 2014 2015 2016 Content Recommendation Social Sharing Data scientists in short supply Organizations want more control and on premise solutions Open Source Machine Learning Recommendation and Prediction MicroservicesAPI Platform agnostic with no lock-in Deploy on premise or in the cloud FinTech Products Barclays PoC Machine Intelligence Ecosystem Global ML Model +
- 7. SELDON.IO Optimization Training Monitor Impact on KPIs Industry Models and Microservices Constant feedback loop PredictionsModelsIngest Data Machine Intelligence Pipeline
- 8. SELDON.IO How to add machine intelligence to your company Build In-House 3rd PartyAPI Open Predictive Platform Control Model Evaluation Time Data Scientists Cost $$$✔️ Too Many $$✗ Limited / Unknown $✔️ Industry Models +Your Own
- 12. SELDON.IO Seldon Microservices Seldon Server Your App or Service GET /predict GET /recommend P1 Vowpal Wabbit P2 XGBoost POST /users /items /actions GET /recommend POST /events GET /predict Internal Microservices API External REST API R1 IBM Watson RN Your algorithm!
- 13. SELDON.IO Fully Integrated Open-Source Ecosystem Microservices API Open Source Libraries Closed APIs Global Models
- 16. SELDON.IO 2016 is the breakout year for open source machine intelligence. Data scientists and decision makers want more control. Open source helps organizations focus on the last 10% of the problem.
- 18. SELDON.IO THANK YOU Alex Housley @ahousley ah@seldon.io http://docs.seldon.io/ Thank you! Questions?
- 19. SELDON.IO How do you make money?(the elephant in the room) OS Enterprise Services MLaaS Partner Ecosystem
Editor's Notes
- Install the Lato font for the best experience: http://www.fontsquirrel.com/fonts/LATO
- INTRO The last time I gave this talk was in Sydney, so I’m happy to halve my carbon footprint by having the chance to share this with the AI community in London! We released Seldon’s open source predictive platform in Feb after four years of R&D from an exceptional team of data scientists. Seldon is tried and tested as a closed API in demanding enterprise environment serving billions of recommendations every month. Last AUGUST- 2 choices I will talk about how OS helped us to survive and thrive. This is not meant to be a an evangelical talk about how I have “seen the light”, I want you share with you some of the things we have learned from moving closed to open. My name is Alex Housley founder and CEO, Seldon.
- [set the scene] You’re in a hot, Sweaty board room, mid-Summer, We were a SaaS predictive API with two PRODUCTS: Recommendation Engine and Predictive Social Sharing. FACED WITH 2 OPTIONS: 1. GET ACQUI-HIRED 2. START SOMETHING NEW We had spent the prev 9 months speaking to potential acquirers. During this process I had a front row seat on the priorities of some of world’s leading media, ecommerce, technology companies. COMMON PATTERN emerged: large companies putting data science and prediction at the very top if their strategic agenda for investment. They were acquisitive and recruiting expensive teams. Made me think: WHAT WOULD BE THE IMPACT OF MAKING THE FULL PREDICTIVE STACK OPEN SOURCE?
- On a wet and windy day in October 2014, our team were sitting on Brighton Beach, discussing the bigger picture of what we could achieve with Seldon. We had come a long way over the previous three years and were serving content recommendations to hundreds of millions of people every month. However, we believed that continuing to ship a black box solution would increasingly face obstacles in adoption by enterprises as machine learning technology became increasingly commoditized, and new applications were developed and adopted everywhere. So we did the most disruptive thing we could imagine and open-sourced the platform and algorithms that we had spent many years, and a couple of million pounds, building. We knew there was a risk that our competitors would steal what makes us valuable, but we also knew that the bigger risk would have been to stop innovating. So we took the leap and pivoted in one of the most exciting ways a technology company can go.
- There are a number of interconnected market forces at work that means 2016 will be a tipping point for machine intelligence: 250 billion billion (250 x 10^18) transistors were produced in 2014. Every second of that year, on average, 8 trillion transistors were produced. That figure is about 25 times the number of stars in the Milky Way. (according to Moores law production should now have doubled.) 58% of job activities can be automated. 47% of jobs will be taken over cognitive machines in the next 10 years Gartner Says 6.4 Billion Connected "Things" Will Be in Use in 2016. Seldon was one of the first companies to open source a machine learning platform. But last year we saw Google open-source TensorFlow, IBM donated SystemML to Apache. Elon Musk and Sam Altman form a non-profit AI research org called OpenAI. Open-source is a huge benefit to enterprise, particularly banks where data privacy and compliance are particularly important, as it gives full control with an on premise deployment.
- Seldon started as a content recommendation engine. Seldon is tried and tested as a closed API in demanding enterprise environment serving billions of recommendations every month, mostly recommending articles on news websites. In 2014 we took the business in a new open-source direction to create Seldon. 2016 – we’re building out a fintech product and are aiming to establish a POC with Barclays. From early discussions there are many parts of the bank that can benefit from Seldon.
- Seldon isn’t an OS library, it’s an end-to-end machine learning PIPELINE. Include best of Open Source and algorithms built ourselves. 1. CONNECT YOUR DATA - Ingesting behavioural data from events that contains metadata and context such as device and location. 2. MODEL BUILDING - Multiple models are built based on desired Goals (Could be a KPI or an action/event). Behavioral data plus algorithms are used to train the predictive models. computationally inefficient to store all the possible alternatives… realtime behavioural data needs to update the models in real-time. value not in the algorithm, the value is in the model. INDUSTRY MODEL - working in media, advertising and ecommerce. 3. OUTPUTS - there are currently two outputs for Seldon - one is a recommendation and another is a prediction (score). FEEDBACK LOOP - Models are optimised in a recursive way… FURTHER INFO Seldon pulls in behavioral data from any digital environment, builds predictive models and outputs recommendations and predictions at SCALE. But have built a generic platform with a broad range of applications including finance, insurance and healthcare.
- Build in house: DEMAND - there are far fewer for truly skilled machine learning and AI developers than big data engineers. Improving internal data science capabilities is increasingly important for companies. So they are hiring or aquahiring teams of data scientists. 3rd party: DATA SECURITY - there are many companies with data control policies that require the hosting of consumer data behind their firewall, which a flexible open source solution will allow. Sometimes there’s no transparency on algorithms. Open Source. MARKET DISRUPTION Open source technologies such as Docker, Hadoop, and Apache Spark, have superseded proprietary operating systems and databases. Meanwhile, most vendors higher up the data science stack (i.e. providing predictive analytics, recommendations, and machine learning APIs) are effectively licensing black box solutions. Seldon wants to reduce barriers to entry and get the Seldon’s technology that we believe in the hands of as many developers as possible. Businesses wanting to solve their own / domain-specific problems. ADD PRIVACY….
- WHY? Data scientists want more CONTROL to solve the problems specific to their business. PARATOS LAW in action – people should be spending 90% of their time solving the 10% of the domain-specific problems that make the biggest impact on their business, but otherwise data scientists are focusing on the remaining 90% and wasting time on reinventing the wheel. I’ll share with you a quick analogy with this DJ MIXER: Each channel on the mixer represents a predictive model. The controls represent hyper-parameters. Data scientist is the DJ listening to the audience and adjusting the controls accordingly. Seldon gives the DJ super powers. Enabling them to play all of the stages of a festival at the same time, so your audiences is not stuck listening to Lionel Richie (collaborative filtering) if you want to listen to Metallica (matrix factorization) But remember, you can always tap the DJ on the shoulder and make special requests.
- 3. OPEN SOURCE SETUP. SaaS platform grows roots, provisioning usually internal process with continuous integration and deployment. Rarely setup new infrastructure from scratch. VIRTUAL MACHINE. DOCUMENTATION. SaaS businesses don’t need as much. Docs on Github so people can commit changes – first pull requests. Documentation gets the highest engagement to see how to use it. [show documentation] COMMUNITY – newsletter, github, detail release notes, clean codebase, future: events, etc. Important for us. CHOOSING A LICENSE. Reason for Apache 2 vs LGPL/GPL – better for business because they don’t have to make modifications open source. SALES CYCLE – pros: more leads, and cons: longer w light touch. Previously contract / trial with decision makers before getting tech in hands of developers. TRACKING – companies can be much further along the funnel because they don’t have to sign up to get started. Example call from major tech company. VALUE CHAIN – where we sit now we have disrupted ourselves. TIMING (OS more strategic and sometimes SaaS is a better option). SAAS – open source is a great driver of SaaS customers. Many companies want to start with SaaS and have a longer term plan to move on prem and work with custom algos. - As SaaS: classically delivering endpoint - In comparison as open source: enable looking into configuration --> make product & onboarding more streamlined - Documentation important for activating developers --> transfer docs/pdf to Github - Interestingly: first pull requests about fixing documentation - Create demo apps to show possibilities - Huge thing: community --> changed way of communication - Inbound instead of outbound - License issues --> solved with Apache 2 license - Sales cycle got longer - Didn’t lose any customers through going open - Found that open source is a good distribution channel - Find right business/revenue model for open source - Deployment --> cloud vs. on premise - Build ecosystem around proactive community & potentially work together with competitors - Architecture enables microservices & API --> interfaces with other ML services - Integration with other open source libraries & closed APIs - Change in cost structure - Conflict in support (free vs. SaaS)? - Measuring open source engagement?
- Two dimensions: horizontal vs vertical (market focus) and product (scale) vs service (consulting) After open-source Seldon in the bottom right, providing services on top of a horizontal platform. ”Integration Services” or “Customer Funded Development” Companies generally seek a position in the top right, unicorn territory. But it doesn’t make sense for us to jump there directly. So we’re first making it much easier to deploy Seldon’s product through SaaS and optimising setup on our horizontal platform and carefully picking some areas to focus on. Our aim is to spawn many unicorns. https://thenounproject.com/term/mule/28242/
- Recommendations using EXTERNAL REST API Predictions takes JSON of /events data and provides regression and classification outputs. Microservices /predictions with Vowpal Wabbit Example /recommendations microservices – including Collaborative Deep Learning from the KDD talk.
- MICROSERVICES – make the Seldon stack completely pluggable with third party code / algos developed by your data scientists IN ANY LANGUAGE (R, Python, etc) that can be put into production and utilise the same pipeline as core Seldon algorithms. OPEN SOURCE LIBRARIES – we already used microservices to connect an OS library called Vowpal Wabbit (Microsoft research) to power our new predict endpoint. This enables regression, binary and multi-class classification. We are connected to the leading machine intelligence libraries such as Torch. CLOSED APIs – third parties who provide We can leverage third party APIs such as IBM Watson for personality insights or text to speech. ENTERPRISE DISTRIBUTIONS – Seldon are already planning integrations with some of the leading enterprise distributions.
- 4. SELDON CONTAINER INFRASTRUCTURE MAIN COMPONENTS – REST API SERVERS, ZOOKEEPER, FLUENTD, KAFKA, SPARK, MEMCACHE, JDBC database… VIRTUAL MACHINE - Setting up infrastructure is complex. Portable - Developer can download it and straight away access the infrastructure. Movie demo. DOCKER / AMIs - technology that allows you to use on different platforms (docker container = shipping container. Part of our infrastructure in each container). Deploy on premise, cloud (AWS, Google) or SaaS. [encourage people to register for AMIs]
- WORK IN PROGRESS Stream based - Apache Kafka as hub data pushed to DBs and processing units as needed Flexible design to allow different techniques to attack the problem Stream algorithms via Flink (or Spark) dependent on latency requirements Batch algorithms via Spark (or Flink) Low latency front end scoring systems evolution of Seldon server with likely input from trading systems expertise Zookeeper for state and control, Luigi pipeline Docker Swarm for deployment Python single machine for agile algorithm development scikit-learn, pandas, pyseldon etc.
- Data scientists and SALES Come and work at Barclays Accelerator
- BEFORE TAKING SOME QUESTIONS, I’LL FINISH WITH A QUESTION. Can we change the way in which people view their competition? London is become a centre of excellence for AI. And companies that collaborate as part of an ecosystems have a competitive advantage. Since Seldon went OS had an open door policy about speaking to “competitors”. Download Seldon’s open source, VM or AMI... visit seldon.io or develors head on over to docs.seldon.io Contact me if you would like to help out or to discuss how we can help add machine intelligence into your business. Or other AI companies that want to partner. THANK YOU
- (quite often this is the elephant in the room – it’s often the first question we get asked) OS is revolutionising the way in which we do business. Since we released Seldon in Feb it is already starting to be used by world’s largest companies. FREE - First, don’t expect more than a small percentage of activated users to pay. ADVISORY. ML is complicated and more projects have unique requirements. Advisory services, enables us to spend more time with customers, understanding their problems so ultimately leads to building a better platform. MLAAS. Finally, we have a SaaS platform! Many companies find us via the OS but SaaS the best starting point. Companies like the MIGRATION PATH [can reference later in call for open models] ECOSYSTEM. Third party technology, APIs and models. Also UNIFY AND MONETISE the DATA. Value in creating an ecosystem. Seldon can offer distribution and monetization.
- There are INFINITE algorithm configurations to choose from. So which ones are best for my business? Example of when different models used: high/low churn, days of week. Userbase or product mix changes. Seasonal changes. YOU CAN USE YOUR OWN ALGORITHMS AND MODELS. The model selection will vary depending on the user type. OLD WAY - AB testing. each test used to be a manual process measure the impact of recommendations on KPIs such as CTR, conversion rate, etc.. NEW WAY – CONTINUOUS TESTING of all models that diverts more traffic towards the model performing best at that specific point in time. Called a MULTI-ARMED BANDIT, inspired by a strategy to play a room full of slot machines. Give a user case of which algorithms would be used high churn news environment. You want to make the case that it’s not 1 algorithm/model but a combination of different models that will maximise your KPI. And that’s why you need Seldon. We A/B test from set of INDUSTRY MODELS to find a combination of model selections that work best for your business for a given user at a given time. TIME SAVED in choosing the best model. Seldon increases the productivity of your data science teams and helps your business to increase profits through rapid prototyping and better KPI performance.