Machine Learning by Sanjay Upadhyay
•
3 gefällt mir•1,137 views
Agile Testing Alliance hosted it's 16th Meetup in Pune on 9th Dec, 2017. Sanjay Upadhyay from Amdocs gave a session on Machine Learning as a part of this meetup. All the rights belongs to the author.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Ähnlich wie Machine Learning by Sanjay Upadhyay
Ähnlich wie Machine Learning by Sanjay Upadhyay (20)
Mehr von Agile Testing Alliance
Mehr von Agile Testing Alliance (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Machine Learning by Sanjay Upadhyay
- 1. Machine Learning Sanjay Upadhyay -SW Test Manager Amdocs 09/12/2017
- 2. Machine Learning Sanjay Upadhyay -SW Test Manager Amdocs 09/12/2017 Presented as a part of ATA Pune 16th Meetup on 9th Dec, 2017
- 3. Information Security Level 2 – Sensitive © 2017 – Proprietary & Confidential Information of Amdocs3 What is learning ??? ”Learning denotes changes in a system that ... enable a system to do the same task more efficiently the next time.” – Herbert Simon “Learning is constructing or modifying representations of what is being experienced.” – Ryszard Michalski “ Learning is making useful changes in our minds.” – Marvin Minsky
- 4. Information Security Level 2 – Sensitive © 2017 – Proprietary & Confidential Information of Amdocs4 • I don’t know, may be learning from machines? • Making machines learn something a.k.a. programming machine software • Learning with help of computers • Learning through online courses (!!!) What is Machine Learning ?
- 5. Information Security Level 2 – Sensitive © 2017 – Proprietary & Confidential Information of Amdocs5
- 6. Information Security Level 2 – Sensitive © 2017 – Proprietary & Confidential Information of Amdocs6 The Learning
- 7. Information Security Level 2 – Sensitive © 2017 – Proprietary & Confidential Information of Amdocs7 • How did our brain process the images? • How did the grouping happen? • Human brain processed the given images - learning • After learning the brain simply looked at the new image and compared with the groups classified the image to the closest group - Classification • If a machine has to perform the same operation we use Machine Learning • We write programs for learning and then classification, this is nothing but machine learning Machine Learning
- 8. Information Security Level 2 – Sensitive © 2017 – Proprietary & Confidential Information of Amdocs8 How exactly do we teach machines? Data as Input Text Files, Spread sheet SQL Data bases Abstracting the data Representation of the data in structural format through the algorithm chosen Generalization The practical application happens here where the learning from the previous step is used to develop an insight
- 9. Information Security Level 2 – Sensitive © 2017 – Proprietary & Confidential Information of Amdocs9 What are the steps used in Machine Learning? There are 5 basic steps used to perform a machine learning task: Collecting data Preparing the data Training a model Evaluating the model Improving the performance
- 10. Information Security Level 2 – Sensitive © 2017 – Proprietary & Confidential Information of Amdocs10
- 11. Information Security Level 2 – Sensitive © 2017 – Proprietary & Confidential Information of Amdocs11 The main advantage of ML • Learning and writing an algorithm • Its easy for human brain but it is tough for a machine. It takes some time and good amount of training data for machine to accurately classify objects • Implementation and automation • This is easy for a machine. Once learnt a machine can process one million images without any fatigue where as human brain can’t • That’s why ML with bigdata is a deadly combination
- 12. Information Security Level 2 – Sensitive © 2017 – Proprietary & Confidential Information of Amdocs12 Applications of Machine Learning Banking / Telecom / Retail Identify: • Prospective customers • Dissatisfied customers • Good customers • Bad payers Obtain: • More effective advertising • Less credit risk • Fewer fraud • Decreased churn rate
- 13. Information Security Level 2 – Sensitive © 2017 – Proprietary & Confidential Information of Amdocs13 Main parts in Machine Learning process Any Machine Learning algorithm has three parts 1. The Output 2. The Objective Function or Performance Matrix 3. The Input Example: Email Spam Classification 1. The Output: Categorize email messages as spam or legitimate. 2. Objective Function: Percentage of email messages correctly classified. 3. The Input: Database of emails, some with human-given
- 14. Information Security Level 2 – Sensitive © 2017 – Proprietary & Confidential Information of Amdocs14 ML in Software Development & Testing • SOFTWARE DESIGN • REQUIREMENT TRACING • CODE GENERATION • SOFTWARE ESTIMATION • SOFTWARE TESTING • GUI TESTING • CI Automation
- 15. Information Security Level 2 – Sensitive © 2017 – Proprietary & Confidential Information of Amdocs15 Journal Citation • WWW.Slideshare.net • https://www.analyticsvidhya.com/blog/2015/06/machine-learning-basics/# • https://www.toptal.com/machine-learning/machine-learning-theory-an-introductory-primer • https://www.google.co.in Machine Learning Courses & Tutorials Coursera - Machine Learning - Stanford University | Coursera + YouTube Series Lecture Collection | Machine Learning - YouTube Machine Learning | Coursera Practical Machine Learning - Johns Hopkins University | Coursera Neural Networks for Machine Learning - University of Toronto | Coursera Udacity - Intro to Machine Learning Course | Udacity DataCamp - Python Machine Learning: Scikit-Learn Tutorial, Introduction to Machine Learning - Online Course EdX - Learning From Data (Introductory Machine Learning) OCW MIT - Machine Learning, Introduction to Neural Networks Hackr.io: Best Machine Learning Tutorials Recommended by the Programming Community
- 16. Thank you
Hinweis der Redaktion
- Just to make sure that we are on same page. has anyone heard about machine learning before? Question to Audience :- What do you think happens when you search for something on Google? how many searches and what all kind of searches would Google handle regularly? how do you think Google can serve so many requests with such accuracy? Do you think there are people sitting in Google offices and continuously deciding which search result is relevant and which is not? this is where machine learning comes into play. Machine learning is a set of techniques, which help in dealing with vast data in the most intelligent fashion (by developing algorithms or set of logical rules) to derive actionable insights (delivering search for users in this case).
- Collecting data: Be it the raw data from excel, access, text files etc., this step (gathering past data) forms the foundation of the future learning. The better the variety, density and volume of relevant data, better the learning prospects for the machine becomes. Preparing the data: Any analytical process thrives on the quality of the data used. One needs to spend time determining the quality of data and then taking steps for fixing issues such as missing data and treatment of outliers. Exploratory analysis is perhaps one method to study the nuances of the data in details thereby burgeoning the nutritional content of the data. Training a model: This step involves choosing the appropriate algorithm and representation of data in the form of the model. The cleaned data is split into two parts – train and test (proportion depending on the prerequisites); the first part (training data) is used for developing the model. The second part (test data), is used as a reference. Evaluating the model: To test the accuracy, the second part of the data (holdout / test data) is used. This step determines the precision in the choice of the algorithm based on the outcome. A better test to check accuracy of model is to see its performance on data which was not used at all during model build. Improving the performance: This step might involve choosing a different model altogether or introducing more variables to augment the efficiency. That’s why significant amount of time needs to be spent in data collection and preparation.
- Supervised Learning / Predictive models: Predictive model as the name suggests is used to predict the future outcome based on the historical data. Predictive models are normally given clear instructions right from the beginning as in what needs to be learnt and how it needs to be learnt. These class of learning algorithms are termed as Supervised Learning. For example: Supervised Learning is used when a marketing company is trying to find out which customers are likely to churn. We can also use it to predict the likelihood of occurrence of perils like earthquakes, tornadoes etc. with an aim to determine the Total Insurance Value. Some examples of algorithms used are: Nearest neighbour, Naïve Bayes, Decision Trees, Regression etc. Unsupervised learning / Descriptive models: It is used to train descriptive models where no target is set and no single feature is important than the other. The case of unsupervised learning can be: When a retailer wishes to find out what are the combination of products, customers tends to buy more frequently. Furthermore, in pharmaceutical industry, unsupervised learning may be used to predict which diseases are likely to occur along with diabetes. Example of algorithm used here is: K- means Clustering Algorithm Reinforcement learning (RL): It is an example of machine learning where the machine is trained to take specific decisions based on the business requirement with the sole motto to maximize efficiency (performance). The idea involved in reinforcement learning is: The machine/ software agent trains itself on a continual basis based on the environment it is exposed to, and applies it’s enriched knowledge to solve business problems. This continual learning process ensures less involvement of human expertise which in turn saves a lot of time!
- SOFTWARE DESIGN :- Both architecture design and detailed design require designers to apply their technical knowledge and experience to evaluate alternative solutions before making commitments to a definite solution. Normally, a designer starts with a guess of the solution, and then goes back and forth exploring candidate design transformations until arriving to the desired solution. Making the right design decisions for each phase is a complex, time-consuming and error-prone activity for designers. Although tools for specification and analysis of designs exist, these tools do not support the designer in making informed decisions based on quality-attribute considerations. Along this line, several AI developments have shown the benefits of improving conventional tools with intelligent agents. REQUIREMENT TRACING Requirements traceability is an important undertaking as part of ensuring the quality of software in the early stages of the Software Development Life Cycle. Swarm intelligence is applied to the requirements tracing problem using pheromone communication and a focus on the common text around linking terms or words in order to find related textual documents. CODE Generation Because of the evolutionary nature of software products, by the time coding is completed, requirements would have changed (because of the long processes and stages of development required in software engineering): a situation that results in delay between requirement specification and product delivery. There is therefore a need for design by experimentation, the feasibility of which lies in automated programming. SOFTWARE Estimation:- Instead of Guessing or using certain Golden rule formula Machine learning can provide better and accurate estimates. SOFTWARE Testing & CI Automation :-A growing interest can be seen in use of AI for GUI testing. There has been some research into how GUI testing could be dealt with the help of AI. The various forms of this technique have been found in a quick glance to ACM library search on the topic. Some of these techniques include generating the GUI based on a model, generating tests based on a model, and automating test case generation to make it possible to regenerate the tests each time GUI changes and making automated oracles, which model the behavior of the user interface. There have also been peeks into generating tests based on artificial intelligence (AI) planning techniques and genetic modeling