Die SlideShare-Präsentation wird heruntergeladen. ×

Accelerating Random Forests in Scikit-Learn

•

29. Aug 2014

• •

Anzeige

Anzeige

Anzeige

Anzeige

Anzeige

Anzeige

Anzeige

Anzeige

Anzeige

Anzeige

Anzeige

Anzeige

Accelerating Random Forests in Scikit-Learn
Gilles Louppe
Universite de Liege, Belgium
August 29, 2014
1 / 26

Motivation
... and many more applications !
2 / 26

About
Scikit-Learn
Machine learning library for Python
Classical and well-established
algorithms
Emphasis on code ...

Outline
1 Basics
2 Scikit-Learn implementation
3 Python improvements
4 / 26

Machine Learning 101
Data comes as...
A set of samples L = f(xi ; yi )ji = 0; : : : ;N

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Accelerating Random Forests in Scikit-Learn

Nächste Präsentationen

Nächste SlideShare

VAE-type Deep Generative Models

VAE-type Deep Generative Models

Wird geladen in …3

×

Hier ansehen

Introducton to Convolutional Nerural Network with TensorFlow

Machine Learning - Introduction to Tensorflow

Andrew Ferlitsch

Effective Numerical Computation in NumPy and SciPy

Gradient Boosted Regression Trees in scikit-learn

TensorFlow example for AI Ukraine2016

ICML2013読み会 Large-Scale Learning with Less RAM via Randomization

Gaussian processing

Accelerating Random Forests in Scikit-Learn

29. Aug 2014

• •

Herunterladen, um offline zu lesen

Daten & Analysen

Random Forests are without contest one of the most robust, accurate and versatile tools for solving machine learning tasks. Implementing this algorithm properly and efficiently remains however a challenging task involving issues that are easily overlooked if not considered with care. In this talk, we present the Random Forests implementation developed within the Scikit-Learn machine learning library. In particular, we describe the iterative team efforts that led us to gradually improve our codebase and eventually make Scikit-Learn's Random Forests one of the most efficient implementations in the scientific ecosystem, across all libraries and programming languages. Algorithmic and technical optimizations that have made this possible include:

- An efficient formulation of the decision tree algorithm, tailored for Random Forests;
- Cythonization of the tree induction algorithm;
- CPU cache optimizations, through low-level organization of data into contiguous memory blocks;
- Efficient multi-threading through GIL-free routines;
- A dedicated sorting procedure, taking into account the properties of data;
- Shared pre-computations whenever critical.

Overall, we believe that lessons learned from this case study extend to a broad range of scientific applications and may be of interest to anybody doing data analysis in Python.

Postdoctoral Research Associate at NYU / CERN

Random Forests are without contest one of the most robust, accurate and versatile tools for solving machine learning tasks. Implementing this algorithm properly and efficiently remains however a challenging task involving issues that are easily overlooked if not considered with care. In this talk, we present the Random Forests implementation developed within the Scikit-Learn machine learning library. In particular, we describe the iterative team efforts that led us to gradually improve our codebase and eventually make Scikit-Learn's Random Forests one of the most efficient implementations in the scientific ecosystem, across all libraries and programming languages. Algorithmic and technical optimizations that have made this possible include:

- An efficient formulation of the decision tree algorithm, tailored for Random Forests;
- Cythonization of the tree induction algorithm;
- CPU cache optimizations, through low-level organization of data into contiguous memory blocks;
- Efficient multi-threading through GIL-free routines;
- A dedicated sorting procedure, taking into account the properties of data;
- Shared pre-computations whenever critical.

Overall, we believe that lessons learned from this case study extend to a broad range of scientific applications and may be of interest to anybody doing data analysis in Python.

Daten & Analysen

Weitere Verwandte Inhalte

Introducton to Convolutional Nerural Network with TensorFlow

Machine Learning - Introduction to Tensorflow

Andrew Ferlitsch

Effective Numerical Computation in NumPy and SciPy

Gradient Boosted Regression Trees in scikit-learn

TensorFlow example for AI Ukraine2016

ICML2013読み会 Large-Scale Learning with Less RAM via Randomization

Gaussian processing

Tensorflow - Intro (2017)

Alessio Tonioni

Support Vector Machines (SVM)

Rabbit challenge 3 DNN Day2

Iclr2016 vaeまとめ

Deep Learning JP

Data Analytics and Simulation in Parallel with MATLAB*

Intel® Software

Optimization (DLAI D4L1 2017 UPC Deep Learning for Artificial Intelligence)

Universitat Politècnica de Catalunya

Introduction of Xgboost

Deep Learning, Keras, and TensorFlow

Oswald Campesato

Introduction to Tensorflow

Automatic Gain Tuning based on Gaussian Process Global Optimization (= Bayesi...

L1 intro2 supervised_learning

Variational Autoencoder

Introducton to Convolutional Nerural Network with TensorFlow

Machine Learning - Introduction to Tensorflow

Andrew Ferlitsch

Effective Numerical Computation in NumPy and SciPy

Gradient Boosted Regression Trees in scikit-learn

TensorFlow example for AI Ukraine2016

K-means Clustering with Scikit-Learn

Numerical tour in the Python eco-system: Python, NumPy, scikit-learn

Exploring Machine Learning in Python with Scikit-Learn

Kan Ouivirach, Ph.D.

Intro to scikit-learn

Clustering: A Scikit Learn Tutorial

Damian R. Mingle, MBA

Authorship Attribution and Forensic Linguistics with Python/Scikit-Learn/Pand...

Realtime predictive analytics using RabbitMQ & scikit-learn

Machine learning with scikit-learn

Machine Learning with scikit-learn

Scikit-learn: the state of the union 2016

Machine learning in production with scikit-learn

Think machine-learning-with-scikit-learn-chetan

Data Science and Machine Learning Using Python and Scikit-learn

Intro to scikit learn may 2017

Francesco Mosconi

Pyparis2017 / Scikit-learn - an incomplete yearly review, by Gael Varoquaux

Pôle Systematic Paris-Region

Intro to machine learning with scikit learn

Introduction to Machine Learning with Python and scikit-learn

Text Classification/Categorization

Scikit-learn for easy machine learning: the vision, the tool, and the project

Converting Scikit-Learn to PMML

K-means Clustering with Scikit-Learn

Numerical tour in the Python eco-system: Python, NumPy, scikit-learn

Exploring Machine Learning in Python with Scikit-Learn

Kan Ouivirach, Ph.D.

Intro to scikit-learn

Clustering: A Scikit Learn Tutorial

Damian R. Mingle, MBA

Authorship Attribution and Forensic Linguistics with Python/Scikit-Learn/Pand...

Everything You Always Wanted to Know About Memory in Python - But Were Afraid...

Tokyo Webmining Talk1

May2010 hex-core-opt

Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...

Universitat Politècnica de Catalunya

Common Design of Deep Learning Frameworks

MLOps Case Studies: Building fast, scalable, and high-accuracy ML systems at ...

Masashi Shibata

Ehsan parallel accelerator-dec2015

Deep Learning meetup

Ivan Goloskokovic

Deep Learning, Scala, and Spark

Oswald Campesato

Google Big Data Expo

Stack Hybridization: A Mechanism for Bridging Two Compilation Strategies in a...

Pytorch for tf_developers

Cray XT Porting, Scaling, and Optimization Best Practices

Afaf Soumia Medjden

Pythran: Static compiler for high performance by Mehdi Amini PyData SV 2014

Designing C++ portable SIMD support

Overview of Chainer and Its Features

Software Frameworks for Deep Learning (D1L7 2017 UPC Deep Learning for Comput...

Universitat Politècnica de Catalunya

Everything You Always Wanted to Know About Memory in Python But Were Afraid t...

09 accelerators

Everything You Always Wanted to Know About Memory in Python - But Were Afraid...

Tokyo Webmining Talk1

May2010 hex-core-opt

Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...

Universitat Politècnica de Catalunya

Common Design of Deep Learning Frameworks

MLOps Case Studies: Building fast, scalable, and high-accuracy ML systems at ...

Masashi Shibata

SCU Moral Relativism Theory Argumentative Essay.pdf

Apache Superset - open source data exploration and visualization (Conclusion ...

Research an advanced nursing practice role and summarize.docx

Types of Assessments in Outpatient and Inpatient Settings.pdf

Research Article Critique List of Articles Discussion.docx

Sensitive to when interacting with a patient.docx

Steps for interview for your ideal future Nursing.docx

2 PUNJAB IN 1965 WAR

Sociology of Healthcare.docx

Spans several states with in the United States.docx

Social determinants of health essay.docx

Something unique to healthcare accounting.docx

Relationships with advanced practice nurses.docx

Someone who works in a care HW.docx

Substance Abuse Treatment Programs.docx

Dance Criteria.docx

RitchelEscobia1

Lecture_9_Poisoning_Attacks_and_Defenses.pptx

Research and writing mental health Counseling.docx

Social media encourages us to be more honest.docx

Street drugs.docx

SCU Moral Relativism Theory Argumentative Essay.pdf

Apache Superset - open source data exploration and visualization (Conclusion ...

Research an advanced nursing practice role and summarize.docx

Types of Assessments in Outpatient and Inpatient Settings.pdf

Research Article Critique List of Articles Discussion.docx

Sensitive to when interacting with a patient.docx

Accelerating Random Forests in Scikit-Learn

1. Accelerating Random Forests in Scikit-Learn Gilles Louppe Universite de Liege, Belgium August 29, 2014 1 / 26
2. Motivation ... and many more applications ! 2 / 26
3. About Scikit-Learn Machine learning library for Python Classical and well-established algorithms Emphasis on code quality and usability Myself @glouppe PhD student (Liege, Belgium) Core developer on Scikit-Learn since 2011 Chief tree hugger scikit 3 / 26
4. Outline 1 Basics 2 Scikit-Learn implementation 3 Python improvements 4 / 26
5. Machine Learning 101 Data comes as... A set of samples L = f(xi ; yi )ji = 0; : : : ;N