SlideShare ist ein Scribd-Unternehmen logo
1 von 46
Downloaden Sie, um offline zu lesen
Kernel, RKHS, and
Gaussian Processes
Caution! No proof will be given.
Sungjoon Choi, SNU
Leveraged
Gaussian	
  Process	
  Regression
Leveraged	
  Gaussian	
  Processes
The	
  original	
  Gaussian	
  process	
  regression	
  
anchors	
  positive training	
  data.
The	
  proposed	
  leveraged	
  Gaussian	
  process	
  regression	
  anchors	
  positive data	
  
while	
  avoiding	
  negative data.
Moreover,	
  it	
  is	
  possible	
  to	
  vary	
  the	
  leverage	
  
of	
  each	
  training	
  data	
  from	
  -­‐1	
  to	
  +1.
Positive and	
  Negative Motion	
  
Control	
  Data	
  for	
  Autonomous	
  
Navigation
Training	
  Phase
Execution	
  Phase
Real	
  World	
  Experiments
Leverage	
  Optimization
Leverage	
  Optimization
• The	
  key	
  intuition	
  behind	
  the	
  leverage	
  optimization	
  is	
  that	
  we	
  
cast	
  the	
  leverage	
  optimization	
  problem	
  into	
  a	
  model	
  selection	
  
problem in	
  Gaussian	
  process	
  regression.	
  
• However,	
  the	
  number	
  of	
  leverage	
  parameters	
  is	
  equivalent	
  to	
  
the	
  number	
  of	
  training	
  data.	
  
• To	
  handle	
  this	
  issue,	
  we	
  propose	
  a	
  sparse	
  constrained	
  leverage	
  
optimization where	
  we	
  assume	
  that	
  the	
  majority	
  of	
  leverage	
  
parameters	
  are	
  +1.	
  
Leverage	
  Optimization	
  (v1)
Using	
  proximal	
  linearized	
  minimization	
  [1],	
  the	
  update	
  rule	
  for	
  solving	
  above	
  
optimization	
  is	
  
[1]  J.  Bolte,  S.  Sabach,  and  M.  Teboulle,  “Proximal  alternating  linearized  minimization  for  nonconvex  and  nonsmooth  problems,”  
Mathematical  Programming,  vol.  146,  no.  1-­2,  pp.  459–494,  2014.
where	
  the	
  proximal	
  mapping	
  becomes	
  soft-­‐thresholding:
where	
   𝛾̅ = 𝛾 − 1.	
  
Leverage	
  Optimization	
  (v2)
We	
  propose	
  a	
  new	
  leverage	
  optimization	
  method	
  by	
  doubling the	
  leverage	
  
parameters	
  to	
  positive	
  and	
  negative	
  parts.	
  
Furthermore,	
  we	
  assume	
  multiple	
  demonstrations	
  are	
  collected	
  from	
  one	
  
demonstrator.	
  
By	
  doing	
  so,	
  we	
  have	
  two	
  major	
  benefits:
1. As	
  the	
  L1-­‐norm	
  regularizer is	
  only	
  on	
  the	
  positive	
  parts	
  of	
  the	
  leverages,	
  
negative	
  parts	
  of	
  the	
  leverages	
  can	
  be	
  optimized	
  more	
  accurately.	
  
2. By	
  doubling	
  the	
  variables,	
  proximal	
  mapping	
  is	
  no	
  longer	
  needed.	
  
Sensory	
  Field	
  Reconstruction
Sensory	
  observations	
  from	
  
correlated	
  sensory	
  fields
Leverage	
  Optimization	
  
Collect	
  observations
Sensory	
  observations	
  from	
  
correlated	
  sensory	
  fields
Proposed	
  Method Previous	
  Method Without	
  Optimization
Sensory	
  Field	
  Reconstruction
Autonomous	
  Driving	
  Experiments
Driving	
  demonstrations	
  with	
  
mixed	
  qualities
Leverage	
  Optimization	
  
Collect	
  observations
Proposed	
  Method
Previous	
  Method
Autonomous	
  Driving	
  Experiments
Proposed	
  MethodWithout	
  Optimization
Autonomous	
  Driving	
  Experiments
Kernel, RKHS, and Gaussian Processes

Weitere ähnliche Inhalte

Was ist angesagt?

[PR12] understanding deep learning requires rethinking generalization
[PR12] understanding deep learning requires rethinking generalization[PR12] understanding deep learning requires rethinking generalization
[PR12] understanding deep learning requires rethinking generalizationJaeJun Yoo
 
Human uncertainty makes classification more robust, ICCV 2019 Review
Human uncertainty makes classification more robust, ICCV 2019 ReviewHuman uncertainty makes classification more robust, ICCV 2019 Review
Human uncertainty makes classification more robust, ICCV 2019 ReviewLEE HOSEONG
 
Leveraged Gaussian Process
Leveraged Gaussian ProcessLeveraged Gaussian Process
Leveraged Gaussian ProcessSungjoon Choi
 
Cost-effective Interactive Attention Learning with Neural Attention Process
Cost-effective Interactive Attention Learning with Neural Attention ProcessCost-effective Interactive Attention Learning with Neural Attention Process
Cost-effective Interactive Attention Learning with Neural Attention ProcessMLAI2
 
Deep Learning in Computer Vision
Deep Learning in Computer VisionDeep Learning in Computer Vision
Deep Learning in Computer VisionSungjoon Choi
 
Corinna Cortes, Head of Research, Google, at MLconf NYC 2017
Corinna Cortes, Head of Research, Google, at MLconf NYC 2017Corinna Cortes, Head of Research, Google, at MLconf NYC 2017
Corinna Cortes, Head of Research, Google, at MLconf NYC 2017MLconf
 
Pelee: a real time object detection system on mobile devices Paper Review
Pelee: a real time object detection system on mobile devices Paper ReviewPelee: a real time object detection system on mobile devices Paper Review
Pelee: a real time object detection system on mobile devices Paper ReviewLEE HOSEONG
 
Using Optimal Learning to Tune Deep Learning Pipelines
Using Optimal Learning to Tune Deep Learning PipelinesUsing Optimal Learning to Tune Deep Learning Pipelines
Using Optimal Learning to Tune Deep Learning PipelinesScott Clark
 
【DL輪読会】Spectral Normalisation for Deep Reinforcement Learning: An Optimisatio...
【DL輪読会】Spectral Normalisation for Deep Reinforcement Learning: An Optimisatio...【DL輪読会】Spectral Normalisation for Deep Reinforcement Learning: An Optimisatio...
【DL輪読会】Spectral Normalisation for Deep Reinforcement Learning: An Optimisatio...Deep Learning JP
 
2019 ICLR Best Paper Review
2019 ICLR Best Paper Review2019 ICLR Best Paper Review
2019 ICLR Best Paper ReviewLEE HOSEONG
 
Model-Based Reinforcement Learning @NIPS2017
Model-Based Reinforcement Learning @NIPS2017Model-Based Reinforcement Learning @NIPS2017
Model-Based Reinforcement Learning @NIPS2017mooopan
 
Super Resolution with OCR Optimization
Super Resolution with OCR OptimizationSuper Resolution with OCR Optimization
Super Resolution with OCR OptimizationniveditJain
 
Variants of GANs - Jaejun Yoo
Variants of GANs - Jaejun YooVariants of GANs - Jaejun Yoo
Variants of GANs - Jaejun YooJaeJun Yoo
 
Captcha-recognition-with-active-deep-learning
Captcha-recognition-with-active-deep-learningCaptcha-recognition-with-active-deep-learning
Captcha-recognition-with-active-deep-learningcrew1274
 
Learning to learn unlearned feature for segmentation
Learning to learn unlearned feature for segmentationLearning to learn unlearned feature for segmentation
Learning to learn unlearned feature for segmentationNAVER Engineering
 
Evolving Reinforcement Learning Algorithms, JD. Co-Reyes et al, 2021
Evolving Reinforcement Learning Algorithms, JD. Co-Reyes et al, 2021Evolving Reinforcement Learning Algorithms, JD. Co-Reyes et al, 2021
Evolving Reinforcement Learning Algorithms, JD. Co-Reyes et al, 2021Chris Ohk
 
Unsupervised visual representation learning overview: Toward Self-Supervision
Unsupervised visual representation learning overview: Toward Self-SupervisionUnsupervised visual representation learning overview: Toward Self-Supervision
Unsupervised visual representation learning overview: Toward Self-SupervisionLEE HOSEONG
 
Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016
Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016
Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016MLconf
 

Was ist angesagt? (20)

[PR12] understanding deep learning requires rethinking generalization
[PR12] understanding deep learning requires rethinking generalization[PR12] understanding deep learning requires rethinking generalization
[PR12] understanding deep learning requires rethinking generalization
 
Human uncertainty makes classification more robust, ICCV 2019 Review
Human uncertainty makes classification more robust, ICCV 2019 ReviewHuman uncertainty makes classification more robust, ICCV 2019 Review
Human uncertainty makes classification more robust, ICCV 2019 Review
 
Leveraged Gaussian Process
Leveraged Gaussian ProcessLeveraged Gaussian Process
Leveraged Gaussian Process
 
Cost-effective Interactive Attention Learning with Neural Attention Process
Cost-effective Interactive Attention Learning with Neural Attention ProcessCost-effective Interactive Attention Learning with Neural Attention Process
Cost-effective Interactive Attention Learning with Neural Attention Process
 
Deep Learning in Computer Vision
Deep Learning in Computer VisionDeep Learning in Computer Vision
Deep Learning in Computer Vision
 
Corinna Cortes, Head of Research, Google, at MLconf NYC 2017
Corinna Cortes, Head of Research, Google, at MLconf NYC 2017Corinna Cortes, Head of Research, Google, at MLconf NYC 2017
Corinna Cortes, Head of Research, Google, at MLconf NYC 2017
 
Pelee: a real time object detection system on mobile devices Paper Review
Pelee: a real time object detection system on mobile devices Paper ReviewPelee: a real time object detection system on mobile devices Paper Review
Pelee: a real time object detection system on mobile devices Paper Review
 
Using Optimal Learning to Tune Deep Learning Pipelines
Using Optimal Learning to Tune Deep Learning PipelinesUsing Optimal Learning to Tune Deep Learning Pipelines
Using Optimal Learning to Tune Deep Learning Pipelines
 
【DL輪読会】Spectral Normalisation for Deep Reinforcement Learning: An Optimisatio...
【DL輪読会】Spectral Normalisation for Deep Reinforcement Learning: An Optimisatio...【DL輪読会】Spectral Normalisation for Deep Reinforcement Learning: An Optimisatio...
【DL輪読会】Spectral Normalisation for Deep Reinforcement Learning: An Optimisatio...
 
2019 ICLR Best Paper Review
2019 ICLR Best Paper Review2019 ICLR Best Paper Review
2019 ICLR Best Paper Review
 
Model-Based Reinforcement Learning @NIPS2017
Model-Based Reinforcement Learning @NIPS2017Model-Based Reinforcement Learning @NIPS2017
Model-Based Reinforcement Learning @NIPS2017
 
Super Resolution with OCR Optimization
Super Resolution with OCR OptimizationSuper Resolution with OCR Optimization
Super Resolution with OCR Optimization
 
Variants of GANs - Jaejun Yoo
Variants of GANs - Jaejun YooVariants of GANs - Jaejun Yoo
Variants of GANs - Jaejun Yoo
 
Captcha
CaptchaCaptcha
Captcha
 
Captcha-recognition-with-active-deep-learning
Captcha-recognition-with-active-deep-learningCaptcha-recognition-with-active-deep-learning
Captcha-recognition-with-active-deep-learning
 
Linear Regression
Linear RegressionLinear Regression
Linear Regression
 
Learning to learn unlearned feature for segmentation
Learning to learn unlearned feature for segmentationLearning to learn unlearned feature for segmentation
Learning to learn unlearned feature for segmentation
 
Evolving Reinforcement Learning Algorithms, JD. Co-Reyes et al, 2021
Evolving Reinforcement Learning Algorithms, JD. Co-Reyes et al, 2021Evolving Reinforcement Learning Algorithms, JD. Co-Reyes et al, 2021
Evolving Reinforcement Learning Algorithms, JD. Co-Reyes et al, 2021
 
Unsupervised visual representation learning overview: Toward Self-Supervision
Unsupervised visual representation learning overview: Toward Self-SupervisionUnsupervised visual representation learning overview: Toward Self-Supervision
Unsupervised visual representation learning overview: Toward Self-Supervision
 
Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016
Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016
Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016
 

Andere mochten auch

Connection between Bellman equation and Markov Decision Processes
Connection between Bellman equation and Markov Decision ProcessesConnection between Bellman equation and Markov Decision Processes
Connection between Bellman equation and Markov Decision ProcessesSungjoon Choi
 
Inverse Reinforcement Learning Algorithms
Inverse Reinforcement Learning AlgorithmsInverse Reinforcement Learning Algorithms
Inverse Reinforcement Learning AlgorithmsSungjoon Choi
 
서버리스 IoT 백엔드 개발 및 구현 사례 : 윤석찬 (AWS 테크에반젤리스트)
서버리스 IoT 백엔드 개발 및 구현 사례 : 윤석찬 (AWS 테크에반젤리스트)서버리스 IoT 백엔드 개발 및 구현 사례 : 윤석찬 (AWS 테크에반젤리스트)
서버리스 IoT 백엔드 개발 및 구현 사례 : 윤석찬 (AWS 테크에반젤리스트)Amazon Web Services Korea
 
Introduction For seq2seq(sequence to sequence) and RNN
Introduction For seq2seq(sequence to sequence) and RNNIntroduction For seq2seq(sequence to sequence) and RNN
Introduction For seq2seq(sequence to sequence) and RNNHye-min Ahn
 
Semantic Segmentation Methods using Deep Learning
Semantic Segmentation Methods using Deep LearningSemantic Segmentation Methods using Deep Learning
Semantic Segmentation Methods using Deep LearningSungjoon Choi
 
Object Detection Methods using Deep Learning
Object Detection Methods using Deep LearningObject Detection Methods using Deep Learning
Object Detection Methods using Deep LearningSungjoon Choi
 
해커에게 전해들은 머신러닝 #1
해커에게 전해들은 머신러닝 #1해커에게 전해들은 머신러닝 #1
해커에게 전해들은 머신러닝 #1Haesun Park
 
TensorFlow Tutorial Part2
TensorFlow Tutorial Part2TensorFlow Tutorial Part2
TensorFlow Tutorial Part2Sungjoon Choi
 
TensorFlow Tutorial Part1
TensorFlow Tutorial Part1TensorFlow Tutorial Part1
TensorFlow Tutorial Part1Sungjoon Choi
 
Visualizing data using t-SNE
Visualizing data using t-SNEVisualizing data using t-SNE
Visualizing data using t-SNE홍배 김
 
How to Become a Thought Leader in Your Niche
How to Become a Thought Leader in Your NicheHow to Become a Thought Leader in Your Niche
How to Become a Thought Leader in Your NicheLeslie Samuel
 
Yingrui Li: Complete Solutions for Now-Generation Bioinformatics
Yingrui Li: Complete Solutions for Now-Generation BioinformaticsYingrui Li: Complete Solutions for Now-Generation Bioinformatics
Yingrui Li: Complete Solutions for Now-Generation BioinformaticsGigaScience, BGI Hong Kong
 
Recent Trends in Neural Net Policy Learning
Recent Trends in Neural Net Policy LearningRecent Trends in Neural Net Policy Learning
Recent Trends in Neural Net Policy LearningSungjoon Choi
 
나는 요리를 오픈소스로 배웠다.
나는 요리를 오픈소스로 배웠다.나는 요리를 오픈소스로 배웠다.
나는 요리를 오픈소스로 배웠다.Rhio kim
 
Heat Tracing Solutions for LNG facilities in Arctic Conditions
Heat Tracing Solutions for LNG facilities in Arctic ConditionsHeat Tracing Solutions for LNG facilities in Arctic Conditions
Heat Tracing Solutions for LNG facilities in Arctic ConditionsDillen Valerie
 
빅데이터 분석을 위한 스파크 2 프로그래밍 : 대용량 데이터 처리부터 머신러닝까지
빅데이터 분석을 위한 스파크 2 프로그래밍 : 대용량 데이터 처리부터 머신러닝까지빅데이터 분석을 위한 스파크 2 프로그래밍 : 대용량 데이터 처리부터 머신러닝까지
빅데이터 분석을 위한 스파크 2 프로그래밍 : 대용량 데이터 처리부터 머신러닝까지위키북스
 

Andere mochten auch (20)

Connection between Bellman equation and Markov Decision Processes
Connection between Bellman equation and Markov Decision ProcessesConnection between Bellman equation and Markov Decision Processes
Connection between Bellman equation and Markov Decision Processes
 
Inverse Reinforcement Learning Algorithms
Inverse Reinforcement Learning AlgorithmsInverse Reinforcement Learning Algorithms
Inverse Reinforcement Learning Algorithms
 
서버리스 IoT 백엔드 개발 및 구현 사례 : 윤석찬 (AWS 테크에반젤리스트)
서버리스 IoT 백엔드 개발 및 구현 사례 : 윤석찬 (AWS 테크에반젤리스트)서버리스 IoT 백엔드 개발 및 구현 사례 : 윤석찬 (AWS 테크에반젤리스트)
서버리스 IoT 백엔드 개발 및 구현 사례 : 윤석찬 (AWS 테크에반젤리스트)
 
Introduction For seq2seq(sequence to sequence) and RNN
Introduction For seq2seq(sequence to sequence) and RNNIntroduction For seq2seq(sequence to sequence) and RNN
Introduction For seq2seq(sequence to sequence) and RNN
 
Semantic Segmentation Methods using Deep Learning
Semantic Segmentation Methods using Deep LearningSemantic Segmentation Methods using Deep Learning
Semantic Segmentation Methods using Deep Learning
 
Object Detection Methods using Deep Learning
Object Detection Methods using Deep LearningObject Detection Methods using Deep Learning
Object Detection Methods using Deep Learning
 
해커에게 전해들은 머신러닝 #1
해커에게 전해들은 머신러닝 #1해커에게 전해들은 머신러닝 #1
해커에게 전해들은 머신러닝 #1
 
TensorFlow Tutorial Part2
TensorFlow Tutorial Part2TensorFlow Tutorial Part2
TensorFlow Tutorial Part2
 
TensorFlow Tutorial Part1
TensorFlow Tutorial Part1TensorFlow Tutorial Part1
TensorFlow Tutorial Part1
 
CNN Tutorial
CNN TutorialCNN Tutorial
CNN Tutorial
 
Visualizing data using t-SNE
Visualizing data using t-SNEVisualizing data using t-SNE
Visualizing data using t-SNE
 
Sesiones inicial
Sesiones inicialSesiones inicial
Sesiones inicial
 
How to Become a Thought Leader in Your Niche
How to Become a Thought Leader in Your NicheHow to Become a Thought Leader in Your Niche
How to Become a Thought Leader in Your Niche
 
Yingrui Li: Complete Solutions for Now-Generation Bioinformatics
Yingrui Li: Complete Solutions for Now-Generation BioinformaticsYingrui Li: Complete Solutions for Now-Generation Bioinformatics
Yingrui Li: Complete Solutions for Now-Generation Bioinformatics
 
Robot Era
Robot EraRobot Era
Robot Era
 
Recent Trends in Neural Net Policy Learning
Recent Trends in Neural Net Policy LearningRecent Trends in Neural Net Policy Learning
Recent Trends in Neural Net Policy Learning
 
10 książek dla Trenera Biznesu
10 książek dla Trenera Biznesu10 książek dla Trenera Biznesu
10 książek dla Trenera Biznesu
 
나는 요리를 오픈소스로 배웠다.
나는 요리를 오픈소스로 배웠다.나는 요리를 오픈소스로 배웠다.
나는 요리를 오픈소스로 배웠다.
 
Heat Tracing Solutions for LNG facilities in Arctic Conditions
Heat Tracing Solutions for LNG facilities in Arctic ConditionsHeat Tracing Solutions for LNG facilities in Arctic Conditions
Heat Tracing Solutions for LNG facilities in Arctic Conditions
 
빅데이터 분석을 위한 스파크 2 프로그래밍 : 대용량 데이터 처리부터 머신러닝까지
빅데이터 분석을 위한 스파크 2 프로그래밍 : 대용량 데이터 처리부터 머신러닝까지빅데이터 분석을 위한 스파크 2 프로그래밍 : 대용량 데이터 처리부터 머신러닝까지
빅데이터 분석을 위한 스파크 2 프로그래밍 : 대용량 데이터 처리부터 머신러닝까지
 

Ähnlich wie Kernel, RKHS, and Gaussian Processes

Software Effort Estimation Using Particle Swarm Optimization with Inertia Weight
Software Effort Estimation Using Particle Swarm Optimization with Inertia WeightSoftware Effort Estimation Using Particle Swarm Optimization with Inertia Weight
Software Effort Estimation Using Particle Swarm Optimization with Inertia WeightWaqas Tariq
 
A parsimonious SVM model selection criterion for classification of real-world ...
A parsimonious SVM model selection criterion for classification of real-world ...A parsimonious SVM model selection criterion for classification of real-world ...
A parsimonious SVM model selection criterion for classification of real-world ...o_almasi
 
Nonlinear Exponential Regularization : An Improved Version of Regularization ...
Nonlinear Exponential Regularization : An Improved Version of Regularization ...Nonlinear Exponential Regularization : An Improved Version of Regularization ...
Nonlinear Exponential Regularization : An Improved Version of Regularization ...Seoung-Ho Choi
 
Chemometric analysis in IR spectroscopy/ infrared spectroscopy
Chemometric analysis in IR spectroscopy/ infrared spectroscopyChemometric analysis in IR spectroscopy/ infrared spectroscopy
Chemometric analysis in IR spectroscopy/ infrared spectroscopyKailashpati Tripathi
 
Bi-activation Function : an Enhanced Version of an Activation Function in C...
Bi-activation Function : an Enhanced Version of an Activation Function in C...Bi-activation Function : an Enhanced Version of an Activation Function in C...
Bi-activation Function : an Enhanced Version of an Activation Function in C...Seoung-Ho Choi
 
Lecture 5: Neural Networks II
Lecture 5: Neural Networks IILecture 5: Neural Networks II
Lecture 5: Neural Networks IISang Jun Lee
 
PR-411: Model soups: averaging weights of multiple fine-tuned models improves...
PR-411: Model soups: averaging weights of multiple fine-tuned models improves...PR-411: Model soups: averaging weights of multiple fine-tuned models improves...
PR-411: Model soups: averaging weights of multiple fine-tuned models improves...Sunghoon Joo
 
On the Performance of the Pareto Set Pursuing (PSP) Method for Mixed-Variable...
On the Performance of the Pareto Set Pursuing (PSP) Method for Mixed-Variable...On the Performance of the Pareto Set Pursuing (PSP) Method for Mixed-Variable...
On the Performance of the Pareto Set Pursuing (PSP) Method for Mixed-Variable...Amir Ziai
 
consistency regularization for generative adversarial networks_review
consistency regularization for generative adversarial networks_reviewconsistency regularization for generative adversarial networks_review
consistency regularization for generative adversarial networks_reviewYoonho Na
 
2020 11 4_bag_of_tricks
2020 11 4_bag_of_tricks2020 11 4_bag_of_tricks
2020 11 4_bag_of_tricksJAEMINJEONG5
 
Accelerated Particle Swarm Optimization and Support Vector Machine for Busine...
Accelerated Particle Swarm Optimization and Support Vector Machine for Busine...Accelerated Particle Swarm Optimization and Support Vector Machine for Busine...
Accelerated Particle Swarm Optimization and Support Vector Machine for Busine...Xin-She Yang
 
A COMPARATIVE STUDY OF BACKPROPAGATION ALGORITHMS IN FINANCIAL PREDICTION
A COMPARATIVE STUDY OF BACKPROPAGATION ALGORITHMS IN FINANCIAL PREDICTIONA COMPARATIVE STUDY OF BACKPROPAGATION ALGORITHMS IN FINANCIAL PREDICTION
A COMPARATIVE STUDY OF BACKPROPAGATION ALGORITHMS IN FINANCIAL PREDICTIONIJCSEA Journal
 
PR-330: How To Train Your ViT? Data, Augmentation, and Regularization in Visi...
PR-330: How To Train Your ViT? Data, Augmentation, and Regularization in Visi...PR-330: How To Train Your ViT? Data, Augmentation, and Regularization in Visi...
PR-330: How To Train Your ViT? Data, Augmentation, and Regularization in Visi...Jinwon Lee
 
Quasi newton artificial neural network training algorithms
Quasi newton artificial neural network training algorithmsQuasi newton artificial neural network training algorithms
Quasi newton artificial neural network training algorithmsMrinmoy Majumder
 
What Makes Training Multi-modal Classification Networks Hard? ppt
What Makes Training Multi-modal Classification Networks Hard? pptWhat Makes Training Multi-modal Classification Networks Hard? ppt
What Makes Training Multi-modal Classification Networks Hard? ppttaeseon ryu
 
Uber Data Analysis - SAS Project
Uber Data Analysis - SAS ProjectUber Data Analysis - SAS Project
Uber Data Analysis - SAS ProjectKushal417
 
Penalized Logistic Regression methods .pptx
Penalized Logistic Regression methods .pptxPenalized Logistic Regression methods .pptx
Penalized Logistic Regression methods .pptxhamdy53
 
Proximal Policy Optimization Algorithms, Schulman et al, 2017
Proximal Policy Optimization Algorithms, Schulman et al, 2017Proximal Policy Optimization Algorithms, Schulman et al, 2017
Proximal Policy Optimization Algorithms, Schulman et al, 2017Chris Ohk
 

Ähnlich wie Kernel, RKHS, and Gaussian Processes (20)

Software Effort Estimation Using Particle Swarm Optimization with Inertia Weight
Software Effort Estimation Using Particle Swarm Optimization with Inertia WeightSoftware Effort Estimation Using Particle Swarm Optimization with Inertia Weight
Software Effort Estimation Using Particle Swarm Optimization with Inertia Weight
 
A parsimonious SVM model selection criterion for classification of real-world ...
A parsimonious SVM model selection criterion for classification of real-world ...A parsimonious SVM model selection criterion for classification of real-world ...
A parsimonious SVM model selection criterion for classification of real-world ...
 
Owa method
Owa methodOwa method
Owa method
 
Nonlinear Exponential Regularization : An Improved Version of Regularization ...
Nonlinear Exponential Regularization : An Improved Version of Regularization ...Nonlinear Exponential Regularization : An Improved Version of Regularization ...
Nonlinear Exponential Regularization : An Improved Version of Regularization ...
 
Chemometric analysis in IR spectroscopy/ infrared spectroscopy
Chemometric analysis in IR spectroscopy/ infrared spectroscopyChemometric analysis in IR spectroscopy/ infrared spectroscopy
Chemometric analysis in IR spectroscopy/ infrared spectroscopy
 
Bi-activation Function : an Enhanced Version of an Activation Function in C...
Bi-activation Function : an Enhanced Version of an Activation Function in C...Bi-activation Function : an Enhanced Version of an Activation Function in C...
Bi-activation Function : an Enhanced Version of an Activation Function in C...
 
ngboost.pptx
ngboost.pptxngboost.pptx
ngboost.pptx
 
Lecture 5: Neural Networks II
Lecture 5: Neural Networks IILecture 5: Neural Networks II
Lecture 5: Neural Networks II
 
PR-411: Model soups: averaging weights of multiple fine-tuned models improves...
PR-411: Model soups: averaging weights of multiple fine-tuned models improves...PR-411: Model soups: averaging weights of multiple fine-tuned models improves...
PR-411: Model soups: averaging weights of multiple fine-tuned models improves...
 
On the Performance of the Pareto Set Pursuing (PSP) Method for Mixed-Variable...
On the Performance of the Pareto Set Pursuing (PSP) Method for Mixed-Variable...On the Performance of the Pareto Set Pursuing (PSP) Method for Mixed-Variable...
On the Performance of the Pareto Set Pursuing (PSP) Method for Mixed-Variable...
 
consistency regularization for generative adversarial networks_review
consistency regularization for generative adversarial networks_reviewconsistency regularization for generative adversarial networks_review
consistency regularization for generative adversarial networks_review
 
2020 11 4_bag_of_tricks
2020 11 4_bag_of_tricks2020 11 4_bag_of_tricks
2020 11 4_bag_of_tricks
 
Accelerated Particle Swarm Optimization and Support Vector Machine for Busine...
Accelerated Particle Swarm Optimization and Support Vector Machine for Busine...Accelerated Particle Swarm Optimization and Support Vector Machine for Busine...
Accelerated Particle Swarm Optimization and Support Vector Machine for Busine...
 
A COMPARATIVE STUDY OF BACKPROPAGATION ALGORITHMS IN FINANCIAL PREDICTION
A COMPARATIVE STUDY OF BACKPROPAGATION ALGORITHMS IN FINANCIAL PREDICTIONA COMPARATIVE STUDY OF BACKPROPAGATION ALGORITHMS IN FINANCIAL PREDICTION
A COMPARATIVE STUDY OF BACKPROPAGATION ALGORITHMS IN FINANCIAL PREDICTION
 
PR-330: How To Train Your ViT? Data, Augmentation, and Regularization in Visi...
PR-330: How To Train Your ViT? Data, Augmentation, and Regularization in Visi...PR-330: How To Train Your ViT? Data, Augmentation, and Regularization in Visi...
PR-330: How To Train Your ViT? Data, Augmentation, and Regularization in Visi...
 
Quasi newton artificial neural network training algorithms
Quasi newton artificial neural network training algorithmsQuasi newton artificial neural network training algorithms
Quasi newton artificial neural network training algorithms
 
What Makes Training Multi-modal Classification Networks Hard? ppt
What Makes Training Multi-modal Classification Networks Hard? pptWhat Makes Training Multi-modal Classification Networks Hard? ppt
What Makes Training Multi-modal Classification Networks Hard? ppt
 
Uber Data Analysis - SAS Project
Uber Data Analysis - SAS ProjectUber Data Analysis - SAS Project
Uber Data Analysis - SAS Project
 
Penalized Logistic Regression methods .pptx
Penalized Logistic Regression methods .pptxPenalized Logistic Regression methods .pptx
Penalized Logistic Regression methods .pptx
 
Proximal Policy Optimization Algorithms, Schulman et al, 2017
Proximal Policy Optimization Algorithms, Schulman et al, 2017Proximal Policy Optimization Algorithms, Schulman et al, 2017
Proximal Policy Optimization Algorithms, Schulman et al, 2017
 

Mehr von Sungjoon Choi

RNN and its applications
RNN and its applicationsRNN and its applications
RNN and its applicationsSungjoon Choi
 
Hybrid computing using a neural network with dynamic external memory
Hybrid computing using a neural network with dynamic external memoryHybrid computing using a neural network with dynamic external memory
Hybrid computing using a neural network with dynamic external memorySungjoon Choi
 
Modeling uncertainty in deep learning
Modeling uncertainty in deep learning Modeling uncertainty in deep learning
Modeling uncertainty in deep learning Sungjoon Choi
 
Gaussian Process Latent Variable Model
Gaussian Process Latent Variable ModelGaussian Process Latent Variable Model
Gaussian Process Latent Variable ModelSungjoon Choi
 
Uncertainty Modeling in Deep Learning
Uncertainty Modeling in Deep LearningUncertainty Modeling in Deep Learning
Uncertainty Modeling in Deep LearningSungjoon Choi
 
Domain Adaptation Methods
Domain Adaptation MethodsDomain Adaptation Methods
Domain Adaptation MethodsSungjoon Choi
 

Mehr von Sungjoon Choi (6)

RNN and its applications
RNN and its applicationsRNN and its applications
RNN and its applications
 
Hybrid computing using a neural network with dynamic external memory
Hybrid computing using a neural network with dynamic external memoryHybrid computing using a neural network with dynamic external memory
Hybrid computing using a neural network with dynamic external memory
 
Modeling uncertainty in deep learning
Modeling uncertainty in deep learning Modeling uncertainty in deep learning
Modeling uncertainty in deep learning
 
Gaussian Process Latent Variable Model
Gaussian Process Latent Variable ModelGaussian Process Latent Variable Model
Gaussian Process Latent Variable Model
 
Uncertainty Modeling in Deep Learning
Uncertainty Modeling in Deep LearningUncertainty Modeling in Deep Learning
Uncertainty Modeling in Deep Learning
 
Domain Adaptation Methods
Domain Adaptation MethodsDomain Adaptation Methods
Domain Adaptation Methods
 

Kürzlich hochgeladen

College Call Girls Nashik Nehal 7001305949 Independent Escort Service Nashik
College Call Girls Nashik Nehal 7001305949 Independent Escort Service NashikCollege Call Girls Nashik Nehal 7001305949 Independent Escort Service Nashik
College Call Girls Nashik Nehal 7001305949 Independent Escort Service NashikCall Girls in Nagpur High Profile
 
Call Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur Escorts
Call Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur EscortsCall Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur Escorts
Call Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur EscortsCall Girls in Nagpur High Profile
 
UNIT-II FMM-Flow Through Circular Conduits
UNIT-II FMM-Flow Through Circular ConduitsUNIT-II FMM-Flow Through Circular Conduits
UNIT-II FMM-Flow Through Circular Conduitsrknatarajan
 
Processing & Properties of Floor and Wall Tiles.pptx
Processing & Properties of Floor and Wall Tiles.pptxProcessing & Properties of Floor and Wall Tiles.pptx
Processing & Properties of Floor and Wall Tiles.pptxpranjaldaimarysona
 
Microscopic Analysis of Ceramic Materials.pptx
Microscopic Analysis of Ceramic Materials.pptxMicroscopic Analysis of Ceramic Materials.pptx
Microscopic Analysis of Ceramic Materials.pptxpurnimasatapathy1234
 
HARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICS
HARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICSHARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICS
HARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICSRajkumarAkumalla
 
Introduction and different types of Ethernet.pptx
Introduction and different types of Ethernet.pptxIntroduction and different types of Ethernet.pptx
Introduction and different types of Ethernet.pptxupamatechverse
 
Structural Analysis and Design of Foundations: A Comprehensive Handbook for S...
Structural Analysis and Design of Foundations: A Comprehensive Handbook for S...Structural Analysis and Design of Foundations: A Comprehensive Handbook for S...
Structural Analysis and Design of Foundations: A Comprehensive Handbook for S...Dr.Costas Sachpazis
 
High Profile Call Girls Nagpur Meera Call 7001035870 Meet With Nagpur Escorts
High Profile Call Girls Nagpur Meera Call 7001035870 Meet With Nagpur EscortsHigh Profile Call Girls Nagpur Meera Call 7001035870 Meet With Nagpur Escorts
High Profile Call Girls Nagpur Meera Call 7001035870 Meet With Nagpur EscortsCall Girls in Nagpur High Profile
 
Extrusion Processes and Their Limitations
Extrusion Processes and Their LimitationsExtrusion Processes and Their Limitations
Extrusion Processes and Their Limitations120cr0395
 
Introduction to IEEE STANDARDS and its different types.pptx
Introduction to IEEE STANDARDS and its different types.pptxIntroduction to IEEE STANDARDS and its different types.pptx
Introduction to IEEE STANDARDS and its different types.pptxupamatechverse
 
247267395-1-Symmetric-and-distributed-shared-memory-architectures-ppt (1).ppt
247267395-1-Symmetric-and-distributed-shared-memory-architectures-ppt (1).ppt247267395-1-Symmetric-and-distributed-shared-memory-architectures-ppt (1).ppt
247267395-1-Symmetric-and-distributed-shared-memory-architectures-ppt (1).pptssuser5c9d4b1
 
Call Girls in Nagpur Suman Call 7001035870 Meet With Nagpur Escorts
Call Girls in Nagpur Suman Call 7001035870 Meet With Nagpur EscortsCall Girls in Nagpur Suman Call 7001035870 Meet With Nagpur Escorts
Call Girls in Nagpur Suman Call 7001035870 Meet With Nagpur EscortsCall Girls in Nagpur High Profile
 
SPICE PARK APR2024 ( 6,793 SPICE Models )
SPICE PARK APR2024 ( 6,793 SPICE Models )SPICE PARK APR2024 ( 6,793 SPICE Models )
SPICE PARK APR2024 ( 6,793 SPICE Models )Tsuyoshi Horigome
 
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escorts
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur EscortsHigh Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escorts
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escortsranjana rawat
 
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...Soham Mondal
 
Software Development Life Cycle By Team Orange (Dept. of Pharmacy)
Software Development Life Cycle By  Team Orange (Dept. of Pharmacy)Software Development Life Cycle By  Team Orange (Dept. of Pharmacy)
Software Development Life Cycle By Team Orange (Dept. of Pharmacy)Suman Mia
 
KubeKraft presentation @CloudNativeHooghly
KubeKraft presentation @CloudNativeHooghlyKubeKraft presentation @CloudNativeHooghly
KubeKraft presentation @CloudNativeHooghlysanyuktamishra911
 
UNIT-III FMM. DIMENSIONAL ANALYSIS
UNIT-III FMM.        DIMENSIONAL ANALYSISUNIT-III FMM.        DIMENSIONAL ANALYSIS
UNIT-III FMM. DIMENSIONAL ANALYSISrknatarajan
 

Kürzlich hochgeladen (20)

College Call Girls Nashik Nehal 7001305949 Independent Escort Service Nashik
College Call Girls Nashik Nehal 7001305949 Independent Escort Service NashikCollege Call Girls Nashik Nehal 7001305949 Independent Escort Service Nashik
College Call Girls Nashik Nehal 7001305949 Independent Escort Service Nashik
 
Call Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur Escorts
Call Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur EscortsCall Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur Escorts
Call Girls Service Nagpur Tanvi Call 7001035870 Meet With Nagpur Escorts
 
UNIT-II FMM-Flow Through Circular Conduits
UNIT-II FMM-Flow Through Circular ConduitsUNIT-II FMM-Flow Through Circular Conduits
UNIT-II FMM-Flow Through Circular Conduits
 
Processing & Properties of Floor and Wall Tiles.pptx
Processing & Properties of Floor and Wall Tiles.pptxProcessing & Properties of Floor and Wall Tiles.pptx
Processing & Properties of Floor and Wall Tiles.pptx
 
Microscopic Analysis of Ceramic Materials.pptx
Microscopic Analysis of Ceramic Materials.pptxMicroscopic Analysis of Ceramic Materials.pptx
Microscopic Analysis of Ceramic Materials.pptx
 
HARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICS
HARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICSHARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICS
HARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICS
 
Introduction and different types of Ethernet.pptx
Introduction and different types of Ethernet.pptxIntroduction and different types of Ethernet.pptx
Introduction and different types of Ethernet.pptx
 
Structural Analysis and Design of Foundations: A Comprehensive Handbook for S...
Structural Analysis and Design of Foundations: A Comprehensive Handbook for S...Structural Analysis and Design of Foundations: A Comprehensive Handbook for S...
Structural Analysis and Design of Foundations: A Comprehensive Handbook for S...
 
High Profile Call Girls Nagpur Meera Call 7001035870 Meet With Nagpur Escorts
High Profile Call Girls Nagpur Meera Call 7001035870 Meet With Nagpur EscortsHigh Profile Call Girls Nagpur Meera Call 7001035870 Meet With Nagpur Escorts
High Profile Call Girls Nagpur Meera Call 7001035870 Meet With Nagpur Escorts
 
Extrusion Processes and Their Limitations
Extrusion Processes and Their LimitationsExtrusion Processes and Their Limitations
Extrusion Processes and Their Limitations
 
Introduction to IEEE STANDARDS and its different types.pptx
Introduction to IEEE STANDARDS and its different types.pptxIntroduction to IEEE STANDARDS and its different types.pptx
Introduction to IEEE STANDARDS and its different types.pptx
 
247267395-1-Symmetric-and-distributed-shared-memory-architectures-ppt (1).ppt
247267395-1-Symmetric-and-distributed-shared-memory-architectures-ppt (1).ppt247267395-1-Symmetric-and-distributed-shared-memory-architectures-ppt (1).ppt
247267395-1-Symmetric-and-distributed-shared-memory-architectures-ppt (1).ppt
 
★ CALL US 9953330565 ( HOT Young Call Girls In Badarpur delhi NCR
★ CALL US 9953330565 ( HOT Young Call Girls In Badarpur delhi NCR★ CALL US 9953330565 ( HOT Young Call Girls In Badarpur delhi NCR
★ CALL US 9953330565 ( HOT Young Call Girls In Badarpur delhi NCR
 
Call Girls in Nagpur Suman Call 7001035870 Meet With Nagpur Escorts
Call Girls in Nagpur Suman Call 7001035870 Meet With Nagpur EscortsCall Girls in Nagpur Suman Call 7001035870 Meet With Nagpur Escorts
Call Girls in Nagpur Suman Call 7001035870 Meet With Nagpur Escorts
 
SPICE PARK APR2024 ( 6,793 SPICE Models )
SPICE PARK APR2024 ( 6,793 SPICE Models )SPICE PARK APR2024 ( 6,793 SPICE Models )
SPICE PARK APR2024 ( 6,793 SPICE Models )
 
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escorts
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur EscortsHigh Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escorts
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escorts
 
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...
 
Software Development Life Cycle By Team Orange (Dept. of Pharmacy)
Software Development Life Cycle By  Team Orange (Dept. of Pharmacy)Software Development Life Cycle By  Team Orange (Dept. of Pharmacy)
Software Development Life Cycle By Team Orange (Dept. of Pharmacy)
 
KubeKraft presentation @CloudNativeHooghly
KubeKraft presentation @CloudNativeHooghlyKubeKraft presentation @CloudNativeHooghly
KubeKraft presentation @CloudNativeHooghly
 
UNIT-III FMM. DIMENSIONAL ANALYSIS
UNIT-III FMM.        DIMENSIONAL ANALYSISUNIT-III FMM.        DIMENSIONAL ANALYSIS
UNIT-III FMM. DIMENSIONAL ANALYSIS
 

Kernel, RKHS, and Gaussian Processes

  • 1. Kernel, RKHS, and Gaussian Processes Caution! No proof will be given. Sungjoon Choi, SNU
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.
  • 22.
  • 23.
  • 24.
  • 25.
  • 26.
  • 27.
  • 28.
  • 29.
  • 30.
  • 32. Leveraged  Gaussian  Processes The  original  Gaussian  process  regression   anchors  positive training  data. The  proposed  leveraged  Gaussian  process  regression  anchors  positive data   while  avoiding  negative data. Moreover,  it  is  possible  to  vary  the  leverage   of  each  training  data  from  -­‐1  to  +1.
  • 33. Positive and  Negative Motion   Control  Data  for  Autonomous   Navigation
  • 38. Leverage  Optimization • The  key  intuition  behind  the  leverage  optimization  is  that  we   cast  the  leverage  optimization  problem  into  a  model  selection   problem in  Gaussian  process  regression.   • However,  the  number  of  leverage  parameters  is  equivalent  to   the  number  of  training  data.   • To  handle  this  issue,  we  propose  a  sparse  constrained  leverage   optimization where  we  assume  that  the  majority  of  leverage   parameters  are  +1.  
  • 39. Leverage  Optimization  (v1) Using  proximal  linearized  minimization  [1],  the  update  rule  for  solving  above   optimization  is   [1]  J.  Bolte,  S.  Sabach,  and  M.  Teboulle,  “Proximal  alternating  linearized  minimization  for  nonconvex  and  nonsmooth  problems,”   Mathematical  Programming,  vol.  146,  no.  1-­2,  pp.  459–494,  2014. where  the  proximal  mapping  becomes  soft-­‐thresholding: where   𝛾̅ = 𝛾 − 1.  
  • 40. Leverage  Optimization  (v2) We  propose  a  new  leverage  optimization  method  by  doubling the  leverage   parameters  to  positive  and  negative  parts.   Furthermore,  we  assume  multiple  demonstrations  are  collected  from  one   demonstrator.   By  doing  so,  we  have  two  major  benefits: 1. As  the  L1-­‐norm  regularizer is  only  on  the  positive  parts  of  the  leverages,   negative  parts  of  the  leverages  can  be  optimized  more  accurately.   2. By  doubling  the  variables,  proximal  mapping  is  no  longer  needed.  
  • 41. Sensory  Field  Reconstruction Sensory  observations  from   correlated  sensory  fields Leverage  Optimization   Collect  observations
  • 42. Sensory  observations  from   correlated  sensory  fields Proposed  Method Previous  Method Without  Optimization Sensory  Field  Reconstruction
  • 43. Autonomous  Driving  Experiments Driving  demonstrations  with   mixed  qualities Leverage  Optimization   Collect  observations