Common Problems in Hyperparameter Optimization

•

0 gefällt mir•1,427 views

This document discusses common problems that can occur in hyperparameter optimization including trusting default values too much, using the wrong evaluation metric, overfitting hyperparameters to validation data, optimizing too few hyperparameters, relying too heavily on manual tuning, using grid search which scales poorly, using random search which has high variance, and provides recommendations such as using Bayesian optimization which can efficiently search large hyperparameter spaces and intelligently determine the most promising configurations to evaluate next.

Ingenieurwesen

Common Problems in Hyperparameter
Optimization
Alexandra Johnson
@alexandraj777

Hyperparameter Optimization
● Hyperparameter
tuning, model tuning,
model selection
● Finding "the best"
values for the
hyperparameters of
your model

Better Performance
● +315% accuracy boost for TensorFlow
● +49% accuracy boost for xgboost
● -41% error reduction for recommender system

● Default values are an implicit choice
● Defaults not always appropriate for your model
● You may build a classifier that looks like this:
Default Values

Choosing a Metric
● Balance long-term
and short-term goals
● Question underlying
assumptions
● Example from
Microsoft

Choose Multiple Metrics
●
● Composite Metric
● Multi-metric

Metric Generalization
● Cross validation
● Backtesting
● Regularization terms

Example: xgboost
● Optimized model
always performed
better with tuned
feature parameters
● No matter which
optimization method

You are not an Optimization Method
● Hand tuning is time
consuming and
expensive
● Algorithms can
quickly and cheaply
beat expert tuning

Grid Search Random Search Bayesian Optimization
Use an Algorithm

No Grid Search
Hyper-
parameters
Model
Evaluations
2 100
3 1,000
4 10,000
5 100,000

Random Search
● Theoretically more
effective than grid
search
● Large variance in
results
● No intelligence

Use an Intelligent Method
Genetic algorithms
Bayesian optimization
Particle-based methods
Convex optimizers
Simulated annealing
To name a few...

SigOpt: Bayesian Optimization Service
Three API calls:
1. Define
hyperparameters
2. Receive suggested
hyperparameters
3. Report observed
performance

Intro
Ian Dewancker. SigOpt for ML: TensorFlow ConvNets on a Budget with Bayesian Optimization.
Ian Dewancker. SigOpt for ML: Unsupervised Learning with Even Less Supervision Using Bayesian Optimization.
Ian Dewancker. SigOpt for ML : Bayesian Optimization for Collaborative Filtering with MLlib.
#1 Trusting the Defaults
Keras recurrent layers documentation
#2 Using the Wrong Metric
Ron Kohavi et al. Trustworthy Online Controlled Experiments: Five Puzzling Outcomes Explained.
Xavier Amatriain. 10 Lessons Learning from building ML systems [Video at 19:03].
Image from PhD Comics.
See also: SigOpt in Depth: Intro to Multicriteria Optimization.
#4 Too Few Hyperparameters
Image from TensorFlow Playground.
Ian Dewancker. SigOpt for ML: Unsupervised Learning with Even Less Supervision Using Bayesian Optimization.
#5 Hand Tuning
On algorithms beating experts: Scott Clark, Ian Dewancker, and Sathish Nagappan. Deep Neural Network Optimization with SigOpt and Nervana
Cloud.
#6 Grid Search
NoGridSearch.com
References - by Section

References - by Section
#7 Random Search
James Bergstra and Yoshua Bengio. Random search for hyper-parameter optimization.
Ian Dewancker, Michael McCourt, Scott Clark, Patrick Hayes, Alexandra Johnson, George Ke. A Stratified Analysis of Bayesian Optimization
Methods.
Learn More
blog.sigopt.com
sigopt.com/research

Weitere ähnliche Inhalte

Was ist angesagt?

Knowledge-based Systems

saimohang

Learning by analogy

Nitesh Singh

This Edureka Recurrent Neural Networks tutorial will help you in understanding why we need Recurrent Neural Networks (RNN) and what exactly it is. It also explains few issues with training a Recurrent Neural Network and how to overcome those challenges using LSTMs. The last section includes a use-case of LSTM to predict the next word using a sample short story Below are the topics covered in this tutorial: 1. Why Not Feedforward Networks? 2. What Are Recurrent Neural Networks? 3. Training A Recurrent Neural Network 4. Issues With Recurrent Neural Networks - Vanishing And Exploding Gradient 5. Long Short-Term Memory Networks (LSTMs) 6. LSTM Use-Case

Recurrent Neural Networks (RNN) | RNN LSTM | Deep Learning Tutorial | Tensorf...

Edureka!

Multi-Layer Perceptrons

Generative models

Feature selection

Scikit Learn intro

Learning set of rules

swapnac12

Semantic nets in artificial intelligence

harshita virwani

Concept learning

Musa Hawamdah

Intro to new Google cloud technologies: Google Storage, Prediction API, BigQuery

Chris Schalk

Stuart russell and peter norvig artificial intelligence - a modern approach...

Lê Anh Đạt

Parametric & Non-Parametric Machine Learning (Supervised ML)

Rehan Guha

▸ Machine Learning / Deep Learning models require to set the value of many hyperparameters ▸ Common examples: regularization coefficients, dropout rate, or number of neurons per layer in a Neural Network ▸ Instead of relying on some "expert advice", this presentation shows how to automatically find optimal hyperparameters ▸ Exhaustive Search, Monte Carlo Search, Bayesian Optimization, and Evolutionary Algorithms are explained with concrete examples

Hyperparameter Optimization for Machine Learning

Francesco Casalegno

Team Name: Outliers This the final project presentation for AIT 582 course of George Mason University. Demand forecasting plays is vital in today's world for the retail business. Proper demand management of store items directly impacts the efficiency of running a retail business. As the retail stores are in direct contact with the clients (shoppers) demands on a daily basis, demand forecasting is a key driver for such businesses. The goal of the project is to identify the best analytical method to solve such problems. The project data consist of 5 years of store-item sales data, and we plan to predict 3 months of sales for 50 different items at 10 different stores. We plan to see how we can deal with the seasonality of the demand for different items? Is forecasting demand for the items for individual stores more efficient or should the analysis should of the combined stores altogether. We plan to see which methods are optimal for such kind of analysis.

Store Item Demand Forecasting - AIT 582

Rahul Pandey

Mihai is the Principal Architect for Platform Engineering and Technology Solutions at IBM, responsible for Cloud Native and AI Solutions. He is a Red Hat Certified Architect, CKA/CKS, a leader in the IBM Open Innovation community, and advocate for open source development. Mihai is driving the development of Retrieval Augmentation Generation platforms, and solutions for Generative AI at IBM that leverage WatsonX, Vector databases, LangChain, HuggingFace and open source AI models. Mihai will share lessons learned building Retrieval Augmented Generation, or “Chat with Documents” platforms and APIs that scale, and deploy on Kubernetes. His talk will cover use cases for Generative AI, limitations of Large Language Models, use of RAG, Vector Databases and Fine Tuning to overcome model limitations and build solutions that connect to your data and provide content grounding, limit hallucinations and form the basis of explainable AI. In terms of technology, he will cover LLAMA2, HuggingFace TGIS, SentenceTransformers embedding models using Python, LangChain, and Weaviate and ChromaDB vector databases. He’ll also share tips on writing code using LLM, including building an agent for Ansible and containers. Scaling factors for Large Language Model Architectures: • Vector Database: consider sharding and High Availability • Fine Tuning: collecting data to be used for fine tuning • Governance and Model Benchmarking: how are you testing your model performance over time, with different prompts, one-shot, and various parameters • Chain of Reasoning and Agents • Caching embeddings and responses • Personalization and Conversational Memory Database • Streaming Responses and optimizing performance. A fine tuned 13B model may perform better than a poor 70B one! • Calling 3rd party functions or APIs for reasoning or other type of data (ex: LLMs are terrible at reasoning and prediction, consider calling other models) • Fallback techniques: fallback to a different model, or default answers • API scaling techniques, rate limiting, etc. • Async, streaming and parallelization, multiprocessing, GPU acceleration (including embeddings), generating your API using OpenAPI, etc.

Retrieval Augmented Generation in Practice: Scalable GenAI platforms with k8s...

Mihai Criveti

Generative Adversarial Network (GAN)

Prakhar Rastogi

Bias and variance trade off

VARUN KUMAR

Lstm

Mehrnaz Faraz

AI: Learning in AI

DataminingTools Inc

Was ist angesagt? (20)

Knowledge-based Systems

Learning by analogy

Recurrent Neural Networks (RNN) | RNN LSTM | Deep Learning Tutorial | Tensorf...

Multi-Layer Perceptrons

Generative models

Feature selection

Scikit Learn intro

Learning set of rules

Semantic nets in artificial intelligence

Concept learning

Intro to new Google cloud technologies: Google Storage, Prediction API, BigQuery

Stuart russell and peter norvig artificial intelligence - a modern approach...

Parametric & Non-Parametric Machine Learning (Supervised ML)

Hyperparameter Optimization for Machine Learning

Store Item Demand Forecasting - AIT 582

Retrieval Augmented Generation in Practice: Scalable GenAI platforms with k8s...

Generative Adversarial Network (GAN)

Bias and variance trade off

Lstm

AI: Learning in AI

Andere mochten auch

Machine learning is full of ideas that are far abstracted away from the underlying data and difficult to understand. Luckily, this represents an amazing opportunity for visualization! These slides dive into the machine learning meta-problem of hyperparameter optimization. We'll show 4 opportunities for visualization in helping people understand, implement, and evaluate hyperparameter optimization strategies.

Plotcon 2016 Visualization Talk by Alexandra Johnson

SigOpt

MLConf 2016 SigOpt Talk by Scott Clark

SigOpt

Alexandra Johnson, Software Engineer, SigOpt Alexandra works on everything from infrastructure to product features to blog posts. Previously, she worked on growth, APIs, and recommender systems at Polyvore (acquired by Yahoo). She majored in computer science at Carnegie Mellon University with a minor in discrete mathematics and logic, and during the summers she A/B tested recommendations at internships with Facebook and Rent the Runway. Abstract Summary: Common Problems In Hyperparameter Optimization: All large machine learning pipelines have tunable parameters, commonly referred to as hyperparameters. Hyperparameter optimization is the process by which we find the values for these parameters that cause our system to perform the best. SigOpt provides a Bayesian optimization platform that is commonly used for hyperparameter optimization, and I’m going to share some of the common problems we’ve seen when integrating into machine learning pipelines.

Alexandra Johnson, Software Engineer, SigOpt, at MLconf NYC 2017

MLconf

2015 Upload Campaigns Calendar - SlideShare

SlideShare

What to Upload to SlideShare

SlideShare

Getting Started With SlideShare

SlideShare

Andere mochten auch (6)

Plotcon 2016 Visualization Talk by Alexandra Johnson

MLConf 2016 SigOpt Talk by Scott Clark

Alexandra Johnson, Software Engineer, SigOpt, at MLconf NYC 2017

2015 Upload Campaigns Calendar - SlideShare

What to Upload to SlideShare

Getting Started With SlideShare

Ähnlich wie Common Problems in Hyperparameter Optimization

Big Data Spain 2018: How to build Weighted XGBoost ML model for Imbalance dat...

Alok Singh

Guiding through a typical Machine Learning Pipeline

Michael Gerke

Automated Machine Learning (AutoML) has received significant interest recently. We believe that the right automation would bring significant value and dramatically shorten time-to-value for data science teams. Databricks is automating the Data Science and Machine Learning process through a combination of product offerings, partnerships, and custom solutions. This talk will focus on how Databricks can help automate hyperparameter tuning. For both traditional Machine Learning and modern Deep Learning, tuning hyperparameters can dramatically increase model performance and improve training times. However, tuning can be a complex and expensive process. In this talk, we'll start with a brief survey of the most popular techniques for hyperparameter tuning (e.g., grid search, random search, and Bayesian optimization). We will then discuss open source tools that implement each of these techniques, helping to automate the search over hyperparameters. Finally, we will discuss and demo improvements we built for these tools in Databricks, including integration with MLflow: Apache PySpark MLlib integration with MLflow for automatically tracking tuning Hyperopt integration with Apache Spark to distribute tuning and with MLflow for automatic tracking Recording and notebooks will be provided after the webinar so that you can practice at your own pace. Presenters Joseph Bradley, Software Engineer, Databricks Joseph Bradley is a Software Engineer and Apache Spark PMC member working on Machine Learning at Databricks. Previously, he was a postdoc at UC Berkeley after receiving his Ph.D. in Machine Learning from Carnegie Mellon in 2013. Yifan Cao, Senior Product Manager, Databricks Yifan Cao is a Senior Product Manager at Databricks. His product area spans ML/DL algorithms and Databricks Runtime for Machine Learning. Prior to Databricks, Yifan worked on two Machine Learning products, applying NLP to find metadata and applying machine learning to predict equipment failures. He helped build the products from ground up to multi-million dollars in ARR. Yifan started his career as a researcher in quantum computing. Yifan received his B.S in UC Berkeley and Master from MIT.

Automated Hyperparameter Tuning, Scaling and Tracking

Databricks

Using Bayesian Optimization to Tune Machine Learning Models: In this talk we briefly introduce Bayesian Global Optimization as an efficient way to optimize machine learning model parameters, especially when evaluating different parameters is time-consuming or expensive. We will motivate the problem and give example applications. We will also talk about our development of a robust benchmark suite for our algorithms including test selection, metric design, infrastructure architecture, visualization, and comparison to other standard and open source methods. We will discuss how this evaluation framework empowers our research engineers to confidently and quickly make changes to our core optimization engine. We will end with an in-depth example of using these methods to tune the features and hyperparameters of a real world problem and give several real world applications.

Scott Clark, Co-Founder and CEO, SigOpt at MLconf SF 2016

MLconf

This poster is presenting a methodology for entity matching of product web offers. It was presented during the 8th Euroscipy conference in end of august of 2015. This poster is presenting Pricing Assistant’s recent work on product matching. The goal was to create a tool capable of determining if two web pages are selling the same product. Our approach combines various techniques from the fields of image analysis, semantic analysis and machine learning. The technique had great results and outperformed existing literature in fields such as skincare, cycling equipment and sporting goods.

Entity matching of web offers, from html to similarity score.

Paul Puget

Foutse_Khomh.pptx

Foutse Khomh

Tuning 2.0: Advanced Optimization Techniques Webinar

SigOpt

CODE TUNINGtertertertrtryryryryrtytrytrtry

kapib57390

Most large-scale online recommender systems like newsfeed ranking, people recommendations, job recommendations, etc. often have multiple utilities or metrics that need to be simultaneously optimized. The machine learning models that are trained to optimize a single utility are combined together through parameters to generate the final ranking function. These combination parameters drive business metrics. Finding the right choice of the parameters is often done through online A/B experimentation, which can be incredibly complex and time-consuming, especially considering the non-linear effects of these parameters on the metrics of interest. In this tutorial, we will talk about how we can apply Bayesian Optimization techniques to obtain the parameters for such complex online systems in order to balance the competing metrics. First, we will provide an in-depth introduction to Bayesian Optimization, covering some of the basics as well as the recent advances in the field. Second, we will talk about how to formulate a real-world recommender system problem as a black-box optimization problem that can be solved via Bayesian Optimization. We will focus on a few key problems such as newsfeed ranking, people recommendations, job recommendations, etc. Third, we will talk about the architecture of the solution and how we are able to deploy it for large-scale systems. Finally, we will discuss the extensions and some of the future directions in this domain.

Ijcai 2020

Viral Gupta

This presentation was made on June 9th, 2020. Video recording of the session can be viewed here: https://youtu.be/OCB9sTUnUug In this meetup with Sanyam Bhutani, Machine Learning Engineer at H2O.ai, he gives a recap of the eight annual ICLR (International Conference on Learning Representations) 2020 - a niche deep learning conference whose focus is to study how to learn representations of data, which is basically what deep learning does. Sanyam goes through a few of his favorite selected papers from this year’s ICLR, note this session may not be able to capture the richness of all papers or allow a detailed discussion. You will be able to find Sanyam in our community slack (https://www.h2o.ai/slack-community/), please feel free to start a discussion with him, if you send a  emoji greeting, you’ll find the answers. Following are the papers we will look into: U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty Your classifier is secretly an energy based model and you should treat it like one ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Reformer: The Efficient Transformer Generative Models for Effective ML on Private, Decentralized Datasets Once for All: Train One Network and Specialize it for Efficient Deployment Thieves on Sesame Street! Model Extraction of BERT-based APIs Plug and Play Language Models: A Simple Approach to Controlled Text Generation BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning Real or Not Real, that is the Question

ICLR 2020 Recap

Sri Ambati

Performance Tuning with XHProf

Salesforce Engineering

Hyperparameter tuning and optimization is a powerful tool in the area of AutoML, for both traditional statistical learning models as well as for deep learning. There are many existing tools to help drive this process, including both blackbox and whitebox tuning. In this talk, we'll start with a brief survey of the most popular techniques for hyperparameter tuning (e.g., grid search, random search, Bayesian optimization, and parzen estimators) and then discuss the open source tools which implement each of these techniques. Finally, we will discuss how we can leverage MLflow with these tools and techniques to analyze how our search is performing and to productionize the best models. Speaker: Joseph Bradley

Best Practices for Hyperparameter Tuning with MLflow

Databricks

MLPerf an industry standard benchmark suite for machine learning performance

jemin lee

Certification Study Group - Professional ML Engineer Session 3 (Machine Learn...

gdgsurrey

housing price prediction.pptx

JINALVASOYA2

Pre-Report.pptx

TANVIBENPATEL

Modeling at Scale: SigOpt at TWIMLcon 2019

SigOpt

Rails Conf 2014 Concerns, Decorators, Presenters, Service-objects, Helpers, H...

Justin Gordon

Evolving The Optimal Relevancy Scoring Model at Dice.com: Presented by Simon ...

Lucidworks

Using Bayesian Optimization to Tune Machine Learning Models

Scott Clark

Ähnlich wie Common Problems in Hyperparameter Optimization (20)

Big Data Spain 2018: How to build Weighted XGBoost ML model for Imbalance dat...

Guiding through a typical Machine Learning Pipeline

Automated Hyperparameter Tuning, Scaling and Tracking

Scott Clark, Co-Founder and CEO, SigOpt at MLconf SF 2016

Entity matching of web offers, from html to similarity score.

Foutse_Khomh.pptx

Tuning 2.0: Advanced Optimization Techniques Webinar

CODE TUNINGtertertertrtryryryryrtytrytrtry

Ijcai 2020

ICLR 2020 Recap

Performance Tuning with XHProf

Best Practices for Hyperparameter Tuning with MLflow

MLPerf an industry standard benchmark suite for machine learning performance

Certification Study Group - Professional ML Engineer Session 3 (Machine Learn...

housing price prediction.pptx

Pre-Report.pptx

Modeling at Scale: SigOpt at TWIMLcon 2019

Rails Conf 2014 Concerns, Decorators, Presenters, Service-objects, Helpers, H...

Evolving The Optimal Relevancy Scoring Model at Dice.com: Presented by Simon ...

Using Bayesian Optimization to Tune Machine Learning Models

Mehr von SigOpt

Optimizing BERT and Natural Language Models with SigOpt Experiment Management

SigOpt

Experiment Management for the Enterprise

SigOpt

Efficient NLP by Distilling BERT and Multimetric Optimization

SigOpt

Detecting COVID-19 Cases with Deep Learning

SigOpt

Metric Management: a SigOpt Applied Use Case

SigOpt

Tuning for Systematic Trading: Talk 3: Training, Tuning, and Metric Strategy

SigOpt

This talk explains how to train deep learning and other expensive models with parallelism and multitask optimization to reduce wall clock time. Tobias Andreassen, who supports a number of our systematic trading customers, presented the intuition behind Bayesian optimization for model optimization with a single or multiple (often competing) metrics. Many times it makes sense to analyze a second metric to avoid myopic training runs that overfit on your data, or otherwise don’t represent or impede performance in real-world scenarios.

Tuning for Systematic Trading: Talk 2: Deep Learning

SigOpt

This talk discusses the intuition behind Bayesian optimization with and without multiple metrics. Tobias Andreassen, who supports a number of our systematic trading customers, presented the intuition behind Bayesian optimization for model optimization with a single or multiple (often competing) metrics. Many times it makes sense to analyze a second metric to avoid myopic training runs that overfit on your data, or otherwise don’t represent or impede performance in real-world scenarios.

Tuning for Systematic Trading: Talk 1

SigOpt

In this webinar, SigOpt ML Engineer Meghana Ravikumar presents on and builds an image classifier trained on the Stanford Cars dataset to evaluate two approaches to transfer learning—fine tuning and feature extraction—and the impact of Multitask optimization, a more efficient form of Bayesian optimization, on these techniques. Once we define the most performant transfer learning technique for Stanford Cars, we will use image augmentation to double the size of the dataset to boost the classifier’s performance. Instead of manually tuning the hyperparameters associated with image augmentation, we will use Multitask Optimization to learn these hyperparameters using the downstream image classifier’s performance as the guide. In conjunction with model performance, we will also explore the features of these augmented images and the downstream implications for our image classifier.

Tuning Data Augmentation to Boost Model Performance

SigOpt

Building on the TWIML eBook, TWIMLcon event and TWIML podcast series that explore Machine Learning Platforms in great detail, this webinar examines the machine learning platforms that power enterprise leaders in AI. SigOpt CEO Scott Clark will provide an overview of critical technical capabilities that our customers have prioritized in their ML platforms. Review these slides to learn about: - Critical capabilities for data, experiment and model management - Tradeoffs between building and buying these capabilities - Lessons from the implementation of these platforms by AI leaders Why focus on these platforms and the capabilities that power them? Nearly every company is investing in machine learning that differentiates products or generates revenue. These so-called "differentiated models" represent the biggest opportunity for AI to transform the business. Most of these teams find success hiring expert data scientists and machine learning engineers who can build these models. But most of these teams also struggle to create a more sustainable, scalable and reproducible process for model development, and have begun building ML platforms to tackle this challenge.

Advanced Optimization for the Enterprise Webinar

SigOpt

SigOpt at Ai4 Finance—Modeling at Scale

SigOpt

Many real world applications - machine learning models, simulators, etc. - have multiple competing metrics that define performance; these require practitioners to carefully consider potential tradeoffs. However, assessing and ranking this tradeoff is nontrivial, especially when the number of metrics is more than two. Often times, practitioners scalarize the metrics into a single objective, e.g., using a weighted sum. In this talk, we pose this problem as a constrained multi-objective optimization problem. By setting and updating the constraints, we can efficiently explore only the region of the Pareto efficient frontier of the model/system of most interest. We motivate this problem with the application of an experimental design setting, where we are trying to fabricate high performance glass substrate for solar cell panels.

Interactive Tradeoffs Between Competing Offline Metrics with Bayesian Optimiz...

SigOpt

As data science workloads grow, so does their need for infrastructure. But, is it fair to ask data scientists to also become infrastructure experts? If not the data scientists, then, who is responsible for spinning up and managing data science infrastructure? This talk will address the context in which ML infrastructure is emerging, walk through two examples of ML infrastructure tools for launching hyperparameter optimization jobs, and end with some thoughts for building better tools in the future. Originally given as a talk at the PyData Ann Arbor meetup (https://www.meetup.com/PyData-Ann-Arbor/events/260380989/)

Machine Learning Infrastructure

SigOpt

SigOpt at Uber Science Symposium - Exploring the spectrum of black-box optimi...

SigOpt

Companies are increasingly building modeling platforms to empower their researchers to efficiently scale the development and productionalization of their models. Scott Clark and Matt Greenwood share a case study from a leading algorithmic trading firm to illustrate best practices for building these types of platforms in any industry. Join in to learn how Two Sigma, a leading quantitative investment and technology firm, solved its model optimization problem.

SigOpt at O'Reilly - Best Practices for Scaling Modeling Platforms

SigOpt

Training and tuning models with lengthy training cycles like those in deep learning can be extremely expensive and may sometimes involve techniques that degrade performance. We'll explore recent research on optimization strategies to efficiently tune these types of deep learning models. We will provide benchmarks and comparisons to other popular methods for optimizing the models, and we'll recommend valuable areas for further applied research.

SigOpt at GTC - Tuning the Untunable

SigOpt

Advanced hardware like NVIDIA technology lowers technical barriers to model size and scope, but issues remain in areas like model performance and training infrastructure management. We'll discuss operational challenges to training models at scale with a particular focus on how training management and hyperparameter tuning can inform each other to accomplish specific goals. We'll also explore techniques like parallelism and scheduling, discuss their impact on model optimization, and compare various techniques. We'll also evaluate results of this approach. In particular, we'll focus on how new tools that automate training orchestration accelerate model development and increase the volume and quality of models in production.

SigOpt at GTC - Reducing operational barriers to optimization

SigOpt

Lessons for an enterprise approach to modeling at scale

SigOpt

SigOpt CEO Scott Clark provides insights for modeling at scale in systematic trading. SigOpt works with algorithmic trading firms that collectively represent $300 billion in assets under management (AUM). In this presentation, Scott draws on this experience to provide a few critical insights to how these companies effectively model at scale. Alongside these insights, Scott shares a more specific case study from working with Two Sigma, a leading systematic investment manager.

Modeling at scale in systematic trading

SigOpt

SigOpt at MLconf - Reducing Operational Barriers to Model Training

SigOpt

Mehr von SigOpt (20)

Optimizing BERT and Natural Language Models with SigOpt Experiment Management

Experiment Management for the Enterprise

Efficient NLP by Distilling BERT and Multimetric Optimization

Detecting COVID-19 Cases with Deep Learning

Metric Management: a SigOpt Applied Use Case

Tuning for Systematic Trading: Talk 3: Training, Tuning, and Metric Strategy

Tuning for Systematic Trading: Talk 2: Deep Learning

Tuning for Systematic Trading: Talk 1

Tuning Data Augmentation to Boost Model Performance

Advanced Optimization for the Enterprise Webinar

SigOpt at Ai4 Finance—Modeling at Scale

Interactive Tradeoffs Between Competing Offline Metrics with Bayesian Optimiz...

Machine Learning Infrastructure

SigOpt at Uber Science Symposium - Exploring the spectrum of black-box optimi...

SigOpt at O'Reilly - Best Practices for Scaling Modeling Platforms

SigOpt at GTC - Tuning the Untunable

SigOpt at GTC - Reducing operational barriers to optimization

Lessons for an enterprise approach to modeling at scale

Modeling at scale in systematic trading

SigOpt at MLconf - Reducing Operational Barriers to Model Training

Kürzlich hochgeladen

Call girls in delhi ✔️✔️🔝 9953056974 🔝✔️✔️Welcome To Vip Escort Services In Delhi [ ]Noida Gurgaon 24/7 Open Sex Escort Services With Happy Ending ServiCe Done By Most Attractive Charming Soft Spoken Bold Beautiful Full Cooperative Independent Escort Girls ServiCe In All-Star Hotel And Home Service In All Over Delhi, Noida, Gurgaon, Faridabad, Ghaziabad, Greater Noida, • IN CALL AND OUT CALL SERVICE IN DELHI NCR • 3* 5* 7* HOTELS SERVICE IN DELHI NCR • 24 HOURS AVAILABLE IN DELHI NCR • INDIAN, RUSSIAN, PUNJABI, KASHMIRI ESCORTS • REAL MODELS, COLLEGE GIRLS, HOUSE WIFE, ALSO AVAILABLE • SHORT TIME AND FULL TIME SERVICE AVAILABLE • HYGIENIC FULL AC NEAT AND CLEAN ROOMS AVAIL. IN HOTEL 24 HOURS • DAILY NEW ESCORTS STAFF AVAILABLE • MINIMUM TO MAXIMUM RANGE AVAILABLE. Call Girls in Delhi & Independent Escort Service – CALL GIRLS SERVICE DELHI NCR Vip call girls in Delhi Call Girls in Delhi, Call Girl Service 24×7 open Call Girls in Delhi Best Delhi Escorts in Delhi Low Rate Call Girls In Saket Delhi X~CALL GIRLS IN Ramesh Nagar Metro best Delhi call girls and Delhi escort service. CALL GIRLS SERVICE IN ALL DELHI … (Delhi) Call Girls in (Chanakyapuri) Hot And Sexy Independent Model Escort Service In Delhi Unlimited Enjoy Genuine 100% Profiles And Trusted Door Step Call Girls Feel Free To Call Us Female Service Hot Busty & Sexy Party Girls Available For Complete Enjoyment. We Guarantee Full Satisfaction & In Case Of Any Unhappy Experience, We Would Refund Your Fees, Without Any Questions Asked. Feel Free To Call Us Female Service Provider Hours Opens Thanks. Delhi Escorts Services 100% secure Services.Incall_OutCall Available and outcall Services provide. We are available 24*7 for Full Night and short Time Escort Services all over Delhi NCR. Delhi All Hotel Services available 3* 4* 5* Call Call Delhi Escorts Services And Delhi Call Girl Agency 100% secure Services in my agency. Incall and outcall Services provide. We are available 24*7 for Full Night and short Time Escort Services my agency in all over New Delhi Delhi All Hotel Services available my agency SERVICES [✓✓✓] Housewife College Girl VIP Escort Independent Girl Aunty Without a Condom sucking )? Sexy Aunty.DSL (Dick Sucking Lips)? DT (Dining at the Toes English Spanking) Doggie (Sex style from no behind)?? OutCall- All Over Delhi Noida Gurgaon 24/7 FOR APPOINTMENT Call/Whatsop / 9953056974

Call Girls in Ramesh Nagar Delhi 💯 Call Us 🔝9953056974 🔝 Escort Service

9953056974 Low Rate Call Girls In Saket, Delhi NCR

Bhosari ( Call Girls ) Pune 6297143586 Hot Model With Sexy Bhabi Ready For Sex At Your Doorstep Booking Contact Details WhatsApp Chat: +91-6297143586 pune Escort Service includes providing maximum physical satisfaction to their clients as well as engaging conversation that keeps your time enjoyable and entertaining. Plus they look fabulously elegant; making an impressionable. Independent Escorts pune understands the value of confidentiality and discretion - they will go the extra mile to meet your needs. Simply contact them via text messaging or through their online profiles; they'd be more than delighted to accommodate any request or arrange a romantic date or fun-filled night together. We provide - 02-may-2024(v.n)

Bhosari ( Call Girls ) Pune 6297143586 Hot Model With Sexy Bhabi Ready For ...

tanu pandey

FEA Based Level 3 Assessment of Deformed Tanks with Fluid Induced Loads

Arindam Chakraborty, Ph.D., P.E. (CA, TX)

LIST OF EXPERIMENTS: 1. Implement simple vector addition in Tensor Flow. 2. Implement a regression model in Keras. 3. Implement a perception in TensorFlow/Keras Environment. 4. Implement a Feed Forward Network in TensorFlow/Keras. 5. Implement an image classifier using CNN in TensorFlow/Keras. 6. Improve the deep Learning model by fine tuning hyper parameters. 7. Implement a Transfer Learning concept in image classification. 8. Using a pre trained model on Keras for transfer learning. 9. Perform Sentimental Analysis using RNN. 10. Implement an LSTM based Auto encoding inTensorflow/Keras. 11. Image generation using GAN. ADDITIONAL EXPERIMENTS 12. Train a deep Learning model to classify a given image using pre trained model. 13. Recommendation system from sales data using Deep Learning. 14. Implement Object detection using CNN. 15. Implement any simple Reinforcement Algorithm for an NLP problem.

CCS335 _ Neural Networks and Deep Learning Laboratory_Lab Complete Record

Asst.prof M.Gokilavani

GAS POWER CYCLES Cycles: Otto, Diesel, Dual, Brayton - Calculation of mean effective pressure - Air standard efficiency - Comparison of cycles INTERNAL COMBUSTION ENGINES Classification - Components and their function - valve timing diagram and port timing diagram - actual and theoretical p-v diagram of two stroke and four stroke engines – carburettor - diesel pump and injector system - battery and magneto ignition system - principles of combustion and detonation in CI engines - lubrication and cooling systems - performance parameters and calculations.

Thermal Engineering Unit - I & II . ppt

DineshKumar4165

VIP Model Call Girls Kothrud ( Pune ) Call ON 8005736733 Starting From 5K to 25K High Profile Escorts In Pune Booking Now open +91- 8005736733 Why you Choose Us- +91- 8005736733 HOT⇄ 8005736733 Mr ashu ji Call Mr ashu Ji +91- 8005736733 (V030524]N) 𝐇𝐨𝐭𝐞𝐥 𝐑𝐨𝐨𝐦𝐬 𝐈𝐧𝐜𝐥𝐮𝐝𝐢𝐧𝐠 𝐑𝐚𝐭𝐞 𝐒𝐡𝐨𝐭𝐬/𝐇𝐨𝐮𝐫𝐲🆓 .█▬█⓿▀█▀ 𝐈𝐍𝐃𝐄𝐏𝐄𝐍𝐃𝐄𝐍𝐓 𝐆𝐈𝐑𝐋 𝐕𝐈𝐏 𝐄𝐒𝐂𝐎𝐑𝐓 Hello Guys ! High Profiles young Beauties and Good Looking standard Profiles Available , Enquire Now if you are interested in Hifi Service and want to get connect with someone who can understand your needs. Service offers you the most beautiful High Profile sexy independent female Escorts in genuine ✔✔✔ To enjoy with hot and sexy girls ✔✔✔ ★providing:- • Models • vip Models • Russian Models • Foreigner Models • TV Actress and Celebrities • Receptionist • Air Hostess • Call Center Working Girls/Women • Hi-Tech Co. Girls/Women • Housewife

VIP Model Call Girls Kothrud ( Pune ) Call ON 8005736733 Starting From 5K to ...

SUHANI PANDEY

Model Call Girl Services in Delhi reach out to us at 🔝 9953056974 🔝✔️✔️ Our agency presents a selection of young, charming call girls available for bookings at Oyo Hotels. Experience high-class escort services at pocket-friendly rates, with our female escorts exuding both beauty and a delightful personality, ready to meet your desires. Whether it's Housewives, College girls, Russian girls, Muslim girls, or any other preference, we offer a diverse range of options to cater to your tastes. We provide both in-call and out-call services for your convenience. Our in-call location in Delhi ensures cleanliness, hygiene, and 100% safety, while our out-call services offer doorstep delivery for added ease. We value your time and money, hence we kindly request pic collectors, time-passers, and bargain hunters to refrain from contacting us. Our services feature various packages at competitive rates: One shot: ₹2000/in-call, ₹5000/out-call Two shots with one girl: ₹3500/in-call, ₹6000/out-call Body to body massage with sex: ₹3000/in-call Full night for one person: ₹7000/in-call, ₹10000/out-call Full night for more than 1 person: Contact us at 🔝 9953056974 🔝. for details Operating 24/7, we serve various locations in Delhi, including Green Park, Lajpat Nagar, Saket, and Hauz Khas near metro stations. For premium call girl services in Delhi 🔝 9953056974 🔝. Thank you for considering us!

Call Now ≽ 9953056974 ≼🔝 Call Girls In New Ashok Nagar ≼🔝 Delhi door step de...

9953056974 Low Rate Call Girls In Saket, Delhi NCR

Top Rated Pune Call Girls Budhwar Peth ⟟ 6297143586 ⟟ Call Me For Genuine Sex Service At Affordable Rate Booking Contact Details WhatsApp Chat: +91-6297143586 pune Escort Service includes providing maximum physical satisfaction to their clients as well as engaging conversation that keeps your time enjoyable and entertaining. Plus they look fabulously elegant; making an impressionable. Independent Escorts pune understands the value of confidentiality and discretion - they will go the extra mile to meet your needs. Simply contact them via text messaging or through their online profiles; they'd be more than delighted to accommodate any request or arrange a romantic date or fun-filled night together. We provide - 01-may-2024(v.n)

Top Rated Pune Call Girls Budhwar Peth ⟟ 6297143586 ⟟ Call Me For Genuine Se...

Call Girls in Nagpur High Profile

Call Girls Pimpri Chinchwad Call Me 7737669865 Budget Friendly No Advance Booking Booking Now open +91- 7737669865 Why you Choose Us- +91- 7737669865 HOT⇄ 7737669865 Mr ashu ji Call Mr ashu Ji +91- 7737669865 (V020524]N) 𝐇𝐨𝐭𝐞𝐥 𝐑𝐨𝐨𝐦𝐬 𝐈𝐧𝐜𝐥𝐮𝐝𝐢𝐧𝐠 𝐑𝐚𝐭𝐞 𝐒𝐡𝐨𝐭𝐬/𝐇𝐨𝐮𝐫𝐲🆓 .█▬█⓿▀█▀ 𝐈𝐍𝐃𝐄𝐏𝐄𝐍𝐃𝐄𝐍𝐓 𝐆𝐈𝐑𝐋 𝐕𝐈𝐏 𝐄𝐒𝐂𝐎𝐑𝐓 Hello Guys ! High Profiles young Beauties and Good Looking standard Profiles Available , Enquire Now if you are interested in Hifi Service and want to get connect with someone who can understand your needs. Service offers you the most beautiful High Profile sexy independent female Escorts in genuine ✔✔✔ To enjoy with hot and sexy girls ✔✔✔ ★providing:- • Models • vip Models • Russian Models

Call Girls Pimpri Chinchwad Call Me 7737669865 Budget Friendly No Advance Boo...

roncy bisnoi

Call Girls in Netaji Nagar, Delhi 💯 Call Us 🔝9953056974 🔝 Escort Service

9953056974 Low Rate Call Girls In Saket, Delhi NCR

Increased aeration of the soil; Stabilized soil structure; Higher and more diversified crop production; Better workability of the land; Earlier planting dates; Reduction of peak discharges by an increased temporary storage of water in the soil decomposition of organic matter; soil subsidence; reduced irrigation efficiency; increased risk of drought. excessive leaching of valuable nutrients from the soil; downstream environmental damage by salty or otherwise polluted drainage water; the presence of ditches, canals, and structures impending accessibility and interfering with other infrastructural elements of the land.

chapter 5.pptx: drainage and irrigation engineering

mulugeta48

VIP Call Girls Ankleshwar 7001035870 Whatsapp Number, 24/07 Booking Escorts Service Available Whatsapp SABANA ☎️ : [+91-7001035870] Escorts Service are always ready to make their clients happy. Their exotic looks and sexy personalities are sure to turn heads. You can enjoy with them, including massages and erotic encounters. Our area Escorts are young and sexy, so you can expect to have an exotic time with them. They are trained to satiate your naughty nerves and they can handle anything that you want. They are also intelligent, so they know how to make you feel comfortable and relaxed Independent Escorts Service They know all the sex positions and can satisfy you in any way that you desire. They can even give you erotic massages to help you relax before your session. This is essential, because a man who is stressed won’t be receptive to the pleasures of sex. They also know how to play with your sexy organs, so you’ll have plenty of foreplay and cuddling. P252024SS SERVICE ✅ ❣️ ⭐➡️HOT & SEXY MODELS // COLLEGE GIRLS HOUSE WIFE RUSSIAN , AIR HOSTES ,VIP MODELS . AVAILABLE FOR COMPLETE ENJOYMENT WITH HIGH PROFILE INDIAN MODEL AVAILABLE HOTEL & HOME ★ SAFE AND SECURE HIGH CLASS SERVICE AFFORDABLE RATE ★ SATISFACTION,UNLIMITED ENJOYMENT. ★ All Meetings are confidential and no information is provided to any one at any cost. ★ EXCLUSIVE PROFILes Are Safe and Consensual with Most Limits Respected ★ Service Available In: - HOME & HOTEL Star Hotel Service .In Call & Out call SeRvIcEs : ★ A-Level (star escort) ★ Strip-tease ★ BBBJ (Bareback Blowjob)Receive advanced sexual techniques in different mode make their life more pleasurable. ★ Spending time in hotel rooms ★ BJ (Blowjob Without a Condom) ★ Completion (Oral to completion) ★ Covered (Covered blowjob Without condom ★ANAL SERVICES.

VIP Call Girls Ankleshwar 7001035870 Whatsapp Number, 24/07 Booking

dharasingh5698

Design For Accessibility: Getting it right from the start

Quintin Balsdon

Intro To Electric Vehicles PDF Notes.pdf

rs7054576148

Top Rated Call Girls In chittoor 📱 {7001035870} VIP Escorts chittoor Escorts Service Available Whatsapp Chaya ☎️ : [+91-7001035870] Escorts Service are always ready to make their clients happy. Their exotic looks and sexy personalities are sure to turn heads. You can enjoy with them, including massages and erotic encounters. Our area Escorts are young and sexy, so you can expect to have an exotic time with them. They are trained to satiate your naughty nerves and they can handle anything that you want. They are also intelligent, so they know how to make you feel comfortable and relaxed Independent Escorts Service They know all the sex positions and can satisfy you in any way that you desire. They can even give you erotic massages to help you relax before your session. This is essential, because a man who is stressed won’t be receptive to the pleasures of sex. They also know how to play with your sexy organs, so you’ll have plenty of foreplay and cuddling. P252024SS SERVICE ✅ ❣️ ⭐➡️HOT & SEXY MODELS // COLLEGE GIRLS HOUSE WIFE RUSSIAN , AIR HOSTES ,VIP MODELS . AVAILABLE FOR COMPLETE ENJOYMENT WITH HIGH PROFILE INDIAN MODEL AVAILABLE HOTEL & HOME ★ SAFE AND SECURE HIGH CLASS SERVICE AFFORDABLE RATE ★ SATISFACTION,UNLIMITED ENJOYMENT. ★ All Meetings are confidential and no information is provided to any one at any cost. ★ EXCLUSIVE PROFILes Are Safe and Consensual with Most Limits Respected ★ Service Available In: - HOME & HOTEL Star Hotel Service .In Call & Out call SeRvIcEs : ★ A-Level (star escort) ★ Strip-tease ★ BBBJ (Bareback Blowjob)Receive advanced sexual techniques in different mode make their life more pleasurable. ★ Spending time in hotel rooms ★ BJ (Blowjob Without a Condom) ★ Completion (Oral to completion) ★ Covered (Covered blowjob Without condom ★ANAL SERVICES.

Top Rated Call Girls In chittoor 📱 {7001035870} VIP Escorts chittoor

dharasingh5698

Call Girls In Mahipalpur (Delhi) Call or Whataap {+91-8377877756} Escorts Provide 24×7 Available With Room ☆☆TIMINGS 24 HOURS OPENS☆☆☆ Booking Now Gentleman Only:-Call Now Best High Class Normal Call Girls Escorts Service In Delhi NCR 24-7 Hours Available Service I provide In Delhi NCR Female Escorts Sex Service 100% Customers Satisfaction Guarantee VIP Profiles Top Grade Service 100% Cooperative All round Service ꧁❤8377877756❤꧂ InCall: – You Can Reach At Our Place in Delhi Our place Which Is Very Clean Hygienic 100% safe Accommodation OutCall: – Service For Out Call You Have To Come Pick The Girl From My Place. We Also Provide Door Step Services. Note: – Pic Collectors Time Passers And Bargainers Stay Away As We Respect The Value Of Your Money And Time And Expect The Same From You. 8377877756. Hygienic: – Full Ac Neat And Clean Rooms Available In Hotel 24 * 7 Hrs In Delhi NCR 8377877756. Place: – South Extension Nehru Place Saket Malviya Nagar Munirka Vasant Kunj Safdarjung Katwaria Sarai Lajpat Nagar Kalkaji Hauz Khas Mahipalpur Dwarka Karol Bagh Noida Gurgaon Faridabad All Outcall Only Hotel Service In Delhi Ncr 8377877756. We are providing. : – House Wife’s : – Private Independent House Wife’s : – Private Independent Collage Going Girls : – Corporate MNC Working Profiles : – Call Center Girls : – Live Band Girls : – Foreigners and Many More: – Independent Models Service type : – Incall Short 2500 : – Outcall Short 3500 : – Incall Night 8500 : – Outcall Night 95,00 For Pictures And Other Details Pls Whatsapp Me Otherwise Call Me Any Time Incall Outcall Both Are Services Available Door Step, House, Apartment, Guest House, Flate, All Star Hotel Available ꧁❤8377877756❤꧂ THANKS FOR VISITING. Booking 24×7 HRS

FULL ENJOY Call Girls In Mahipalpur Delhi Contact Us 8377877756

dollysharma2066

KubeKraft presentation @CloudNativeHooghly

sanyuktamishra911

Booking open Available Pune Call Girls Koregaon Park 6297143586 Call Hot Indian Girls Waiting For You To Fuck Booking Contact Details WhatsApp Chat: +91-6297143586 pune Escort Service includes providing maximum physical satisfaction to their clients as well as engaging conversation that keeps your time enjoyable and entertaining. Plus they look fabulously elegant; making an impressionable. Independent Escorts pune understands the value of confidentiality and discretion - they will go the extra mile to meet your needs. Simply contact them via text messaging or through their online profiles; they'd be more than delighted to accommodate any request or arrange a romantic date or fun-filled night together. We provide - 01-may-2024(v.n)

Booking open Available Pune Call Girls Koregaon Park 6297143586 Call Hot Ind...

Call Girls in Nagpur High Profile

Call Girls Walvekar Nagar Call Me 7737669865 Budget Friendly No Advance Booking Booking Now open +91- 7737669865 Why you Choose Us- +91- 7737669865 HOT⇄ 7737669865 Mr ashu ji Call Mr ashu Ji +91- 7737669865 (V020524]N) 𝐇𝐨𝐭𝐞𝐥 𝐑𝐨𝐨𝐦𝐬 𝐈𝐧𝐜𝐥𝐮𝐝𝐢𝐧𝐠 𝐑𝐚𝐭𝐞 𝐒𝐡𝐨𝐭𝐬/𝐇𝐨𝐮𝐫𝐲🆓 .█▬█⓿▀█▀ 𝐈𝐍𝐃𝐄𝐏𝐄𝐍𝐃𝐄𝐍𝐓 𝐆𝐈𝐑𝐋 𝐕𝐈𝐏 𝐄𝐒𝐂𝐎𝐑𝐓 Hello Guys ! High Profiles young Beauties and Good Looking standard Profiles Available , Enquire Now if you are interested in Hifi Service and want to get connect with someone who can understand your needs. Service offers you the most beautiful High Profile sexy independent female Escorts in genuine ✔✔✔ To enjoy with hot and sexy girls ✔✔✔ ★providing:- • Models • vip Models • Russian Models

Call Girls Walvekar Nagar Call Me 7737669865 Budget Friendly No Advance Booking

roncy bisnoi

STEAM NOZZLES AND TURBINES Flow of steam through nozzles, shapes of nozzles, effect of friction, critical pressure ratio, supersaturated flow - impulse and reaction principles, velocity diagram, work done and efficiency – types of compounding - governors. AIR COMPRESSORS Classification - working principle - type of compressors, work of compression with and without clearance - volumetric efficiency - isothermal and isentropic efficiency of reciprocating compressors - multistage air compressor with inter cooling.

Thermal Engineering -unit - III & IV.ppt

DineshKumar4165

Kürzlich hochgeladen (20)

Call Girls in Ramesh Nagar Delhi 💯 Call Us 🔝9953056974 🔝 Escort Service

Bhosari ( Call Girls ) Pune 6297143586 Hot Model With Sexy Bhabi Ready For ...

FEA Based Level 3 Assessment of Deformed Tanks with Fluid Induced Loads

CCS335 _ Neural Networks and Deep Learning Laboratory_Lab Complete Record

Thermal Engineering Unit - I & II . ppt

VIP Model Call Girls Kothrud ( Pune ) Call ON 8005736733 Starting From 5K to ...

Call Now ≽ 9953056974 ≼🔝 Call Girls In New Ashok Nagar ≼🔝 Delhi door step de...

Top Rated Pune Call Girls Budhwar Peth ⟟ 6297143586 ⟟ Call Me For Genuine Se...

Call Girls Pimpri Chinchwad Call Me 7737669865 Budget Friendly No Advance Boo...

Call Girls in Netaji Nagar, Delhi 💯 Call Us 🔝9953056974 🔝 Escort Service

chapter 5.pptx: drainage and irrigation engineering

VIP Call Girls Ankleshwar 7001035870 Whatsapp Number, 24/07 Booking

Design For Accessibility: Getting it right from the start

Intro To Electric Vehicles PDF Notes.pdf

Top Rated Call Girls In chittoor 📱 {7001035870} VIP Escorts chittoor

FULL ENJOY Call Girls In Mahipalpur Delhi Contact Us 8377877756

KubeKraft presentation @CloudNativeHooghly

Booking open Available Pune Call Girls Koregaon Park 6297143586 Call Hot Ind...

Call Girls Walvekar Nagar Call Me 7737669865 Budget Friendly No Advance Booking

Thermal Engineering -unit - III & IV.ppt

Common Problems in Hyperparameter Optimization

1. Common Problems in Hyperparameter Optimization Alexandra Johnson @alexandraj777

2. What are Hyperparameters?

3. Hyperparameter Optimization ● Hyperparameter tuning, model tuning, model selection ● Finding "the best" values for the hyperparameters of your model

4. Better Performance ● +315% accuracy boost for TensorFlow ● +49% accuracy boost for xgboost ● -41% error reduction for recommender system

5. #1 Trusting the Defaults

6. ● Default values are an implicit choice ● Defaults not always appropriate for your model ● You may build a classifier that looks like this: Default Values

7. #2 Using the Wrong Metric

8. Choosing a Metric ● Balance long-term and short-term goals ● Question underlying assumptions ● Example from Microsoft

9. Choose Multiple Metrics ● ● Composite Metric ● Multi-metric

10. #3 Overfitting

11. Metric Generalization ● Cross validation ● Backtesting ● Regularization terms

12. Metric Generalization ● Cross validation ● Backtesting ● Regularization terms

13. Metric Generalization ● Cross validation ● Backtesting ● Regularization terms

14. #4 Too Few Hyperparameters

15. Optimize all Parameters at Once

16. Include Feature Parameters

17. Include Feature Parameters

18. Example: xgboost ● Optimized model always performed better with tuned feature parameters ● No matter which optimization method

19. #5 Hand Tuning

20. What is an Optimization Method?

21. You are not an Optimization Method ● Hand tuning is time consuming and expensive ● Algorithms can quickly and cheaply beat expert tuning

22. Grid Search Random Search Bayesian Optimization Use an Algorithm

23. #6 Grid Search

24. No Grid Search Hyper- parameters Model Evaluations 2 100 3 1,000 4 10,000 5 100,000

25. #7 Random Search

26. Random Search ● Theoretically more effective than grid search ● Large variance in results ● No intelligence

27. Use an Intelligent Method Genetic algorithms Bayesian optimization Particle-based methods Convex optimizers Simulated annealing To name a few...

28. SigOpt: Bayesian Optimization Service Three API calls: 1. Define hyperparameters 2. Receive suggested hyperparameters 3. Report observed performance

29. Thank You!

30. Intro Ian Dewancker. SigOpt for ML: TensorFlow ConvNets on a Budget with Bayesian Optimization. Ian Dewancker. SigOpt for ML: Unsupervised Learning with Even Less Supervision Using Bayesian Optimization. Ian Dewancker. SigOpt for ML : Bayesian Optimization for Collaborative Filtering with MLlib. #1 Trusting the Defaults Keras recurrent layers documentation #2 Using the Wrong Metric Ron Kohavi et al. Trustworthy Online Controlled Experiments: Five Puzzling Outcomes Explained. Xavier Amatriain. 10 Lessons Learning from building ML systems [Video at 19:03]. Image from PhD Comics. See also: SigOpt in Depth: Intro to Multicriteria Optimization. #4 Too Few Hyperparameters Image from TensorFlow Playground. Ian Dewancker. SigOpt for ML: Unsupervised Learning with Even Less Supervision Using Bayesian Optimization. #5 Hand Tuning On algorithms beating experts: Scott Clark, Ian Dewancker, and Sathish Nagappan. Deep Neural Network Optimization with SigOpt and Nervana Cloud. #6 Grid Search NoGridSearch.com References - by Section

31. References - by Section #7 Random Search James Bergstra and Yoshua Bengio. Random search for hyper-parameter optimization. Ian Dewancker, Michael McCourt, Scott Clark, Patrick Hayes, Alexandra Johnson, George Ke. A Stratified Analysis of Bayesian Optimization Methods. Learn More blog.sigopt.com sigopt.com/research

Common Problems in Hyperparameter Optimization

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (6)

Ähnlich wie Common Problems in Hyperparameter Optimization

Ähnlich wie Common Problems in Hyperparameter Optimization (20)

Mehr von SigOpt

Mehr von SigOpt (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Common Problems in Hyperparameter Optimization