SlideShare ist ein Scribd-Unternehmen logo
1 von 93
Downloaden Sie, um offline zu lesen
The Hiring Problem:
 Going Beyond Secretaries

  Sergei Vassilvitskii (Yahoo!)

    Andrei Broder (Yahoo!)
    Adam Kirsch (Harvard)
      Ravi Kumar (Yahoo!)
Michael Mitzenmacher (Harvard)
       Eli Upfal (Brown)
The Secretary Problem
Interview n candidates for a position one at a time. After each
interview decide if the candidate is the best.
Goal: maximize the probability of choosing the best candidate.
The Secretary Problem
Interview n candidates for a position one at a time. After each
interview decide if the candidate is the best.
Goal: maximize the probability of choosing the best candidate.
The Secretary Problem
Interview n candidates for a position one at a time. After each
interview decide if the candidate is the best.
Goal: maximize the probability of choosing the best candidate.
The Secretary Problem
Interview n candidates for a position one at a time. After each
interview decide if the candidate is the best.
Goal: maximize the probability of choosing the best candidate.
The Secretary Problem
Interview n candidates for a position one at a time. After each
interview decide if the candidate is the best.
Goal: maximize the probability of choosing the best candidate.
The Secretary Problem
Interview n candidates for a position one at a time. After each
interview decide if the candidate is the best.
Goal: maximize the probability of choosing the best candidate.
The Secretary Problem
Interview n candidates for a position one at a time. After each
interview decide if the candidate is the best.
Goal: maximize the probability of choosing the best candidate.
The Secretary Problem
Interview n candidates for a position one at a time. After each
interview decide if the candidate is the best.
Goal: maximize the probability of choosing the best candidate.
The Secretary Problem
Interview n candidates for a position one at a time. After each
interview decide if the candidate is the best.
Goal: maximize the probability of choosing the best candidate.
The Secretary Problem
Interview n candidates for a position one at a time. After each
interview decide if the candidate is the best.
Goal: maximize the probability of choosing the best candidate.




This is not about hiring secretaries, but about decision
making under uncertainty.
The Hiring Problem
The Hiring Problem
A startup is growing and is hiring many employees:
  Want to hire good employees
  Can’t wait for the perfect candidate
The Hiring Problem
A startup is growing and is hiring many employees:
  Want to hire good employees
  Can’t wait for the perfect candidate
Many potential objectives.
  Explore the tradeoff between number of interviews & the
  average quality.
The Hiring model
Candidates arrive one at a time.
Assume all have iid uniform(0,1) quality scores - For
applicant i denote it by iq .
  (Can deal with other distributions, not this talk)
The Hiring model
Candidates arrive one at a time.
Assume all have iid uniform(0,1) quality scores - For
applicant i denote it by iq .
  (Can deal with other distributions, not this talk)
During the interview:
  Observe iq
  Decide whether to hire or reject
Strategies
Strategies
Hire above a threshold.
Strategies
Hire above a threshold.
Hire above the minimum or maximum.
Strategies
Hire above a threshold.
Hire above the minimum or maximum.
Lake Wobegon Strategies:
  “Lake Wobegon: where all the women are strong, all the
  men are good looking, and all the children are above
  average”
Strategies
Hire above a threshold.
Hire above the minimum or maximum.
Lake Wobegon Strategies:
  Hire above the average (mean or median)
Strategies
Hire above a threshold.
Hire above the minimum or maximum.
Lake Wobegon Strategies:
  Hire above the average
Side note: [Google Research Blog - March ‘06]:
  “... only hire candidates who are above the mean of the
  current employees...”
Threshold Hiring
Set a threshold t , hire if iq ≥ t .
Threshold Hiring
Set a threshold t , hire if iq ≥ t .
Threshold Hiring
Set a threshold t , hire if iq ≥ t .

     t
Threshold Hiring
Set a threshold t , hire if iq ≥ t .

     t
Threshold Hiring
Set a threshold t , hire if iq ≥ t .

     t
Threshold Hiring
Set a threshold t , hire if iq ≥ t .

     t                 t
Threshold Hiring
Set a threshold t , hire if iq ≥ t .

     t                 t
Threshold Hiring
Set a threshold t , hire if iq ≥ t .

     t                 t
Threshold Hiring
Set a threshold t , hire if iq ≥ t .

     t                 t               t
Threshold Hiring
Set a threshold t , hire if iq ≥ t .

     t                 t               t
Threshold Hiring
Set a threshold t , hire if iq ≥ t .

     t                 t               t
Threshold Analysis
Set a threshold t , hire if iq ≥ t .

                                            1+t
Easy to see that average quality approaches
              1                              2
Hiring rate       .
            1−t
Threshold Analysis
Set a threshold t , hire if iq ≥ t .

                                            1+t
Easy to see that average quality approaches
              1                              2
Hiring rate       .
            1−t




Quality stagnates and does not increase with time.
Maximum Hiring
Hire only if better than everyone already hired.
Maximum Hiring
Hire only if better than everyone already hired.
Maximum Hiring
Hire only if better than everyone already hired.
Maximum Hiring
Hire only if better than everyone already hired.
Maximum Hiring
Hire only if better than everyone already hired.
Maximum Hiring
Hire only if better than everyone already hired.
Maximum Hiring
Hire only if better than everyone already hired.
Maximum Hiring
Hire only if better than everyone already hired.
Maximum Hiring
Hire only if better than everyone already hired.
Maximum Hiring
Hire only if better than everyone already hired.
Maximum Hiring
Hire only if better than everyone already hired.
Maximum Hiring
Hire only if better than everyone already hired.
Maximum Hiring
Hire only if better than everyone already hired.
Maximum Hiring
Hire only if better than everyone already hired.
Maximum Analysis
Start with employee of quality q
Let hi be the i-th candidate hired
Maximum Analysis
Start with employee of quality q
Let hi be the i-th candidate hired
Focus on the gap: gi = 1 − (hi )q    gi
Maximum Analysis
Start with employee of quality q
Let hi be the i-th candidate hired
Focus on the gap: gi = 1 − (hi )q    gi


Conditioned on gn−1 :
Maximum Analysis
Start with employee of quality q
Let hi be the i-th candidate hired
Focus on the gap: gi = 1 − (hi )q    gi


Conditioned on gn−1 :
      gn ∼ U nif (0, gn−1 )
Maximum Analysis
Start with employee of quality q
Let hi be the i-th candidate hired
Focus on the gap: gi = 1 − (hi )q    gi


Conditioned on gn−1 :
      gn ∼ U nif (0, gn−1 )
                    gn−1
     E[gn |gn−1 ] =
                      2
Maximum Analysis
Start with employee of quality q
Let hi be the i-th candidate hired
Focus on the gap: gi = 1 − (hi )q    gi


Conditioned on gn−1 :
      gn ∼ U nif (0, gn−1 )
                    gn−1
     E[gn |gn−1 ] =
                      2
              1−q
     E[gn ] = n
                2
Maximum Analysis
Start with employee of quality q
Let hi be the i-th candidate hired
Focus on the gap: gi = 1 − (hi )q                  gi


Conditioned on gn−1 :
      gn ∼ U nif (0, gn−1 )
                    gn−1
     E[gn |gn−1 ] =
                      2
              1−q
     E[gn ] = n               Very high quality!
                2
Maximum Analysis
Start with employee of quality q
Let hi be the i-th candidate hired
Focus on the gap: gi = 1 − (hi )q                      gi


Conditioned on gn−1 :
      gn ∼ U nif (0, gn−1 )
                    gn−1
     E[gn |gn−1 ] =
                      2
              1−q
     E[gn ] = n               Extremely slow hiring!
                2
Lake Wobegon Strategies
Lake Wobegon Strategies
 Above the mean:
                                       1
   Average quality after n hires: 1 − √
                                        n
Lake Wobegon Strategies
 Above the mean:
                                       1
   Average quality after n hires: 1 − √
                                        n
 Above the median:
                                      1
   Median quality after n hires: 1 −
                                      n
Lake Wobegon Strategies
 Above the mean:
                                       1
   Average quality after n hires: 1 − √
                                        n
 Above the median:
                                      1
   Median quality after n hires: 1 −
                                      n

 Surprising:
   Tight concentration is not possible
   Hiring above mean converges to a log-normal distribution
Hiring Above Mean
Start with employee of quality q
Let hi be the i-th candidate hired
Focus on the gap: gi = 1 − (hi )q    gi
Hiring Above Mean
Start with employee of quality q
Let hi be the i-th candidate hired
Focus on the gap: gi = 1 − (hi )q                      gi


Conditioned on gn    : (in+1 )q ∼ U nif (1 − gn , 1)
Hiring Above Mean
Start with employee of quality q
Let hi be the i-th candidate hired
Focus on the gap: gi = 1 − (hi )q                       gi


Conditioned on gn     : (in+1 )q ∼ U nif (1 − gn , 1)
                        n+1       1
Therefore:     gn+1   ∼     gn +     U nif (0, gn )
                        n+2      n+2
Hiring Above Mean
Start with employee of quality q
Let hi be the i-th candidate hired
Focus on the gap: gi = 1 − (hi )q                       gi


Conditioned on gn     : (in+1 )q ∼ U nif (1 − gn , 1)
                        n+1       1
Therefore:     gn+1   ∼     gn +     U nif (0, gn )
                        n+2      n+2
                                U nif (0, 1)
                      = gn   1−
                                   n+2
Hiring Above Mean
               U nif (0, 1)
gn+1 = gn   1−
                  n+2
Hiring Above Mean
               U nif (0, 1)
gn+1 = gn   1−
                  n+2
                     t
                             U nif (0, 1)
Expand: gn+t = gn         1−
                    i=1
                             n+i+1
Hiring Above Mean
                 U nif (0, 1)
gn+1 = gn     1−
                    n+2
                      t
                                U nif (0, 1)
Expand: gn+t = gn            1−
                     i=1
                                n+i+1


Therefore:
                            n
                                     U nif (0, 1)
             gn ∼ (1 − q)         1−
                            i=1
                                        i+1
Hiring Above Mean
                 U nif (0, 1)
gn+1 = gn     1−
                    n+2
                      t
                                U nif (0, 1)
Expand: gn+t = gn            1−
                     i=1
                                n+i+1


Therefore:
                            n
                                     U nif (0, 1)
             gn ∼ (1 − q)         1−
                            i=1
                                        i+1

                            n
                                         1            1
       E[gn ] = (1 − q)           1−            = Θ( √ )
                            i=1
                                     2(i + 1)          n
Hiring Above Mean
Conclusion:
                                         1
  Average quality after n hires: 1 − Θ( √ )
                                          n

  Time to hire n employees: Θ(n3/2 )


  Very weak concentration results.
Hiring Above Median
Start with employee of quality q
When we have 2k + 1 employees.
  Compute median mk of the scores
  Hire next 2 applicants with scores above mk
Hiring Above Median


      mk
Hiring Above Median


             mk




New hires:        with quality above mk
Hiring Above Median


      mk
Hiring Above Median


        mk+1
Hiring Above Median


      mk
Hiring Above Median


                mk



Inductive hypothesis:   U nif (mk , 1)
Hiring Above Median


                 mk



 Inductive hypothesis:      U nif (mk , 1)




New hires:               are U nif (mk , 1)
Hiring Above Median


      mk



             U nif (mk , 1)
Hiring Above Median


        mk+1

               U nif (mk , 1)
Hiring Above Median


        mk+1


               U nif (mk+1 , 1)
Hiring Above Median


                    mk+1


                           U nif (mk+1 , 1)
 Induction holds.
Hiring Above Median


        mk+1

               U nif (mk , 1)
Hiring Above Median


              mk+1

                     U nif (mk , 1)

The median:
Hiring Above Median


                            mk+1

                                          U nif (mk , 1)

The median: smallest of k + 1 uniform r.v., each U nif (mk , 1)
Hiring Above Median


                            mk+1

                                          U nif (mk , 1)

The median: smallest of k + 1 uniform r.v., each U nif (0, gk )
Hiring Above Median


                            mk+1

                                          U nif (mk , 1)

The median: smallest of k + 1 uniform r.v., each U nif (0, gk )

                     gk+1 |gk ∼ gk Beta(k + 2, 1)
Hiring Above Median
                                  k

Expand (like the means): gk ∼ g         Beta(i + 1, 1)
                                  i=1
Hiring Above Median
                                  k

Expand (like the means): gk ∼ g         Beta(i + 1, 1)
                                  i=1

                                 1
                      E[gn ] = Θ( )
                                 n
Hiring Above Median
                                  k

Expand (like the means): gk ∼ g         Beta(i + 1, 1)
                                  i=1

                                  1
                       E[gn ] = Θ( )
                                  n
                                                    log n
Caveat: this is the median gap! The mean gap is: Θ(       )
                                                      n
Hiring Above Median
Conclusion:
                                        log n
  Average quality after n hires: 1 − Θ(       )
                                          n

  Time to hire n employees: Θ(n2 )


  Again, very weak concentration results.
Extensions
Interview preprocessing:
  Self selection: quality may increase over time
Errors:
  Noisy estimates on iq scores.
Firing:
  Periodically fire bottom 10% (Jack Welch Strategy)
  Similar analysis to the median.
Many more...
Thank You

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Parallel Algorithms
Parallel AlgorithmsParallel Algorithms
Parallel Algorithms
 
daa-unit-3-greedy method
daa-unit-3-greedy methoddaa-unit-3-greedy method
daa-unit-3-greedy method
 
Dynamic programming
Dynamic programmingDynamic programming
Dynamic programming
 
Randomized algorithms ver 1.0
Randomized algorithms ver 1.0Randomized algorithms ver 1.0
Randomized algorithms ver 1.0
 
Job sequencing with deadline
Job sequencing with deadlineJob sequencing with deadline
Job sequencing with deadline
 
2-Approximation Vertex Cover
2-Approximation Vertex Cover2-Approximation Vertex Cover
2-Approximation Vertex Cover
 
Daa notes 3
Daa notes 3Daa notes 3
Daa notes 3
 
Traveling salesman problem
Traveling salesman problemTraveling salesman problem
Traveling salesman problem
 
Activity selection problem
Activity selection problemActivity selection problem
Activity selection problem
 
All pair shortest path
All pair shortest pathAll pair shortest path
All pair shortest path
 
Semaphores
SemaphoresSemaphores
Semaphores
 
Shortest Path in Graph
Shortest Path in GraphShortest Path in Graph
Shortest Path in Graph
 
Searching Algorithm
Searching AlgorithmSearching Algorithm
Searching Algorithm
 
9. chapter 8 np hard and np complete problems
9. chapter 8   np hard and np complete problems9. chapter 8   np hard and np complete problems
9. chapter 8 np hard and np complete problems
 
0/1 knapsack
0/1 knapsack0/1 knapsack
0/1 knapsack
 
Time and space complexity
Time and space complexityTime and space complexity
Time and space complexity
 
Greedy method by Dr. B. J. Mohite
Greedy method by Dr. B. J. MohiteGreedy method by Dr. B. J. Mohite
Greedy method by Dr. B. J. Mohite
 
Regular expressions
Regular expressionsRegular expressions
Regular expressions
 
Dijkstra’s algorithm
Dijkstra’s algorithmDijkstra’s algorithm
Dijkstra’s algorithm
 
String matching, naive,
String matching, naive,String matching, naive,
String matching, naive,
 

Andere mochten auch

"Culture eats strategy for breakfast"
"Culture eats strategy for breakfast""Culture eats strategy for breakfast"
"Culture eats strategy for breakfast"Tobias Dahlberg
 
Bubble sort
Bubble sortBubble sort
Bubble sortBadge301
 
Randomized Algorithms
Randomized AlgorithmsRandomized Algorithms
Randomized AlgorithmsKetan Kamra
 
Topic 1.4: Randomized Algorithms
Topic 1.4: Randomized AlgorithmsTopic 1.4: Randomized Algorithms
Topic 1.4: Randomized AlgorithmsKM Bappi
 
Culture eats Technology
Culture eats TechnologyCulture eats Technology
Culture eats TechnologyMax Jester
 
Recruiting and Selection Essentials
Recruiting and Selection EssentialsRecruiting and Selection Essentials
Recruiting and Selection EssentialsBen Eubanks
 
Hiring in Today's Digital Age
Hiring in Today's Digital AgeHiring in Today's Digital Age
Hiring in Today's Digital AgeCoalmarch
 
Recruiting + Hiring with Social Media | HR Handbook Series
Recruiting + Hiring with Social Media | HR Handbook SeriesRecruiting + Hiring with Social Media | HR Handbook Series
Recruiting + Hiring with Social Media | HR Handbook SeriesHarlan/Thompson HR
 
managing labor relations
managing labor relationsmanaging labor relations
managing labor relationszeba khan
 
Company Formation: Hiring
Company Formation: HiringCompany Formation: Hiring
Company Formation: HiringMichael Skok
 
A Guide to Hiring for your Startup
A Guide to Hiring for your StartupA Guide to Hiring for your Startup
A Guide to Hiring for your StartupYevgeniy Brikman
 
Is Your Culture Eating Your Strategy for Breakfast?
Is Your Culture Eating Your Strategy for Breakfast?Is Your Culture Eating Your Strategy for Breakfast?
Is Your Culture Eating Your Strategy for Breakfast?McKinley Advisors
 
Centralization and Decentralization
Centralization and DecentralizationCentralization and Decentralization
Centralization and DecentralizationTejas Atyam
 
Recruiting Metrics - Strategic and Tactical KPIs for Talent Acquisition
Recruiting Metrics - Strategic and Tactical KPIs for Talent AcquisitionRecruiting Metrics - Strategic and Tactical KPIs for Talent Acquisition
Recruiting Metrics - Strategic and Tactical KPIs for Talent AcquisitionMaia Josebachvili
 
Culture Eats Strategy for Breakfast - Greenspot by DartGroup Amsterdam - Cont...
Culture Eats Strategy for Breakfast - Greenspot by DartGroup Amsterdam - Cont...Culture Eats Strategy for Breakfast - Greenspot by DartGroup Amsterdam - Cont...
Culture Eats Strategy for Breakfast - Greenspot by DartGroup Amsterdam - Cont...Sirous Kavehercy
 
Recruitment & selection
Recruitment & selectionRecruitment & selection
Recruitment & selectionNISHA SHAH
 

Andere mochten auch (20)

"Culture eats strategy for breakfast"
"Culture eats strategy for breakfast""Culture eats strategy for breakfast"
"Culture eats strategy for breakfast"
 
Recruitment
RecruitmentRecruitment
Recruitment
 
Bubble sort
Bubble sortBubble sort
Bubble sort
 
Randomized Algorithms
Randomized AlgorithmsRandomized Algorithms
Randomized Algorithms
 
Topic 1.4: Randomized Algorithms
Topic 1.4: Randomized AlgorithmsTopic 1.4: Randomized Algorithms
Topic 1.4: Randomized Algorithms
 
Culture eats Technology
Culture eats TechnologyCulture eats Technology
Culture eats Technology
 
Recruiting and Selection Essentials
Recruiting and Selection EssentialsRecruiting and Selection Essentials
Recruiting and Selection Essentials
 
Randomized Algorithm
Randomized AlgorithmRandomized Algorithm
Randomized Algorithm
 
Hiring in Today's Digital Age
Hiring in Today's Digital AgeHiring in Today's Digital Age
Hiring in Today's Digital Age
 
Recruiting + Hiring with Social Media | HR Handbook Series
Recruiting + Hiring with Social Media | HR Handbook SeriesRecruiting + Hiring with Social Media | HR Handbook Series
Recruiting + Hiring with Social Media | HR Handbook Series
 
managing labor relations
managing labor relationsmanaging labor relations
managing labor relations
 
Company Formation: Hiring
Company Formation: HiringCompany Formation: Hiring
Company Formation: Hiring
 
P&G Hiring Process
P&G Hiring ProcessP&G Hiring Process
P&G Hiring Process
 
A Guide to Hiring for your Startup
A Guide to Hiring for your StartupA Guide to Hiring for your Startup
A Guide to Hiring for your Startup
 
Job design
Job designJob design
Job design
 
Is Your Culture Eating Your Strategy for Breakfast?
Is Your Culture Eating Your Strategy for Breakfast?Is Your Culture Eating Your Strategy for Breakfast?
Is Your Culture Eating Your Strategy for Breakfast?
 
Centralization and Decentralization
Centralization and DecentralizationCentralization and Decentralization
Centralization and Decentralization
 
Recruiting Metrics - Strategic and Tactical KPIs for Talent Acquisition
Recruiting Metrics - Strategic and Tactical KPIs for Talent AcquisitionRecruiting Metrics - Strategic and Tactical KPIs for Talent Acquisition
Recruiting Metrics - Strategic and Tactical KPIs for Talent Acquisition
 
Culture Eats Strategy for Breakfast - Greenspot by DartGroup Amsterdam - Cont...
Culture Eats Strategy for Breakfast - Greenspot by DartGroup Amsterdam - Cont...Culture Eats Strategy for Breakfast - Greenspot by DartGroup Amsterdam - Cont...
Culture Eats Strategy for Breakfast - Greenspot by DartGroup Amsterdam - Cont...
 
Recruitment & selection
Recruitment & selectionRecruitment & selection
Recruitment & selection
 

The Hiring Problem