JUDCon London 2011 - Bin packing with drools planner by example
•Download as ODP, PDF•
0 likes•2,834 views
Scheduling processes into cloud servers.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to JUDCon London 2011 - Bin packing with drools planner by example
Similar to JUDCon London 2011 - Bin packing with drools planner by example (20)
More from Geoffrey De Smet
More from Geoffrey De Smet (20)
Recently uploaded
Recently uploaded (20)
JUDCon London 2011 - Bin packing with drools planner by example
- 2. Geoffrey De Smet Bin packing with Drools Planner Scheduling processes on cloud servers
- 3. Every organization has planning problems.
- 15. Per teacher
- 22. Minimize server cost http://www.flickr.com/photos/torkildr/3462607995/
- 24. Minimize server cost http://www.flickr.com/photos/torkildr/3462607995/
- 25. Which processes do we assign to this server?
- 26. How did we find that solution?
- 27. First fit by decreasing size
- 28. First fit by decreasing size
- 29. First fit by decreasing size
- 30. First fit by decreasing size
- 31. Another case
- 32. Try FFD again
- 33. Try FFD again
- 34. Try FFD again
- 35. Try FFD again
- 36. FFD failure
- 37. NP complete
- 40. Organizations rarely optimize planning problems.
- 51. Open source (ASL 2.0)
- 53. Demo Drools Planner CloudBalance example
- 54. Domain model
- 55. Server
- 56. Server public class Server { private int cpuPower ; private int memory ; private int networkBandwidth ; private int cost ; // getters }
- 57. Process
- 58. Process
- 59. Process is a planning entity @PlanningEntity public class Process { private int requiredCpuPower ; private int requiredMemory ; private int requiredNetworkBandwidth ; ... // getters, clone, equals, hashcode
- 60. Process has a planning variable @PlanningEntity public class Process { ... private Server server ; @PlanningVariable @ValueRangeFromSolutionProperty ( propertyName = "serverList" ) public Server getServer() { return server ;} public void setServer(Server server) { ... } }
- 61. CloudBalance
- 62. Solution CloudBalance: problem facts public class CloudBalance implements Solution<HardAndSoftScore> { private List<Server> serverList ; public List<Server> getServerList() { return serverList ; } ... // clone, equals, hashcode
- 63. Solution CloudBalance: planning entities public class CloudBalance implements Solution<HardAndSoftScore> { ... private List<Process> processList ; @PlanningEntityCollectionProperty public List<Process> getProcessList() { return processList ; } ...
- 64. Solution CloudBalance: getProblemFacts public class CloudBalance implements Solution<HardAndSoftScore> { ... // Used in score constraints public Collection<Object> getProblemFacts() { List<Object> facts = new ArrayList<Object>(); facts.addAll( serverList ); return facts; } ...
- 65. Solution CloudBalance: score public class CloudBalance implements Solution<HardAndSoftScore> { ... private HardAndSoftScore score ; public HardAndSoftScore getScore() { ... } public void setScore(HardAndSoftScore score) { ... } }
- 67. Each Solution has 1 Score
- 68. Each Solution has 1 Score
- 69. Better score => better solution
- 70. Better score => better solution
- 71. Best score => best solution
- 72. Best score => best solution
- 76. Soft constraint: server cost rule "serverCost" when // there is a server $s : Server( $c : cost) // there is a processes on that server exists Process(server == $s ) then // lower soft score by $c insertLogical( new IntConstraintOccurrence( "serverCost" , ConstraintType. NEGATIVE_SOFT , $c , $s )); end
- 77. Hard constraint: CPU power rule "requiredCpuPowerTotal" when // there is a server $s : Server( $cpu : cpuPower) // with too little cpu for its processes $total : Number(intValue > $cpu ) from accumulate ( Process(server == $s , $requiredCpu : requiredCpuPower), sum( $requiredCpu ) ) then // lower hard score by ($total - $cpu) end
- 78. Solver configuration by XML < solver > < solutionClass > ... CloudBalance</ solutionClass > < planningEntityClass > ... Process</> < scoreDrl > ... ScoreRules.drl</ scoreDrl > < scoreDefinition > < scoreDefinitionType >HARD_AND_SOFT</> </ scoreDefinition > <!-- optimization algorithms -> ... </ solver >
- 79. Solving XmlSolverConfigurer config = new XmlSolverConfigurer( " ... SolverConfig.xml" ); Solver solver = config.buildSolver(); solver.setPlanningProblem( cloudBalance ); solver.solve(); cloudBalance = ( CloudBalance) solver.getBestSolution();
- 81. Brute Force
- 82. Brute Force config < solver > ... < bruteForce /> </ solver >
- 83. Brute force scalability 6 processes = 15 ms 9 processes = 1.5 seconds * 100
- 84. Brute force scalability 12 processes = 17 minutes 6 processes = 15 ms 9 processes = 1.5 seconds * 100 * 680
- 85. Plan 1200 processes with brute force?
- 86. First Fit
- 87. First Fit config < solver > ... < constructionHeuristic > < constructionHeuristicType >FIRST_FIT</> </ constructionHeuristic > </ solver >
- 89. First Fit results CPU: OK RAM: OK Network: OK Cost active servers: shown
- 91. First Fit Decreasing config < solver > ... < constructionHeuristic > < constructionHeuristicType >FIRST_FIT_DECREASING</> </ constructionHeuristic > </ solver >
- 92. DifficultyComparator public class ProcessDifficultyComparator implements Comparator<Process> { public int compare(Process a, Process b) { // Compare on requiredCpuPower * requiredMemory // * requiredNetworkBandwidth } } @PlanningEntity (difficultyComparatorClass = ProcessDifficultyComparator. class ) public class Process { ... }
- 93. First Fit Decreasing scalability
- 94. First Fit Decreasing results
- 95. Local Search
- 96. Local Search comes after Construction Heuristics < solver > ... < constructionHeuristic > < constructionHeuristicType >FIRST_FIT_DECREASING</> </ constructionHeuristic > < localSearch > ... < localSearch > </ solver >
- 97. Local Search needs to be terminated < solver > ... < termination > < maximumMinutesSpend >20</ maximumMinutesSpend > </ termination > ... </ solver >
- 98. Selecting moves < localSearch > < selector > < selector > < moveFactoryClass >GenericChangeValueMoveFactory</> </ selector > < selector > < moveFactoryClass >GenericSwitchAllValuesMF</> </ selector > </ selector > ... tabu search, simulated annealing or ... </ localSearch >
- 99. Tabu Search < localSearch > < selector > ... </ selector > < acceptor > <!-- Untweaked standard values --> < solutionTabuSize >1000</ solutionTabuSize > < propertyTabuSize >7</ propertyTabuSize > </ acceptor > < forager > <!-- Untweaked standard value --> < minimalAcceptedSelection >1000</> </ forager > </ localSearch >
- 100. Or Simulated Annealing < localSearch > < selector > ... </ selector > < acceptor > <!-- Tweaked value --> < simulatedAnnealingStartingTemperature > 0hard/400soft</> </ acceptor > < forager > <!-- Untweaked standard value --> < minimalAcceptedSelection >4</> </ forager > </ localSearch >
- 103. Organizations rarely optimize planning problems.
- 105. Real-time planning
- 107. Demo Real-time planning CloudBalance example
- 108. Summary
- 110. Adding constraints is easy and scalable
- 111. Switching/combining algorithms is easy
- 112. Try an example!
- 114. Google+: Geoffrey De Smet