Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Andere mochten auch
Andere mochten auch (7)
Ähnlich wie WWW 2012
Ähnlich wie WWW 2012 (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
WWW 2012
- 1. Tamas Jambor, Jun Wang, Neal Lathia (@jamborta,@seawan,@neal_lathia) University College London
- 2. Recommender systems exists in time. Many aspects of the system can only be understood over time.
- 3. Understanding and controlling dynamic aspects of the systems to gain control over certain objectives.
- 4. Predictability over time Allocating resources over time Stable system over time
- 5. Describe and estimate the system dynamics Build and analyze a feedback loop controller Test the dynamics with real data
- 6. More data -> better performance Frequent training -> Up-to-date recommendation
- 7. Performance loss over time
- 8. Use all the data -> best performance Use part of the data -> frequent and regular updates
- 9. Relationship between the number of training samples (computation) and performance.
- 10. How certain input now would change the future state of the system.
- 11. r m F(t) k y(t) y = displacement F = force applied m = mass of the block r = damping constant k = spring constant
- 12. Using Newton’s second law F (t ) = my ' ' (t ) ◦ Spring force: f1 = ky (t ) r ◦ Damping force: f 2 = ry ' (t ) m F(t) This gives the equation k y(t) F (t ) = ky (t ) + ry ' (t )
- 13. r F(t) k y(t) F (t ) = ky (t ) + ry ' (t ) y = performance F = number of training samples k = constant related to the new training samples r = constant related to the change in the new training samples
- 14. Generate an input signal Log-normal random walk to add noise Maximum likelihood estimation
- 15. Build a control loop based on system dynamics
- 16. Controlled Recommender System Controller Manipulated Variable Control control (e.g., number of input Actuator new training samples) Function Dynamics error +- Monitor Performance variable Reference performance
- 17. Reference Value Disturbance yr (t ) e(t ) = yr (t ) − y (t ) n(t ) + e(t ) u (t ) Performance ∑ Controller Dynamics Rec. Error Control y (t ) - Signal y (t ) Rec. Accuracy Measurement
- 19. Rise time Settling time Overshoot
- 20. Type Rise time Settling time Overshoot (%) (sec) (sec) P 5 10 0 PD 1 5 8.21 PI 0 10 36.8 PID 0 10 15.7 P controller PD controller PI controller PID controller
- 21. How the controller can perform with unseen data.
- 22. Simulate system for 30 days (once a day) 3 reference values (RMSE 0.95, 0.9, 0.86) Measure ◦ Settling time ◦ Error (with respect to reference) – RMSE-SS ◦ Stability (with respect to reference) – SD-SS ◦ Overshoot
- 23. Type Settling RMSE-SS SD-SS Overshoot time (error) (stability) (%) P (0.86) 23 0.0104 0.0034 0 PD (0.86) 20.8 0.0059 0.0037 0.18 PI (0.86) 18.6 0.0055 0.0047 0.49 PID (0.86) 18.6 0.0045 0.0045 0.72 PD controller
- 24. Type Settling RMSE-SS SD-SS Overshoot time (error) (stability) (%) P 20 0.0068 0.0052 0.50 PD 18.8 0.0054 0.0055 0.98 PI 20.6 0.0137 0.0075 2.50 PID 17.8 0.0091 0.0059 1.86 P controller PD controller PI controller PID controller
- 25. Computation Performance loss Computational gain
- 26. Fixed computational cost would also fix update frequency (given the resources available) System can provide regular updates independent from the rate of the new samples Resources are used to their limits (e.g. EC2)
- 27. RECOMMENDATION recommendation SERVER usage data RECOMMENDER INPUT CONTROLLER feedback TRAINING DATA PROCESSOR RECOMMENDATION request data ASSESSOR USAGE DATA PROCESSOR
- 28. Certain aspects of the system are to be controlled Stability and predictability are important Use resources to their limits
- 29. Optimise rather than stabilise over time Obtain system dynamics Describe objective to be achieved (finite or infinite horizon)
- 30. Describe system dynamics Design a feedback loop Evaluate control system on real data
- 31. Thank you.
Hinweis der Redaktion
- Most of the time systems is viewed and evaluated in a static context Snapshot at time t How the system evolves Items are changing People are changing Change the outcome
- Understand the system dynamics Select important aspects of the system Gain control over certain objectives
- Understand and describe system -> predict over time Example: we know the system at time t, we know the change, we know what it will be at t+1 Organise resources accordingly Controlling the system brings stability
- Describe Select Build feedback Add controller Test
- Performance loss is calculated with respect to training four times a day. Red line twice a day – midnight and midday Blue line once a day – only at midnight
- Use all and train whenever you can Part of data would give more frequent and regular updates
- the relationship between the number of training samples and the overall performance Number of training samples related to performance Gain control performance
- Balance the mass to keep the distance from the wall constant Pull and release
- The acceleration a of a body is parallel and directly proportional to the net force F and inversely proportional to the mass m First order differential equation
- The analogy to recommender systems is that Y is the performance of the system K – new training samples R – change in new training samples
- Explain input and put through a recsys for 30 days Three scenarios, new data samples are increasing, deceasing and stable (showing the increasing one). Learn parameter R-squared, the coefficient of determination, to measure the accuracy of the prediction.
- General concept Dynamics
- Three components
- Rise time Settling time Overshoot – maximum percentage the system might go over the reference
- Based on analysis Picked to get the fastest rise time Rise time – PI, PID Overshoot – PI, PID not really good PD is the best
- Analysis and results are consistent Settling time – best PI and PID Overshoot – highest for PID and PI P is zero Overall PD has the best properties
- Reduce performance would reduce computation Red line – performance loss, computational gain Blue line – performance loss, computational gain once the required performance has reached, the computational cost becomes stable.
- EC2 pay per instance, use it all but keep regular updates
- How it works now - subscription and purchase is used Data that can be collected – PVR recording logs, subscription packages, purchase data, live TV watch, Facebook shares, content information Identify the usefulness of the signal per user basis Enhance recommendation by requesting additional information Reduce data and computation
- Certain aspects – need to fix aspects of the system Stability would bring predictability More optimal resource allocation
- First two steps are similar Example here – optimise performance over time Can be used for cold start problem Good quality recommendation from the beginning
- Sequence of design Dynamics – feedback loop (analytical) Put into production – evaluate on real data Plug-in different dynamics and different controllers