Slides of talk at the Workshop on ICT for Food Quality and Healthy Nutrition

1. Personalized food recommendations: combining
recommendation, visualization and augmented reality
techniques for healthier food decision-making
1
https://augment.cs.kuleuven.be/
Katrien Verbert, Francisco Gutiérrez
Augment/HCI - @katrien_v

2. recommender systems – visualization – intelligent user interfaces
Learning analytics
Media
consumption
Research Information
Systems
Food &
health
Augment prof. Katrien Verbert
ARIA prof. Adalberto Simeone
Computer
Graphics
prof. Phil Dutré
Language
Intelligence &
Information
Retrieval
prof. Sien Moens
Motivational
design
Prof. Vero Vanden Abeele
Accessibility Prof. Kathrin Gerling
Human-computer interaction group

3. Augment/HCI team
Robin De Croon
Postdoc researcher
Katrien Verbert
Associate Professor
Francisco Gutiérrez
Postdoc researcher
Tom Broos
PhD researcher
Martijn Millecamp
PhD researcher
Nyi Nyi Htun
Postdoc researcher
Houda Lamqaddam
PhD researcher
Jeroen Ooge
PhD researcher
Oscar Alvarado
PhD researcher
http://augment.cs.kuleuven.be/
Diego Rojo Carcia
PhD researcher
Leen Van Houdt
PhD researcher

4. Collaborative filtering – Content-based filtering –
Knowledge-based filtering - Hybrid
Recommendation techniques

5. Two ways to recommend healthier food choices
5
Recipes
recommend recipes for healthier meal
(or meal plans / sets of recipes)
Food items
suggest healthier food items themselves

6. 6Trattner, C., & Elsweiler, D. (2017). Food recommender systems: important contributions,
challenges and future research directions. arXiv preprint arXiv:1711.02760.
Survey of 18 food
recommender systems
Wide variety of CB, KB, CF,
hybrid techniques
Datasets: Allrecipes,
Yummly, Kochbar ….

7. Challenge 1: users tend to like unhealthy recipes
7
Trattner, C., & Elsweiler, D. (2017). Food recommender systems: important contributions,
challenges and future research directions. arXiv preprint arXiv:1711.02760.

8. Our approach: take into account nutritional scores
8
Gutiérrez, F., Htun, N. N., Charleer, S., De Croon, R., & Verbert, K. (2019, January). Designing
Augmented Reality Applications for Personal Health Decision-Making. In Proceedings of the 52nd
Hawaii International Conference on System Sciences.

9. Challenge 2: motivating users
Users need to be motivated to select healthier products / recipes
Our approach: visualize nutritional facts and impact on weight using a
predictive model
9

10. Challenge 3: need for in-situ recommendations
Recommendations and visualizations need to be delivered at critical moment of
decision: when users hold the food product in their hands
Our approach: using augmented reality techniques to support decision-making
in grocery stores
10

11. 11
Food labels are
considered an
essential element in
strategies against
unhealthy diets and
obesity. (Cecchini and
Warin, 2016)

12. 12

13. 13
PHARA: Personal Health Augmented Reality assistant

14. 14
https://wms.cs.kuleuven.be/cs/onderzoek/augment/demos

15. 15
Iterative design
Gutiérrez, F., Cardoso, B. D. L. R. P., & Verbert, K. (2017, August). PHARA: a Personal Health Augmented Reality
Assistant to Support Decision-Making at Grocery Stores. In Healthrecsys@ recsys (pp. 10-13).

16. 16
Visualization components

17. 17
AR components
Users' Decision Behavior in
Recommender Interfaces: Impact of
Layout Design (Chen and Tsoi, 2011)

18. 18
a) MS HoLolens (HMD) c) iPhone 6S (Smartphone)
b) NSW Joycon (controller)
Devices

19. List Layout
Stack Layout
19
AR components
Smartphone HoloLens

20. 20
AR components
List Layout
List Layout
Smartphone HoloLens

21. 21
AR components
List Layout
Grid Layout
Smartphone HoloLens

22. 22
AR components
List Layout
Pie Layout
Smartphone HoloLens

23. ● Within Subjects
● n = 28 (1F, 27M) Ages from 22 to 38 (M = 25.81, SD = 4.57)
● Number of conditions: 2 (HoloLens, Smartphone)
● Task 1:
- Select two products that you would like to have for dinner...
- Select two similar products...
- Select and replace with two alternatives...
- Reflection...
● Task 2:
- Imagine that you have some friends coming over... select two products… and put them in
the basket.
● Post-Questionnaires
○ TAM (Technology Acceptance)
○ NASA-TLX (Task Load Index)
23
User study

24. 24
Results
food choices overtime
Choice 1 Choice 2 Choice 3 Choice 4
Choice 1 Choice 2 Choice 3 Choice 4

25. 25
Results
Gutiérrez Hernández, Francisco; Htun, Nyi Nyi; Charleer, Sven; De Croon, Robin; Verbert, Katrien; 2019. Designing
augmented reality applications for personal health decision-making. Proceedings of the 2019 52nd Hawaii
International Conference on System Sciences; 2019; pp. 1738 - 1747

26. 26
Results

27. 27
Results

28. 28
Conclusions
● PHARA allows users to make informed decisions, and
resulted in selecting healthier food products.
● Stack layout performs better with HMD devices with a
limited field of view, like the HoloLens, at the cost of some
affordances.
● The grid and pie layouts performed better in handheld
devices, allowing to explore with more confidence,
enjoyability and less effort.

29. 29
Future work
● Long-term goal: influence healthy grocery store shopping
and in-restaurant purchase behavior.
● Evaluate the potential of our approach in real-life settings.
● Integrate other motivational design techniques.
● Wine decision-support scenario coming up.

30. 30
Phara wine
http://www.bigdatagrapes.eu/

31. AHMOSE: Augmented by human model selection
3
1
https://ahmose.ml/

32. 32
Thank you!
Katrien Verbert
Francisco Gutiérrez
[Augment/HCI - KU Leuven]
Katrien.verbert@kuleuven.be
@katrien_v | Twitter
https://augment.cs.kuleuven.be/