The Grey Area is an award-winning RoboCup@Home Education team from Macau Puiching Middle School that will be giving a lecture on their development towards becoming world champions. They will discuss how they used techniques like OpenPose for pose detection, OpenVINO and YOLO for human and object detection in their research and development. The lecture will cover topics like hardware development, object relationship detection, pose detection, and person follower software development. It will take place on August 03, 2020 from 15:00-17:00 GMT+8.

1. The Grey Area is an award-winning RoboCup@Home Education team from
Macau Puiching Middle School.
- RoboCup@Home Education Online Challenge 2020 - Gold Award (Junior)
- RoboCup@Home Education Challenge 2019 - World Champion (Junior)
Team members:
Joshua Lei, Cristiano Afonso da Silva, Thomas Leong, Sebastien Sin
Coaches:
Thomas Lao, Kinda Lam
RoboCup@Home Education
ONLINE CLASSROOM
Invited Lecture Series
Development Towards World Champion
Speakers: The Grey Area | Macau Puiching Middle School
Time: August 03, 2020 (Mon) 15:00~17:00 (GMT+8)
https://www.robocupathomeedu.org/learn/online-classroom/invited-lecture-series
Highlights
● OpenPose to detect human postures
● OpenVINO and YOLO for human
and object detection

2. RoboCup@Home Education | www.RoboCupatHomeEDU.org
Development Towards World Champion
● Speakers: The Grey Area | Macau Puiching Middle School
● Host: Jeffrey Tan | @HomeEDU
● Date and Time:
○ August 03, 2020 (Mon) 15:00~17:00 (GMT+8 China/Malaysia)
○ August 03, 2020 (Mon) 03:00~05:00 (EDT New York)
○ August 03, 2020 (Mon) 09:00~11:00 (CEST Italy/France)
○ Web: https://www.robocupathomeedu.org/learn/online-classroom/invited-lecture-series
** Privacy reminder: Video will be recorded and published online **
RoboCup@Home Education Online Classroom
2

3. RoboCup@Home Education | www.RoboCupatHomeEDU.org
RoboCup@Home Education is an educational initiative in RoboCup@Home that promotes educational
efforts to boost RoboCup@Home participation and artificial intelligence (AI)-focused service robot
development.
Under this initiative, currently there are 4 efforts in active operation:
1. RoboCup@Home Education Challenge events (national, regional, international)
2. Open Source Educational Robot Platforms for RoboCup@Home (service robotics)
3. OpenCourseWare for the learning of AI-focused service robot development
4. Outreach Programs (local workshops, international academic exchanges, etc.)
Web: https://www.robocupathomeedu.org/
FB: https://www.facebook.com/robocupathomeedu/
RoboCup@Home Education
3

4. RoboCup@Home Education | www.RoboCupatHomeEDU.org
Special Online Challenge Tracks
● Open Platform Online Classroom [EN]
● Open Platform Online Classroom [CN]
● Standard Platform Pepper 2.9 Online
Classroom [EN]
● Standard Platform Pepper 2.5 Online
Classroom [CN]
More details:
https://www.robocupathomeedu.org/learn/online
-classroom
Invited Lecture Series
● Robotics Development with MATLAB
● Robot Localisation: An Introduction
● World Representation Through Artificial
Neural Networks: An Introduction
● Introduction to NLP
● Development Towards World Champion
Regular Online Classroom Tracks
● Introduction to Service Robotics [EN]
○ 6 weeks
○ ROS, Python
○ Speech, Vision, Navigation, Arm
RoboCup@Home Education Online Classroom
4

5. RoboCup@Home Education | www.RoboCupatHomeEDU.org
The Grey Area | Macau Puiching Middle School
5
The Grey Area is an award-winning RoboCup@Home Education team from Macau
Puiching Middle School.
- RoboCup@Home Education Online Challenge 2020 - Gold Award (Junior)
- RoboCup@Home Education Challenge 2019 - World Champion (Junior)
Team members:
- Joshua Lei
- Cristiano Afonso da Silva
- Thomas Leong
- Sebastien Sin
Coaches:
- Thomas Lao
- Kinda Lam

6. Robocup@home education
sharing section
The grey area

7. Table of contents
1. Team Introduction (25 mins)
2. Detection Tutorial (40 mins)
3. Research & development (30 mins)
4. Q&A (20 mins)

8. Installation

9. Team Introduction

10. Our team

11. Our team
2019 - The journey of deep learning 2020 - The Grey Area

12. Our team
The journey of deep thinking - RoboCup@home education sydney

13. Our team
The grey area - The origin of our name

14. School environment

15. School environment

16. School environment

17. School environment

18. School environment

19. Testing environment

20. Testing environment

21. Time management

22. Time management Work for at least 3 hours per day

23. Learning journey

24. How we learn under this pandemic?
Notion & Zoom

25. How we learn under this pandemic?

26. Learning journey

27. Experience and memories

28. Experience and memories
Joshua - Mission 3 in
Robocup Sydney

29. Experience and memories
Rootwalker -
1. Forget to release the
emergency button when
mission 1
2. Messy project tree
https://github.com/supercatex/home_edu/tree
/master/scripts

30. Experience and memories
Cristiano - The journey of overcoming tough times

31. Research and
Development

32. Hardware development

33. Hardware -Introduction

34. Hardware
structure

35. Hardware -Versions
Version 1 Version 2 Version 3

36. System
structure

37. Manipulator

38. The Manipulator
Camera vision
Each servo of the
manipulator

39. The Manipulator -camera vision
Proportion calculation

40. The Manipulator -each servo of the manipulator
Cosine
theorem

41. Object relationship

42. Object relationship -Introduction
Find the relation between humans and its surrounding objects

43. Object relationship -Software development
Part 1
Using YOLO to find all
the humans and objects
in the frame
Part 2
Find the main character
(human) in the frame
Part 3
Reduce far away and
small objects in the
frame

44. Object relationship -Software development
Part 1 Using YOLO to find all the humans and objects in the frame

45. Object relationship -Software development
Part 2 Find the main character (human) in the frame

46. Object relationship -Software development
Part 2 Find the main character (human) in the frame

47. Object relationship -Software development
Part 3 Reduce far away and small objects in the frame

48. Object relationship -Application
Detecting the surrounding objects of a person

49. Object relationship -Application
Finding the main character and its surroundings

50. Pose detection

51. Pose detection -Introduction
Notice the danger of humans in different situations

52. Evaluation of Emergency detecting system
Average value
method
SVM
(Support vector
machine)
Version 1 Version 2
No Machine Learning
Have Machine
Learning
CNN
(Convolution
Neural
Network)
Version 3
Have Machine
Learning
Disadvantage: when it comes to
the situation that the person is
sleeping on the bed, this method
cannot distinguish the
differences whether the person
has fallen or the person is just
sleeping
Disadvantage: when it
comes to some
different
environment, it shows
a low accuracy
Advantage: it shows
geart accuracy
regarding different
environment

53. Average value method
從Open Pose Library
獲取肢體特徵點
利用肢體特徵點
得知人體的動作
求出所有點的平均值
判斷點是否在畫面下半
部分
把影像分為上下兩部分

54. Training Dataset
The higher amounts of
images in dataset, the
more accuracy the model
will be
Training dataset
amounts: 3085
images
Amount of Photos
Accuracy of model

55. Emergency detecting system
Danger
Safe

56. Training Dataset
Dataset amounts: 3085 images
Danger (1086 images) Warning (1399 images) Safe (600 images)

57. Testing dataset
We have used photos in
different backgrounds to
improve the accuracy of
our program
Testing dataset
amounts:
197 images
Variety of Photos
Accuracy of model

58. SVM
best
model

59. training degree original dataset training score original dataset testing score new dataset testing score
1 0.962666667 0.964476021 0.5721649484536082
2 0.981877777 0.975133214 0.5567010309278351
3 0.993333333 0.975133214 0.6288659793814433
4 0.998222222 0.973357015 0.6494845360824743
5 0.9991111111 0.966252220 0.6804123711340206
6 0.999555555 0.962699822 0.6958762886597938
7 0.999555555 0.959147424 0.7010309278350515
8 0.999555555 0.953818827 0.7061855670103093
9 0.999555555 0.946714031 0.7164948453608248
10 0.999555555 0.941385435 0.7268041237113402
11 0.999555555 0.939609236 0.7371134020618557
12 0.999555555 0.939609236 0.7422680412371134
13 0.999555555 0.939609236 0.7422680412371134
14 0.999555555 0.936056838 0.7422680412371134
15 0.999555555 0.930728241 0.7422680412371134
16 0.999555555 0.927175843 0.7474226804123711
17 0.999555555 0.923623445 0.7525773195876289
18 0.999555555 0.923623445 0.7474226804123711
19 0.999555555 0.921847246 0.7525773195876289
20 0.999555555 0.920071047 0.7422680412371134
rising
trend
rising
trend
remain the
same
drop trend
drop trend
drop trend
rising
trend
rise
and
drop
remain the
same
remain the
same
remain the
same
drop trend
best

61. Person follower

62. Before😰 After😍
No image processing Image processing
VS

63. What the robot sees

64. History
Ver 1.0
It’s a clone of the officinal one by
turtlebot. It can’t identify a human
either an object
Ver 1.1
Added screen size crop to avoid
other objects
Ver 2.0
Using YOLO to identify a human
or an object. But still can’t
recognize a target person
Ver 2.1
Using dlib trackers to track the
target person, but low
performance
Ver 2.2
Using OpenVino’s model to track,
which has the highest
performance

65. Old follower flow-chart

66. The PID controller
The PD controller

67. Openvino
https://docs.openvinotoolkit.org/2020.3/_models_intel_person_reidentification_retail_0249_description_person_reidentification_retail_0
249.html

68. YOLO

69. Person Follower -Software Development
Part 1
Detect only persons
with YOLO
Part 2
Run the person
reidentification model
on each person after
cropping it from the
original frame with the
box
Part 3
Using cosine similarity
to find out the target
person

70. New follower flow-chart

71. Ending

72. Suggestions
● Good Time management
● Use apps (eg. Notion) to jog down notes and references
● Work hard

73. Q&A

74. Web: https://www.robocupathomeedu.org/
FB: https://www.facebook.com/robocupathomeedu/
GitHub: https://github.com/robocupathomeedu/
Online Classroom: https://www.robocupathomeedu.org/learn/online-classroom
Contact: oc@robocupathomeedu.org
RoboCup@Home Education
ONLINE CLASSROOM
Invited Lecture Series

75. RoboCup@Home Education | www.RoboCupatHomeEDU.org
1. The Grey Area (Macau) | Macau Puiching Middle School
○ August 03, 2020 (Mon) 15:00~17:00 (GMT+8)
2. UBC Open Robotics (Canada) | The University of British Columbia
○ August 10, 2020 (Mon) 19:00~21:00 (PST)
3. Danny Ng Wee Kiat - UTAR (Malaysia) | Universiti Tunku Abdul Rahman
○ August 22, 2020 (Sat)
4. FantasyIT_Papper & Micro agent service (China) | Changhe High School
○ August 4th week
5. LASR (United Kingdom) | University of Leeds
○ September 02, 2020 (Wed) 10:30~12:30 (UK)
6. RoboBreizh (Fance) | Lab-STICC, ENIB
○ September 30, 2020 (Wed) 15:00~16:00 (UTC+2)
Special Online Challenge 2020 Invited Lecture Series