This document describes the development of an agricultural robot sprayer and evaluation of different user interfaces for human-robot interaction. It discusses the technical specifications of the robot platform used and modifications made to integrate a sprayer. Various user interface designs were implemented and tested in field experiments, including web-based and augmented reality interfaces. Preliminary findings showed that interfaces providing views from multiple cameras led to better task performance and fewer collisions compared to a single camera view. Further work is still needed to address additional technical challenges of agricultural robotics and improve the usability of interfaces.
The 7 Things I Know About Cyber Security After 25 Years | April 2024
Agrirobot presentation by George Adamides
1. Agricultural Robot Sprayer and
Evaluation of User Interfaces in Field
Experiments
George Adamides
Senior Agricultural Research Officer
Agricultural Research Institute
2. Presentation overview
Technical characteristics of the
Agricultural Robot Sprayer
User Interfaces for HRI Design &
implementation
Field Experiment design &
implementation
Findings
Conclusion
3. Summit XL by Robotnik
Technical characteristics of the Agricultural Robot Sprayer
http://www.robotnik.es/en/products/mobile-robots/summit-xl
4. Summit XL technical characteristics
• The SUMMIT XL is a medium-sized high mobility all-terrain
robot with extreme performance.
• The Summit XL has skid-steering kinematics based on 4
high power motor wheels
– Dimensions 693x626x417 mm
– Weight 30 Kg
– Load capacity 20 Kg
– Speed 3 m/s
– Traction system 4 wheels
– Batteries 8x3.2V LiFePO4 (~5 hours autonomy)
• 45 minutes for full charge
– Temperature range 0oC +50oC
– Max climbing angle 45o
– Controller ROS embedded PC with Linux Real Time
– Communications WiFi 802.11n
– Connectivity Internal: USB, RS232, GPIO y RJ45
– External: USB and power supply 12 VDC
Technical characteristics of the Agricultural Robot Sprayer
6. Building the Robot Sprayer
Technical characteristics of the Agricultural Robot Sprayer
September 2012
7. The Sprayer
• Serena electric sprayer
Weight (with full tank) 13.6 kg
Measurements: 315x145x400
Water flow rate with a fan nozzle 26 liters/h
Battery life 11.30 h
Full cycle charging 10h
Capacity 10 liters
Technical characteristics of the Agricultural Robot Sprayer
8. Sprayer installation
• Modbus IO
• Sprayer case
• ROS module programming
<?xml version="1.0"?>
<launch>
<!-- start modbus_io node -->
<param name="modbus_io_node/ip_address" value="192.168.2.185" />
<param name="digital_outputs" value="8"/>
<param name="digital_inputs" value="8"/>
<param name="analog_outputs" value="2"/>
<param name="analog_inputs" value="2"/>
<node pkg="modbus_io" type="modbus_io_node" name="modbus_io_node" output="scr
een"/>
</launch>
ssh summit@192.168.0.200
summit> ping 192.168.0.185
3. Launch the modbus_io module
roslaunch modbus_io test_io.launch
4. Test the digital outputs
rosservice call /modbus_io/write_digital_output 0 false
rosservice call /modbus_io/write_digital_output 0 true
rosservice call /modbus_io/write_digital_output 1 false
rosservice call /modbus_io/write_digital_output 1 true
...
Outputs 5,6,7 and 8 are RELAYS
5. Once it is working, modify the summit_xl_complete launch file in order to launch t
Sprayer case and installation by AgroWise
9. Building the Robot Sprayer – modified version
December 2012
Technical characteristics of the Agricultural Robot Sprayer
10. Initial trial-out findings
• Issues with the camera
– Viewing angle
– Drops on dome cover
• Issues with PC screen
– Lighting/shading/reflection
• Issues with wireless
connection
– Distance
• Issues with Bluetooth
connection
– PS3 (distance)
Technical characteristics of the Agricultural Robot Sprayer
11. Improving the Agrirobot
• Hardware
– Installation of two USB cameras to improve
Peripheral vision and End-effector vision
– Bigger wifi antenna
– Moved higher the spraying nozzle
• Software
– Installation and configuration of the
mjpeg_server ROS module and the Apache
webserver
– Installation and programming the
pr2_keyboard_teleop ROS module for the
Summit XL navigation and the sprayer on/off
state
Technical characteristics of the Agricultural Robot Sprayer
17. Presentation overview
Technical characteristics of the
Agricultural Robot Sprayer
User Interfaces for HRI - Design
and Implementation
Field Experiments Design &
Implementation
Findings
Conclusion
18. Reality Based Interaction (RBI) styles [1]
New interaction styles that draw strength
by building on users’ pre-existing
knowledge of the everyday, non-digital
world to a much greater extent than
before.
Examples of RBI: VR, AR, TUI, ubiquitous and
pervasive computing, handheld or mobile
interaction…
[1] Jacob, Robert JK, et al. "Reality-based interaction: a framework for post-WIMP
interfaces." Proceedings of the SIGCHI conference on Human factors in computing systems.
ACM, 2008.
User Interfaces for HRI – Design and Implementation
19. User Interface for Robot Teleoperation –
Development Phases
User Interfaces for HRI – Design and Implementation
Spraying
20. Designing for HRI Awareness
HRI Awareness [2]
Given one human and one
robot working on a task
together, HRI awareness is
the understanding that the
human has of the location,
activities, status, and
surroundings of the robot;
and the knowledge that
the robot has of the
human’s commands
necessary to direct its
activities and the
constraints under which it
must operate.
LASSO technique [3]
• Location Awareness
• Activity Awareness
• Status Awareness
• Surroundings
Awareness
• Overall Mission
Awareness
[3] Jill L. Drury, Holly A. Yanco & Keyes, B 2007,
'LASSOing HRI: analyzing situation awareness in
map-centric and video-centric interfaces',
Proceedings of the ACM/IEEE international
conference on Human-robot interaction.
[2] Scholtz, J.; Young, J.; Drury, J.L.; Yanco, H.A., "Evaluation of human-robot interaction
awareness in search and rescue," Robotics and Automation, 2004. Proceedings. ICRA '04.
2004 IEEE International Conference on, vol.3, no., pp.2327,2332 Vol.3, 26 April-1 May 2004
22. Phase 1. Using ROS command line
~> roslaunch usb_cam low_res.launch
User Interfaces for HRI – Design and Implementation
~> export ROS_MASTER_URI=http://V3:11311
~> rostopic list
~> rosrun image_view image_view image:=/logitech_usb_webcam/image_raw
Robot PC Remote PCssh summit@V3
23. • Using ROS
command line to
display Peripheral
and End-Effector
cameras
• Web interface of
Axis Ethernet
camera
ROS environment with three
cameras
User Interfaces for HRI – Design and Implementation
24. • Installed the mjpeg_server module
• Installed the Apache web server
The mjpeg_server is a streaming
server that subscribes to requested
image topics in ROS and publishes
those topics as MJPEG streams via
HTTP
Preparing for phase 2
User Interfaces for HRI – Design and Implementation
25. Phase 2. First attempt for Web UI in HTML
User Interfaces for HRI – Design and Implementation
26. Improved version of the Web UI
in PHP
UI Design and Implementation by Istognosis
27. UI for driving
Main camera & Peripheral
camera
UI for spraying (rejected)
Main camera & spraying
camera
UI for spraying
Main camera & ssupport
cameras
28. Phase 4. Using a patriot wireless tracker &
digital glasses
User Interface Design and Implementation
by Istognosis
29. Presentation overview
Technical characteristics of the
Agricultural Robot Sprayer
User Interfaces for HRI – Design &
Implementation
Field Experiments Design &
implementation
Findings
Conclusion
35. Experiment design &
implementation
1 PC Screen + PS3 + Main Camera Only
2 PC Screen + PS3 + Main & Support Cameras
3 PC Screen + Keyboard + Main Camera Only
4 PC Screen + Keyboard +Main & Support Camera
5 AR Glasses + PS3 + Main Camera Only
6 AR Glasses + PS3 + Main & Support Cameras
7 AR Glasses + Keyboard + Main Camera Only
8 AR Glasses + Keyboard + Main & Support Cameras
Field experiments Design & Implementation
Tasks were randomized and
were conducted in two day
visits. Four tasks were carried
out on day 1 and the
remaining tasks on day 2 (not
consecutive, period between
experiments varied 2 to 10
days)
25 participants:
Agronomists
Agricultural Technicians
Agricultural Laborers
36. Experiment procedures
• Consent form
• Pre-Questionnaire
– Demographics, Immersion Tendency Questionnaire,
General Self-Efficacy Scale, Santa Barbara Sense of
Direction Scale, CEW Fluency Scale
• Briefing and getting familiar with the UIs
• Post-Questionnaire after each run
– SUS, Presence, NASA TLX
– Metrics (collisions, path divergence, targets
sprayed, targets missed, percent completed,
duration)
• Experiment duration ~3 hours per participant
to complete 8 tasks
Field experiments Design & Implementation
37. The Agrirobot in the field
Photo Album
Field experiments Design & Implementation
69. Presentation overview
Technical characteristics of the
Agricultural Robot Sprayer
User Interfaces for HRI – Design &
Implementation
Field Experiments Design &
Implementation
Findings
Conclusion
70. Preliminary results
NASA Task Load Index per UI
Findings
User Interfaces
with PS3 controller
User Interfaces
using keyboard
controller
71. Preliminary results
Effectiveness: Number of grape clusters sprayed per UI
Findings
User Interfaces
with main and
Support (3) cameras
User Interfaces
with main (1) camera
72. Preliminary findings
Mean number of collisions per UI
UI # Cameras N Minimum Maximum Mean Std.
Deviation
UI1 - 1
Camera
25 0 3 ,76 ,779
UI2 - 3
Cameras
25 0 4 ,60 ,913
UI3 - 1
Camera
25 0 7 1,36 1,777
UI4 - 3
Cameras
25 0 2 ,56 ,861
UI5 - 1
Camera
25 0 5 1,28 1,242
UI6 - 3
Cameras
25 0 4 ,84 ,987
UI7 - 1
Camera
25 0 4 ,80 1,118
UI8 - 3
Cameras
25 0 3 ,28 ,678
74. In Summary – Problems faced and
overcome
• Transformation of a “off-the-shelf”
robot into a robotic sprayer
• Pilot trials revealed issues with WiFi,
Bluetooth, camera view points
• A lot of –smaller or bigger- practical,
“non-research” problems, turned into
valuable experience for the future
75. In Summary - what we did
• Designed and implemented several
user interfaces
• Used WIMP and RBI interaction
styles
• Field experiments
76. Conclusions
• Yes, it is feasible! (Agri Robot tele-operation )
• The user interface design does make a
difference
• There are many small, practical issues to
resolve
– Agricultural task are demanding and take place in a
difficult environment
– Many issues to overcome
• PS3 Bluetooth, WiFi, monitor shading/light, web
cameras
• Robot wheels, sprayer hose
• ROS module programming
• Promising findings
77. Future work
• Incorporation of sensor information in the
UI to include ultrasound sensor
information, battery-life (under development)
• Robot improvements
– Servos for extending sprayer antenna and
USB/Ethernet cameras control/rotation
– Sprayer antenna with multiple nozzles
• Learnability issues need further study
• Long hours?
• Cost-benefit analysis