1. www.ict.csiro.au
Elliot Duff
Mining
Robotics
Project
Autonomous
Systems Laboratory
CSIRO ICT Centre
QCAT Brisbane
Mine Integration Plan

2. www.ict.csiro.au
Projects, Past Present and Future
Sensor Networks
for Coal Prep plants
ACARP C11009 (03-04)
Open Cut Automation
ACARP C11054
Traffic Management
Truck path planning
ACARP C13041 (2004-05)
Excavator Guidance
Laser based measurement
ACARP C14043 (2005-06)
Dragline Swing Assist
Algorithm Refinement
ACARP C13040 (04-05)
Digital Terrain Mapping
Calibration and Integration
ACARP C13034 (2004-05)
Rope Shovel Automation
Auto digging on scale model
ACARP C12030 (2003-04)
Trayscan
Truck Load Volume
Transcale ((2000-2004)
Automated LHD's
Commercial Prototype
AMIRA/Caterpillar (99-03)
Automated Explosive Loading
Proof-of-Concept
ORICA (2000-01)
Shovel Dipper Volume
Laser Scanning Feasibility
BHP (1998)
Automated Haulage
GPS and INS guidance
Komatsu & Caterpillar
Dragline Swing Assist
Feasibility demonstration
ACARP C5003 (1996-98)
Digital Terrain Mapping
Proof-of Concept
ACARP C10039 (2001-02)
Dragline Swing Assist
ACIRL Scale Model
ACARP ME3 (1994-95)
Dragline Swing Assist
Production field trials
ACARP C9028 (2001-02)
Automated Explosive Loading
Commercial Prototype
ORICA (2005-)
Rope Shovel Automation
On production shovel
ACARP proposal
Shovel Force Feedback
Based upon DTM and performance
ACARP proposal
Boom Monitoring
Wireless sensor network
ACARP
Automated Dragline
Integration of DSA with DTM
ACARP proposal
Wireless Sensor Network
for Mine Environment
ACARP proposal
ELAP
Prototype
ORICA (2002)
Automated Explosive Loading
Feasibility Study - Vision and Laser
Scanning ORICA (1998-99)
Rope Shovel Automation
Auto loading / face mapping
ACARP C14058 (2005-06)
Surface Mining
Completed
Commercialized
Current
Proposed
HMC Automation
Auto Tramming
Alcoa (2005-06)
Software development
RTX & DDX
Hardware development
IMU & IMU

3. www.ict.csiro.au
Dragline Swing Assist (DSA)
• Since 1994 we have been automating a large part (80%) of a
dragline’s cycle (swing, dump and return).
• We have installed two systems on production machines
• In 2003 we ran a two-week production trial on the BE1350
• This work has been supported by five ACARP projects
Optimized swing path

4. www.ict.csiro.au
Dragline Digital Terrain Maps (DTM)Dragline Digital Terrain Maps (DTM)
 We have installed a protype system on a BE1350
dragline at Boundary Hill (Callide)
 It assists the operator in excavating to specific bench
heights with increased precision and confidence.
 It displays a high resolution image of spoil pile and
the surrounding terrain.
 DTM can also guides the operator in correct
movement and positioning of the dragline tub.
 This work has been supported by two ACARP
projects

5. www.ict.csiro.au
DTM MasterplanDTM Masterplan
OverviewOverview
Images include
cross-section,
plan view or 3D.
Operator
prompts include
a tub positioning
reference and
critical dig lines
The elevation image is shown.
Plan package will also utilise
colours and effects to highlight
variations between the dig and
actual plan

6. www.ict.csiro.au
DTM – Direct Operator Benefits…DTM – Direct Operator Benefits…
 It visually prompts the operator where to dig and dump
overburden as per the dig plan.
 Masterplan clearly and immediately highlights
variations between the actual dig and the planned
excavation.
 Greatly aids in reducing excessive rehandle.
 It assists the operator in excavating to specific bench
heights with increased precision and confidence.
 It displays a high resolution image of spoil pile and the
surrounding terrain.
 DTM can also guides the operator in correct
movement and positioning of the dragline tub.

7. www.ict.csiro.au
Rope Shovel Automation (RSA)
 Automation of these machines could lead to significant productivity
improvements
 In 2003/04 we demonstrated autonomous digging on a scale model
(1/7 th) machine (this work was supported by ACARP)
 We are now running a second ACARP project to further progress the
system and autonomously dump into trucks (scale model)

8. www.ict.csiro.au
Shovel Traffic Control
A combination of sensors will:
• Generate a DTM of the
local environment
• Enable the swing loader to
locate each haul truck as it
approaches
• Provide communication
from the swing loader to
each haul truck
Develop a system which schedules and directs haul truck drivers
to an optimum loading position for a swing loader.
Develop a system which schedules and directs haul truck drivers
to an optimum loading position for a swing loader.
Handover
Zone
Work Zone
Incoming
Trucks
Outgoing
Trucks
Bench
Obstacle

9. www.ict.csiro.au
Excavator Guidance and Measurement
Laser scan of terrain and bucket location

10. www.ict.csiro.au
Truck Volume Measurement (Trayscan)Truck Volume Measurement (Trayscan)
Trayscan is an automated system that measures the volume of
material in the tray of haul-trucks using laser-scanning technology.
This technology provides the ability to measure the volume in-situ,
whilst the truck is moving. Commercialised through Transcale.
 A 3D profile is
generated, as the
haul truck drives
under scanning
laser.
 The in-situ volume
is estimated from a
comparison of
empty and full
trucks.

11. www.ict.csiro.au
Load Haul Dump (LHD) Automation
 Demonstrated full speed operation at NorthParkes Mine
 System based upon reactive navigation requiring no guidance
infrastructure
 Navigation technology licensed to DAS (CEPL) with commercial
system launched April 2003
 Pre-production prototypes developed for a three type of LHD including
electric machines
Automated CSIRO loader tested at
Northparkes mine
Automated Tamrock electric LHD
from Northparkes at QCAT

12. www.ict.csiro.au
Automation of Explosive Charging (AOC)
 We have developed a robotic charging unit to automatically charge
blast holes in underground metal mines (ORICA)
 A scanning laser is used to create a 3D map in order to locate a ring of
blast holes
 A vision system is used to guide a hose into the collar of each hole

13. www.ict.csiro.au
Autonomous Hot Metal Carrier
• 30 tonne vehicle for
navigation sensor
• moving molten
aluminium
• Demonstrate full
automation within the
next 6 months
• The vehicle is based on
a large fork-lift truck
• Will use laser scanners
as the primary

14. www.ict.csiro.au
Who are we ?
Team Complement
• 7.5 Research staff
• 4 Engineering support
– 2 engineers
– 2 technicans
• 1 BDM (theme resource)
• 1 Executive officer (theme
resource)
• 2 Postdocs
• 4 Co-supervised PhD
students
• Visitors (current)
– 1 IAESTE intern
– 5 undergraduate students
Disciplines/Skills
• Electrical engineering
• Mechanical engineering
• Computer Science
• Aerospace engineering
• Physics & Applied maths

15. www.ict.csiro.au
What we do?
The intelligent connection of perception to actionThe intelligent connection of perception to action
 Robot navigation
 Relative & reactive navigation
 Visual homing
 Visual servoing
 Multiple cameras with sensor
fusion
 Ranging and measurement
 Laser imaging
 3D imaging
 Omni directional vision
 Stereo vision
 Agent based systems
 Low-cost distributed sensor systems
 Ad-hoc wireless networks
 Real time computing
 Distributed systems
 Real time libraries
 Embedded systems
 High-performance control systems
 Hydraulic
 Electro mechanical

16. www.ict.csiro.au
Team Project Activities
 External focus
 Mining Robots
• Unique issues facing mining sector:
• Dirty, dusty, wet and hot
• Constantly changing environment (the aim of mining!)
 Strategic Focus
 Competent vision-based navigation and control,
• tested on our platforms: ground, air, underwater
 Wireless sensor and actuator networks
• Robot interaction & Environmental monitoring
Deliver prototypes of complex engineered systems for non-
traditional robotics applications (field/outdoor robotics)
Deliver prototypes of complex engineered systems for non-
traditional robotics applications (field/outdoor robotics)

17. www.ict.csiro.au
Autonomous Underwater Vehicles (AUV)
Starbug: Sea trials in Moreton Bay

18. www.ict.csiro.au
Autonomous Land Vehicle
• Experimental platform for
land based navigation
• Based on a ride-on
lawnmower
• Equipped with vision, laser
scanner and magnetic
compass
• All computing on-board

19. www.ict.csiro.au
Autonomous Aerial Vehicles
Mark II – Mantis (2003)
Mark I (1999-2002)
• Experimental platform for testing CSIRO
autonomous flight control system
• Equipped with stereo vision, low-cost
inertial sensors, magnetic compass and
standard GPS
• All computing on-board
• We have developed a vision-based velocity
and height estimation system
• Currently investigating asset management
and search support opportunities.

20. www.ict.csiro.au
Cable-Array Robot
• A set of spatially distributed
motors which actuate cables
connected to an end effecter
– Similar to a Stewart platform
• By changing the length of the
cables, the end effecter can
be moved
• Examples include:
– SkyCam – used in football
stadiums for giving an
overhead view
– NIST robot crane
– Quay cranes
Schematic view of a cable-array robot
end effecter
actuator
cables

21. www.ict.csiro.au
Robotics & Sensor Networks
Old thinking New thinking
Robot
carries/controls small
number of expensive
sensors
Sensors are
everywhere, robot
asks for information
Scientists go into
field to gather data
Robot goes into field
to “harvest” data
from sensors
Robot goes into field
to gather data itself

22. www.ict.csiro.au
ADC/DIO Power Solar
Programmer/
Serial port
Connectors/
Mounting holes
AdHoc Radio Networks
Fleck 1
 Temperature
 ADC
 DIO
 Serial

23. www.ict.csiro.au
AdHoc Radio Networks
Fleck2
 temperature
 motion
 compass
 GPS
 DIO/ADC
 SD/MMC
 Video

24. www.ict.csiro.au
EiMU
• Three solid-state gyros
• Two dual-axis accelerometers
• Three Honeywell
magnetometers
• HC12 based micro-controller
• RS232 and Canbus outputs
• Small: 50x50x50mm
• Lightweight: 65g
• Complementary filters for
fusion

25. www.ict.csiro.au
Navigation Technology
• Laser Sensing
– Reactive Navigation
– Opportunistic Localization
• Visual Sensing
– Optical Flow (speed)
– Stereo (range / height)

26. www.ict.csiro.au
Navigation Technology
Odometry: vision vs groundtruth

27. www.ict.csiro.au
DDX: Distributed Data eXchange
Local store
C
P
C
C
CP
P
Local store
internet
Log file
• Real-time publish and subscribe model
• Shared memory within machine
• Time stamping
• Event handling & synchronization
• Multicast between machines
CPU2CPU1