2. flight and require constant monitoring from the human The payload functions are represented by the mission
operator (HO). Their use up to now has been described as functional system, i.e. those systems which are mission
the use of “goggles in the sky” by a human operator on the specific, and they are defined in [Ref. 1] (chp1, pp1), as
ground. those functions that include the set of all functions that
By automatic is meant a repetitive action that does not directly relate to the mission of a given vehicle
require external influence or control, but which repeats The payload of each platform is mission specific and it
itself based on some set conditions. A well known will vary with the specific mission requirement and it will
automatic system is feedback control system, which need to be fitted on the air vehicle and its control defined
automatically adjusts the input in order to obtain the and implemented. Examples of possible sensors that could
desired outputs be fitted on a UAV are:
The major difference between automatic and • Electro-Optical including Optical and IR sensors for
autonomous systems is that autonomous systems can obstacle detection and target detection
change their behaviour in response to unanticipated • Ground Moving Target Indicator (GMTI) radar and
events, whereas automatic systems would produce the Synthetic Aperture Radar (SAR)
same outputs regardless of any changes experienced by • GPS/INS navigation system.
the system or its surroundings. In robotic applications, The vehicle management system (VMS), on the other
automatic actions allow the machines to be operationally hand, is defined in [1] as the collection of functions that
autonomous, but they do not allow them to have are required for the vehicle to understand, plan, control,
decisional autonomy. and monitor the air vehicle operations. They usually
An autonomous action or event is defined as an action represent the safety critical functionalities required for the
or event that is ‘independent in mind or judgment’, a self- safe employment of the platform; hence they include all
directed, self-governing entity, not controlled by others or the flight-critical and safety-related functions.
by outside forces. The mission system functional capabilities can be
Autonomous UAVs are herein defined as air vehicle classified according to the level of autonomy that each of
systems with embedded autonomous functionalities. them would require if the HO was not included in the
In this research the focus is on identifying loop.
methodologies to allow the level of autonomy for UAVs Three major levels of autonomy have been identified:
to increase and the implementation of a dynamic, • Low Level - The low level of autonomy can be
responsive behaviour on the platform itself to reduce the considered the reactive side of the UAV, where,
operator workload and provide independence from pre- given an event, the UAV will automatically react
programmed flight. according to pre-defined limits and following given
dynamic models. This level is characterised by the
3. UAV Mission System group of functionalities that would be required in
order to fly the platform remotely. These include, but
The main gaps in current UAV systems are are not limited, to the flight control system, the
represented by the reliance on datalink, the ‘dumbness’ of actuator function, the engine or propulsion control,
the control and payload systems, which means that they and the aircraft flight mechanics and air data
are not reactive to the environment, and the latency of acquisition.
information to the ground control station. In order to • Medium Level – The medium level of autonomy
understand where the identified technological gaps can captures the ‘reflective’ capability that should be
present a significant drawback within the UAV systems, embedded on board of the platform in order to enable
the general UAV functionalities are captured. it to reason about its internal and external state. That
A UAV mission system is seen to provide the is the ability of the platform to detect internal faults
translation of mission objectives into quantifiable, or malfunctions, to self-regulate when subjected to
scientific descriptions that provide a measure to judge the unexpected events while carrying out the mission.
performance of the platform, i.e. the system in which the The functionalities of the mission systems that have
mission objectives are transformed into system been identified to belong to this class are:
parameters. According to the AGARD report on o The flight path command and performance
Integrated Vehicle Management Systems [Ref. 1], the envelope protection, such as the waypoint
mission management functions of modern aerospace following system and the guidance and
vehicle can be split into two different functional elements: navigation functions;
the payload functions and the vehicle management o The health manager and fault tolerant control, in
function. order to detect and react to possible system
failures and malfunctions;
112
3. o The power management system, included systems.
optimising the power consumption and maximising By developing those on the platform, the air vehicle is
the mission time. envisaged to achieve the required level of autonomy for
• High Level – The high level represents the most the drone. The key capabilities and the associated system
sophisticated layer of autonomy that it is desirable to where it can be built in pointed out are reported in Table
provide on board the air vehicle. It provides the 1. A possible application of these fields is provided in the
platform with decision making capabilities and with implementation of the systems reported in Table 1.
the ability of interoperate with other platforms and /or
systems in order to gather and analyse information, as
well as the ability to reason and act upon the 7DEOH /LQN EHWZHHQ 8$9 UHTXLUHG FDSDELOLWLHV DQG
conclusions it draws. The functional systems which IXQFWLRQDO VXEVVWHPV
have been identified to require a higher level of
autonomy are: UAV capabilities Functional subsystem
o the fault detection and identification (ID), i.e. the React to changes in Mission Planner
ability for the platform to detect the malfunction, environment and be capable of Mission Goal Manager
reason about it and take an appropriate action re-planning Sensor Manager
without any HO intervention; Navigation through complex Flight Path
o the situation awareness manager, responsible to terrain, possibly at high speed
maintain and update the state of the world (i.e. the Dynamic allocation of on board Power Manager
representation of the environment in which the resources, i.e. fuel, power, Sensor Manager
UAV is operating) and communicate detected sensors.
changes to the mission manager. Interoperability with other Sensor and Integrated Signal
A high level of autonomy is also required to make systems and platforms and Data Processing Manager
decisions and to provide interoperability with other Constant maintenance of Sensor Manager, Integrated
systems. Therefore, the mission goal manager and sensor situational awareness Signal and Data Processing
manager, as well as the integrated signal and data Manager
processing managers fall into this layer. The mission goal Situational awareness Manager
manager is responsible for redefining the mission Perform autonomous Mission Planner
objectives and goals and assessing their validity while manoeuvre at the limits of the Flight path envelope protection
cooperating with other air platforms or systems. The flight envelope to maximise
sensor manager needs to monitor, select, and allocate the performance
sensor when the target is detected and decide both the Provide on board data and Integrated Signal and Data
required accuracy and precision of information for the information processing Processing Manager
given task, and also the data that need to be recorded in capability
order to successfully support the mission execution. Data Operate outside communication Mission Goal Manager
may come from the own sensor platforms mounted on the link limits Mission Planner
UAV or from other air vehicles or systems. The integrated Sense and avoid threats, Sensor Manager
signal and data processing manager has the task of collision avoidance Mission Planner
processing the multiple source information, including the Autonomous reconfiguration of Fault Tolerant/Reconfigurable
operator commands and reasoning about their soundness systems in case of fault control
and applicability within the mission context, and also to Fault Detection and ID Manager
process them into appropriate information useful by the
other subsystems. Figure 1 captures the envisaged mission 5. Artificial intelligence techniques applicable
system architecture and the three major autonomy levels
to UAVs
as described above.
It is important to observe that the application of
Conventional control systems and deterministic
artificial intelligence technique is envisaged to be used
optimisation techniques provide a means to deal with
mainly in the high-level autonomy layer.
uncertainty within certain boundaries. However, the
multitude of case scenarios faced by the UAV even before
4. Identified mission level functionalities for it starts its mission requires a real time adaptive system
UAVs capable of reacting to unforeseen events to the best of its
possibilities. Machines are now required to ‘think’ and
In this section the key mission-level capabilities for a ‘learn’ from the environment, as it has been pointed out in
UAV are discussed and linked to the above functional the previous paragraphs.
113
4. Surveillance/IFF
UAV Mission System
Target Detection and Tracking
Mission Objectives
Battle Damage
Assessment/Reconnaissance/
Search and Rescue
Mission
)LUH &RQWURO :HDSRQ
requirements
6VWHP
Mission Communication
Mission Design Payload Functions
Integration
Sensor Control/Sensor Fusion
Vehicle Management Aerial Refuelling/Rearming
Functions
Deployment of Humanitarian
Aids
/RZ /HYHO $XWRQRP 0HGLXP /HYHO $XWRQRP $XWRQRP
+LJK /HYHO $XWRQRP
• )OLJKW &RQWURO - )OLJKW 3DWK &RPPDQGV (QYHORSH • )DXOW 'HWHFWLRQ DQG ,'
• )OLJKW PHFKDQLFV 3URWHFWLRQ • 0LVVLRQ 3ODQQHU
• $FWXDWRU FRQWURO - 0XOWLIXQFWLRQ ,QWHJUDWHG 1DYLJDWLRQ • 6LWXDWLRQ $ZDUHQHVV 0DQDJHU
• 3URSXOVLRQ (QJLQH 6VWHP 'HFLVLRQ 0DNLQJ VXEVVWHPV
&RQWURO - +HDOWK 0DQJHU • 0LVVLRQ *RDO 0DQDJHU
- )DXOW 7ROHUDQW 5HFRQILJXUDEOH &RQWURO • 6HQVRU 0DQDJHU
- 3RZHU 0DQDJHPHQW • ,QWHJUDWHG 6LJQDO DQG 'DWD 3URFHVVLQJ
0DQDJHU
• Monitoring – identify changes in a system’s observed
)LJXUH 8$9 RQ ERDUG PLVVLRQ VVWHPV state;
Therefore, the introduction of artificial intelligence • Risk Analysis – identify issues within a given course
methodology in order to implement, enhance, and improve of action and plan/provide mitigations, such as
the UAV’s autonomous functionalities seems to be a alternative routes in case of detected obstacles;
natural evolution of more conventional feedback control • Data Analysis/Processing – for instance data mining,
theory. identification of trends, extrapolation of data;
The following branches of AI seem to be the most • Optimisation – streamlining a system or object, such
significant for potentially providing autonomous as the route plan or fuel consumption for given
capabilities for the air vehicle. They were adapted manoeuvre, to achieve the best performance, resource
partially from [2], pp. 248, and they are as follows: allocation;
• Interpretation – consists of data analysis coupled with • Classification – Assignment of a category to an
domain knowledge forming high-level conclusions; object, for instance threat identification;
• Prediction – projecting probable consequences of • Control of Systems – Governing the behaviour of
given situations; complex systems. Manipulation of a system’s
• Diagnostic – Determining the cause of malfunctions interaction with the world adjusting (actuating) the
in complex situations based on observable symptoms. control surfaces, for instance to maintain a flight path
Identify abnormalities in the observed states of the The applicability of those fields to the functional
UAV’s systems and possibly suggest remedies to subsystems previously outlined is shown in the following
mitigate fault; table and it is based on the author’s engineering judgment.
• Design – finding a configuration of system
components, which can meet the required 7DEOH $SSOLFDWLRQ RI $, ILHOGV· WR IXQFWLRQDO VXEVVWHPV
performance while satisfying the constraints; AI Field Functional Subsystem
• Planning/Scheduling – devise a series of actions to Interpretation • Mission Planner
achieve certain goal and co-ordinate them • Situation Awareness Manager
sequentially, for instance sensor pointing time or • Fault Detection and ID
route planning; • Mission Goal Manager
• Decision Support/Decision Making – advise the HO, • Sensor Manager
both on the ground or on a fighter aircraft acting as Prediction • Mission Planner
mission manager, to aid them in difficult cognitive
• Situation Awareness Manager
tasks;
• Fault Detection and ID
114
5. • Mission Goal Manager no longer sufficient to provide the additional capabilities
• Sensor Manager of the UAV, it is still crucial to the implementation of
Diagnostic • Fault Detection and ID adaptive and reconfigurable control subsystems. Also,
• Fault optimisation is more and more being adopted for
Tolerant/Reconfigurable subsystems requiring resource allocation, planning and
Control scheduling. In fact, heuristic searches and genetic
• Health Manager algorithms are found to be well suited providing a quick
and optimal solution when faced with multiple variable
Planning and • Mission Planner
scenarios and incomplete information.
Scheduling • Mission Goal Manager
Over the years, several AI techniques have been
• Sensor Manager
developed, each claiming to provide significant
Decision • Mission Planner advantages over the others. No individual technique has
Support/Making • Situation Awareness Manager proved to be the answer to the problem of creating
• Fault Detection and ID machine autonomy. Therefore, it is necessary to blend the
• Mission Goal Manager different methodologies and provide a new level of
• Sensor Manager integration in order to create hybrid systems. Only by
Monitoring • Mission Planner combining different methodologies and matching them to
• Situation Awareness Manager the systems requirements it will be possible to move
• Fault Detection and ID autonomy forward.
• Mission Goal Manager In this research, the focus has been on identifying
• Sensor Manager methodologies that could allow an increased capability for
• Fault a dynamic, responsive behaviour and its implementation
Tolerant/Reconfigurable on the platform itself so to reduce the operator workload
Control and provide independence from pre-programmed flight.
• Health Manager Therefore, this paper has focused on techniques, which
Risk Analysis • Mission Planner are believed to be matured enough or most appropriate for
• Situation Awareness Manager the application to UAV.
The AI methodologies selected to provide autonomy to
• Fault Detection and ID
the UAV were:
• Mission Goal Manager
• Artificial Neural Networks (ANN or NN) ([7],
• Sensor Manager
[2],[11]);
Data Analysis & • Sensor Manager • Fuzzy Logic (Fuzzy) ([16],);
Processing • Integrated Signal and Data • Genetic Algorithms (GA) (16], [11], [2]);
Processing Manager • Reinforcement Learning (RL) ([4], [8], [13]);
Optimisation • Sensor Manager • Temporal Logic (TL) ([15], [2]);
• Power Manager • Knowledge Based Systems (KBS);
• Situation Awareness Manager • Rule Base Systems (RBS) [14], [10], [11]);
Classification • Mission Planner • Case Based Reasoning (CBR) ([9], [11], [2]);
• Situation Awareness Manager • Constrain Satisfaction Problem (CSP) ([3], [11]);
• Fault Detection and ID • Model Based Reasoning (MBR) ([2], [11]).
• Mission Goal Manager In order to understand what techniques could be
• Sensor Manager applied to the given AI fields in order to enable more
Control of Systems • Health Manager autonomous functionalities on board of the platform, the
• Flight Protection selected techniques were tabulated against the identified
Commands/Envelope AI fields and they were rated according to their
protection applicability. The following table (Table 3) shows how
• Multifunction Integrated each technique could contribute to the set tasks which
Navigation System need to be performed.
• Fault tolerant/Reconfigurable In order to provide interpretation and diagnostic
Control capabilities to the UAV systems, it is important to
introduce data fusion techniques and data mining
procedures to quickly process and analyse the data.
It is important to observe that each subsystem requires
Information and data fusion become particularly important
multiple AI fields in order to satisfy the required level of
in the sensor management system, where the sensor
autonomy and that even though control theory in itself is
information are collected and processed in order to
115
6. provide the current world state situation awareness and advantages and drawbacks of implementing the individual
update the belief functions on board of the UAV. systems with the suggested techniques and analyse the
The following is a qualitative and judgmental analysis. issues associated with their integration. Moreover, by
The analysis carried out in the previous paragraphs led implementing specific systems, different techniques may
to the definition of a functional UAV mission system become more appealing compared to the one suggested.
architecture with possible AI techniques associated with Therefore, more research should be carried out on
those subsystems. The result is shown in Figure 2 and overall UAV functional system design and to their
Figure 3. Figure 2 captures the high level decision making integration. The key difference to current systems is the
functionalities, usually performed by the pilot or the HO. inclusion of artificial intelligence techniques within the
Figure two represents the subsystems such as the guidance functional domain.
system that require a medium level of autonomy and the
most inner part of the control loop, which can be fully 7. References
automatic. The architecture has been subdivided into three
components each of which groups together the functional [1] AGARD Advisory Report, ‘Integrated Vehicle Management
subsystems according to their required autonomy level System’, AR 343, NATO, April 1996.
and to their functional flow of information. [2] Luger, G. F., ‘Artificial Intelligence: Structures and
The AI techniques selected for each subsystem are Strategies for Complex Problem Solving’, Addison Wesley,
drawn from Table 3. IV Edition, 2002
[3] Barták, R., ‘Constraint Processing’, IJCAI_07 Tutorial
[4] Berenji, H. R., et alt.,‘Co-evolutionary Perception-based
7DEOH 0DSSLQJ RI $, 7HFKQLTXHV WR $, )LHOGV Reinforcement Learning For Sensor Allocation in
Autonomous Vehicles’, IEEE, 2003
NN Fuzzy GA RL TL KBS RBS CBR CSP MBS [5] Dufrene, W. R., ‘Approach for Autonomous Control of
Scheduling/ Unmanned Aerial Vehicle Using Intelligent Agents for
Planning Knowledge Creation’, IEEE, 2004
[6] Grelle, C., Ippolito, L., Loia, V., and Siano, P., ‘Agent –
Decision
based architecture for designing hybrid control systems’,
Support/
Information Sciences, Vol. 176, pp. 1103-1130, 2006.
Decision
[7] Haykin, S., ‘Neural Networks: A comprehensive
Making
Foundation’, Prentice Hall, II Edition
Diagnostic [8] Harmon, M. E., and Harmon, S., ‘Reinforcement Learning:
Risk analysis A Tutorial’.
[9] Lees, B., ‘9th UK Workshop on Case-Based Reasoning:
Data Proceeding’, SGAI, December 2004
Analysis/ [10] Nilsson, N. J., ‘Introduction to Machine Learning: An early
Processing draft of a proposed textbook’, Robotics Laboratory, Dep. Of
Monitor Computer Science, Stanford University, Dec 1996
Optimisation [11] Russell, S. and Norvig, P., ‘Artificial Intelligence: A
Modern Approach’, Prentice Hall, 2003
Interpretation [12] Shirazi, M. A., and Soroor, J., ‘An intelligent agent-based
Classification architecture for strategic information system application’.
Knowledge Based Systems, 2006.
Control [13] Ten Hagen, S. and Kröse, B., ‘A Short Introduction to
System, Reinforcement Learning’, 7th Belgian-Dutch Conference on
Prediction Machine Learning, pp 7-12, 1997
[14] Tunstel, E., et al., ‘Rule-based reasoning and neural
Design
networks for safe off-road robot mobility’, Expert Systems,
Vol.19, No 4, September 2002
Legend: [15] Vila, L., ‘A Survey on Temporal Reasoning in Artificial
Intelligence’, AI Communications, Vol. 7, No. 1, March
Highly applicable
1994
Potentially applicable [16] Johnson, J. and Picton, P., ‘Concepts in Artificial
Not applicable Intelligence: designing Intelligent Machines, Volume 2’,
Butterworth-Heinemann editions in association with the
Open University, 2001
7. Future work
There are several issues that have not been addressed
in this paper. Future research should investigate the
116
7. Mission Mission
Objectives Requirements
Fault Detection
& ID
Mission Goal Manager CBR & CSP
GOAL Manager:
• Update Goals; Flight Path Commands
• Eliminate the one
reached/unachievable; Mission Planner
• Generate new goals
CBR, CSP, RL, GA CSP, RL, GA
PAYLOAD Integrated Signal and Data Processing
Integrated
INS/ Signal Integrated Situation
GPS Processing Data Fusion Awareness
Processing Manager
OUTPUTS:
Radio • Range KBS, GA, RL
Alt
117
• Range Rate KBS
• Angle ANN, GA,
• Potential ID,
EO Size, Shape
RL
World KBS MBS on Air
Database vehicle
SAR/
GMTI
High Level Control On-Board PC
Sensor Manger
Fault
TL, RL, CBR, GA Detection &
ID
CBR, GA, NN MBS on Sensor
)LJXUH $, WHFKQLTXHV DVVRFLDWHG WR 8$9 IXQFWLRQDO VXEVVWHPV UHTXLULQJ KLJK OHYHO RI DXWRQRP
8. Mid-Level Command Terrain database
Commands from
Mission Manager Multifunction
Flight Path Commands integrated navigation UAV States
Adaptive Mode
Trajectory Generator Transition
RL, CSP, TL KBS, RL, TL, ANN
Flight Control/ UAV States
Loads/Propulsion/Flight
Mechanics Flight
Low Level Flight
mechanics/Propulsi Control
Envelope Protection on/FCS/ Actuators
RL, CSP Actuato
FCS r Model UAV
NN, 6-
GA, DOF
Fuzzy
118
Model
Flight Health Manager
Mechanics/Propulsion/ CBS, MBS, TL Sensor
FCS/Electrical/Air Data Inputs
Engine
Model
DATALINK/INTERFACE
Fault Tolerant Control
Reconfiguration
MBS & CBS, TL, ANN,
Fuzzy
UAV States
MBS for MBS for MBS for
FCS Engine Actuators
)LJXUH $, WHFKQLTXHV DVVRFLDWHG WR 8$9 IXQFWLRQDO VXEVVWHPV UHTXLULQJ PHGLXP DQG ORZ OHYHO DXWRQRP
![flight and require constant monitoring from the human The payload functions are represented by the mission
operator (HO). Their use up to now has been described as functional system, i.e. those systems which are mission
the use of “goggles in the sky” by a human operator on the specific, and they are defined in [Ref. 1] (chp1, pp1), as
ground. those functions that include the set of all functions that
By automatic is meant a repetitive action that does not directly relate to the mission of a given vehicle
require external influence or control, but which repeats The payload of each platform is mission specific and it
itself based on some set conditions. A well known will vary with the specific mission requirement and it will
automatic system is feedback control system, which need to be fitted on the air vehicle and its control defined
automatically adjusts the input in order to obtain the and implemented. Examples of possible sensors that could
desired outputs be fitted on a UAV are:
The major difference between automatic and • Electro-Optical including Optical and IR sensors for
autonomous systems is that autonomous systems can obstacle detection and target detection
change their behaviour in response to unanticipated • Ground Moving Target Indicator (GMTI) radar and
events, whereas automatic systems would produce the Synthetic Aperture Radar (SAR)
same outputs regardless of any changes experienced by • GPS/INS navigation system.
the system or its surroundings. In robotic applications, The vehicle management system (VMS), on the other
automatic actions allow the machines to be operationally hand, is defined in [1] as the collection of functions that
autonomous, but they do not allow them to have are required for the vehicle to understand, plan, control,
decisional autonomy. and monitor the air vehicle operations. They usually
An autonomous action or event is defined as an action represent the safety critical functionalities required for the
or event that is ‘independent in mind or judgment’, a self- safe employment of the platform; hence they include all
directed, self-governing entity, not controlled by others or the flight-critical and safety-related functions.
by outside forces. The mission system functional capabilities can be
Autonomous UAVs are herein defined as air vehicle classified according to the level of autonomy that each of
systems with embedded autonomous functionalities. them would require if the HO was not included in the
In this research the focus is on identifying loop.
methodologies to allow the level of autonomy for UAVs Three major levels of autonomy have been identified:
to increase and the implementation of a dynamic, • Low Level - The low level of autonomy can be
responsive behaviour on the platform itself to reduce the considered the reactive side of the UAV, where,
operator workload and provide independence from pre- given an event, the UAV will automatically react
programmed flight. according to pre-defined limits and following given
dynamic models. This level is characterised by the
3. UAV Mission System group of functionalities that would be required in
order to fly the platform remotely. These include, but
The main gaps in current UAV systems are are not limited, to the flight control system, the
represented by the reliance on datalink, the ‘dumbness’ of actuator function, the engine or propulsion control,
the control and payload systems, which means that they and the aircraft flight mechanics and air data
are not reactive to the environment, and the latency of acquisition.
information to the ground control station. In order to • Medium Level – The medium level of autonomy
understand where the identified technological gaps can captures the ‘reflective’ capability that should be
present a significant drawback within the UAV systems, embedded on board of the platform in order to enable
the general UAV functionalities are captured. it to reason about its internal and external state. That
A UAV mission system is seen to provide the is the ability of the platform to detect internal faults
translation of mission objectives into quantifiable, or malfunctions, to self-regulate when subjected to
scientific descriptions that provide a measure to judge the unexpected events while carrying out the mission.
performance of the platform, i.e. the system in which the The functionalities of the mission systems that have
mission objectives are transformed into system been identified to belong to this class are:
parameters. According to the AGARD report on o The flight path command and performance
Integrated Vehicle Management Systems [Ref. 1], the envelope protection, such as the waypoint
mission management functions of modern aerospace following system and the guidance and
vehicle can be split into two different functional elements: navigation functions;
the payload functions and the vehicle management o The health manager and fault tolerant control, in
function. order to detect and react to possible system
failures and malfunctions;
112](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)