1. Norman Morrison - 0.1 -
Tracking Filter Engineering
The Gauss-Newton and Polynomial Filters
Typesetter:
When typesetting this book
Please include a blank line wherever I do
Please indent wherever I do
Please do not indent if I do not indent
Please set an item âlandscapeâ whenever I do
---------------------------------------
I have included typesetting requests from time to time in red
Please follow my requests
Sincerely
Norman Morrison
2. Norman Morrison - 0.2 -
Tracking Filter Engineering
The Gauss-Newton and Polynomial Filters
Tracking Filter Engineering
The Gauss-Newton and Polynomial Filters
Norman Morrison
Department of Electrical Engineering,
University of Cape Town, South Africa
Formerly Bell Labs Ballistic Missile Defense,
Whippany, New Jersey, USA
4. Norman Morrison - 0.4 -
Tracking Filter Engineering
The Gauss-Newton and Polynomial Filters
Behold, I will cause breath to enter into you, and ye shall live.
And I will lay sinews upon you, and I will bring up flesh upon you,
and cover you with skin, and put breath in you,
and ye shall live.
Ezekiel XXXVII
5. Norman Morrison - 0.5 -
Tracking Filter Engineering
The Gauss-Newton and Polynomial Filters
Acknowledgements
Many people made this book possible.
Dr. Richard Lord, formerly of the University of Cape Town and now of the Karroo Array
Telescope (KAT), stepped forward to bring his remarkable creativity and outstanding
programming skills to bear on our tracking problems. I will always be grateful to him.
Johan Kannemeyer, an extremely inventive software engineer at Reutech Radar Systems,
programmed and tested many of the algorithms. He also made many suggestions, all of
which were adopted. It was Johan who came up with the ratio matrix that is discussed in
Chapter 7.
Professors Linda Haines and Tim Dunne of the University of Cape Townâs Department
of Statistical Sciences kept their doors open for me and made available whatever time it
took to discuss my problems. They also read parts of the manuscript, found errors and
made suggestions, all of which were adopted. Without their help this book would not
have been written.
Professors Daya Reddy and Ronnie Becker of the University of Cape Townâs
Department of Mathematics and Applied Mathematics read parts of the manuscript,
found errors and offered many excellent suggestions.
Professor Mike Inggs of the University of Cape Townâs Department of Electrical
Engineering posed a problem to me that ultimately led to the writing of this book. He also
read parts of the manuscript and made suggestions. It was Mike who connected me with
the IET and with the University of Cape Townâs Open Access Database.
Professor Pieter Willem van der Walt, formerly Dean of Engineering at the University of
Stellenbosch and now Technology Executive of Reutech Radar Systems, read an early
version of the manuscript and set me on the right path.
Pieter-Jan Wolfaardt, Technology Executive of Reutech Radar Systems, constantly
challenged me with his brilliant mind and his fresh ideas. It was Pieter-Jan who brought
me into their radar project, and I will always be beholden to him for his support and
encouragement.
Ivan Gibbons, Chief Engineer of Denel Aerospace Systems, and Pieter Reyneke, Denel
software engineer, quickly caught on to the merits of Gauss-Newton. Ivan encouraged me
to keep on going, and Pieter gave up his time and programming skills to derive needed
results.
6. Norman Morrison - 0.6 -
Tracking Filter Engineering
The Gauss-Newton and Polynomial Filters
Drs. Yossi Etzion and Michael Maidanik of IAI also quickly caught on to the merits of
Gauss-Newton. Yossi made a number of suggestions, all of which were adopted, and
Michaelâs enthusiastic understanding of the advantages of Gauss-Newton served as a
constant reason for me to keep on going against the odds.
Jean-Paul de Conçeaçao, a graduate student at the University of Cape Town, asked a
question that caused me to rewrite the entire book. In his Masterâs thesis, Jean-Paul
successfully implemented a Gauss-Newton filter in an FPGA.
Roaldje Nadjiasngar, also a graduate student at the University of Cape Town, read the
manuscript, programmed a number of the algorithms, found errors that we fixed and then
verified that the algorithms were correct. Roaldje has since made use of Gauss-Newton in
his doctoral thesis with great success.
Professor Stephen Hodgart of the University of Surrey revived my interest in tracking
filters during our walks on Table Mountain, after I had been away from them for many
years. Thank you, Stephen.
Professor Dov Hazony, my doctoral thesis adviser at Case Western Reserve University in
Cleveland, Ohio, taught me, almost fifty years ago, what the word âcompleteâ means. I
can only hope that I have succeeded.
The late Professor Richard Duffin of the University of Pittsburgh was an enduring
inspiration to so many us. While writing this book I thought often of him, and also of the
late Professor Brian Hahn of the University of Cape Town who died under tragic
circumstances, much too long before his time. I learnt a great deal from both Dick and
Brian. They were good teachers and good friends, and I miss them both.
I was privileged, almost half a century ago, to work with many brilliant people at Bell
Labs in New Jersey who first introduced me to tracking filters, among them Drs. Marvin
Epstein, Paul Buxbaum, Alphonse Claus and Jack Riordan. While writing this book,
memories of those friends and what they taught me were constantly in my mind.
My very special thanks to Professor Hugh Griffiths of University College London, IET
Series Editor, and to Nicki Dennis and Paul Deards, both IET Commissioning Editors.
Hugh and Nicki gave me the encouragement I needed at a most critical time and Paul
helped with the shortening and reorganization of the typescript. My thanks also to Helen
Langley, IET Production Controller, for her seemingly endless patience, and to the many
others at the IET who participated in the publication of this book
ââș
A number of people found errors and omissions in the manuscript of this book and I have
made every effort to correct them and to ensure that it is error-free. However, I assume
full responsibility for any and all errors, omissions and incorrect statements that might
still remain.
7. Norman Morrison - 0.7 -
Tracking Filter Engineering
The Gauss-Newton and Polynomial Filters
The Greek alphabet
8. Norman Morrison - 0.8 -
Tracking Filter Engineering
The Gauss-Newton and Polynomial Filters
Preface
Why would anyone write a book about Gauss-Newton tracking filters? There are many
reasons. Here are just a few.
â The Kalman filter has become an extremely complex topic with a bewildering
8-pt
number of mutations. By contrast, Gauss-Newton filters are easy to understand and
diamonds
easy to implement. What we are offering here is a simpler, more practical approach to
tracking filter engineering that will appeal to both experienced practitioners and to
newcomers.
â Gauss-Newton filters work, and they work extremely well. We and our colleagues
have established that â by using them in a variety of operational hardware as well as
in simulations.
â Once launched, Kalman filters1 have only one type of memory â expanding â
something that is impossible to change.
By contrast, Gauss-Newton filters can have fixed length, variable length or expanding
length memories and any of these can be changed at will during filter operation.
Gauss-Newton filter models can also be changed during filter operation â if need be
from one cycle to the next.
This ability to change their memory lengths and filter models opens up powerful new
ways by which to track manoeuvring targets, ways that are difficult, if not impossible,
to implement with Kalman filters.
â Gauss-Newton filters can be used when the observations are stage-wise correlated,
something that cannot be done with Kalman filters, and while it is true that
observations in filter engineering are often stage-wise uncorrelated, that is not always
the case.
â Perhaps most important of all, Gauss-Newton filters are not vulnerable to the
instability problems that plague the extended Kalman filters â in which,
unpredictably, the estimation errors become inconsistent with the filterâs covariance
matrix.
Gauss-Newton filters therefore do not require what is known in Kalman parlance as
âtuningâ â searching for and including a Q matrix that one hopes will prevent
1
The words âKalman filterâ mean different things to different people. When we use them here we are
referring specifically to the mainstream version appearing in References 3, 6, 22, 23, 24, 25, 30, 31, 48, 50,
53, 54, 55, 56, 57, 173 and 174. The equations of the mainstream version appear in Appendix 11.1.
9. Norman Morrison - 0.9 -
Tracking Filter Engineering
The Gauss-Newton and Polynomial Filters
something that is both unpredictable and almost inexplicable, and in the process
degrading the accuracy of the Kalman estimate.
This in turn means that in practice Gauss-Newton filters are more accurate, and that
they are easier to implement, require shorter project time, are extremely robust and,
best of all, unconditionally stable.
And what can one say about execution time?
Using Gauss-Newton on a 1958-era machine, the extraction of a satelliteâs orbital
parameters from a set of radar observations would have added about one hour of
processing time after the observations had been obtained.
However, on todayâs readily available desk-top or lap-top machines, that hour has shrunk
to a few hundred milliseconds, and as computer performance and other technologies such
as FPGA continue to advance, it is safe to say that in the not too distant future it will have
shrunk to tens of milliseconds or even less.
ââș
As envisaged here, tracking filters are computer programs that extract estimates of the
states of dynamic processes in near real-time from sequences of discrete real
observations.
Tracking filter engineering is the collection of techniques, knowledge and activities that
we employ when we create and operate such filters.
The following is a partial list of fields in which tracking filters are used:
Air-traffic control, artificial intelligence, astronomy, atmospheric re-entry,
ballistic missile defence, chemical engineering, civil engineering, control
engineering, econometrics, electrical engineering, GPS and WAAS, industrial
engineering, inertial navigation, mechanical engineering, missile engineering,
neural networks, physics, pilotless-aircraft engineering, radar and tracking,
robotics, satellite engineering, space navigation, statistics, telecommunications,
telescope engineering and wind-power engineering.
This book will be of interest to three groups of people:
8-pt â Practitioners working in the above or similar fields.
diamonds
â Graduate-level newcomers wishing to learn about Gauss-Newton and
polynomial filters, and how they can be used in filter engineering.
â University lecturers who might wish to include material on Gauss-Newton
and polynomial filters in graduate-level courses on tracking filter engineering.
10. Norman Morrison - 0.10 -
Tracking Filter Engineering
The Gauss-Newton and Polynomial Filters
ââș
Tracking filter engineering came into existence around 1958 at the start of the
Satellite Era.2
Engineers and scientists had long been fitting curves to data by least squares, but this was
different. Now there were huge tracking radars making observations on artificial
satellites, from which estimates of orbital parameters â known as Keplerians â were to be
extracted, and the filtering algorithms for doing so were far more sophisticated than
simple least squares.
Astronomers had been estimating Keplerians of objects in orbit since the time of Gauss,
using what was known as the Gauss-Newton algorithm. However, such calculations were
done by hand and it took many months to obtain results, whereas with artificial satellites
the need was for the observations to be processed by computers and for the estimates to
be available in near real-time.
Running Gauss-Newton on a 1958-era machine would have taken roughly an hour to
extract a satelliteâs Keplerians from a set of radar observations after the latter had been
obtained. Such a delay was clearly too far removed from real-time, and so Gauss-Newton
could not be used.
In its place two new filters were devised â both related to Gauss-Newton but
computationally different. The first of these was published by Swerling in 1958 3 and the
second by Kalman4 and Bucy5 in 1960/61, both of which could extract Keplerians in near
real-time on the existing machines.6
Starting from the extraction of Keplerians, the use of the Kalman filter spread rapidly to
the many other fields listed above where today it occupies the dominant place in filter
engineering. The Kalman filter has also come to occupy the dominant place in filter-
engineering curricula throughout the academic world.
ââș
But while all of this was taking place, something else was also happening.
2
Sputnik-1 was placed in orbit by the Soviet Union on October 4th, 1957. See Ref. 40.
3
Peter Swerling (American mathematician and radar theoretician), 1929 â 2000. See References 6, 7, 8, 9,
91.
4
Rudolf Emil Kalman (Hungarian-born American scientist), 1930 â . See References 1, 2, 6, 92.
5
Richard S. Bucy (American mathematician), 1935 â . See Reference 93.
6
See Chapter 11 for what we mean by âSwerling filterâ and âKalman filterâ.
11. Norman Morrison - 0.11 -
Tracking Filter Engineering
The Gauss-Newton and Polynomial Filters
Since 1958, computer technology has made advances that are difficult to comprehend,
and the end is still nowhere in sight. Consider for example the following item that
appeared not long ago the Intel corporate website:
âIntel remains at the forefront of Mooreâs Law. Our 22 nanometer technology-based Intel
microprocessors will enable never-before-seen levels of performance, capability, and
energy-efficiency in a range of computing devices.â
And so, while it may have been appropriate to reject Gauss-Newton in 1958, it is no
longer appropriate to do so today. In this book we accordingly do something different.
Many books have been written about the Kalman filter â the author is aware of at least
fifteen. However, to our knowledge nobody has yet written a book about the Gauss-
Newton filters and their remarkable fit to certain important areas of filter engineering.
This then is our attempt to do so, and to present those filters in a readable and self-
contained way.
ââș
The Gauss-Newton filters possess attributes that make them particularly well-suited for
use in tracking filter engineering, among them the following:
10-pt â Ideal for tracking both manoeuvring and non-manoeuvring targets.
diamonds
â Can be used when the observations are stage-wise correlated.
â Are not vulnerable to the instability phenomena that plague both
the Kalman and Swerling extended filters, and hence require no tuning.
â Require no initialization in the all-linear case and very little in the
three nonlinear cases.
â Possess total flexibility with regard to
â Memories that can be configured as fixed length, variable length or
8-pt
expanding length â and if need be, reconfigured cycle by cycle.
diamonds
â Filter models that can also be reconfigured, if need be, from cycle by
cycle.
10-pt â Offer immediate access to the residuals which can then be used to
diamond run goodness-of-fit tests.7
7
Goodness-of-fit tests are used by the Master Control Algorithms (MCAâs) to control a Gauss filterâs
memory length and filter model when tracking manoeuvring targets, in such a way that its performance is
14. Norman Morrison - 0.14 -
Tracking Filter Engineering
The Gauss-Newton and Polynomial Filters
10-pt â In two of them we have implemented prefiltering using the polynomial filters,
diamond and in which the main filters are Gauss-Newton, Kalman and Swerling â all three of
which process precisely the same observations.
These two programs
8-pt â Demonstrate how prefiltering works.
diamonds
â Enable you to witness instability in both the Kalman and Swerling filters 10
and the fact that Gauss-Newton, operating on exactly the same data, is always
stable.
â Make it possible to time the executions of the three main filters.
â Enable you to compare their performance in a way that is âapples-to-
applesâ.
ââș
The polynomial filters discussed in this book were first devised when we worked at Bell
Labs Ballistic Missile Defense in Whippany New Jersey from 1964 to 1968. They have
been written up twice before â once in our first textbook (see Reference 6) and again in
Reference 24.
However, over the succeeding years we have learned a great deal more about them and
continue to do so11 and in this book we provide a complete discussion which includes
much of that new material.
ââș
We have tried to keep the book as generic as possible, i.e. applicable to multiple fields.
However, in some places we were forced to place it in a specific context, and the field
that we selected was that of radar and tracking.
Some of our discussion is thus slanted towards target motion sensing using two types of
radar â those with constantly rotating antennas (track-while-scan or TWS radars) and
those with steerable antennas (tracking radars).
In this regard we beg forgiveness from those readers who use filters in other fields.
However, we assume very little knowledge of radar and tracking and instead place our
primary emphasis on filtering techniques, and so we hope that the book can be read and
used by practitioners and students who are interested in using tracking filters in almost
any field.
10
The Swerling filter becomes unstable whenever the Kalman filter does, and in all regards their
performances are essentially identical.
11
See References 112 and 118.
15. Norman Morrison - 0.15 -
Tracking Filter Engineering
The Gauss-Newton and Polynomial Filters
ââș
The material in the book has been taught on six occasions on three continents to a mix of
practicing filter engineers, graduate students and university lecturers.
It is well suited for a complete single-semester graduate course of 24 lectures or else for
partial incorporation into other graduate courses, and contains more than a sufficiency of
examinable material that such courses require. The fourteen computer programs and the
end-of-chapter problems and projects will enable students to envisage and apply the
concepts that are discussed in the chapters.
We have made every effort to keep the book readable and friendly. We can only hope that
we have succeeded.
Norman Morrison
Cape Town, July 2012
16. Norman Morrison - 0.16 -
Tracking Filter Engineering
The Gauss-Newton and Polynomial Filters
Viewing the Track-While-Scan Video Clips
In order to see, first hand, how well-suited these filters are to tracking manoeuvring
targets, we suggest at this point that you turn on your computer and retrieve the down-
loadable material. (See downloading instructions at the start of Chapter 1.)
Once that is done, please read the following document so that you understand fully what
you will be viewing:
Video_ClipsTWSDocumentsReadme.pdf
Then please take a few minutes to view the track-while-scan video clips that are
contained in the folder
Video_ClipsTWSFlights.
This will enable you to see how effectively the Gauss-Newton filters under MCA-1
control (Master Control Algorithm, Version-1) are able to perform when tracking
manoeuvring targets.
Words containing the letter Z
We lived in the USA for thirty years and were under the impression that the use of the
letter z in words like maximize and minimize was a distinctly American thing, and that on
our side of the Atlantic one would spell them as maximise and minimise.
Imagine our surprise when we consulted the Shorter Oxford Dictionary and found that it
also uses the letter z in many such cases. Throughout the book we have attempted to
follow the usage in that dictionary.
17. Norman Morrison - 0.17 -
Tracking Filter Engineering
The Gauss-Newton and Polynomial Filters
Contents