Invited talk at 2014 Fall AGU in San Francisco

1. Balancing Fact and Formula in the
Science of Complex Systems: The
Example of 1/f Spectra
Nick Watkins
NickWatkins@mykolab.com

2. • Slides 1-24 were given as an invited talk on
19th December 2014 at the Fall AGU in San
Francisco, beginning the Session on
Intermittency and Dynamical Complexity in
Space Plasmas from the Sun to Interplanetary
and Planetary Environments.
• I have also included the spares prepared in
case of questions and being developed for
future talks.

3. Thanks
• Tom, Giuseppe and Marius for inviting me
• Holger Kantz for hosting me in Dresden since
September 2013
• Co-authors Tim Graves, Christian Franzke,
Bobby Gramacy, Scott Osprey and Paulo
Davini
• Discussions with all of the above and Sandra
Chapman, Eli Barkai, Igor Sokolov, Rainer
Klages and Aleksei Chechkin among others

4. Theme
Hurst
effect
Will today distinguish
three things often taken
as same
• Observed growth of
range in time series:
“Hurst effect”

5. Theme
1/f
Hurst
effect
Will today distinguish
three things often taken
as same
• Observed growth of
range in time series:
“Hurst effect”
• Observation of a
singularity at zero in
Fourier spectra: “1/f”

6. Theme
(S)LRD
1/f
Hurst
effect
Will today distinguish three
things often taken as same
• Observed growth of range
in time series: “Hurst
effect”
• Observation of a singularity
at zero in Fourier spectra:
“1/f”
• The long range dependence
seen in stationary 1/f case:
(S)LRD.
• Using 1/f as a diagnostic of
LRD assumes stationarity

7. Fact: Anomalous growth of range
Hurst
Effect
Hurst, Nature, 1957
“I heard about the … Nile … in '64, ... the variance doesn't draw like
time span as you take bigger and bigger integration intervals;
it goes like time to a certain power different from one. … Hurst …
was getting results that were incomprehensible”. – Mandelbrot, 1998

8. Formula: Long Range Dependence
(S)LRD
Hurst
Effect
• Mandelbrot, van Ness, and
Wallis, 1965-69
• First [history in Graves et al,
arXiv, 2014a] demonstration
that Hurst effect could be
explained by stationary long
range dependent process
• Model, fractional Gaussian
noise [see also Kolmogorov’s
“Wiener Spiral”], had singular
spectral density at lowest
frequencies.
'( ) ~ 
S f f

9. The 1/f “paradox”
If spectral density '( )
then i) it is singular as
and ii) if we define an autocorrelation
function via ( ) ( ) ( )
and use Wiener-Khinchine theorem to
get from Fourier transform of
~
0

  


  

S f f
f
x t x t
S
falls off as power law, and
'( )
then
summed lags "blow up"
its
( )

   
f

10. Ionosphere
Magnetosphere
1 1
2 2
, 22fBm: ( ) ( ) ( )~ ( )
 
 
  
 
 
  
H
H R
H
X t t s s dL s
Infinite range
memory kernel
Gaussian

11. Fractional motions and noises
0 100 200 300 400 500 600 700 800 900 1000
-18
-16
-14
-12
-10
-8
-6
-4
-2
0
2
Fractional Brownian motion, H=0.7
1 1
2 2
, 22fBm: ( ) ( ) ( )~ ( )
 
 
  
 
 
  
H
H R
H
X t t s s dL s
Build a nonstationary, self similar
walk … (used wfbm in Matlab)
fractional motion
2 1  H

12. Fractional motions and noises
0 100 200 300 400 500 600 700 800 900 1000
-18
-16
-14
-12
-10
-8
-6
-4
-2
0
2
Fractional Brownian motion, H=0.7
1 1
2 2
, 22fBm: ( ) ( ) ( )~ ( )
 
 
  
 
 
  
H
H R
H
X t t s s dL s
Build a nonstationary, self similar
walk … (used wfbm in Matlab)
0 100 200 300 400 500 600 700 800 900 1000
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
Fractional Gaussian noise
fractional motion
Then differentiate to give a
stationary LRD noise
fractional noise
2 1  H 2 1  H

13. Ionosphere
Magnetosphere
[…], if infinite dependence is necessary it does not mean that
IBM's details of ten years ago influence IBM today, because
there's no mechanism within IBM for this dependence. However,
IBM is not alone. The River Nile is [not] alone. They're just one-
dimensional corners of immensely big systems. The behaviour of
IBM stock ten years ago does not influence its stock today through
IBM, but IBM the enormous corporation has changed the
environment very strongly. The way its price varied, went up or
went up and fluctuated, had discontinuities, had effects upon all
kinds of other quantities, and they in turn affect us. –
Mandelbrot, interviewed in 1998 by B. Sapoval for Web of Stories
One resolution of 1/f paradox
In modern fractional Langevin models fGn is noise term
e.g. reviews of Metzler et al, PCCP, 2014; Watkins GRL,
2013.

14. Ionosphere
Magnetosphere
[…], if infinite dependence is necessary it does not mean that
IBM's details of ten years ago influence IBM today, because
there's no mechanism within IBM for this dependence. However,
IBM is not alone. The River Nile is [not] alone. They're just one-
dimensional corners of immensely big systems. The behaviour of
IBM stock ten years ago does not influence its stock today through
IBM, but IBM the enormous corporation has changed the
environment very strongly. The way its price varied, went up or
went up and fluctuated, had discontinuities, had effects upon all
kinds of other quantities, and they in turn affect us. –
Mandelbrot, interviewed in 1998 by B. Sapoval for Web of Stories
One resolution of 1/f paradox
In modern fractional Langevin models fGn is noise term
e.g. reviews of Metzler et al, PCCP, 2014; Watkins GRL,
2013.
• Resolution of apparent paradox is that world as a whole is
Markovian, the memory is a consequence of looking at a piece
of it. Generalises the Mori-Zwanzig approach.

15. Often see 1/f spectra and heavy tails
Ionosphere
Magnetosphere
Ground-based
magnetometers
sense ionospheric
currents
0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
x 10
5
0
500
1000
1500
2000
2500
3000
1978 AE data: threshold percentile=99
Time, t [minutes]
AE[nanoTesla]
AE data
threshold
Slow 1/f region in AE power spectrum
Tsurutani et al, GRL, 1991
See also Hush et al, Poster
SH53A-4209 this afternoon
Moscone South

16. Joint inference on LRD and heavy tails.
Tim Graves’ PhD developed Bayesian method: tested on α-stable ARFIMA(0,d,0)
where heavy tails & LRD co-exist, see Graves et al, arXiv, submitted CSDA 2014b;
& my talk from AGU, 2012 on Slideshare.
1.5 0.15d 
Test on Synthetic ARFIMA data

17. Other models for “1/f”
Ionosphere
Magnetosphere
Selecta H
Selecta N

18. 1/f without (S)LRD
(S)LRD
1/f
Hurst
effect
• Before (S)LRD models,
Mandelbrot [1963-67]
had proposed other 1/f
models which were not
stationary LRD in same
sense as fGn.
• Solved 1/f paradox by a
different route. Still
little known in the
geosciences [but see
Klemes, WRR, 1974].

19. Ionosphere
Magnetosphere
• Abrupt state changes
• Fat distributions of switching times: “Levy” (E[t^2] = ∞) case.

20. The conditional spectrum:
Magnetosphere
Mandelbrot 1967 reviewed in N2, Selecta, 1999

21. The conditional spectrum:
Magnetosphere
Mandelbrot 1967 reviewed in N2, Selecta, 1999
• “Numerical … 1/f … spectrum … need not … estimate … Wiener-
Khinchine spectrum”. Instead “depends on conditioning length
T”. Unlike stationary LRD model, singularity is an artefact.

22. Hurst effect from state changes
Ionosphere
Magnetosphere
Franzke et al, in review, Sci. Rep., 2014
• Interestingly, the classic Lorenz 63 model (columns 1 and 2) can
generate Hurst effect in some measures, such as DFA, even
without long tailed waiting times between regime shifts. Confirms
that Hurst effect is easier to generate than 1/f or full blown S(LRD).

23. Formula versus fact
“Like the ear, the eye is very
sensitive to features that the
spectrum does not reflect. Seen
side by side, different 1/f noises,
Gaussian [i.e. fGn], dustborne [i.e.
fractional renewal] and multifractal,
obviously differ from one another”-
Mandelbrot, Selecta N, 1999.
“Nothing can be more fatal to
progress than a too confident
reliance on mathematical symbols;
for the student is only too apt to …
consider the formula and not the
fact as the physical reality”.
Thomson (Kelvin) & Tait, 1890
edition.

24. Summary
• The 1/f paradox: The "1/f" spectral shape seen
throughout physics, & why it seems paradoxical.
• Many faces of 1/f: Mandelbrot promoted two
resolutions of paradox in the mid 60s: stationary long
range dependence (S)LRD , or what he dubbed
“conditionally stationary” renewal models. History is
in Graves et al, arXiv, 2014a. One “formula”
consistent with different “facts”.
• Ways forward ? Mandelbrot’s Selecta (Vols. N &H)
urged use of our eyes as well as formalism, and I’ve
advertised two new results in this spirit: Bayesian
method for choosing between (S)LRD models, Graves
et al, arXiv, 2014b, and work on a dynamical origin
for Hurst effect in the Lorenz model (Franzke et al,
Scientific Reports, in review, 2014 ).

25. SPARE SLIDES

26. THE ROAD TO MULTIFRACTALS

27. So what were the other models ?
• Additive models extending fGn like fractional
hyperbolic model of Mandelbrot & Wallis [1969].
• Multiplicative, multifractal models exhibiting
volatility bunching as well as 1/f spectra and fat
tails-1972 (turbulence), 1990s (finance).
• And the class he referred to as “dustborne”: the
least known of his papers, from 1965-67, though
closely related to the Alternating Fractional
Renewal Process, the CTRW and modern work on
weak ergodicity breaking.

28. Multiplicative multifractal cascades
Many systems have aggregation, but not by an additive
route. Classic example is turbulence.
One indicator: lognormal or stretched exponential pdf …
Selecta
Volume N
1999

29. Multifractals and volatility clustering
another: correlations between the absolute value
of the time series- or here, in ionospheric data,
the first differences.
0 0.5 1 1.5 2 2.5 3 3.5 4
x 10
4
-600
-400
-200
0
200
400
600
increments,r
First differences of AE index January-June 1979
-100 -80 -60 -40 -20 0 20 40 60 80 100
-0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
lag
acf
AE data: acf of returns
-100 -80 -60 -40 -20 0 20 40 60 80 100
-0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
lag
acf
AE data: acf of squared returns

32. CONDITIONAL STATIONARITY, NON-
ERGODICTY ETC

33. The infrared catastrophe as mirage:
Ionosphere
Magnetosphere
Reviewed in N2, Selecta, 1999

34. The infrared catastrophe as mirage:
Ionosphere
Magnetosphere
• Rather than representing a true singularity at lowest frequencies,
as seen in stationary LRD, in this case he described the infrared
catastrophe as a ``mirage“ arising from fact that here “measured
square Fourier moduli do not estimate a Wiener-Khinchine
density”. Same formula changes meaning because of different facts
Reviewed in N2, Selecta, 1999

35. Ionosphere
Magnetosphere
(Fifth Berkeley Symposium on
Probability, 1965)

36. Non-ergodicity:
Ionosphere
Magnetosphere
• M 1967 illustrated this first in case of single jump, infinite interval

37. Non-ergodicity:
Ionosphere
Magnetosphere
• Mandelbrot 1967 then discussed case of many state changes with
power law waiting times

38. FRACTIONAL STABLE MODELS

39. Distinct from fBm and fGN:
Ionosphere
Magnetosphere
Mandelbrot 1967 was prepared in the same period as Mandelbrot
and van Ness on fBm and fGn, which it cites as ``to be published".
In it contrast is clearly drawn between the conditionally
stationary, non-Gaussian renewal process as a 1/f model and
his stationary, Gaussian (fGn) model:

40. LRD and its controversies
Ionosphere
Magnetosphere
• Mandelbrot [1965]; M and Van Ness[1968] proposed use of
fractional Brownian motion. Non stationary self similar model which
generalises Wiener process, has spectral index between -1 and -3.
• … and its derivative, fractional Gaussian noise,
which is stationary, and long range dependent.
• Unlike the stable amplitude distribution we just saw, the power
spectra of fBm and fGn are singular at zero frequency. In Bm (and fBm)
this is a result of its nonstationarity … [Selecta, N2, 1999]

41. Ionosphere
MagnetosphereMandelbrot & Wallis [1969] first attempt to unify long range memory
kernel of fGn with heavy tailed amplitude fluctuations - called it
“fractional hyperbolic” model because of its power law tails.
Anticipated the versatile linear fractional stable noises, but it
didn’t satisfy him completely for problems he was looking at.
Additive fractional stable class

42. HISTORY

43. A neglected paper … ?
Ionosphere
Magnetosphere
Mandelbrot 1967 received far less attention than either papers on heavy tails in finance
in early 1960s or the series with Van Ness and Wallis in 1968-69 on stationary fractional
Gaussian models for LRD. Only about 20 citations in its first 20 years !
Was apparently unknown to Vit Klemes [Water Resources Research, 1974], who
essentially reinvented it to criticise fBm. Still seems a relatively little known paper. Not
cited by Beran et al [2013], and while listed in the citations of Beran [1994] I can’t find
in the text. Some exceptions, e.g. Grigolini et al, Physica A, 2009.
Although he revisited the paper with new commentary in Selecta Volume N [1999]
dealing with multifractals and 1/f noise, Mandelbrot neglected to mention it
explicitly in his popular and historical accounts of genesis of LRD such as Mandelbrot
and Hudson [2008].
Why ? Because it wasn’t as popular as fBm/fGn ? Or because it wasn’t as “beautiful” ?

44. … Whose time has come ?
Ionosphere
Magnetosphere
• Should we pay more attention to this class of models ? As
the AFRP [Lowen and Teich’s book], and as models for weak
ergodicity breaking [c.f. Niemann et al, 2013], we already
are ! Also links to the CTRW.
• Particular value of looking back nearly 50 years to how
Mandelbrot saw these models is to see how they fit into
“the panorama of grid-bound self-affine variability” as he
later put it [Selecta, 1999, N1].
Helps link maths and physics, the formula & the fact.
And inform future work.

47. Ionosphere
Magnetosphere
And so my argument has always [sic] been that each of these
causal chains is totally incomprehensible in detail, probably
exponentially decaying. There are so many of them that a very
strong dependence may be perfectly compatible. Now I would like
to mention that this is precisely the reason why infinite dependence
exists, for example, in physics. In a magnet- because two parts far
away have very minor dependence along any path of actual
dependence. There are so many different paths that they all
combine to create a global structure. In other words, there is no
global structure in one dimension, but there's one in two and three
dimensions etc. for magnets -the basis of Onsager's work and the
whole theory. And in economics there is nothing comparable to
these calculations, but the intuition of what they represent is the
same – BBM, op cit
A critical phenomenon ?