Updated (version 2.3) Easy (Complexity Theory), Probability & Disorder,Causality & Feedback,System Intricacy, Emergence (a 3D Space Perspective)-Ver 2.3.ppt

1
Probability, Causality, Intricacy, and Emergence
“Complexity Space”
An Easy & Structured Approach to the CONCEPTS of :
(Complexity Theory), (Probability & Disorder),
(Causality and Feedback) and (Complex Systems)
Complexity Theory
HABIB’s Complexity 3D Perspective
Harmonizing the Concepts of Probability, Causality, Intricacy, and Emergence.
Downloadable (for free) for Non-members
(and is : Virus, Malignancy, and Macro Free)
VERSION 2.3 , September 5th 2022
To get the Latest Version: Open https://www.slideshare.net/EmadfHABIB2/
You will Find ONLY ONE File Named :
“UPDATED (Version <whatever>) Easy (Complexity Theory) … “ ,
While other files are named “Outdated” or have a Completely Different Name (Other Presentations)
Eng. Emad Farag HABIB

2
Easily forming a "Structured-Knowledge" Idea
About the CONCEPTS of :
Probability, Causality, Intricacy, and Emergence.
Complexity Theory
Apology: bad Presentation-form!: EG Rivals: Severe Microwave Tortures & DROWSY Fabrications

3
Easily forming a "Structured-Knowledge" Idea
About the CONCEPTS of :
Probability, Causality, Intricacy, and Emergence.
Via a Basic Starting Point: the “Deteministic” Universe !
Then: 4 Additions:
[ Probability, Causality, Intricacy, and Emergence ]
Complexity Theory

4
TOC
Probability, Causality, Intricacy, and Emergence:
CONCLUSION !
4-Realms: ~ A Pre-requisite : and A Special Request
Quotes (Complexity)
Emphasis: on Notions / Distinctions
Exploring the 4-Realms
Complexity Space (A Coherent Perspective)
Open Questions, CONCLUSION, Learned Lessons
Appendices ( Defining … Applications) / ( Very Draft Slides) / Counter-Vagues
Acronyms & Abbreviations / Otherness(relationships) / Proposed Opinions / Cybernetics & Physics
Quotes & Proverbs (Science & Intellect)
References
Complexity Theory
Farag HABIB

5
Eng Emad Farag HABIB
“CONCLUSION”
A Proposed 3D Perspective that may prove Useful in :
 Focusing our EFFORTS: on the Concerned “Constituent” of the Complex Sy
 If Main Concern is the (System/Environment)-Boundary Region (X) :
• i.e. The Macro & Statistical Aspects of the Complex System .
 If Main Concern is the (Inter-Components) Region (Y) :
• i.e.: the Links, Causality, and Feedback Aspects . ( cf System Links & Interrelations )
 If Main Concern is the (Intra-Component) Region (Z) :
• i.e.: the Components themselves: ( cf Intricacy: [Diversity/ Numerosity/ and Nestedness] , … etc . )
 Classifying Complexity ISSUES: Measures, Phenomena & Concepts :
 Classify Issues in a Structured-Knowledge Scheme (cf MIT paper of "Seth Lloyd")
 Attaining better INSIGHT: to Complexity Dynamics :
Insight to Details: e.g.: how the (Non-Equilibrium) Condition begins and proceeds,
then how the counter-process begins and proceeds (cf the CAS slides in Appendix F)
Or attaining insight to “Big Picture”: e.g.: how to COMBINE many Measures :
to evaluate the (Overall Complexity) of the Concerned system :
e.g. to Compare Two Systems or Two system-states.

6
4-Realms :
Probability
Causality
Intricacy
Emergence
4-Realms :
Probability
Causality
Intricacy
Emergence

7
You Better have some idea about :
Fractals
Best Source: https://en.wikipedia.org/wiki/Fractal - Wikipedia.html
MAS: Multi-Agent Systems
Best Source: https://en.wikipedia.org/wiki/Multi-agent system - Wikipedia.html
Complex Systems
Best Source: Book “What Is a Complex System” : by Ladyman & Wiesner
published with Yale University Press : and you can find their “Reading Sample”
and Chapter(s) via google: “WhatisacomplexsystemReadingSample.pdf”
CAS : Complex Adaptive Systems
Best Source: https://en.wikipedia.org/wiki/Complex adaptive system -
Wikipedia.html
4-Realms :
Probability
Causality
Intricacy
Emergence

8
Many Slideshare Powerpoint
Presentations gives an Excellent Introduction to
Complexity:
e.g.: Slideshare : “Complexity Theory Basic Concepts” , by John Cleveland
4-Realms :
Probability
Causality
Intricacy
Emergence

9
Preliminary Notes:
Probability:
In the sense of “INHERENT” & Intrinsic Probability .
Causality:
Entities + LINKS , surrounded by an Environment // Open (non-Equilibrium) Systems
Complexity (Intricacy):
Definition(s) by “Jochen Fromm” :
“Roughly, by a complex system I mean one made up of a large number of parts that
interact in a non-simple way.”
Complexity is synonym to “DEPENDENCE” (of Factors, Entities-interactions, and Links
that Contribute to System dynamics ) , The opposite to Complexity in NOT simplicity,
but (INDEPENDENCE)
Emergence:
In the sense of producing a higher-Complexity ENTITIES !
4-Realms :
Probability
Causality
Intricacy
Emergence

10
( ~ A Special Request )
If some Terms & Concepts mentioned in this presentation:
like :
“Random”, Complexity, Chaos,
Non-deterministic Universe, Equifinality,
Emergence, “Evolution”, Simulation,
“Artificial Life” … :
In their Scientific Context :
Cause you any troubles/issues related to Religion or Politics :
Or: If you think that : this presentation is a “Unification Endeavor” !
Or is the “Theory of Everything” !
PLEASE STOP Reading this Presentation .
( You can just suffice by reading only one slide : “What Complexity is NOT” )
Complexity Theory
ad Farag HABIB
4-Realms :
Probability
Causality
Intricacy
Emergence

11
Users’ Guide : Where to Start from ?
Are You a [ Novice/ Acquainted/ Expert ] ?
Complexity Theory
ad Farag HABIB
If You have Some Idea about Complexity Theory,
i.e. You are Acquainted with the Topic,
then Continue and go on to the next slide !
( and Please Give special Attention to the slides titled : “Difficult First” ! Slides,
They present what you need to know and comprehend to expand you knowledge)
Continue to next slide
If Yoy are an Expert : then only one thing can be said :
Please : Review/ Comment/ Advice/ Rebuke! : and inform the author.
( May like to revisit the Presentation’s Conclusion ? )
If You are a Novice to Complexity Theory,
just getting an Idea about the subject:
You can Skip the “Difficult First” Slides,
and Start Directly by : “Complexity <> Randomness”
4-Realms :
Probability
Causality
Intricacy
Emergence

12
( Quotes )
“Complexity science is so important in today's world ..
Many of the most important problems
in Engineering, Medicine, and Public Policy
are now addressed with the ideas and methods of complexity science.”
James Ladyman (University of Bristol), Karoline Wiesner (Universität Potsdam), August 2020 ,
DOI:10.12987/yale/9780300251104.001.0001
And Author’s book “What is a complex system?” (published with Yale University Press)
“Complexity is A MULTI-FACETED Phenomenon,
involving a variety of features .. “
( same a/m authors )
“A variety of Different Measures would be required
to capture all our intuitive ideas
about what is meant by complexity”
The late Physics Nobel Laureate : “MurrayGell-Mann”
Complexity Theory
ad Farag HABIB
4-Realms :
Probability
Causality
Intricacy
Emergence
Importance of Complexity:
Complexity “Space” :

13
( Quotes )
“ ... to begin thinking along the LINES of complexity theory.
Future Scholars and Scholar-Practitioners
will need to think and act Differently
when facing Complexity. “
John R. Turner and Rose M. Baker :
Complexity Theory An Overview with Potential Applications for the Social Sciences ; doi:10.3390/systems7010004
“Focusing on Information Flow
will help us to understand better
how cells and organisms work.”
Nobel Laureate Paul Nurse
Complexity Theory
ad Farag HABIB
4-Realms :
Probability
Causality
Intricacy
Emergence
Complexity Importance & “Space” :
Complexity Core-Issue is “Information Flow” :

14
Difficulty First ! ..
Next 5 Slides will Emphasis
The “Complexity Science”
Is it fake? Is It a “Conspiracy Theory” Science ??!!
Or a real Science & “Extension”
to “Deterministic” Sciences ?
Next 5 Slides: Will Emphasis the location of the “Science of Complexity”
In the Histroy & Story of Science in General …
Given first in “Difficult” Statements !,
then in “more-Difficult” Mathematical Statements !!
But later-on: a much more Easier slides will follow …
Complexity Theory
4-Realms :
Probability
Causality
Intricacy
Emergence

15
Emphasis: on Basic Notions (#1 of 5)
Through the Ages & Centuries :
Humanity have achieved a great success in tackling “Matter” & “Energy”
but it is still so hampered regarding “INFORMATION” !
i.e. : Humanity have well tackled the issues of “Matter” & “Energy”,
In what is called Classical Sciences :
But then we had to deal with a totally different issue :
the issue of ( “INFORMATION” )
Ubiquitly-encoutered in what is called Contemporary Sciences …
Complexity Theory
4-Realms :
Probability
Causality
Intricacy
Emergence

16
So: Keep in mind the following while reading next slides
On One SIDE:
Classical (Traditional) Sciences, Matter & Energy,
Direct-Causality, Causal Universe, Analogue
On the Other SIDE:
Contemporary Sciences, Information,
“Circular Causality” ! , “Equifinality” !! , “Digital” !!!
In addition to :
( -ve Feedback) for Stability & Regulation ,
( +ve Feedback) for Flexibility & Emergence .
Complexity Theory
4-Realms :
Probability
Causality
Intricacy
Emergence

17
[ Probability, Causality, Intricacy, Emergence ]
For Centuries Humanity had to deal with the
( Macro & Meso )-scale objects,
Discovering the notions of “Matter” & “Energy”
Then ( Micro )-scale objects revealed the notion of ( INFORMATION ! )
A difficult start (in ~1900) led to discovery of : Quantum ( ~~ Digital !) & Uncertainty,
Expanding our Intellect to a Probabilistic Realm
Then Systems , Information Theory (1948), Cybernetics, …
Then: Complexity or Intricacy: Climate Change, Immune system,
Stock Exchanges,…
Then: Emergence : Living Creatures capabilities :
not just ( ADAPTING ) to a varying environment, Nor ( Regenerate itself: Autopoiesis )
But even-more: EMERGE new higher-complexity Entities !
Complexity Theory
4-Realms :
Probability
Causality
Intricacy
Emergence

18
In Mathematical Notations:
Complexity developed via long phases of “MATHEMATICALLY REPRESTATIONS”:
[ Statistics & Arithmetic Mean, RMS/ then: Geometric/ then: Sequence, Digital/
Then: Discrete Math.: Edges (Links) rather than Nodes! / then: Emergence]
From the Deterministic Realm:
To Statistics: that just “optionally” summarizes much in few:
Using a Single-number (or max 2): Average, Standard Deviation, …
(an Arithmetic Average, plus a “Root-mean Square” average)
To Probability: representation of Randomity of the “Gas Molecules”
Via Entropy : the simplest “Sampling & Counting” measure (2^N) (a Geometric Average: DIGITAL !! )
Then To Systems & Cybernetics: representing “Circular Causality”
Feedback / Information content of System Signals (Information Flow) /
number of System State ( Varieties: DIGITAL !! )
Complexity Theory
4-Realms :
Probability
Causality
Intricacy
Emergence

19
In Mathematical Notations:
[ Arithmetic Mean, RMS / Geometric / Sequence, Digital / Complexity/ Emergence]
Then To Complexity : that was “En-passant” discovered :
due to the use of Digital “Computers” !!
In fact: computers were first used for pure “Computational” purposes,
But with the development of “Artificial Neural Networks” we discovered Nature’s way of doing things ! :
Which is the way of “ organized Complexity” ! : “Multi-agent Systems” : stressing the
Importance of “Edges” or “Connections” , not just of “Nodes” and “Entities” ..
Then To Emergence : the Pioneering notion of “Autopoiesis” (1973)
And Genetics, Genetic Algorithms, in addition to CAS (Complex Adaptive Systems)
So ! : If we are to deal with the Issues of :
Probability, Causality, Intricacy, Emergence
We MUST use Digital notions !!!!
Complexity Theory
4-Realms :
Probability
Causality
Intricacy
Emergence

20
Distinctions
If You have NOT ever heard about Complexity,
You can Skip the next 2 Slides
But If You have ALREADY heard about Complexity:
You can focus on a set of “Important Distinctions”
( mentioned in next 2 Slides)
While reading the rest of the Presentation
( and with my apology regarding “Acronyms” used,
You can see the “Abbreviations & Acronyms slides” at the end of this presentations
If faced with any difficulty )
Complexity Theory
4-Realms :
Probability
Causality
Intricacy
Emergence

21
Distinctions
on “Important Advanced Distinctions”
related to this Presentations (1 of 2)
Complexity <> Randomness
Complexity: 2 Types :
Type1: Easily-understood: disorganized Complexity ( e.g. : Gas Molecules)
Type2: Difficult-to-understand: organized Complexity ! ( e.g. : Fractals )
-ve Feedback vs +ve Feedback
-FDBK: common in most “technologies” ( Control: Regulation & Servo)
+FDBK: Nature’s way of “EMERGING” NEW entities ( Flexibility & Emergence)
Complexity Theory
4-Realms :
Probability
Causality
Intricacy
Emergence

22
Distinctions
on “Important Advanced Distinctions”
related to this Presentations (2 of 2)
ONLY CAS are capable of Self-Organization “SLFO”,
While MAS are capable of PSLFO only (Pseudo-SLFO: just a mere “Adaptation”)
SLFO: is possible via 2 different routes (Don’t mix’m up)
HCMX  EMRG  SLFO  (+FDBK)  [CRTCL / EDGKA, SOC]
Only CAS are capable of “EMRGENCE”,
Producing “Higher Complexity ENTITIES [ via ADPT & EMRG to HCMX Entities ]
MAS are capable of “ADAPTATION” only,
Producing NOT Entities, but Links: just a (possibly) More Complex SYSTEM or LINKS
Only CAS are capable of “EMRGENCE.Entities”,
Both CAS & MAS are capable of “EMRGENCE.Properties”
Complexity Theory
4-Realms :
Probability
Causality
Intricacy
Emergence

23
Complexity Theory
Complexity <> Randomness
A Simple Curve showing the relation between Complexity and Randomness :
4-Realms :
Probability
Causality
Intricacy
Emergence

24
Reference: www Scholarpedia : “Complexity” Olaf Sporns (2007): Indiana University, Bloomington, IN doi:10.4249/scholarpedia.1623
Complexity vs Randomness (Probability) :
4-Realms :
Probability
Causality
Intricacy
Emergence

25
Reference: “Complex Adaptive Systems: Emergence and Self-Organization, 2009 Kaisler & Madey”
Complexity vs Randomness (Probability) : in 3 Cosmos:
4-Realms :
Probability
Causality
Intricacy
Emergence

26
Get ready to explore
the 4-Realms:
Get ready to view :
the Complexity 3D Space
A 3D-Space that describes Complexity :
Via 3 Axes
After just considering the following Question …
Complexity Theory
4-Realms :
Probability
Causality
Intricacy
Emergence

27
ORDER
DISORDER
+Feedback
(Causality)
-Feedback
(Causality)
+Intricacy
-Intricacy
Question:
Is Our Universe “Deterministic” ?!
Physics, Classical Mechanics, …
EMRG

28
ORDER
DISORDER
+Feedback
(Causality)
-Feedback
(Causality)
+Intricacy
-Intricacy
Answer:
unfortunately No!
It is NOT A Deterministic World !,
Physics, Classical Mechanics, …
EMRG
And … get Prepared for “4 updates” …

29
ORDER
update #1 of 4
There is something called “Disorder”,
Probability, Intrinsic Randomity, ..
+Feedback
(Causality)
-Intricacy
DISORDER
+Intricacy
EMRG

30
ORDER
update #2 of 4
In addition to “DIRECT causality” ,
That is usually dealt-with in Physical Sciences:
There is also “Circular Causality:
Feedback
+Feedback
(Causality)
-Intricacy
DISORDER
+Intricacy
EMRG -Feedback
(Causality)

31
ORDER
update #3 of 4
Also: “Intricacy” !
Physical Sciences fail to solve “complex” problems :
Starting from the 3-Bodies problem in Mechanics ! -Intricacy
DISORDER
+Intricacy
EMRG
+Feedback
(Causality)
-Feedback
(Causality)

32
ORDER
( Example on #3 ):
The 3 Bodies- Problem
The Deterministic Realms
Suffers great difficulties at
a complexity of ONLY 3 Objects !
+Feedback
(Causality)
DISORDER
+Intricacy
EMRG
At a Complexity of 3 ( Only) !
The Famous
Three-bodies problem
is UNSOLVABLE
via Deterministic-Realm tools

33
ORDER
update #4 of 4
and
There is also “Emergence”
Related to an under-development term
of
“SPROUTING”
{ CF “Open Questions” slide }
DISORDER
EMRG
+Feedback
(Causality)
-Feedback
(Causality)
-Intricacy
+Intricacy

34
Let’s explore
these 4-Realms:
The “Complexity Space”
Complexity Space Axes :
What are the Axes Limits ( “Start” & “End” values ) ?
And What “Planar Notions” are related to these Axes ?
Complexity Theory
4-Realms :
Probability
Causality
Intricacy
Emergence

35
ORDER
4-Realms :
Probability
Causality
Intricacy
Emergence
DISORDER
+Feedback
(Causality)
-Feedback
(Causality)
+Intricacy
-Intricacy
EMRG
The 4-Realms
More than just “Deterministic”
1
2
4
3
5
6
7
1 - 7
Suggested
reading
Sequence

36
ORDER
4-Realms :
Probability
Causality
Intricacy
Emergence
DISORDER
+Feedback
(Causality)
-Feedback
(Causality)
+Intricacy
-Intricacy
EMRG
The 4-Realms
Axes Start & End Points (Extremes)
“COMPLICATED” , i.e. Not Complex but “Complicated”
(e.g.: a “car” : just a complication that is “REDUCIBLE”,
Fully-Determinate
Dataseries: Shannon Entropy =0,
Dataseries: Compression Ratio=100%
Fully-Indeterminate
Dataseries: Shannon Entropy =1,
Dataseries: Compression Ratio=0%
Circular Causality
+FDBK: Amplification, Reproduction .
Circular Causality
-FDBK: Regulation, Servo Mechanisms
“Complex” Systems
(Large Number of INTERACTING Heterogeneous Elements,
CAS Entities, system is IRREDUCIBLE)
Emergence: Applicable only for
(High Complexity Entities) ,
1
2
4
3
5
6
7

37
ORDER
4-Realms :
Probability
Causality
Intricacy
Emergence
DISORDER
+Feedback
(Causality)
-Feedback
(Causality)
+Intricacy
-Intricacy
EMRG
The 4-Realms
Axes Details (Points, Ranges, Trends)
A System that is Succumb to “Divide&Conquer”
is Not a (Complex-System)
Order/disorder Mix
( =Max. Complexity )
+FDBK:
VIMP: Has “LIMIT”(s) :
Spatio-temporal limitations,
Limited-resources, Saturation, Traffic, ..
-FDBK: with (order, rank, degree) of:
-1: in ALL Engineering Sys. (s.c.: -2: “internal” FDBK loops)
-n(SubGroups): [Disagreements/ Opposition/ Conflict]
-2n(SubGroups): Double[Disagreements/ ..]
-nn: Group-Self-Regulation [ ANN, MAS, .. ]
Intricacy
Maximum at ( Order/disorder Mix )
Emergence
Producing (even-higher CMX, more –FDBK)
(via initial +FDBK)

38
The 4-Realms
Planar Notions ( Order/Disorder Intricacy )
@ zero-Causality (= Direct-Causality)
Next slide: the ( Order/Disorder Intricacy ) PLAN
Complexity = MAXIMUM @ : Mix of ( Order + Disorder )
Next slide Will show 2 facts :
[ Complexity & Randomity are not the same thing / Max Complexity occurs @ 50-50 Randomity-Order ]
Can refer to Famous “Measures” :
Shannon Entropy: measures Randomity & surprise not complexity,
“Kolmogorov Complexity”: mistakes Randomity for Complexity ! Despite its name !
( a Humble Advice:
if you haven’t heard about neither Disorder nor Complexity: please Google & read any simple page,
then re-read the last 2-slides : Axes: Extremes & Points)
Next 3 slides: will show 3 important Concepts:
Shown clearly In the 3 “Planar Views” of the 3-Axes :
ORDER
DISORDER
Next slide: ( Order/Disorder Intricacy ) PLAN

39
ORDER
4-Realms :
Probability
Causality
Intricacy
Emergence
DISORDER
+Intricacy
The 4-Realms
Planar Notions ( Order/Disorder Intricacy )
@ zero-Causality (= Direct-Causality)
Order/disorder Mix
( =Max. Complexity )
Maximum at ( Order/disorder Mix )
100% Disorder = 0 Complexity // 100% Order = 0 Complexity

40
The 4-Realms
Planar Notions ( Intricacy Causality )
@ (average)-Randomity
Next slide: ( Intricacy Causality ) PLAN
-veFeedback = Regulation , +veFeedback = Reproduction
Will show the Process of “EMERGENCE”
How (High Complexity) Bio-beings are capable of
“EMERGING” to a ( Higher Complexity ) Being !,
Via both +Feedback & -ve Feedback !!
+Intricacy
Next slide: ( Intricacy Causality ) PLAN

41
ORDER
4-Realms :
Probability
Causality
Intricacy
Emergence
DISORDER
+Feedback
(Causality)
+Intricacy
EMRG
The 4-Realms
Planar Notions ( Intricacy Causality )
@ (average)-Randomity
Best Exists at the “Mix of Order-Disorder” ,
And in “–ve Causality beings” only
(while still being capable of +Feedback:
In Emergence & Reproduction )
-Feedback
(Causality)

42
+Feedback
(Causality)
-Feedback
(Causality)
The 4-Realms
Planar Notions ( Order/Disorder Causality )
@ zero-Intricacy (Physics, MAS, ANN, ..)
Next slide: ( Order/Disorder Causality ) : shows many interesting facts:
#1 : MAS (Multi-Agent Systems):
#2 : ENGINEERING SYSTEMS:
#3 : FRACTALS:
( Later on in Appedix F : “CAS” : the Notion of “Complex Adaptive Systems” )
Next slide: ( Order/Disorder Causality ) PLAN

43
Gases Liquids Solids Crystals
4-Realms :
Probability
Causality
Intricacy
Emergence
DISORDER
+Feedback
(Causality)
-Feedback
(Causality)
+Intricacy
The 4-Realms
Planar Notions ( Order/Disorder Causality )
@ zero-Intricacy (Physics, MAS, ANN, ..)
==LIMIT ==
+(more ?!)
+1
+FDBK:
Too much -FDBK:
-2 r/n
- r/n
-1,-2, ..
Slight -FDBK
Direct Causality
(non-causal)
ORDER
Engineering Systems
MAS
ANN
Fractals
Ashby’s 4-galvanometers
Ashby’s Homeostat hunt
Disagreements, Conflicts, Disputes

44
Next Slides :
Will Exploit the 3D “Complexity Space”
To gain more Insight of “Complexity Theory”
Via a (Coherent Perspective)

45
A Coherent Perspective to Complexity:
X: Ordemess:
System Orderness: Environ-Sys
Y: Feedback:
Causality: Sys-SubSys
Z: Intricacy:
System “Complexity”: SubSys
[Diversity, Nestedness, and Numerosity]

46
Complexity Space
(A Coherent Perspective)
Viewing Complexity as a 3D Information Space
(# 1 of 4: Complexity Phenomena & Examples)
Axis X Y Z
Axis-Title Orderness Causality (Feedback) Intricacy
System Part
("Scope")
Environ / Sys Sys / Subsys Subsys / Subsys
Main Phenomena Macro Properties,
Pattern formation.
Feedback
(Coded Symbolic).
Self-Organization
(Subsys, Elements).
Examples Thermodynamics(PV=
nRT),Fractals,
Swarms, Flocks
Comm: Sampling
Rates (2X), mRNA,
Regulatory (Signaling)
Pathways? (Physiology)
Immune Antibodies Diversification
(Germinal Centers)
Quantification Entropy measure:
(T.D., Shannon)
Hard!, Indirect via:
[Non-Linearity & (Info-
)Agents Formation]
Measures of: Sophistication,
Hierarchical C., Tree subgraph.
Main Feature Notion of ~Gestalt Notion of ~Classes Notion of ~Elements

47
Complexity Space
(# 2 of 4: Complexity Measures : Types & Examples)
Axis X Y Z
System Part
("Scope")
Complexity
Measures
How to Describe the
system
How to Build the system System's Degree of Organization
(Elements-wise).
Measures
Examples
Information/ Entropy/
Algorithmic Complexity/
Min. Description
Length/ Renyi/ Fractal
(macro) Dimension
Logical Depth/
Thermodynamic D./
Computational
Complexity (,Time,
Space)/ Information-
Based C.
Fractal D. (micro!)/ Sophistication/
Effective Measure C./ Hierarchical
C./ Tree Subgraph/
Homogeneous.

48
Complexity Space
(# 3 of 4: Complexity as Evident in Sys Scale & Linguistics)
Axis X Y Z
System Part
("Scope")
~Scale ~macro ~meso ~micro
Follows, Guided
by, ..
Simple Rules!
( Statistical)
Communication Rules Balance/Duality: [Specifity/
Diversification]
Limits? Spatio-Temporal
Limits: Saturation,
Clipping,..
Communication
Limits, Smartness of
Agents
None!! : Pure Random ! // then
select/elect by -ve Feedback ?
Info "Types"
(semiotics)
Syntactic (~Form,
Objects)
Semantic
(~Correlations,
relations)
Pragmatic (~Subjective,
Beholder, User)
Entropy
Concentration
theorems
Sequence space
(Alphabet)
Classes of
Sequences (=Types)
Elements (Symbols)
Comm. Ex. : a "data string" (aggr.) its interpretation its measurement

49
Next Slide :
Dear Reader is advised to re-read the 3 Previous Slides:
Noticing the many Similarities suggesting a 3D Complexity-Space:
for example: [ Gross phenomena “Scale: Macro, meso, micro”,
Complexity Measures “3 groups”,
Information types (Info Qualitative Aspects) "Semiotics: 3 issues”
, ... etc ]
Then the Reader can move-on to next Slide :
(titled: Slide # 4 of 4: Viewing ALL )

50
A Coherent Perspective to Complexity
Axis X Y (Y-Z Shared !) Z Info Aspects
System Part
("Scope")
Environ / Sys Sys / Subsys (Inter-Subsys) Subsys / Subsys Info Domains
Main Phenomena Macro Properties, Pattern
formation.
Feedback
(Coded Symbolic).
(Building &
Organizing)
( the SubStr)
Self-Organization
(Subsys, Elements).
Info Usage,
Outcomes
Examples Thermodynamics(PV=
nRT),Fractals, Swarms,
Flocks
Comm: Sampling Rates (2X), mRNA,
Regulatory (Signaling) Pathways?
(Physiology)
(mRNA Vaccines
Marvel! )
Immune Antibodies Diversification
(Germinal Centers)
Info Norms
Quantification Entropy measure: (T.D.,
Shannon)
Hard!, Indirect via: [Non-
Linearity & (Info-)Agents
Formation]
Transfer Entropy ,
…
Measures of: Sophistication,
Hierarchical C., Tree subgraph.
Info
Measures
Main Feature Notion of ~Gestalt Notion of ~Classes Notion of ~Typicality Notion of ~Elements I. Concern
Complexity
Measures
How to Describe the system How to Build the system ( Str / Shared Info) System's Degree of Organization
(Elements-wise).
MIT paper: Info
Measures
Measures
Examples
Information/ Entropy/
Algorithmic Complexity/
Min. Description Length/
Renyi/ Fractal (macro)
Dimension
Logical Depth/ Thermodynamic
D./ Computational Complexity
(,Time, Space)/ Information-
Based C.
(Algorithmic Mutual Info/
Channel Capacity/
Correlation/ Stored Info/
Transfer/ Organization )
Fractal D. (micro!)/ Sophistication/
Effective Measure C./ Hierarchical C./ Tree
Subgraph/ Homogeneous.
MIT Paper by
"Seth Lloyd"
[#3: Str. /
Shared Mutual
Info. ]
~Scale ~macro ~meso (meso-micro) ~micro Info ~Scale
Follows, Guided
by, ..
Simple Rules!
( Statistical)
Communication Rules ( [Speciality/
Numerosity] )
Balance/Duality: [Specifity/
Diversification]
Info "Envelops"
Limits? Spatio-Temporal Limits:
Saturation, Clipping,..
Communication Limits,
Smartness of Agents
( N.A. ! : already
between 2 Extremes)
None!! : Pure Random ! // then
select/elect by -ve Feedback ?
Info
Asymptotes
Info "Types"
(semiotics)
Syntactic (~Form, Objects) Semantic (~Correlations,
relations)
( Learning ) Pragmatic (~Subjective,
Beholder, User)
I. Qualitative
Aspects
Entropy
Concentration
theorems
Sequence space
(Alphabet)
Classes of Sequences (=Types) (Max. Entropy
Distribution? )
Elements (Symbols) I. (Entropy)
Concentration
Comm. Ex. : a "data string" (aggr.) its interpretation its measurement an example
(Action By), the
"Computer"
Sys (not Environ) De-centralized !! (SubSys) De-centralized : just the
(Elements), No "Organizer" !!
Info
Computation
~ ~ Western
Science-Schools
German Science-School:
Constructivism ?
British Science-School:
Empricism ?
American Science-School:
Pragmatism ?
Knowledge
Approach ?
Notes Pattern formation: can be
Scale-free!
VIMP: +veFDBK LIMITS!: e.g. :
Resources, Saturation, Traffic, ..
(Shared Features : can be
considered Y or Z),
~"Transition Features"
Traditional (Classical) Science:
ceases at a Complexity of 3 Elements
!!
Eng. Emad Farag
Habib, Nov 2021
Abbrev.: Information/ System/ Diversification/ Aggregate/ ThermoDynamics/ Feedback/ Complexity (C.) / Communication (Comm.)/ Example/ Not Applicable/ Very Important/ Dimension

51
( A Coherent Perspective to Complexity )
Notions of: Space, Dimension, Aspect, Feature
Space.Dimension.Aspect.Feature
“Complexity Space” : is a 3D Space: comprising 3-Dimensions:
X: Orderness
Y: Causality (Feedback)
Z: Intricacy (formerly: complexity)
X-Axis: Aspects [ Order/Disorder , Stability/Flexibility, Robustness/Resilliance ]
Entropy, Shannon-Entropy, ..
Y-Axis: Aspects [ Causality , Feedback, Correlations & Links (causal) ]
Feedback , Info. Computationality [ Direct Info vs Symbolic ] , Non-Linearity
Z-Axis: Aspects [ Diversification , Nestedness , Numerosity, and Self Organization ]
Measures of: Distance, Attribute, and Shannon Entropy / “Interconnections Distribution” (across agents, Local)

52
Details: of “Space.Dimension.Aspect.Feature” :
Any Space has Dimensions ( e.g. 3D ),
Each Dimension has “Aspects” ( e.g. 1 to 5 Aspects )
Each Dimension (has/can have) “Features” ( e.g. 0 to 7 Features )
{ and Features can have “Synonym(s)” or “aka(s)” }

53
The X Axis: Orderness : has the following Aspects:
Non-determinism : [ Disorder-Order // Probability-Deteminism // Randomness-Regularity ]
Open systems, Non-Equilibrium : [ Open vs Closed ]
Edge of Chaos : [ Stability vs Flexibility ]
State Properties : [ Uni-, Bi-, Multi-State-Variable(s) ]
Pattern Formation : [ Scale-dependent vs Scale-free ]
The Y Axis: Causality : has the following Aspects:
Feedback : [+veFDBK vs -veFDBK ]
Info. Computationality : [ Direct vs Coded (Symbolic) Information ]
Non-Linearity : [ Non-linear vs Linear ]
(Info)-Agents Smartness: [ Social/ Cognitive/ Bio/ Inanimate ]
The Z Axis: Intricacy : has the following Aspects:
Horizontal Diversification : [ Diverse vs Homogeneous ]
Vertical Nestedness : [ Nested vs Flat ]
Numerosity : [ Numerous vs Oligo ]
Interconnections : [ Existing/ Emerged ]
(Info)-Agents : Formation/ Consistency, Coherence/ Interconnections Distributions ( @ agents, Local )
Self Organization : [ Spontaneous vs Equilibrium-based ]
Adaptation : [ System vs Environment ]

54
Complexity SPACE: 3D [ Disorder, Causality, Intricacy ]: It's all about "INFORMATION", Eng. Emad Farag HABIB, Dec 2021
DIMENSION
ASPECT FEATURE Terminology notes/aka
Sys.Scope (Info Flow): Dim.: Axis Title : [ Axis Limits ] Notes
Sys/Environ: X: Disorder
Open systems, Non-Equilibrium : [ Open vs Closed vs Isolated ]
Stability
Non-determinism : [ Disorder-Order // Probability-Deteminism // Randomness-Regularity ]
Disorder Stats/ Info: Variance, S.D., Coefficient of Variation / S
Edge of Chaos : [ Stability vs Flexibility ] aka: Robustness vs Resilliance
Sensitive Dependence on Initial Conditions
State Properties : [ Uni-, Bi-, Multi- : State-Variable(s) ] System aka: Order (in, of) the system ?
Macro Properties Formation: TD [Temperature & Pressure]
Phase Transitions TD phase transitions have "gaps!"
Pattern Formation [ Scale-dependent vs Scale-free ] aka: Order (by) the system ?
Fractals (S.Free) , Hexagonal honeycomb (S.Dep.)
Sys/Subsys: Y: Causality
Non-Linearity : [ Non-linear vs Linear ] Linear only at the (origin)
Non Linear Feedback ! a misnomer -ve Feedback <> Non-Linearity
Non Linear "Chaos" ! a misnomer the 2 phenomena are "intersecting"
(true) Non Linear Dynamics : O.D.E. is non-linear (but usually deterministic)
"Power Laws" (a s.c. of Non Linearity)
"Dependance" Correlations (a s.c. of Non Linearity), also "modularity"
"+ve Feedback" (a s.c. of Non Linearity)
Feedback : [+veFDBK vs -veFDBK ] links to: Direct Causality
Info. Computationality : [ Direct vs Coded (Symbolic) Information ]
(Info)-Agents Smartness: [ Social/ Cognitive/ Bio/ Inanimate ]
Being (Type, level, ..) ?
Intricacy ?!
Subsys/Elements: Z: Intricacy Intricacy not Complexity
Horizontal Diversification : [ Diverse vs Homogeneous ] aka: Speciality, Heterogeneosity, Modularity, Uniqueness
Diversity within a type:measures [Shannon, distance, attribute, H Conc. Theorems]
Diversity across types: [<same>]
Diversity of community composition [<same>]
Vertical Nestedness : [ Nested vs Flat ] aka: Inclusion-Embedding, Hierarchy
Numerosity : [ Numerous vs Oligo ] aka: counting, Number of Entities, Quantitative Intricacy
Interconnections [ Existing/ Emerged ] some: become Causal Links …
Self Organization : [ Spountaneous vs Equilibrium-based ] of subSystems/ Elements
(Info)-Agents Formation, Consistency, Coherence/ Interconnections Distributions (@ agents, Local)
Adaptation [ System vs Environment ] sys-adaptation, vs sys-affects-its-environ
Abbrev: Versus/ Standard Deviation/ thermodynamics/ Feedback/ special case/ also known as/

55
Open Questions & Learned Lessons

56
Open Questions
• The term “Complexity” ?!! :
• Complexity Quanta?!!
• ~Evidences on 3D
The term “Complexity” ?
The term “Complexity” : used to mean both “the Whole Science” and an “Information Dimension” :
We can propose to define the use of the term “Complexity” as follows:
“Complexity” = the whole Science
While using “Intricacy” = the Information Dimension
the z-axis is to be renamed as “System’s Intricacy”
Previous slides have detailed System’s “Intricacy” (and renamed the “Complexity” Dimension) :
( also: Appendix C will elaborate the System Intricacy Dimension ) .
Complexity Quanta?!! (aka: Intricacy Quanta, cf later)
Faced Boldly by “Gaps” in the Order-Disorder Axis: in cases of [ Gases/ Liquids/ Solids ] ,
And by the gaps in the Feedback Axis: in cases of FDBK = [ -1, -2, .. ] (only, i.e. there is NO such thing as 1.5 feedback )
We may expect “Complexity” (as a Whole, the “aggregate phenomenon” ) to be quantized & having “Quanta” values?
~Evidences Supporting? such 3D perspective ro Complexity (0226)
- The Blatant (Axiomatic, Basic, Elemetal, Green) Notion of: 3 [ Macro, meso, and μicro ] in Science.
- The easily-noted borderlines suggesting Notion of: 3 (Aggregates, assemblages, segregations) in Complex Systems :
[ System & Environ // VS Systems “Gestalt”, Whole & Entities // VS Inter-Entities ]
- The (missing! /explaining! ) mathematical terms when studying “TD Aspects of Info Processes” :
The Math terms that link ( Subsys to Sys to Environment ) {cf: Joseph T. Lizier et al}
- Very Consistent & Coherent with many other (3D) findings : [ Measures’ groups/ Semiotic Types (Info Aspects) / .. ]
- Most (Recently Proposed) Notions groups ( sometimes 5, 7, 8 .. ): fit exactly within such 3D str.
[ CMX Features: [ Numerosity/ DORD/ HTROG {Entities} / FDBK/ -EQLM ] { cf Wiesner & Ladyman 2020} ]

57
Open Questions
• Vague-Axes !
• Surfaces ?!
• +ve Feedback Limits (Ceiling) ?
Vague Axes !
Admitting that Complexity Science is still “new”: with no rigorous Mathematical Framework yet:
We observe also that “Axes” are not as “sharp” as common in physical sciences !
The 4 axes have some “vague” or gray areas around them ! ( sometimes parallel to them ! ) :
Order-Disorder Axis: seems “clouded” with a “parallel axis” ?! of ( Stability, Robustness, Equilibrium, ..) !
( Ref: Sheard and Mostashari 2009 “complexity .. (is) more flexibility .. than ..order, more stability than disorder” )
Feedback Axis: is also “clouded” : Computationality, Number of ( non Coded-Information flow ) Links !
Complexity Axis: “clouded”: fact that: NO single measure captures “Complexity”: each measure captures one feature only !
Emergence Axis: “clouded” with what is called “Pseudo-Self-Organzation” and with “simple forms” of order : e.g.:sand piles,, )
Surfaces ?!
Having Complexity a function of both [ Probability & Computationality ]
We find that Systems behave as if it is “Respecting” some space-surfaces:
similar to ( Phase-transitions in Thermodynamics )
(Ref: Ladyman & Wiesner : "Measuring Features of Complex Systems“ ,
And by Same Authors: The Book “What is a complex system?”, published with Yale University Press )
+ve Feedback Limits (Ceiling) ?
Evident In Fractal & Chaos: when dynamics (Sys-Environ) pushes matters in this direction:
Sharp “Curbs” or “Limits” act as a (ceiling) to non-permissible phenomena: e.g.: “over-lumping!” and the like
Ref#2: no doi (chapter of a book): but: other paper:
https://doi.org/10.1088/2632-072X/ac371c

58
Open Questions
• The 3D Perspective:
Anatomical ≠ Physiological
Counter-intuitive 0604
( to be better shown by a diagram : Axes locations )
What is Counter-intuitive :
CAUSALITY: is not coincident with ( Max Order) , but with the (Mix , diversification, panoramic-,..)
COMPLEXITY: is not coincident with ( Max Disorder) , but with the (Mix , diversification, panoramic-,..)
EMRG: in not at ( zero Intricacy), not (max) , but somewhere inbetween

59
Open Questions
• The 3D Perspective:
Anatomical ≠ Physiological
In the 3D Perspective: Form ≠ Essence
The a/m Question of Vague Axis: seems related in someway
to the well-known & Frequently-met Differences between Scientific Notions,
the Difference Between : [ Form & Essence ]
System: [ micro-meso-macro ] Perspective
= [ Form, Anatomy, SpatioTemporal, Layout Diagrams, Scope & Scale Perspective ]
VERSUS
System: [ Component-State-Process ] Perspective
[ Essence, Physiology, System Function(s), Schematic Diagrams, Phase Change & Transition Diagrams ]
( refering again to the fact that: NO single measure captures “Complexity”, cf previous slide ) (0515)
“Sprouting”(0729 , since ~ 0722)
Sprout / ( Germinal Centers ) / free + guided / entropy + rules /
Feedback : ( must have BOTH: Reference_value & Feedback_value ) :
Results in : Stronger than the [ challenge / adverse conditions / .. ] : must ( new info )
Possibly linked Disciplines/Sciences : [ Evolution / Immunology/ Pathology ( ROS) / Space ???!! / Free Societies .. ]
“Rules” ( since ~ 0825)
How Rules are formed ( from “Higher” Rules: e.g.: Conservation ,, ) /// Rule Types: [ Mandatory, ~Statistical, ~Optional, ?? ]
+
-

60
Open Questions
• Reverse Engineering of
Complex Adaptive Systems ??
Employing “Reverse Engineering” methods ?
To The scientific investigation of the COMPLEXITY Theory ?
Using the 3D space Heurestic Model ,
with its Axes, and Axes-Aspects
We can : use “Reverse Engineering” :
Starting with : A Test (Experimental, Prototype) system ,
Doing a Perturbation in one of its aspects,
Then using “AI” to analyze the time-series Entropies & other measures ??
[ of course: After confirming that the (“Complexity Space” ) Perspective is valid in the first place ]
Instead of the current trend: to slowly enhance the set of existing Measures to suit Complexity Theory ?
January 4th 2022
Points-Sequence on the FDBK Axis is [non-causal, direct, -ve, +ve] ?
i.e. : ( “Direct Causality” is not a midway, not “in-between” )
and (+FDBK is simply a “~too much circular causality” , despite starting as –ve ! )
-veFDBK is not the opposite to +veFDBK
-veFDBK means : not only DIRECT causality, but also ( Circular) and more-or-less in a ( LEASHING ) sense/direction
and +veFDBK = circular (also) , up to the utmost of (UNLEASHING) all info-path to reach their limit !
Too much (Dependence) leads to the (over) +FDBK Condition ?
[ A basic or pre-knowledge : regarding feedback : in NATURE: no-one assigns –veFDBK or +veFDBK to a newly emerged link ,
Contrary to MAN-MADE (Engg) system : where we simply impose a “summation point” with a –ve sign for the FDBK
So , links “can” (in principle) at some point : become suddenly inverted to a +FDBK
January 5th

61
In a bid to answer the a/m Question: Complexity “Vague Axes”
The Perspective of (System Scale) : is Different from : The Perspective of (system Function) ?
Macro
Meso
micro
System-Perspective
#1
:
Scale
System
Sub-
system(s)
Entities
System-Perspective #2 : Function
Component Process State
Thermodynamics: Substances Phases, Mixtures Reversible, Irreversible Processes State Functions (Properties)
Immunity: Cell Types [T, B, N, Mϕ, ..] Proliferation, Phagocytosis, .. Cells States[Naïve, Activated,..]
Nutrition: Nutrients & Whole Foods Effects [Nourishment, Side-Effects] Wellbeing, Protection
Physiology: Systems&Organs Processes [Metabolism, Regeneration] Healthy, Disease(Illness)
1
Macro-state
caused
by
n
micro-states
n
micro-states
causes
1
Macro-state
Population VERSUS Statistic
Effects
are
Caused
by
Causes
Causes
Lead
to
Effects
Cause VERSUS Effect
Analysis,
Contains
Synthesis,
Comprises
Subsystem VERSUS Gestalt
n SubSystems
1 Whole
(Systems. Gestalt)
n micro-states
1 Macro-State
n Processes
1 System-Process
(Task? )
Pharmaceuticals (BNF): Body System (Drug) Mechanism of Action Disorder, Disease

62
The Perspective of (System Scale) : is Different from : The Perspective of (system Function) ?
Thermodynamics: Substances Phases, Mixtures Reversible, Irreversible Processes State Functions (Properties)
Immunity: Cell Types [T, B, N, Mϕ, ..] Proliferation, Phagocytosis, .. Cells States[Naïve, Activated,..]
Nutrition: Nutrients & Whole Foods Effects [Nourishment, Side-Effects] Wellbeing, Protection
Physiology: Systems&Organs Processes [Metabolism, Regeneration] Healthy, Disease(Illness)
Pharmaceuticals (BNF): Body System (Drug) Mechanism of Action Disorder, Disease
Component Process State 0604
ONE system: CPS
(Process= Sys State1 to State2 )
(in : System Modeling, while Analysis-Synthesis : Sys & SubSys’s : LOD)
Physiology
Nutrition (Plant Processing)
TD
Complexity [ Intricacy/ Causality/ Disorder ]
TWO systems: CSP
( S= the state of sys1, sys2 // P= joint process)
(in: Interaction, Communication, Collaboration, Partnerships, Synergism )
Immunity
Pharma (Therapeutic, Treatment)
Notes:
- This will easily solve the ( ENTROPY meanings table )
- Such Scientific Opinion (Perspective): is based on a (TD background), then (Physiology Tasks in 2011)

63
Learned Lessons

64
Learned Lessons
Tip: To deal Successfully with Complexity:
Identify well: which System-“Scale” you are Concerned in: (micro/ meso/ macro)?
And pay special attention to “Info. Flow” and to system’s “interconnections” :
Details: Focus on Which System Scale ( aka: “Level of Detail” LOD) :are you concerned with ?
This is easily deduced from specifiying which Phenomena and dynamics you concerned with.
Also which type of Information flow and Information (Entropy) “Measures” are there ?
Finally: What are the System’s: Environment, Boundaries, and interconnections between its components .
Links <> “Causal Links”
In Any “System” : What matters is “Causal Links”, not just “Any” Links:
Links with Information Flow.
(can refer to Appendix E for Types of Interconnections)
Complex System Entities: Disordred Complexity vs “Ordered Complexity”
Entities Intricacy: [Horizontal Diversification & Numerosity] contribute to Disordered C. (mainly )
While Entities Intricacy: [Vertical Nestedness] contributes to “Ordered Complexity” (mainly ).

65
Learned Lessons
“Chaos” <> “Edge of Chaos”
Chaos can occur in non-complex systems (as an exceptional phenomena) ,
While Edge of Chaos is “habitual” in complex systems.
“Feedback” does not mean a Non-Linear System
Because -ve Feedback exists in numerous Linear (or easily-linearized) systems,
While (only) +ve Feedback leads to non-linearity .
( it is very important also to distinguish “Causal Links” from “Non-causal Links” )

66
“CONCLUSION”
A Proposed 3D Perspective that may prove Useful in :
 Focusing our EFFORTS: on the Concerned “Constituent” of the Complex Sy
 If Main Concern is the (System/Environment)-Boundary Region (X) :
• i.e. The Macro & Statistical Aspects of the Complex System .
 If Main Concern is the (Inter-Components) Region (Y) :
• i.e.: the Links, Causality, and Feedback Aspects . ( cf System Links & Interrelations )
 If Main Concern is the (Intra-Component) Region (Z) :
• i.e.: the Components themselves: ( cf Intricacy: [Diversity/ Numerosity/ and Nestedness] , … etc . )
 Classifying Complexity ISSUES: Measures, Phenomena & Concepts :
 Classify Issues in a Structured-Knowledge Scheme (cf MIT paper of "Seth Lloyd")
 Attaining better INSIGHT: to Complexity Dynamics :
Insight to Details: e.g.: how the (Non-Equilibrium) Condition begins and proceeds,
then how the counter-process begins and proceeds (cf the CAS slides in Appendix F)
Or attaining insight to “Big Picture”: e.g.: how to COMBINE many Measures :
to evaluate the (Overall Complexity) of the Concerned system :
e.g. to Compare Two Systems or Two system-states.

67
Appendices:
Appendix A: “What’s wrong with this Presentation ?!”
Appendix B: Defining : Complexity , Complex System, Complex Adaptive
System
Appendix C: What is Meant by Intricacy ?
Appendix D: Linking Information & Entropy (IT & TD)
Appendix E: Links & Interrelations in Systems
Appendix F: CAS systems
Appendix G: Measures of [ Causality, Entropy, and Complexity ]
Appendix H: Complexity Theory Applications
Complexity Theory

68
Appendix A: “What’s wrong with
this Presentation ?!”
Complexity Theory

69
Appendix A: “What’s wrong with this Presentation ?!”
This PPT: is not for Quick “Sequential Reading” :
This Presentation can be browsed initially : to get an idea of the unusual “Complexity” subject and
of the perspective it assigns to Science. it is also recommended that You include the (Appendices,
these slides) in this initial browsing.
Then: You may start re-reading it all-over again ! , discovering new notions in some slides based on
what you have read in the presentation as a whole ,
Then: if you feel that some topics seem interesting to you, you may “Google” such topics, and you
may wish to re-read the Presentation a third time !,
This simply stems from the Nature of the Presentation’s Subject : “Complexity” (cf next Definition):
Complexity Definition(s) :
“To give a precise and exact definition is inherently difficult,
because something is complex if it can not be described in a simple way.” , “Jochen Fromm”
An Extra “additional” difficlty does indeed exist ! , mandating me (the Presentation author) an apology :
CONTENT-wise: some slides are still draft: as I prefer putting them Online in a bid to get comments, reviews,and criticism.
FORMAT-wise: Non Appealing: most slides are still needing much more formatting: due to being done under torture in
EG!. So please focus more on content rather than form.

70
Appendix B: Defining : Complexity ,
Complex System, Complex Adaptive
System
Complexity Theory

71
Appendix B: Defining : Complexity, Complex System, Complex Adaptive System
Defining : Complexity , Complex System, Complex Adaptive System
Complexity (CMX) ≠ Complex System (CMXS) ≠ Complex Adaptive System (CAS)
#1: Complexity = Numerosity & Diversity .
And it may also be defined as :
CMX Causes: { Sheard and Mostashari (2011) }
[ -LTI / EMRG / KAZZ/ ADPT/ LNXK/ SLFO/ Decentralized/ open/ political (as opposed to scientific)/
NESTD (multi-scale) / and many pieces ]
CMX Effects: (perceived as complex):
[ UNK/ difficult to understand/ UPRDICT/ -CTRL/ -STBL/ unrepairable; unmaintainable, costly/ -CZL (unclear)/
taking too long to build. ]
CMX 5 Conditions: { Wiesner & Ladyman (2019) }
[ NUMRS / LNKX/ DORD/ -EQLM (Open system) / FDBK ]
CMX 8 Products ( ~ ‘emergent’ properties)
[ -LTI/ SLFO/ RBST.Order/ NESTD/ RBST.Fn / ADPT/ MDUL/ Mem ]
Ref: “Wiesner & Ladyman”, Measuring complexity, And their book “What is a complex system?”, published with Yale University Press )
Sheard and Mostashari (2011), Jochen Fromm (2004)

72
#2: Complex System = Numerosity, Diversity, and Interconnections .
CMXS = NUMRS + DVRS + LNKX
Features: [ Numerosity/ DORD/ HTROG {Entities} / FDBK/ -EQLM ] { Wiesner & Ladyman (2020) }
Products: [ SPNT, SLFO/ -LTI/ ROBST/ NESTD/ MDUL/ history and memory/ ADPT]
{ Products are “collective” results of the Conditions (Features) } .
Some Products are pre-requisites to others : ex. :
Memory  ROBST
NESTD, MDUL  ADPT
#3: Complex Adaptive System = 5 Features ( Hallmarks) :
CAS = [ EMRG + (SLFO/COPR) + SPCL + NESTD ] { Jochen Fromm (2004) }
Ref: “Wiesner & Ladyman”, Measuring complexity, And their book “What is a complex system?”, published with Yale University Press )
Sheard and Mostashari (2011), Jochen Fromm (2004)

73
“What is Complexity” ?
Ref. : Olaf Sporns (2007), Scholarpedia, doi:10.4249/scholarpedia.1623

74
Appendix C: What is Meant by
Intricacy ?
Complexity Theory

75
Complex Systems : [ Numerosity/ Disorder/ Diversity/ Feedback/ Non-Equilibrium ]
( Ref: “Wiesner & Ladyman”, Measuring complexity,
And their book “What is a complex system?”, published with Yale University Press )
Details of : [ Numerosity/ Diversity/ Nestedness/ InterConnections ]
Let’s move on to
What is meant by the
”Numerosity/ Diversity/ Nestedness” Aspects
Also showing: Already-Existing “Interconnections” :
(shown in red dashed-lines : )
And showing the : Newly Emerged “Interconnections”
(shown in magenta bold dashed-lines : )

76
Let’s now detail
How New “Interconnections” are formed
As a “Complex System”
( i.e: already [Numerosity/ Disorder/ Diversity/ Feedback] )
is “exposed” to any “Non-Equilibrium” Condition (e.g. a shortage of nourishment)
New Information-processing Interconnections Emerge !
( and sometimes : even Spontaneously without such a condition )
Next Slide: show an example of a suste, with Already-Existing “Interconnections”
Followed by Slides showing : Newly-Emerged “Interconnections”
( Reader can also refer to “Appendix D: Links & Interrelations”

77
“COMPLEX SYSTEM”
ENVIRONMENT
Environ-Feedback
Non-Equilibrium (Open System)
Computation
System Intricacy (Vertically)
(Organized Complexity)
Self-Organization
Pattern-Formation
Diversity
Speciality,
Heterogeneity,
~Modularity.
Numerosity
Element 50%
40%
10%
Percentage
Of Computation
(exemplary
values)
SubSys
SubStr,
Cluster
System Intricacy (Horizontally)
(Apparent Dis-Org. Complexity)
Nestedness
Inclusion-Embedding,
Hierarchy
Element

78
ENVIRONMENT
Environ-Feedback
Computation
Self-Organization
Pattern-Formation
Diversity
Speciality,
Heterogeneity,
~Modularity.
Numerosity
Element 50%
40%
10%
Percentage
Of Computation
(exemplary
values)
SubSys
SubStr,
Cluster
Nestedness
Hierarchy
Element
Interconnecions
Formation

79
ENVIRONMENT
Environ-Feedback
Computation
Self-Organization
Pattern-Formation
Diversity
Speciality,
Heterogeneity,
~Modularity.
Numerosity
Element 50%
40%
10%
Percentage
Of Computation
(exemplary
values)
SubSys
SubStr,
Cluster
Nestedness
Hierarchy
Element
Interconnecions
Formation
Existing Interconnections
Emerged Interconnections

80
Appendix D:
Linking Information & Entropy
(IT & TD)
Main Reference: Information-theoretic bound on the energy cost of stochastic
simulation , Wiesner et al, 2011, arxiv.org/phys/0905.2918
Complexity Theory

81
“Appendix D”
If you are interested in Entropy: next slides shows you
A List of Concepts that you must be aware of …
 Next Slides: List of Concepts,
including what you MUST know if you are interested in the Entropy Notion.
Then:
DA: BASICS: Counting : BCR, famous 2x2 Matrix (still TODO ~0220)
DB: LINK H-p: H link to p (Probability): Understanding What Quantity does H measure: Calc (+ evaluation)
DC: P.DISTR. “SELECTION”: What Select a Probability Distribution ? (the MEP).
DD: LINK H-(Info Production/Erasure):
DE: Open Questions in the ( Probability/ Information/ Entropy/ Thermodynamics ) Notions:
and recommended reference(s)

82
Core
Notion
of
Entropy
Main Ref.: A. LESNE, “Shannon entropy: a rigorous notion at the crossroads between probability, information theory, dynamical systems and statistical physics”
Information Theory
Dynamical Systems
Probability
Statistical Physics

83
Core
Notion
of
Entropy
Concept of “Chance”
Information Theory
Dynamical Systems
Probability
Statistical Physics
Concept of “Shannon Entropy”
Concept of “Frequency”, “Histogram”
Concept of “Macro” Properties
Concept of “Probability Distribution”
Concept of “micro, meso, macro”
Concept of “Useful Work”
Concept of “Exergy” , “Free Energy”
Concept of “#States” (#Varieties)
Concept of “System Response Components”
Concept of “Stochastic Processes”
Concept of “Open systems”
Concept of “Complex System”, ..
Concept of “Events” Concept of “#digits” , “compression” , “algorithm”
Concept of “Info Content”, “Energy Cost”
Concept of “Symbolic Sequence & Distribution”
Concept of (order vs disorder)
Concept of “Causality”
Core Notion of Entropy
4 Sciences Concepts & Notions (Slide #1 of 4)

84
Core
Notion
of
Entropy
Chance”: #Times will Occur Concerned Event /#Total Events //
Theoretical Calc & Equal Likelihood
Concept of “Shannon Entropy”
Concept of “Frequency”
Concept of “Macro” Properties
Concept of “Probability Distribution”
Concept of “micro, meso, macro”
Concept of “Useful Work”
Concept of “Exergy” , “Free Energy”
Concept of “#States” (#Varieties
Concept of “System Response Compo
Concept of “Stochastic”
Concept of “Open systems”
Concept of “Complex System”, ..
Concept of “Events” Concept of “#digits” , “compression” , “algorithm”
Concept of “Info Content”, “Energy Cost”
Concept of “Symbolic Sequence & Distribution”
Concept of (order vs disorder)
Information Theory
Dynamical Systems
Probability
Statistical Physics

85
Core
Notion
of
Entropy
"p“ = chance of an event to occur
Shannon Entropy = Σ pi log pi ( =Σ log(p) )
p measures the Frequency of a Certain Event
(vs other events)
Macro Properties vs Microsates
Probability Distribution: Why? MEP!
micro, meso, macro`
useful mechnical work
Exergy / Gibbs
system, #States (#Varieties) / CMX,
Dynamics: "ALL" System response(s) are a summation
( VIMP: decaying exponential or secondary, sinosoida
Dynamics: t, stochastic
(Time(t) , Sequence) / non-determinstic
Open systems (non-equilm)
misc: Complex System, Fractals, scale-free, and power la
#Events, #sampling space #digits / compressions / algorithmic
Info Content
& Energy Cost of Storing and Earsing
Symbolic Sequence & Distribution
H measures % (order vs disorder)
“p”
Classical “p”: #ways for Concerned Event to occur / #ways for all Events
( only if sure of “Equal Likelihood” in case we decided to do an “Experiement” )
Frequency “p”: #times Event Occurred / #trials
( Approach,,) , ref #586p5
MEP:
MEP / MOX { Lagrange: J = F + λ C } // n other MOX cases
Information Theory
Dynamical Systems
Probability
Statistical Physics

86
( Quote about MEP )
“The success of the maximum entropy approach
provides evidence for the fact that
Thermodynamic Laws
are based on Universal Statistical Laws
( governing the structure and features of emergent behaviours ) ,
rather than on Specific Physical Laws (Jaynes 1957a;Jaynes 1957b). ”
ANNICK LESNE, 2011
“Shannon entropy: a rigorous notion at the crossroads between probability, information theory, dynamical systems and statistical physics”
DOI: 10.1017/S0960129512000783
Complexity Theory
d Farag HABIB
A new perspective to Thermodynamic Laws:
Stated another way:
Matters are NOT :
Physical Laws  Thermodynamic Laws , Aided by a “side-tool” of Mathematical Statistics,
In addition to a by-chance “instance” of a Notion named Shannon Entropy
( encountered mainly in the context of Information theory ),
But ARE :
Entropy Notion  Governs the Macrostate of the Emergent behaviours ,
Including the “selection” of the Statistical Entropy ,
apparent as a “Thermodynamic Laws” & “Physical Laws” !!
{ in short : It is NOT : Physics  Thermodynamics, plus a sideline “mention” of Stats & Entropy !
but It turnedout to be : Entropy  Stats  Thermodynamics & Physics }

87
Complexity Theory
d Farag HABIB
Discovering the “More General Law” : Philosophy of Science :
Summary:
[ Special-case Laws discovered first  then the ”More general case” Law ] :
Details:
In the Philosophy of Science : It is amazing to find out that decades of researches to discover
some rule ( for example: the Normal distribution Curve & Formula ), can later prove to
have a much “shorter” route , and to be just a special case of a more general law ! .
Furthermore: the derivation of the (Discovered First) Law from the (Discovered-Later) Law is more than
easy and is usually done in one paper only ! . We have seen (in early slides) how to derive
the Gaussian Probability Distribution from the MEP Principle in a single A4-paper .
( And this interesting fable occured many times in the philosophy of Science, and in very
essential laws : a notable example is Kepler’s Laws & Newton’s Laws :
When decades of research by many brilliant scientists was concluded into discovering Kepler
laws of Planetary motion, to later discover a much shorter route ( and more rigorous )
derivation : from Newton’s laws of motion …

88
Entropy Meaning
Understanding the Meaning of Entropy (in Different Sciences): (ref: "Crossroads", Annick Lesne 2011 )
Topic High Entropy String (H=0.9, H=15,..) Low Entropy String (H=0.1,H=0.3,..)
Basics:
Information High Information = Less repeated pattern Low Information = More repeated pattern
Predictability Low Predictability = High Uncertainity High Predictability = Low Uncertainity
Typicality of Disorder Low "Typicality" (High Rarity) Disorder High "Typicality" (Low Rarity) Disorder
Unevenness High Unevenness = Symmetry-breaking Low Unevenness = High-Symmetry
Ex: words like: "aztdn", "odrcr" (from "Wenglish") words like: "ABCDEFGH", ~ "qu….."
IT:
#Digits Large #Digits required to store the info Few #Digits required to store the info
Shannon Entropy (Math.) less Correlated String: Entropy "H" (H~=log2(N) ) Correlated String: "h" (h << log2(N), Dep.)
Indep. Of String-symbols more Independent Sequence more Dep. Seq.[Symbols'-Distr/ Time-Correl.]
Redundancy Scarcely Redundant (Highly distinct) Highly Redundant (scarcely distinct)
%Compressibility Scarcely Compressible (Highly informative) Highly Compressible (scarcely informative)
Missing Info (average) = average I. required to specify the outcome x when the receiver knows the distribution p = amount of uncertainty represented by a pro
Large Missing Information = Large P.Distr. Uncertainity Little Missing Information = Low P.Distr. Uncertainity
Algorithmic Length Large (long) Algorithm to regenerate a String Small (short) Algorithm to regenerate a String
#Ways to (compose) string Few #Ways Many #Ways
Context Uncommon string (within current context) Common string (within current context)
Ex: # : 3.1623 , 3.1103755(another context: √10, π in Octal) # : 4444444, 2468
Probability: [ 1: ELH // 2: P.Distr. : Random Var X, p(x) // 3: Sequences: X, p(x), Types, SubTypes! ]
Uniformity (Elements-wise) more Equal-likelihood Elements less Equal-likelihood Elements
Uniformity (Classes-wise) Similar Classes DisSimilar Classes
Distribution: Event-described ! Distr. Is composed (fully) of Common Events Distr. Is composed (fully) of Rare Events
#States (Possible): TODO Expectation, @states, H,,
Large #: 3(added)dice=4.17 > 1 die=2.58> coin=1 Small #States: coin tossing ( log2|x|=1)
Ex: P. Distr "in/of" string: #Digits to Describe the string
"Normal" (inside 6σ set of values/events) "Normal" (outside 6σ set of values/events)
Dynamical Systems:
#Categories,Elements Large #Categories & Sparse #Elements Few #Categories & Dense #Elements
Ex: Bio. Molecules Protein Structures, Immune-System Cell-Types Simple Structures
VIMP: in Immune System: Healthy: Entropy "booms" @∆ T-Cells & B-Cells ! Eldery ?: minor ∆H: even @large ∆ of Immune threats
Stat. Physics:
( Concerning: Entropy Production "by/via" a dissipative system, rather than Entropy "in/of" the system : Thermodynamic "S" rather than Statistical "H" )
Microstate Molecules: Gas M. are ALL at the same state Molecules: Gas M. are at Different states
Macrostate System: Unable to do useful (mechanical) Work System: Able to do useful (mechanical) Work
Gases Gas in One thermodynamic Compartment Gas in Two thermodynamic Compartments
Ex: P. Distr "by/via" system: S= #Digits of Emergence ! (to Estimate possible Useful work, as opposed to "pure Dissipation")

89
Entropy Meaning (for thorough reading … )
Abbrev.: "NFP" = Noise, Fully-random, Pseudorandom //CFS: Certain, Fully-determined, Sure Information
0 Topic H=Max High Entropy String ( H=0.9 , H=15, ... ) Low Entropy String (H=0.1 , H=0.3, ... ) H=0 (p=1)
Basics:
Information NEVER-repeated pattern !, NFP Information,
High Information = Less repeated pattern Low Information = More repeated pattern Fully-repeated pattern "AAAA .."
Predictability Full Uncertainity: Climate after 1000years
Low Predictability = High Uncertainity High Predictability = Low Uncertainity Certain, Sure, Fully-Predictable
Typicality of Disorder Full Disorder High Disorder = Low "Typicality" (High Rarity) Less Disorder = High "Typicality" (Low Rarity) Zero Disorder
Symmetry Full Irregular Symmetry-breaking = High Irregularity full-symmetry = Low Irregularity Zero Irregularity
Ex: PRBS & Pseudorandom Characters (Codes) (PRBS: Engineering: Psuedo Random Binary Sequences: for ultimate System Testing)
words like: "aztdn", "odrcr" (from "Wenglish")
words like: "ABCDEFGH", ~ "qu….." CFS Word/Character: e,g, all are
IT:
(Defintions): average missing I = average I. required to specify the outcome x when the receiver knows the distribution p = amount of uncertainty represented by a prob
NEVER-repeated pattern !, NFP Information,
#Digits storing a (non-decimal fraction) (non-circular, and ignoring its "fraction context")
Large #Digits required to store the info Few #Digits required to store the info Certain Digit: ex: "True"
Shannon Entropy (Math.)Fully Uncorrelated (Irrelevant) String: a mix of many/unknown languages
Uncorrelated String: : "H" (~ log2(N) , i.i.d.) Correlated String: "h" (<< H ~ log2(N), Dep.) Fully-dep. String (ex: X2=aX1 : X2
Indep. Of String-symbols Fully Indep. Sequence more Independent Sequence more Dep. Seq.[Symbols' Distr/ time Correl.] Certain Sampling!: ex: ~Sampling.C
Redundancy zero redundancy Scarcely Redundant (Highly distinct) Highly Redundant (scarcely distinct) Fully-redundant trivial string
%Compressibility zero compression Scarcely Compressible (Highly informative) Highly Compressible (scarcely informative) certain string
Algorithmic Length impossible regenerate! Large (long) Algorithm to regenerate a String Small (short) Algorithm to regenerate a String trivial algorithm
#Ways to (compose) string
Fully-random = infinite #ways: Random Sampling Bins!
Many #Ways Few #Ways
Context out-of-context string Uncommon string (within current context) Common string (within current context) sure string
Ex: # : 3.1103755, 3.1623 (another context: π in Octal, √10)
# : 4444444, 2468
Probability:
Uniformity (Elements-wise)
Fully Equal-likelihood Elements
more Equal-likelihood Elements less Equal-likelihood Elements
Distribution: Event-described ! Distr. Is composed (fully) of Common Event(s) Distr. Is composed (fully) of Rare Event(s)
#States (Possible) ∞#Categories or 0#ElementsLarge #States: 3dice(added 4.17)> 1die(2.58)> coin(1)
Small #States: coin tossing (H=1) trivial tossing (sure "True": H=0)
"Uniform" "Normal" (inside 6σ set of values) "Normal" (outside 6σ set of values) ~Delta-function Distribution ?!
Dynamical Systems:
Components #Categories,Elements
∞#Categories or 0#ElementsLarge #Categories & Sparse #Elements Few #Categories & Dense #Elements 0 #Categories or ∞#Elements
Ex: Bio. Molecules Unsorted Genome details of all living creatures
Immune-System Cell-Types, Protein Structures
Simple Structures
VIMP: Immune System:Pre-occupied Sys: System: Fake Categories exists !!
( Entropy "booms" @change in T-Cells & B-Cells ! )
( minor ∆H, even @large ∆ of Elements: ex: Allergic Parient Infection ?
A Waning Sys : Immune System
Stat. Physics:
( Concerning: Entropy Production "by/via" a dissipative system, rather than Entropy "in" or "of" the system )
Microstate Already "in" a state of: 1 Category only & Abundant (ALL!) #Elements ( = no further Entropy-Production possible)
Gas molecues are ALL at the same state Gas molecues are at Different states
Macrostate Unable to do useful (mechanical) Work Able to do useful (mechanical) Work
Ex: Gas in One thermodynamic Compartment Gas in Two thermodynamic Compartments
Ex: P. Distr "by/via" system: #Digits of Emergence ! (to Estimate possible Useful work, as opposed to "pure Dissipation")
"Uniform" "Normal" (outside 6σ) "Normal" (inside 6σ) ~Delta-function Distribution ?!

90
Abbrev.: "NFP" = Noise, Fully-random, Pseudorandom //CFS: Certain, Fully-determined, Sure Information
0 Topic H=Max High Entropy String ( H=0.9 , H=15, ... ) Low Entropy String (H=0.1 , H=0.3, ... ) H=0 (p=1)
Basics:
Information NEVER-repeated pattern !, NFP Information,
Predictability Full Uncertainity: Climate after 1000years
Low Predictability = High Uncertainity High Predictability = Low Uncertainity Certain, Sure, Fully-Predictable
Typicality of Disorder Full Disorder High Disorder = Low "Typicality" (High Rarity) Less Disorder = High "Typicality" (Low Rarity) Zero Disorder
Symmetry Full Irregular Symmetry-breaking = High Irregularity full-symmetry = Low Irregularity Zero Irregularity
Ex: PRBS & Pseudorandom Characters (Codes) (PRBS: Engineering: Psuedo Random Binary Sequences: for ultimate System Testing)
words like: "aztdn", "odrcr" (from "Wenglish")
words like: "ABCDEFGH", ~ "qu….." CFS Word/Character: e,g, all are "A"
IT:
(Defintions): average missing I = average I. required to specify the outcome x when the receiver knows the distribution p = amount of uncertainty represented by a probability distribution
NEVER-repeated pattern !, NFP Information,
#Digits storing a (non-decimal fraction) (non-circular, and ignoring its "fraction context")
Large #Digits required to store the info Few #Digits required to store the info Certain Digit: ex: "True"
Shannon Entropy (Math.)Fully Uncorrelated (Irrelevant) String: a mix of many/unknown languages
Uncorrelated String: : "H" (~ log2(N) , i.i.d.) Correlated String: "h" (<< H ~ log2(N), Dep.) Fully-dep. String (ex: X2=aX1 : X2=shift X1 code by a constant vaue )
Indep. Of String-symbols Fully Indep. Sequence more Independent Sequence more Dep. Seq.[Symbols' Distr/ time Correl.] Certain Sampling!: ex: ~Sampling.Counting.BCR: with replacement.MutuallyExclusive: sampling Event A = Certain Event B
Redundancy zero redundancy Scarcely Redundant (Highly distinct) Highly Redundant (scarcely distinct) Fully-redundant trivial string
%Compressibility zero compression Scarcely Compressible (Highly informative) Highly Compressible (scarcely informative) certain string
Algorithmic Length impossible regenerate! Large (long) Algorithm to regenerate a String Small (short) Algorithm to regenerate a String trivial algorithm
#Ways to (compose) string
Fully-random = infinite #ways: Random Sampling Bins!
Many #Ways Few #Ways
Context out-of-context string Uncommon string (within current context) Common string (within current context) sure string
Ex: # : 3.1103755, 3.1623 (another context: π in Octal, √10)
# : 4444444, 2468
Probability:
Uniformity (Elements-wise)
Fully Equal-likelihood Elements
more Equal-likelihood Elements less Equal-likelihood Elements
Distribution: Event-described ! Distr. Is composed (fully) of Common Event(s) Distr. Is composed (fully) of Rare Event(s)
#States (Possible) ∞#Categories or 0#ElementsLarge #States: 3dice(added 4.17)> 1die(2.58)> coin(1)
Small #States: coin tossing (H=1) trivial tossing (sure "True": H=0)
"Uniform" "Normal" (inside 6σ set of values) "Normal" (outside 6σ set of values) ~Delta-function Distribution ?!
Dynamical Systems:
Components #Categories,Elements
∞#Categories or 0#ElementsLarge #Categories & Sparse #Elements Few #Categories & Dense #Elements 0 #Categories or ∞#Elements
Ex: Bio. Molecules Unsorted Genome details of all living creatures
Immune-System Cell-Types, Protein Structures
Simple Structures
VIMP: Immune System:Pre-occupied Sys: System: Fake Categories exists !!
( Entropy "booms" @change in T-Cells & B-Cells ! )
( minor ∆H, even @large ∆ of Elements: ex: Allergic Parient Infection ?)
A Waning Sys : Immune System: Losing Important Categories !!
Stat. Physics:
( Concerning: Entropy Production "by/via" a dissipative system, rather than Entropy "in" or "of" the system )
Microstate Already "in" a state of: 1 Category only & Abundant (ALL!) #Elements ( = no further Entropy-Production possible)
Gas molecues are ALL at the same state Gas molecues are at Different states
Macrostate Unable to do useful (mechanical) Work Able to do useful (mechanical) Work
Ex: Gas in One thermodynamic Compartment Gas in Two thermodynamic Compartments
Ex: P. Distr "by/via" system: #Digits of Emergence ! (to Estimate possible Useful work, as opposed to "pure Dissipation")
"Uniform" "Normal" (outside 6σ) "Normal" (inside 6σ) ~Delta-function Distribution ?!
Notes:
I. Cognitive Being : Tends to exist More in Higher Info Setting
rather than Lower Info :
II. The limit is Max. Info:note-worthy:
III. However: ( Max Info, Random, -veFDBK “wise-selection” :
Genetics (Heridity) & Immunity (Germinal Centers: Virus
Variant Antibodies Production):
IV: Info “by/via” = - Info “in/of” system :
V:
a = -b (uniformity-wise)
a=towards uniformity, energy "paid by" system itself ,
b=away from uniformity, energy paid by other systems
0314

91
Draft Ideas:
I. ENTROPY + Automatic Deduction of Parameters :
I:
The main problem with ApEn & SampEn: is defining the parameters / it is
required (research-wise) to find a way to estimate parameters (regioursly)
from the datasets .
This is not just an IMPROVEMENT , is a MUST : it is illogical to try to
analyze datasets without having the pre-requisite quantity: datasets’ time-
scale (time-constant) .. To be able to “catch” its real dynamics &
correlations .
(main pusher of the idea:
ApEn & SampEn are not fast enough …
{ Ref: RangeEn ? Principe p ? .. }
0320 ( 8PM , before reading a word from Chapter 10 “Principe” on
CorrEntropy )

92
Entropy Meaning ( Ex: “Immune System” )
Examples : Factors Contributing to Increasing the Value of Calculated Entropy :
[ I: #Events Considered // II: Coarse Graining & Local Averaging / III: #SubCategories and
Occupancy ]
I. “Entropy is thus extremely sensitive to the set of possible events considered”
II. sensitive to Coarse Graining & Local Averaging :
III. ( DIYK PPT of “Immune System”, E.F.HABIB, in September 2021 )
I. sensitive to the set of possible events considered:
II. sensitive to Coarse Graining & Local Averaging :
III. Immune System Example: 2 factors: "Severe" = "Several" + "Sparse“
"Severe" H = "Several" Categories + "Sparse“ Occupancy
In plain English: Severe Entropy occurs due to a change in Categories/Elements having : Several SubCategories
(not Oligo) and/or Sparse Occupancy (not dense): T Cells & B Cells Densities: exactly follow such Rules …
See my other presentation on “Immune System” & Entropy Optimization : titled
( Immunity & Cybernetics: Immune System “Mystries” & the Science of Cybernetics }
September 2021

93
Entropy Meaning ( Ex: “Immune System” )
{ in plain English: Severe Entropy occurs due to a change in Categories/Elements
having : Several SubCategories (not Oligo)
First: we must review some Definitions:
[ Gross Total/ #Categories/ Category size/ and #SubCategories ]
+ Gross Total: Total Entities in the System ( Immune S. = ~9000 WBC/uL serum)
+ #Categories: count of Cell-Types ( Immune System ~10 Categories )
+ Category Size: ranges from Important, (small categories: DC Cells )~ to less-important (large
categories: Ab’s, Cx)
+ #SubCategories: ranges from hardly (2 Subcategories) to (~7 Subcategories): Ab’s
Ex: Ab’s [ Non-Specific, Specific, Neutralizing immunity, Opsonizing, Blocking Cell Receptors, Free,
Variants (Ongoing, anew) ]
( noting that what matters is: the “Actual, true, real” #SubCategories )
Ie: Distinguish: [ theoretical-vs-actual = Proposed vs True = Immaginary vs Real = Aptitude vs
Potential = Possibilities vs Probabilties ]
List [ Body Cells/ Ab/ cx/ AG/ NK/ M/ N/ DC/ T/ B ] 20210900 & 20210300
Data from Reference: Ganong (17th Edition, 1995 : p474 )
summary:
total 9K, body cells=30k
details:
(relevant) body cells=30,000 //
WBC: total 9,000 (all-in-all white blodd cells)
N 5400 // M 540 //
Dendrtic Cells 50 // T200// B250
{ WBC: 9000 (cell / uL) : N=5400 , M=540, Leuko=2750 (ref: #209 : Ganong 17th p474) {

94
ENTROPY
So, a List of “MUST Know Topics” is :
(You can Google 'm for more information)
Macro, meso, micro
Entropy Concept
Irreversibility
4 [ Replacement/ N.R., Order/ N.O. ]
Simple Examples: Coin, Dice
μ,σ : Average & Standard Deviation
Entropy= Summation ( pi * log (pi) )
Simple Ex: FDBK in Engg & Bio
Simple Ex: 3-body motion, ants
Simple Ex: Swarms, Flocks, ..
Then You may at a later stage consider
the following :
If you are of an Engineering, Science,
or Computer Background: You may stress-more
all concepts of Stat. Mech. , Probability & IT.
If you are of a Medical, Pharmacuetical,
Desntistry, Veteranery Medicine, or Biology
Specialities: You may stress-more all concepts
of Gibbs Energy and Complexity.
If you are of a Social, Economical or Political
Concerns: You may stress-more all concepts of
Information Theory and Complexity ,

95
ENTROPY Concepts #1 of 2
Notion of Entropy & 4 Sciences: List of Important Concepts (30)
Science
# Concept Side Topics (not mainstream of this science topic in particular)
Defintion (Basic) Symbol, Quantity
Statistical Mechanics:
Math Basics
Physical Basic Quanities T
Physical Derived Quanities P,Q,W,H
1 Temperature "Heat" Content (a measure of -) T
2 TD Property defines System State (Properties=Physical Quanities) -
3 Entropy (TD) Irreversibility (a measure of -) S
4 Useful Mechanical Work Availability (a measure of -) W
5 Gibbs Free Energy G
6 Exergy Ability to Exert Energy B
7 Stochastic Processes Sequenced Non-deterministic Proceeses t, X(t)
Probability:
8 #SampleSpace Events # of ALL possible events N
9 #Events # of CONCERNED events r
10 p1: chance (=w/o experiement) p= (r / N) p
11 p2: freq. (=w. experiement/Obs) p= (f / N) p
12 p3: personal (experience) - p
13 Random Variable variable value=result/outcome of a random experiement/trial X
14 Probability Distribution curve: X-value vs Frequency f(X)
15 MEP Applicable Distribution is the "maximum" Entropy value J
16 Statistic (a Quantity) a 1-value Summary (of the distribution) μ,σ
Information Theory:

96
ENTROPY Concepts #2 of 2
Science
Defintion (Basic) Symbol, Quantity
Information Theory:
17 #Digits #Digits required to store a value (deterministic !) n
18 Entropy (Shannon) #Digits to store info (random, average) H
19 Information "Distribution" curve: H-value vs Entropy H(X)
20 Compression % after removing redundancy %ratio
21 Algorithm shortest Algorithm AIC
22 Energy Cost of Storing and Earsing Q
Dynamical Systems:
23 #System States number of possible system states (dimensionless) V
24 System Response: 2 Components only ! (LTI) c(t), C(s)
25 Info Flow in Non-Equilm (Open) Systems -
26 Feedback -
27 Causality -
28 Complexity -
29 Complex System -
30 Special Distr.(s)&Phenomena: -
Reference(s):
Abbrev.: General System Theory/ Linear Time-Invariant/

97
ENTROPY – What You need to know ?
Science
.Context MUST Know Topics ( Google 'm ! )
Statistical Physics:
Math Basics
(Basics)
Physical Basic Quanities
(Basics)
Physical Derived Quanities
(Basics) Macro, meso, micro
1 Temperature Heat Transfer
2 TD Property State (of a System)
3 Entropy (TD) 2nd Law Entropy
4 Useful Mechanical Work 1st&2nd Law Irreversibility
5 Gibbs Energy
6 Exergy Energy
7 Stochastic Processes TD Processes
Probability:
8 #SampleSpace Events Random.Chance 4 [ Replacement/ N.R., Order/ N.O. ]
9 #Events Random.Chance
10 p1: chance (=w/o experiement) Estimating Probability Simple Examples: Coin, Dice
11 p2: freq. (=w. experiement/Obs) Estimating Probability
12 p3: personal (experience) Estimating Probability
13 Random Variable Random Variable
14 Probability Distribution Random Variable
15 MEP (Optimization) ( Advanced )
16 Statistic (a Quantity) Probability Distrib. μ,σ : Average & Standard Deviation
Information Theory:

98
ENTROPY – What You need to know ?
Science
.Context MUST Know Topics ( Google 'm ! )
Information Theory:
17 #Digits (Arithmatics)
18 Entropy (Shannon) Info/Communication Entropy= Summation ( pi * log (pi) )
19 Information "Distribution" Info/Communication
20 Compression Info/Communication
21 Algorithm Computation/ Programming
22 Energy Cost of Info Processing
(Information Theoretic)
Dynamical Systems:
23 #System States Cybernetics
24 System Response: 2 Components only ! (LTI)
System Dynamics ( Advanced )
25 Info Flow in Non-Equilm (Open) Systems
GST
26 Feedback GST Simple Ex: FDBK in Engg & Bio
27 Causality(Logic)
28 Complexity
(Complexity Science) Simple Ex: 3-body motion, ants
29 Complex System
(Complexity Science) Simple Ex: Swarms, Flocks, ..
30 Special Distr.(s)&Phenomena:
(Complexity Science)
Reference(s): A. LESNE, “Shannon entropy: a rigorous notion at the crossroads between probability, inform
Abbrev.: ThermoDynamics/ General System Theory/ Linear Time-Invariant/ Equilibrium/
Eng. Emad Farag Habib Feb2022

Updated (version 2.3) Easy (Complexity Theory), Probability & Disorder,Causality & Feedback,System Intricacy, Emergence (a 3D Space Perspective)-Ver 2.3.ppt

Updated (version 2.3) Easy (Complexity Theory), Probability & Disorder,Causality & Feedback,System Intricacy, Emergence (a 3D Space Perspective)-Ver 2.3.ppt

Recommended

Recommended

More Related Content

Similar to Updated (version 2.3) Easy (Complexity Theory), Probability & Disorder,Causality & Feedback,System Intricacy, Emergence (a 3D Space Perspective)-Ver 2.3.ppt

Similar to Updated (version 2.3) Easy (Complexity Theory), Probability & Disorder,Causality & Feedback,System Intricacy, Emergence (a 3D Space Perspective)-Ver 2.3.ppt (20)

More from EmadfHABIB2

More from EmadfHABIB2 (20)

Recently uploaded

Recently uploaded (20)

Updated (version 2.3) Easy (Complexity Theory), Probability & Disorder,Causality & Feedback,System Intricacy, Emergence (a 3D Space Perspective)-Ver 2.3.ppt