1
Key Trends:
6TiSCH in Perspective
Pascal Thubert
Cisco Systems
January 17th, 2016

2
Trends
Deterministic
Networking
Clock Synchronization
SDN
(Controller)
Fog

4
Industrial Internet

5
Converge Control Networks to IP
Make IP operations more efficient
Emulating existing Industrial protocols
Beyond Control and Automation
Optimize processes (by 1%?)
Leveraging IT, Live big data and Analytics

6
Control loops and Movement detection
Deterministic: global optimization of routes for jitter an latency – thus computed centrally -,
flow (over) provisioning, with static multipath (packet replication and elimination) hard slot allocation.
Large Scale Monitoring
Stochastic: self-healing -thus distributed- routing, RPL
Background resource optimization
Management
a separate topology – a RPL instance - that does not break.
alerts
bursty, unexpected, on-demand slot allocation, prioritization
Dynamic resource optimization

7
• Not Process Control but “Missing Measurements”
Reliability and availability are important, which implies
Scheduling and admission control
• Scalability
10’s of thousands of new devices
• Deployment cost factor is key
For Emerson this is the second layer of automation:
Typically missing are the measurements you need to
monitor the condition of the equipment--temperature,
pressure, flow and vibration readings you can use to
improve site safety, prevent outages and product losses,
and reduce maintenance costs of equipment such as
pumps, heat exchangers, cooling towers, steam traps and
relief valves.

8
• Maintenance and operation
cost represent 75% of the
Total equipment cost
Downtime
Maintenance & operation COST
Corrective maintenance
Programmed
Maintenance
Preventive
Maintenance
Learning Machine
Predictive maintenance
 Deployment of Wireless sensors is seen as an efficient way to achieve it

9
WWAN: GSM – LTE
WLAN: 802.11
WPAN: 802.15.4, ISA100.11a, WirelessHART

10
• IEC based on HART 7.1.
• TDMA
• fixed time slots (10ms)
• Mesh only
• Shipped YE-2008.
• Vendor driven
• Emerson, E&H, ABB,
Siemens
• IEC based on 2011 revision
• TDMA+CSMA
• Var. time slots
• Star, mesh and hybrid topology
• IPv6, 6LoWPAN, TCP-friendly
• Shipped mid-2010
• Mostly user driven
• Honeywell, Yokogawa, Invensys
IEC 62601
WIA-PA
IEC 62591 IEC 62734
ISA100.11a
• Alternate from China
• Star, mesh and hybrid
topology
• Standardization work
started in 2006.

11
Field is after next % point of operational optimization:
Requires collecting and processing of live “big data”, huge amounts of missing
measurements by widely distributed sensing and analytics capabilities.
Often sharing the same medium as critical (deterministic) flows
Achievable by combination of the best of IT and OT technologies together, forming
the IT/OT convergence, aka Industrial Internet.
Architectural approach, standards, Industry adoption needed to embrace radical
changes happening in IT networking technologies. Products are following.
Secured-by-default model required throughout network lifecycle.
The next problem is to extend Deterministic Industrial technologies to share
bandwidth with non-deterministic traffic, reaching higher scales at lower costs.

12

13
• Battery-operated and Scavenging
Wireless HART, ISA100.11a – Silo’ed
802.15.4 TSCH – and then TSCH over other PHYs
• Powered
802.15.1 WISA (ABB) – Evolved into WSAN-FA,
802.11 iPCF (Siemens) – Time Sliced but Proprietary
Other Wi-Fi evolutions – Collaboration Cisco/Rockwell
U-LTE - Going to IMS band opens a huge potential

14
6TiSCH
WiHART WIA-PAISA100
Common Network
Management and Control
HART WIA-PAISA100
MGT /
CTRL.
APPLI.
HART WIA-PAISA100
Better Co-ExistenceTRIPLE OPEX + interferences
Wia-PAISA100NETW. Wireless
HART

15
• Radio Mesh: Range extension with Spatial reuse of the spectrum
• TSCH with Centralized routing, optimized for Time-Sensitive flows
 Mission-critical data streams (control loops)
 Deterministic reach back to Fog for virtualized loops
 And limitations (mobility, scalability)
• RPL Distributed Routing and scheduling for large scale monitoring
 Enabling co-existence with IPv6-based Industrial Internet
 Separation of resources between deterministic and stochastic
Leveraging IEEE/IETF standards (802.15.4, 6LoWPAN …)

16
Sender
(Talker)
NIC
NIC
Per-Flow State
Controller (PCE)
Flow ID
?
Service (northbound)
interface
Receiver
(Listener)
?
UNI
interface
Control (southbound)
interface
UNI
interface 16
draft-ietf-6tisch-architecture
draft-ietf-6tisch-6top-interface
draft-ietf-6tisch-coap
draft-ietf-6tisch-architecture

17
Wireless DetNet requires scheduled radios such as TSCH and LTE/5G
6LoWPAN is NOT a silo’ed solution like WirelessHART and ISA100.11a
6LoWPAN is how you do IPv6 over low-Power networks
providing an Adaptation Layer and a novel IPv6 Neighbor Discovery
6LoWPAN was updated under Cisco lead to fit in ISA100.11a
Now generalizing on all Low-Power technologies at the IETF 6lo WG
6TiSCH puts together 6LoWPAN for IPv6, RPL for Low-Power Routing
and TSCH for Deterministic Wireless Networking capability
6TiSCH generalizes ISA100.11a and WirelessHART to enable the vision
of the Industrial Internet

18
6TiSCH basics

19
Active IETF WG, 6 WG docs in or close to last call
Defines an Architecture that links it all together
Align existing standards
(RPL, 6LoWPAN, PANA, RSVP, PCEP, MPLS) over 802.15.4 TSCH
Support Mix of centralized and distributed deterministic routing
Design 6top sublayer for L3 interactions
Open source implementations (openWSN…) and PlugTests
Multiple companies and universities participating

20
Deterministic IPv6 over IEEE802.15.4 TimeSlotted Channel Hopping (6TiSCH)
The Working Group will focus on enabling IPv6 over the TSCH mode of the
IEEE802.15.4 standard. The scope of the WG includes one or more LLNs, each
one connected to a backbone through one or more LLN Border Routers (LBRs).
Active drafts
https://tools.ietf.org/html/rfc7554
http://tools.ietf.org/html/draft-ietf-6tisch-terminology
http://tools.ietf.org/html/draft-ietf-6tisch-architecture
http://tools.ietf.org/html/draft-ietf-6tisch-minimal
draft-dujovne-6tisch-6top-sf0-00
draft-wang-6tisch-6top-sublayer (6P protocol)

21
Centralized route and
track computation
and installation
Management and
Setup
Discovery
Pub/Sub
Authentication for
Network Access
Wireless ND
(NPD proxy)
Time Slot
scheduling and
track G-MPLS
forwarding
Distributed route and
track computation and
installation
Distributed route and
track computation and
installation
IEEE 802.15.4 TSCH
6LoWPAN HC
IPv6
RPL
6top
TCP UDP ICMP CCAMP
PCEP/PCC CoAP/DTLS AAA 6LoWPAN ND
}

22
BBR A
BBR B
BBR C
BBR D

23
BBR A
BBR B
BBR C
BBR D

24
6TiSCH
Backbone
Router
Intelligent device
Management
Wireless Controller
Backbone
Router
Limit of IPv6 subnet(s)Limit of IPv6 subnet
End to end IPv6 routing
NAC
/Firewall
VPN Concentrator

25
6TiSCH
6TiSCHSensor
Actuator
Backbone
Router
Management
Wireless Controller
Backbone
Router
Single Multilink IPv6 Subnet with free inner mobility (same subnet == no renumbering)
Virtual Service Engine
(virtual PLC, PCE …)
+ IPv6 ND registrar
+ Network Function
Virtualization (NFV)
NAC
/Firewall
VPN Concentrator
IPv6 ND registration and proxy operation
Full
virtualization
and chaining

26
Low Power TSCH mesh is a complex technology adapted to:
• Mesh: Range extension with Spatial reuse of the spectrum
• IPv6-based Industrial Internet
 Stochastic routing for large scale monitoring (RPL)
 Separation of resources between deterministic and stochastic (TSCH)
 Leveraging IEEE/IETF standards (802.15.4, 6LoWPAN …)
• Fog-based Centralized optimization for Deterministic flows
 Mission-critical data streams (control loops)
 Reach Back to Fog deterministically for virtualized loops
 And limitations (mobility, scalability)

27
Determinism
and Very High Probability

28
In mathematics and physics, a deterministic system is a system in which no randomness is
involved in the development of future states of the system. A deterministic model will thus always
produce the same output from a given starting condition or initial state.
[In philosophy] A deterministic system is a conceptual model of the philosophical doctrine of
determinism applied to a system for understanding everything that has and will occur in the
system, based on the physical outcomes of causality. In a deterministic system, every action, or
cause, produces a reaction, or effect, and every reaction, in turn, becomes the cause of
subsequent reactions. The totality of these cascading events can theoretically show exactly how
the system will exist at any moment in time.
What is
Deterministic?(per Wikipedia definition)

29
End-to-End latency enforced by timed pause at station
Periodic trains along a same path and same schedule
Trains stay in station till scheduled departure time
Collision avoidance on the rails guaranteed by schedule
29

30
A bus every T. minutes, Stop A to Stop B in X. minutes
Worst end-to-end delivery time < (T + X)
Every packet in an Airbus 380 takes a bus across the fabric
Engines react a guaranteed time after pilot asynchronous command
30

31
The Law of large numbers says:
1. Long term, the casino will win.
2. Long term, for any value of X, some player will win more
than X.
• That’s in theory an unbounded peak
• The object of DetNet is to remove chance from the picture.
• We have always been in the business of optimizing average
throughput and latency. (The law of large numbers.)
=> DetNet optimizes for worst-case latency.
31

32
• People buy part time ownership,
Say a week every year, week 30.
• During that week they own the flat,
they will not find another family in
the kitchen or the bedroom, they
should/will not find any clue of other
occupancy.
• Now if they do not come one year,
they may sub-rent, someone else
may opportunistically use the flat.

33
Perfect timing. QoS beyond QoS, for new level of guarantees for IT.
Hard bound latency. Optimum use of constrained medium.
High ratio of critical flows for traffic known a priori and asynchronous.
Sharing physical resources with classical best effort networking

34

35
Controlling time of emission
Can achieve ~10ms sync on 802.15.4
Can guarantee time of delivery
Protection the medium
ISM band crowded, no fully controlled
all sorts of interferences, including self
Can not guarantee delivery ratio
Improving the Delivery ratio
Different interferers => different mitigations
Diversity is the key

36
Code diversity
Code Division Multiplex Access
Network Coding (WIP)
Frequency diversity
Channel hopping
B/W listing
Time Diversity
ARQ + FEC (HARQ)
TDM Time slots
Spatial diversity
Dynamic Power Control
DAG routing topology + ARCs
Duo/Bi-casting (live-live)
Use all you can!

37
16channeloffsets
e.g. 31 time slots (310ms)
A
BC
D
E
F
G
H
I
J
• Schedule => direct trade-off between throughput, latency and power consumption.
• A collision-free communication schedule is typical in industrial applications.
• But requires network synchronization, and de-sync means long isolation

38
Converged Plant Network
High availability
Flow Isolation
Guaranteed Bandwidth
IP based Control Network
Autonomic, zero touch commissioning
Time Sensitive Networking for critical apps
Packet Reliability
IPv6-based Wireless Field Network
Deterministic, Autonomic, Secure
Large Scale for Monitoring (RPL)
Backward Compatible (with ISA100 or HART)
10’s
of K
10’s
100’s
Control
network
Converged plant net
Wireless Field network

39
Forwarding Models

40
A
B
X Y
U
V
1TX
1TX
1RX
1RX
shaping
QoS
RED
1+1 TX
Packets
are
serialized

41
15.4 PHY
15.4e TSCH
CoAP
UDP
IPv6
6LP-HC
6top
15.4 PHY
15.4e TSCH
CoAP
UDP
IPv6
6LP-HC
6top
15.4 PHY
15.4e TSCH
CoAP
UDP
IPv6
6LP-HC
6top
15.4 PHY
15.4e TSCH
CoAP
UDP
IPv6
6LP-HC
6top
UYA X

42
1TX
1TX
1RX
1RX
Dest MAC set to
broadcast 0xFFFF
DSCP Deterministic
Dest MAC restored by
6top at final destination
Dest MAC not changed
DSCP Deterministic
2TX 2RX
A
B
X Y
U
V

43
1TX
1TX
1RX
1RXAdditional retries
Using L3 bundle
Dest MAC set to Y
DSCP Deterministic
Transmission failure
Over track slots
Retries exhausted over
Track L2 bundle
2TX 2RX
A
B
X Y
U
V
If arrival in time
Dest MAC reset to
broadcast 0xFFFF
DSCP Deterministic
Placed back on track
Dest MAC set to
broadcast 0xFFFF
DSCP Deterministic
1TX 1RX

44
15.4 PHY
15.4e TSCH
CoAP
UDP
IPv6
6LP-HC
6top
15.4 PHY
15.4e TSCH
CoAP
UDP
IPv6
6LP-HC
6top
15.4 PHY
15.4e TSCH
CoAP
UDP
IPv6
6LP-HC
6top
15.4 PHY
15.4e TSCH
CoAP
UDP
IPv6
6LP-HC
6top
UYA X

45
ISA100.11a /
WIHART
15.4e TSCH 15.4e TSCH 15.4e TSCH15.4e TSCH
15.4 PHY 15.4 PHY 15.4 PHY15.4 PHY
CoAP
UDP
IPv6
6LoWPAN-HC
6top
CoAP
UDP
IPv6
6LoWPAN-HC
6top
15.4 PHY
15.4e TSCH
CoAP
UDP
IPv6
6LoWPAN-HC
6top
15.4 PHY
15.4e TSCH
ISA100.11a /
WIHART
UDP
IPv6
6LoWPAN-HC
6top
CoAP
Dest MAC set to
broadcast
0xFFFF
Dest MAC not
changed
Dest MAC restored
by 6top

46
1TX
1TX
1RX
1RX
Dest MAC restored
by 6top
Dest MAC set to X
DSCP not Deterministic
2TX 2RX
P
B
X Y
U
V
A
Q
Packet to be routed via Y
Dest MAC set to Y
Placed of deterministic track
Packet extracted from track due
to dmac == Y and/or DSCP and
then routed to a next hop != V
No frame in slot
Associated to
deterministic

47
A
B
X Y
U
V
1TX
1RX
1RX
Next
fragment
follows
n TX
First
fragment
installs
states

48
15.4 PHY
15.4e TSCH
CoAP
UDP
IPv6
6LP-HC
6top
15.4 PHY
15.4e TSCH
CoAP
UDP
IPv6
6LP-HC
6top
15.4 PHY
15.4e TSCH
CoAP
UDP
IPv6
6LP-HC
6top
15.4 PHY
15.4e TSCH
CoAP
UDP
IPv6
6LP-HC
6top
UYA X
1st1st
NextNext

49
• ARQ (retry)
F
GI
IN
IN
OUT
or

50
• Replication
E
F
GI
IN
OUT
OUTand

51
• Elimination
E
F
GI
IN
IN
OUT
or

52
Network Characteristics

53

54
Channeloffset
Upto16channel
slotOffset, 0 .. N
The matrix repeats itself over time. CDU is defined by 6TiSCH and represents the global characteristics of the
network, channel unused, duration of the timeslot and number of timeslots per iteration.
6TiSCH defines a new global concept that is called a Channel distribution/usage (CDU) matrix; a
Channel distribution/usage (CDU) matrix is a matrix of so-called “cells” with an height equal to the
number of available channels (indexed by ChannelOffsets), and a width in timeslots that is the period of
the network scheduling operation (indexed by slotOffsets).
Timeslot width, typically 10-15ms each in 802.15.4e
Cell (slotOffset,
channelOffset)

55
A Slotframe is a MAC-level abstraction that is internal but common to all nodes and contains a series of
timeslots of equal length and priority. It is characterized by a slotframe_ID, and a slotframe_size. Multiple
slotframes can coexist in a node’s schedule, i.e., a node can have multiple activities scheduled in different
slotframes, based on the priority of its packets/traffic flows. The timeslots in the Slotframe are indexed by
the SlotOffset; the first timeslot is at SlotOffset 0.
Channeloffset
Upto16channel
slotOffset, 0 .. N
TimeSlot
Slotframe
CDU
matrix
Cell (slotOffset,
channelOffset)
Timeslot width, typically 10-15ms each in 802.15.4e

56
slotOffset, 0 .. N
High Priority
RCV XMIT RCVSlotframe 1
slotOffset, 0 .. M
Low Priority
RCV RCV RCV XMIT XMITSlotframe 2
Will receive
Will receive if nothing to xmit
Will xmit
Will receive
Will receive
Decision to transmit / rcv
made at the beginning of
time slot based on
existing frames in queues
and slotframe priorities

57
A Chunk is a well-known list of cells, well-distributed in time and frequency, within the CDU matrix; a chunk
represents a portion of the CDU. The partition of the CDU in chunks is globally known by all the nodes in the
network to support the appropriation process, which is a negotiation between nodes within an interference
domain. A node that manages to appropriate a chunk gets to decide which transmissions will occur over the
cells in the chunk within its interference domain. Ultimately, a chunk represents some amount of bandwidth
and can be seen as the generalization of a transmission channel in the time/frequency domain.
Chunk 1
Chunk 2
...
Chunk 8
CDU matrix, 6 channels, 7 timeslots, 8 chunks, represented from ASN= 0

58
11 12 13
25242322
3534333231
464544434241

59

60
Parent schedule
Child 1 schedule
Child 2 schedule
Child 3 schedule
Xmit mcast rcv rcv xmit rcv xmit
rcv rcv xmit
rcv xmit rcv
rcv xmit
Slotframe
Say a parent
appropriates the
pink chunk from
the CDU matrix
above. Now the
parent gets to
decide when the
pink cells are
used and by
which node.
A device maintains a schedule that dictates its
transmissions and receptions inside the slotframe.
The RPL parent
allocates timeslots
between itself and
its children. The
parent takes the
timeslots off a
chunk that it has
appropriated.

61

62
6TiSCH finally defines the peer-wise concept of a bundle, that is needed for the communication between
adjacent nodes.
A Bundle is a group of equivalent scheduled cells, i.e. cells identified by different [slotOffset,
channelOffset], which are scheduled for a same purpose, with the same neighbor, with the same flags, and
the same slotframe.
The size of the bundle refers to the number of cells it contains. Given the length of the slotframe, the size
of the bundle translates directly into bandwidth, either logical, or physical. Ultimately a bundle represent a
half-duplex link between nodes, one transmitter and one or more receivers, with a bandwidth that amount
to the sum of the timeslots in the bundle. Bundles thus come in pairs, an incoming and an outgoing.
A bundle is globally identified by (source MAC, destination MAC, trackID)
timeslot
bundle

63
We use a well-known constant as trackId for a L3 link, that I’ll refer as NULLT.
So an IP link between adjacent nodes A and B comprises 2 bundles, (macA, macB, NULLT) and (macB,
macA, NULLT).
timeslot
bundle
timeslot
bundle
outgoingIncoming
IP layer
AA
B

64
Along a that has a segment …A->B->C… , there are 2 bundles in node B,
incoming = (macA, macB, trackId) and outgoing = (macB, macC, trackId).
timeslot
bundle
timeslot
bundle
outgoingIncoming
C
B
A

65
Synchronizing the Network

66

67
• T1 is stamped on the fly at XMIT time
• Or sent is a following Packet
• 1-Way is good enough for short
transmission delays vs. clock precision
• 2-ways account fors transmission delays
but expects symmetry.
• It takes 3 sources to be really sure

68
IETF
TICTOC
ITU-T
Q13/15
IEEE1588-
2008
(PTPv2)
IEEE
802.1AS
IETF
NTP
AVB
Profile
ATIS
Telcordia
ProfiNet: IEC 61158 Type10
DeviceNet: IEC 62026-3
ControlNet: IEC 61158 Type2
IEC
Profiles
Telecom
Profile(s)
On-going

69
1. Scans all channels to detect
Extended Beacon
2. Synchronizes
3. Parses beacons for schedule
4. Performs Join Process (AAA)
5. Renumbers and DADs
6. Route setup (eg RPL DAO)
6TiSCH forms a single multilink
subnet, no loss of sync, no
renumbering
Industrial WSN Handheld moves:

70
Neighbors are discovered at L2 from 15.4e MAC Extended Beacon
Time parent selection based on a join priority field in the EB
Join priority as a distance but no real loop avoidance
=> time loops and
=> node isolation
When a new node B joins the network
it needs to wait for an EB to align itself to the sequence of slots
EBs are sent using TSCH
Node cannot know in advance the channel to listen
it may take a long time for B before catching an EB
The problem exacerbates for bootstrapping networks with many hops
After a node joins
synchronization should be maintained using data/ack and keepalive handshakes

71
One RPL instance dedicated to time parenting
RPL Rank used as join priority, set once once joined that timing instance
0xFF (Infinite) as long as not joined that instance => not a suitable time parent
RPL provides a better loop avoidance yet may let temporary loops
If DAGMaxRankIncrease is non-zero
8.2.2.4. Rank and Movement within a DODAG Version
If multiple control messages are lost
Loops will be detected reactively on traffic
3.2.7. Data-Path Validation and Loop Detection
11.2. Loop Avoidance and Detection
Q: Proscribe DAGMaxRankIncrease > 0 ?

72
GM 1
GM 2
GM 3
GM 4
Each Grand Master is a time source, and a root in a RPL graph

73
GM 1
GM 2
GM 3
GM 4
6TiSCH architecture describes RPL’s preferred parent tree to distribute time.

74
GM 1
GM 2
GM 3
GM 4
This baseline has the benefit that RPL manage the routes and maintain loopless time
distribution as the network changes and reforms (vs. BMCA which is too naïve WRT routing)

75
75
Multiple uncoordinated DODAGs with
independent roots (differing DODAGIDs)
DODAGs partition the network
No connectivity between DODAGs
Can be used to distribute time:
e.g. roots are GPS-based time sources
Nodes may jump to alternate DODAG
A single DODAG with a single root
May be used for time synchronization
Time may drift with distance if large mesh
A single DODAG with a virtual root that
coordinates LLN sinks (with the same
DODAGID) over a backbone network
May be used for time synchronization
Time can be synchronized over the backbone
e.g. Precision Time Protocol (IEC 61588, IEEE 1588v2)
6TiSCH
LLN
6TiSCH LLN
6TiSCH
LLN
Common time
source
Floating
root

76
6TiSCH
towards an IPv6-based
Industrial Internet
Pascal Thubert
ENG Labs
Sept 04th, 2013

77
• The any-to-any paradigm
Art: Any pair of global addresses can reach one another
IoT: Many devices of all sorts, no hotfixes
Corridors to the cloud?
• The end-to-end paradigm
Art: Intelligence at the edge, dumb routing nodes
Infinite bandwidth to all powerful clusters?
• The best-effort paradigm
Art: Stochastic packet distribution and RED
Can we make the whole Internet deterministic?

78
Trillion devices in the Internet
The core technologies will not change
Leakless Autonomic Fringe
Leakless Route Projection
Million devices in a Subnet
New models for the subnet protocols
IPv6 ND, RPL, Service Discovery …
Fiber + Copper + Wi-Fi Backbone
Femto + 802.11 + 802.15.4 + … Access
Unhindered Mobility
Location / ID Separation (LISP, NEMO)
10’s
of K

79
Data: Capillary Wireless
Acquisition of large amounts
of live Big Data
Information: DiM/Fog to
add descriptive meta-tags,
allowing for compression
and correlation
Knowledge: Prediction
from self-learned models
and Knowledge Diffusion
Wisdom: Machine Learnt
Expertise, auto-Generating
Prescriptions and Actionable
Recommendations
(Data)
ACQUISITION
(Knowledge)
DIFFUSION
(Wisdom)
OPEN LOOP
(Information)
AGGREGATION
5G
femtocell Data in
Motion / Fog
Learning
Machines
/ Cloud
6TiSCH

80
Aka Time Sensitive Networking (TSN)
For traffic known a priori (control loops…)
Time Synchronization and Global Schedule
NP-complete optim. problem => centralized computation
Fully controlled local network

81
We are in for a change:
Basic Internet structures and expectations reconsidered
Revising classical end-to-end and any-to-any
Revising best effort to new levels of guarantees (deterministic)
New function placement, new operations
Merge at the Information layer,
Gossip at the knowledge layer
Impacts network, transport, security models

Thank you.