This slides indicate an introduction on the definition of real time and RTOSes, then you can find information on introducing RT Linux approaches and comparing them with each other, then finally you can see a latency measurement test done by "Linutronix" in the slides
1. Title: RT Linux
Seyed Navid Ashrafi
Telecommunication Networks Group
Technische Universität Berlin
Summer Semester 2018
35-minute presentation
NES SS2018 1
2. Over view
>> Introduction to RTOS
- Definitions and history
- Usages & requirements
NES SS2018 2
>> PREEMPT_RT patch
- Overview
- Features
>> Latency measurement test
>> Conclusion
>> Different approaches
- Xenomai
- PREEMPT_RT
- Cons & pros
3. What is real time?
• It is about fast operation time
• It is about performance
• Its about DETERMINISM
- You need to execute an operation in a specific time frame and you need a
guarantee for that.
- As fast as specified, not as fast as possible
- RT-Linux: Deterministic timing behavior in Linux.
- Depends on the system
NES SS2018 3
4. Usage and requirements of RT-Linux
• Usages:
- Industrial applications
- Multimedia systems
- Aerospace system
- Automative
- Financial services
• Requirements:
- Deterministic timing behavior
- Preemption
- priority inheritance
NES SS2018 4
5. Soft Realtime??
• It is in contrast with deterministic behavior
• We concentrate on 100% and 95% hard RT.
• 100% hard RT: Realtime requirements should be met 100% otherwise
it will lead to an error condition (industry)
• 95% hard RT: Realtime requirements should be met 95% (data
collection)
NES SS2018 5
6. Critical section
• Concurrent access to a shared resource can lead to
unexpected or error condition
• There are over 10,000 individual critical section in a
typical Linux 2.6 kernel
• In the non-preemptible design of Linux 2.6
kernel, any critical section can block preemption
• In the Preemptible design the effort was to reduce the total size and
number of non-preemptible critical sections.
NES SS2018 6
8. Dual-kernel approach
• First approaches which are still existing.
• Linux is running on top of a microkernel as a low priority real time task.
• The microkernel does the real time stuff.
• You need to maintain the microkernel and a hardware abstraction layer on Linux
to make it run on the microkernel:
- Big, time consuming effort.
- Reason why most of these approaches
are always one step behind
Linux fast development
NES SS2018 8
9. Single-kernel approach
• A way to make Linux it self real-time capable.
• You would have to touch every single file in the kernel.
• No microkernel or hardware abstraction layer (you can just work with
Linux standard tools).
NES SS2018 9
10. Xenomai
Xenomai 1.0
• Announced in 2001 – as portability framework for RTOS applications
• Required a real-time basis
• Development of ADEOS layer for Linux and RTAI
• Merged with RTAI
Xenomai 2.0
• Departed from RTAI in 2005 – incompatible design goals
• Evolved ADEOS to I-pipe layer (also used by RTAI)
• Ported to 6 architectures
Xenomai 3.0
• Released in 2015 after 5 years of development
• Rework of in-kernel core (now POSIX-centric)
• Support for native Linux
NES SS2018 10
13. Migration between schedulers
• creating realtime task in application, over a normal Linux task
• 2 threads for one task which share a lot of states
• only one of them can run at the same time
• Migration to RT: suspend Linux task, resume Cobalt thread
• Migration to Linux (on syscall, fault/trap, signal):
suspend Cobalt thread, resume Linux task
• Every things that we do not handle in Xenomai will be directed
to Linux
NES SS2018 13
15. Issues of dual-kernel approach
• Special API
• Special tools and libraries
• Microkernel needs to be ported for new HW and new Linux versions
• Bad scaling on big platforms
• Various number of not palatable patches
• Changes to critical subsystems regularly cause regressions
• Porting efforts consume core developer resources
-Most work done by Philippe and Gilles so far
-Time would be better spent on feature improvements...
NES SS2018 15
16. Xenomai application
• Machine control systems, PLCs
• Printing machines (manroland)
• Printers / copying machines
• Network switches (e.g. Ruggedcom)
• Magnetic resonance tomographs (Siemens Healthcare)
• OROCOS (OSS robotics framework)
• Robotic research projects
• … (many, many incognito applications)
NES SS2018 16
17. Preempt_RT
• Founded 14 years ago by: Thomas Gleixner & Ingo Molnar
• Huge community
• In-kernel approach
• Main idea: only code that, absolutely must be non-preemptible, is allowed
to be non-preemptible
• Most of the features already made it into the “mainline”
• Interrupt handlers run in a kernel thread which has two advantages:
- The kernel thread is interruptible
- It shows up in a process list with a PID
• First mainline integration 2006
NES SS2018 17
18. Preempt_RT Goals
• 100% Preemptible kernel
- not possible yet, but let’s try regardless
- Removal of interrupts and preemption disabling
- Fast “worst case” time.
• Quick reaction time
- Bring latencies down to a minimum
• Minimize non-Preemptible kernel code
- at the meantime Minimize code changing amount
NES SS2018 18
21. Fully Preemptible kernel (the RT patch)
• Preemption almost everywhere
• Spinlocks turn into mutexes
• No hard interrupt context
• Preemptible critical section
• Preemptible interrupt handlers
• Preemptible “interrupt disable” code sequence
• Priority inheritance
• Deferred operation
• Latency reduction
NES SS2018 21
22. Mutex
• Critical sections are still protected without disabling preemption
• A Mutex guarantees data integrity while extending the operation of the
spinlock
- we have priority inheritance
- All locks are assumed to be preemptible
- This design eliminates the possibility that an inefficient locking
implementation in a driver can introduce un-detected latency into
the system.
- Only a small subset of locks stay unpreemptible which are manageable.
NES SS2018 22
23. Priority inversion
• When a high priority task wants to run but it can not because a lower
priority task is holding that lock
• It is bad but we can not get rid of it, what we can get rid of is
unbounded priority inversion
NES SS2018 23
26. local_irq_disable
asmlinkage void
do_entInt(unsigned long type, unsigned long vector,
unsigned long la_ptr, struct pt_regs *regs)
{
struct pt_regs *old_regs;
local_irq_disable();
switch (type) {
case 0:
#ifdef CONFIG_SMP
NES SS2018 26
handle_ipi(regs);
return;
#else
irq_err_count++;
printk(KERN_CRIT "Interprocessor interrupt?
"You must be kidding!n");
#endif
break;
1 2
27. Preempt_disable
NES SS2018 27
• Local_irq_disable younger sibling
• Also does not give a hint to what does it do!
• Has the exact same problems of local_irq_disable
• Preempt_enable_no_resched
- only should be used within preempt_disable locations
28. Why do we still need Co-kernel?
Functional limitations of In-kernel
• Emulation of RTOS scheduling behavior limited
by Linux scheduler
• Not all kernel+libc code paths used by In-kernel approaches
are necessarily hard real-time under PREEMPT-RT
• No detection of non-RT behavior of your application
Performance limitations of In-kernel
• Co-kernel is usually more light-weight on low-end platforms
(limited caches vs. code path lengths)
• PREEMPT-RT can have unwanted impact
co-located non-RT workloads
Xenomai adds value to Linux
• it keeps the realtime stuff away from the complexities
of Linux kernel
• Co-kernel approach can be beneficial
for low latencies and real-time application architecture
NES SS2018 28
29. Latency measurement on xenomai & preempt_RT
• Made by Jan Altenberg from Linutronix GmbH
• On two ARM Cortex A9 SOC platforms
• IRQ test with 10KHz frequency with the latency box
• 100% CPU load with “hackbench”
• They did the test in usespace and kernelspace
• 12-hour duration
NES SS2018 29
30. Hackbench
• Starts n groups of 20 clients and 20 servers
• Each client sends 100 messages to each server via a socket
connection
NES SS2018 30
41. Test results
• Which approach to choose?
- If timing is vital, choose Xenomai, it separates the realtime interrupt
handling from complexities of Linux kernel
• You can go with PREEMPT_RT using FIQ
• Xenomai kernelspace is the best for 100% hard performance
• Xenomai userspace is slower than it’s kernelspace
• Tests only can help you in the field of latency which is dependent on
the hardware
• No guarantee for results repeatation
NES SS2018 41
42. Conclusion
• PREEMPT_RT is getting integrated in Linux mainline
- Download here: https://www.kernel.org/
• PREEMPT_RT Increases Linux stability and quality
• PREEMPT_RT is simple to use
• Xenomai has it’s own advantages over PREEMPT_RT
• Microkernels are hard to handle
• For the most use cases the latency of the both approaches is almost
the same
• FIQ offers very fast latency but brings limitations
NES SS2018 42
