Tulipp: Towards Ubiquitous Low-power Image Processing Platforms

1. This project has received funding from
the European Union’s Horizon 20 20
research and innovation programme
under grant agreement No 688403
www.tulipp.eu
TULIPP
Place :
Date :
PEGPUM Workshop
https://www.hipeac.net/events/activities/7435/pegpum/
Project Overview
HiPEAC’17, Stockholm
24th of January 2017
Philippe Millet, Thales, Coordinator

2. 8 Partners
• Thales : Coordinator and Medical
use case
• Sundance : Hardware
• Hipperos : Operating system
• Synective Labs : ADAS use case
• Efficient Innovation :
Management
• Fraunhofer IOSB : UAV use case
• Ruhr Universität Bochum : FPGA
tools
• NTNU : Performance tools

3. Outline
• Project objectives
• Main outcomes
• Advisory Board and Ecosystem

4. WP presentation
WP7: Management, Coordination
LABEL : Marketing, Ecosystem and Pre-normalisation
WP6: IP protection, Dissemination, Communication, Advisory Board
and Exploitation preparation
WP1: Reference platform definition
(Interfaces & implementation Rules)
Instantiations
WP2:
Hardware
WP4:
Programming
Toolchain
WP3:
Runtime, API,
Libraries & OS
feedback WP5 : Usecases description
and Integration and platform
validation

5. Starter Kit

6. Handbook Overview

7. Guidelines

8. Guidelines
Advice: Exploit both vectorization and multithreading for high performance on multicore
processors with vector units such as the ARM Cortex A9. On these architectures, utilizing
all hardware execution resources is key to achieve high performance [2] [4, 5].
Recommended implementation method: Use OpenMP. OpenMP is a widely supported
parallel programming API that enables programmers to express vectorization and
multithreading operations concisely using compiler directives. Programmers need not
worry about specifying scheduling and synchronization operations in code. These are
handled transparently by the OpenMP runtime system. See the official OpenMP
examples[6] to understand in more detail about exploiting vectorization and
multithreading simultaneously.

9. Reference Platform
Towards Ubiquitous Low-Power Image Processing Platforms
Component tools
Operating System
Processor
Toolchain
Reference Platform
Memory
IO
Processor

10. • The use case is taken from the many
applications that now enter our cars, doing
emergency braking, lane keeping, etc., with
direction towards autonomous cars
• The chosen image processing algorithm for the
use case is pedestrian detection that typically is
used for emergency braking and driver
assistance systems.
• Pedestrian detection is today mostly made by
either using Viola/Jones classifiers or Deep
Learning
• Viola Jones classifying has been chosen due to
its more challenging memory access patterns
ADAS use case

11. Pedestrian
detection
Safety
application
Car
integration
The Use Case
ADAS use case

12. Medical imaging use case
• Real-Time X-Ray imaging for surgery
• Reduce radiation dose by 75%
• Add noise removal processing with critical
real-time constraints

13. Medical imaging use case
TDLP
RAW IMAGE
THALES Processing
Unit
CI / ICS
UI
GigE-Vision + Msg
THALES Flat panel detector
Customer system
UI
GigE-Vision + Msg
CI / ICS
Nano Processing Unit
Inside the detector
Based on SoC (credit card size board)
Customer system
THALES Flat panel detector
Before Tulipp
After Tulipp

14. Unmanned Aerial Vehicle (UAV) use case
• Performs real-time stereo depth estimation to do obstacle /
collision avoidance (for an UAV), i.e. to detect obstacles in
direction of flight
• Based on dual cameras

15. Unmanned Aerial Vehicle (UAV) use case
• Uses state-of-the art stereo algorithms (image correlation)
• Produces a distance image, i.e. where the image data
shows the distance to each object

16. TULIPP Hardware Node
• Now – Zynq SoC SoM • Soon– Zynq MPSoC SoM
40mm x 50 mm – Z7030 40mm x 50 mm - ZU5EV

17. TULIPP System Node

18. Low-Power Image Processing RTOS
Needs OS
- high reliability,
- low power,
- hard real-time
- high performance
This kind of RTOS is not (yet) available today, as current GPOS (e.g.
Linux) or RTOS lack one or more of the required features and
performance.
Specific Image processing Needs
- supporting the hardware accelerators
- the libraries needed for image processing.

19. STHEM: The TULIPP Tool-chain
Status:
• Xilinx SDSoC has been extended to
support the current platform
• Support for HIPPEROS OS is underway
Insights:
• Significant effort has been invested
into the development of vendor tools
• STHEM fills the productivity gaps
between existing tools
Support uTilities for Heterogeneous
Embedded image processing (STHEM) •Supports development for all platform
components
•Map source files of the application to the
appropriate tool chain
•Retrieve OS configuration from the developer
Development and Mapping
•Boot OS with selected configuration (if
needed due to changed configuration)
•Update files (binaries, bitfiles, etc.)
•Initialise the reconfigurable logic (if needed)
•Start the application with the requested
instrumentation
Runner
•Analyses performance results and presents
findings to the developer
Analyser

20. TULIPP in the news
20

21. Advisory Board and EcoSystem
Advisory
Board
(WP6)
Reference
Platform
(WP1)
Tulipp Guide, implementation and demos
Ask for review / advise
Roles in the project:
Provide information about standards
Give feedback on the approach
Early adopters
Sys temEco
AB: first step to
the ecosystem

22. Advisory Board Members
22

23. Advisory Board and EcoSystem
• Pledge
• Interconnectivity
• Faster time-to-market
• Open standards