This project aims to develop ubiquitous low-power image processing platforms. It has several objectives including defining a reference platform, instantiating it through use cases, and demonstrating performance improvements. Several partners from industry and academia are involved. Key tasks include selecting hardware components, developing interfaces and tools, and validating the platform using applications like medical imaging, automotive driver assistance, and unmanned aerial vehicles. An initial hardware instance was selected using the Sundance EMC2 board with an ARM CPU and FPGA. The UAV use case involves real-time stereo depth estimation for obstacle avoidance.
Unlocking the Potential: Deep dive into ocean of Ceramic Magnets.pptx
TULIPP overview
1. This project has received funding from
the European Union’s Horizon 20 20
research and innovation programme
under grant agreement No 688403
www.tulipp.eu
TULIPP
Towards Ubiquitous Low-power Image
Processing Platforms
I. Tchouchenkov
15.05.2018
2. Partners
• Thales : coordinator and Medical use case
• Sundance : hardware
• Hipperos : Operating system
• Synective Labs : ADAS use case
• Efficient Innovation : Management
• Fraunhofer IOSB : UAV use case
• Ruhr Universität Bochum : FPGA tools
• NTNU : performance tools
3. Main objectives:
• Objective 1: Define a reference platform for low-
power image processing applications
• Objective 2: Instantiate the reference platform
through use cases applications
• Objective 3: Demonstrate and plan improvements
of defined key performance indicators
• Objective 4: Start-up and manage an ecosystem of
stakeholder to extend image processing norms
4. Project objectives
Towards Ubiquitous Low-Power Image Processing Platforms
Component tools
Operating System
Processor
Toolchain
Reference Platform
Memory
IO
Processor
5. WPs
WP7: Management, Coordination
LABEL : Marketing, Ecosystem and Pre-normalisation
WP6: IP protection, Dissemination, Communication, Advisory Board
and Exploitation preparation
WP1: Reference platform definition
(Interfaces & implementation Rules)
Instantiations
WP2:
Hardware
WP4:
Programming
Toolchain
WP3:
Runtime, API,
Libraries & OS
feedback WP5 : Usecases description
and Integration and platform
validation
6. Advisory Board and EcoSystem
• Guaranty
• Interconnectivity
• Faster time-to-market
• open standards
7. Tasks and WP2 objectives
Objectives:
1. The reference platform instantiation [based on the recommendations given in WP1 and coordinated with
WP3-WP5]
2. A holistic iterative development and optimization concept for low-power high-performance
image processing boards [taking into account results of WP3 - WP5].
Tasks:
T2.1 Components and parameters [M03 - M15] Leader: FHG / Participants: RUB, SUN, THL, NTNU, SYN
T2.2 Internal Components Interfaces [M06 to M12] Leader: FHG / Participants: SUN, THL, NTNU, SYN
T2.3 Development of Tulipp Platform [M12 - M34] Leader: SUN / Participants: FHG, THL, NTNU, SYN
8. Solution: Hardware Selection as a Project
1. REQUIREMENT ANALYSIS: The first step in selection understands the user’s
requirements within the framework and the environment in which the system is
being installed.
2. SYSTEM SPECIFICATION: The system specification must be clearly defined. These
specification must reflects the actual application to be handled by the system.
3. EVALUATION AND VALIDATION: The evaluation phase ranks various vendor
proposals and determines the one best suited to the user’s requirements. It looks
into items such as price, availability and technical support.
4. VENDER SELECTION: This step determines the vender with the best combination
of reputation, reliability, service record, training, delivery time, lease/finance
terms.
5. POST INSTALLATION REVIEW: The step checks how the user‘s requirements were
fulfilled.
9. Analysis of Use Cases
Sizes,
mm
Weight,
g Interfaces Resolution
Power
Consumption
Progr.
Language
Input Output minimal optimal
Medical
Imaging
PCIe 1x 2.0
minimum
1x Gigabit
Ethernet 1024x1024 1344x1344
< 10 W, better
5W
C/C++,
OpenCL
Automotive Camera Link Ethernet… 640x480 1024x512 Few watts
C/C++,
OpenCL,
CUDA
UAV 120 x 120 < 300
2 x Camera
Link
USB,CAN,
Ethernet 376x240 640x480
< 10 W, better
5W
C/C++,
OpenCL,
OpenMP
Input, MBits/sec Output, MBits/sec
Latency,
msecs
minimal preferred minimal preferred
Medical
Imaging 420 (2 bytes/pixel) 870 900 940 < 170
Automotive 222 (3 bytes/pixel) 378
<1 for control
250 for video
<1 for control
400 for video < 150
UAV 7 (1 byte/pixel) 73
<1 for control
8 for video
<1 for control
80 for video
< 100
(optimal 10)
1. REQUIREMENT ANALYSIS (partially)
2. SYSTEM SPECIFICATION (partially)
16. First instance of the Tulipp Hardware Node
Sundance EMC2-Z7030 (Z7015) with a dual-core ARM-A9 and Kintex-7 FPGA
Advantages:
PC/104 form factor board
Integrated 1Gb Ethernet, USB2.0, sATA-2
PCI Express 2.0 and integrated PCI Express switch
Infinite number of the boards can be stacked for large I/O solutions
Expandable with any VITA57.1 FMC I/O Module for more flexibility
Latest Xilinx SDSoC development environment integrated
Has an upgrade path to the Zynq UltraScale+
17. WP5: Unmanned Aerial Vehicle
(UAV) Use Case
• Uses state-of-the art stereo
algorithms (image
correlation)
• Produces a distance image,
i.e. where the image data
shows the distance to each
object
• Performs real-time stereo depth
estimation to do obstacle /
collision avoidance (for an UAV),
i.e. to detect obstacles in
direction of flight
• Based on dual cameras
18. Implementation of the obstacle
avoidance
Obstacle Stereo camera EMC2 Board
RS232 (-12V +12V)USB 2.0
DJI Matrice
MAX3223
3.3V TTL
Find contours
Histogram
Obstacle avoidance
U-Map
Short-Term-Map
API control