Tutorial at International Workshop on Smart Info-Media Systems in Asia (SISA2015) http://www.ieice-sisa.org/?page_id=583#ts2

1. Developing Video Signal
Processing Algorithms for
Embedded Vision
Systems
Shogo MURAMATSU
Dept. of Elec. & Electronic Eng.
Niigata University

2. Contents
 Introduction to Embedded Vision
 Tutorial Material with MATLAB®/Simulink ®
EmbVision Tutorial
 Example Activities
Smart Dormitory Program (PBL Example)
NSOLT Project (Research Example)
 Conclusions
2015/8/26 SISA 2015 @ Chiba Institute of Tech.
 Introduction to Embedded Vision
 Tutorial Material with MATLAB®/Simulink®
EmbVision Tutorial
 Example Activities
Smart Dormitory Program (PBL Example)
NSOLT Project (Research Example)
 Conclusions

3. The IoT Era is Beginning.
 Diversification of Sensing Environment
2015/8/26 SISA 2015 @ Chiba Institute of Tech.
Embedded Systems will be Deployed Broadly.

4. Seeing is Believing.
 Rapid Proliferation of Vision Systems
2015/8/26 SISA 2015 @ Chiba Institute of Tech.
Embedded
System
Computer
Vision
Embedded
Vision
Embedded Vision Systems gather Attention.

5. Contents
 Introduction to Embedded Vision
 Tutorial Material with MATLAB ® /Simulink ®
 EmbVision Tutorial
 Example Activities
 Smart Dormitory Program (PBL Example)
 NSOLT Project (Research Example)
 Conclusions
2015/8/26 SISA 2015 @ Chiba Institute of Tech.

6. From DSP First to SI First
 DSP First: A Multimedia Approach (1998)
 Presents basic DSP concepts in an intuitive
style by multimedia signals with MATLAB.
 Evolution of MATLAB makes the System
Integration (SI) First Approach available.
 Embedded System Development
 Web + Database Management
 Network Connection
DB
2015/8/26 SISA 2015 @ Chiba Institute of Tech.

7. EmbVision Tutorial
 Educational Material
for new members in
our Lab.
(Junior or Senior)
 Estimated to take 12H
SISA 2015 @ Chiba Institute of Tech.2015/8/26
Today, condensed to
50min!
http://msiplab.eng.niigata-u.ac.jp/embvision/en/

8. Demo with Raspberry Pi™
 Gradient Filtering on
Raspberry Pi
(Final Exercise)
2015/8/26 SISA 2015 @ Chiba Institute of Tech.

9. Overview of EmbVision Tutorial
 One can implement user-defined System
objects on Raspberry Pi.
2015/8/26 SISA 2015 @ Chiba Institute of Tech.
Part 1 Part 2 Part 3 Part 4 Part 5 Part 6
Intensity
Adjustment
Color space
Conv.
Grad.
Filter
System
Object™
Video
Stream
Process.
MATLAB
System
block
Unit
Test
Raspberry
Pi
MATLAB Simulink

10. Part 1:
Image I/O and Pixel Processing
 Target
 Read, display, and write images
 Simple pixel processing
 Exercises
 Intensity Adjustment
 Color Space Conversion
2015/8/26 SISA 2015 @ Chiba Institute of Tech.
RGB2
HSV
HSV2
RGB
𝑆 ← 2𝑆

11. Part 2:
Filtering and Frequency Analysis
 Target
 1-D signal and 2-D image filtering
 Frequency analysis
 Exercises
 Horizontal Differential Filter
 Magnitude and Direction of Gradient
2015/8/26 SISA 2015 @ Chiba Institute of Tech.

12. Part 3:
Class Definition and Unit Testing
 Target
 Object-oriented programming in MATLAB
 Unit testing framework in MATLAB.
 Exercises
 HSV2RGB Class
 Gradient Filter Class
2015/8/26 SISA 2015 @ Chiba Institute of Tech.

13. Part 4: Video Stream Processing
- MATLAB -
 Target
Read, display and write videos in MATLAB
Video stream processing in MATLAB
 Exercises
Sobel Gradient Filter
Frame Difference
2015/8/26 SISA 2015 @ Chiba Institute of Tech.

14. Part 5: Video Stream Processing
- Simulink -
 Target
 Read and display videos with Simulink
 Use MATLAB System blocks on Simulink
 Video stream processing with Simulink
 Exercises
 Prewitt Gradient Filter
 Sobel Gradient Filter
2015/8/26 SISA 2015 @ Chiba Institute of Tech.

15. Part 6: Video Stream Processing
- Raspberry Pi™ -
 Target
 Simulate Simulink model in external mode
 Deploy Simulink model on Raspberry Pi
 Exercises
 Sobel Gradient Filter
 Deploy a created new model
(Option)
2015/8/26 SISA 2015 @ Chiba Institute of Tech.

16. DIY
 List of Expected Extension Examples
 Raspberry Pi Camera Module Control
 Fixed-point Implementation
 Parallel Implementation (for Quad Core on RasPi2)
 Detection & Recognition of Objects
 Speech & Audio Signal Processing
 GPIO/I2C Controls
 Network Application Development
 etc.
2015/8/26 SISA 2015 @ Chiba Institute of Tech.

17. Contents
 Introduction to Embedded Vision
 Tutorial Material with MATLAB®/Simulink®
EmbVision Tutorial
 Example Activities
Smart Dormitory Program (PBL Example)
NSOLT Project (Research Example)
 Conclusions
2015/8/26 SISA 2015 @ Chiba Institute of Tech.

18. Smart Dormitory Program
 Project-Based Learning (PBL) Program at Facul.
of Eng., Niigata Univ.
 Financially Supported by MEXT
 MEXT: Ministry of Education, Culture, Sports, Science and
Technology
 15 Research Groups are working on their research
activities.
 The 2nd group is Cyber Physical System (CPS)
Dormitory
 Members are from different grades and departments.
 Develop an IoT App. with embedded vision systems
2015/8/26 SISA 2015 @ Chiba Institute of Tech.

19. CPS Dormitory
 Develops a System for Estimation and Visualization of
Customer Congestion at a Student Cafeteria
 BeagleBone Black boards as Sensor Nodes
 ThingSpeak as an IoT data collection plathome
Installation of a Sensor Node Deployment Map of Sensor Nodes
2015/8/26 SISA 2015 @ Chiba Institute of Tech.

20. Presentation at the 4th MEXT
Science Inter-College
2015/8/26 SISA 2015 @ Chiba Institute of Tech.
@Kobe Convention Center
29th Feb. - 1st Mar., 2015

21.  Nonseparable Oversampled Lapped Transform
 Sparsity-Aware Image and Volume Data Restoration
(a)
(b)
NSOLT Project
[Muramatsu,ICASSP2014]
2015/8/26 SISA 2015 @ Chiba Institute of Tech.
Simulink Model

22. Zynq® Implementation
 Xilinx®’s All Programmable SoC
ARM® CPU and FPGA are connected to each
other through AXI4 interface on a single chip
2015/8/26 SISA 2015 @ Chiba Institute of Tech.
Xilinx’s Zynq Inside of Zynq

23. Workflow of Co-implementation
Generation of HDL
(HDLCoder™)
Generation of C
(Embedded Coder® )
Configuration Build
Zynq
2015/8/26
System Model of NSOLT
(MATLAB/Simulink)
Isolation of HW and SW in the model
SISA 2015 @ Chiba Institute of Tech.

24. Contents
 Introduction to Embedded Vision
 Tutorial Material with MATLAB®/Simulink®
EmbVision Tutorial
 Example Activities
Smart Dormitory Program (PBL Example)
NSOLT Project (Research Example)
 Conclusions
2015/8/26 SISA 2015 @ Chiba Institute of Tech.

25. Conclusions
 This tutorial explained
 How to develop and evaluate video signal
processing algorithms with MATLAB/Simulink
 How to implement visual applications on
embedded systems with MATLAB/Simulink
 Two example activities were introduced
 CPS Dormitory Prog. as an Educational Activity
 NSOLT Project as a Research Activity
2015/8/26 SISA 2015 @ Chiba Institute of Tech.