講者： 工研院資通所闕志克所長

1. Technology Development Directions for
Taiwan’s AI Industry
Tzi-cker Chiueh 闕志克
Information and Communications Labs

2. Introduction to AI System
• AI ~ Deep Neural Network (DNN)-based Machine Learning
• AI system: a system with analysis and synthesis capabilities powered by DNN-
based machine learning
– Autonomous driving vehicle, drone, robot, personal virtual assistant, etc.
• Machine learning: a universal algorithm for building a functional mapping
between sample inputs and associated outputs
– A new paradigm of software development
– Learn from many normal people
vs. Design with few gifted experts
• From AI Winter to AI Everywhere
– Algorithmic breakthrough that enables
training of deep neural network
– Large high-quality training data set, e.g., ImageNet
– Availability of high-performance GPU

3. Overarching Strategies
• 產業AI化: Apply modern AI techniques to improving the value and
efficiency of existing industry segments,
– As common a tool as MatLab
– Medicine: diagnostics, nursing care
– Manufacturing: defect detection, equipment maintenance, robotic manipulation
– Finance: credit assessment, personal investment, trading algorithm
– Commerce: advertisement, retail analytics, logistics planning
• AI產業化: Convert modern AI techniques into new systems and
products that enable applications of AI
– High-performance DNN training
– Real-time low-power DNN inferencing
– DNN-based systems: autonomous driving vehicle, autonomous drone, robot,
personal virtual assistant

4. Machine Learning Basics
• Supervised Learning: from sample input-output pairs
– Labeling a training data set  knowledge acquisition
– Training to get a functional model  knowledge transfer
& abstraction
– Applying a learned model  knowledge application
– Ask the right question: Set up a proper optimization
objective function: “Like this?”
– Training corresponds to multi-variable non-linear optimization
• Universal Approximation Theorem
• Gradient descent-based search
• Unsupervised Learning
– Clustering
– Factor analysis
– Auto encoding

5. Training and Inference of Neural Network
Training: Forward/Backward Propagation 
Inference: Forward Propagation 

6. Key Technical Challenges in DNN
• Training of DNN model
– Quality: how to acquire high-quality training data set
• Label correctness and diversity
• Semi-automatic training data collection and labeling
– Speed:
• Reduce the number of rounds required in the training process
– Round  Epoch  Batch
• Reduce the computation overhead associated with each training round
• Speed and power consumption of applying DNN model (inference)
– Real-time: autonomous driving
– Embedded system: low power and low cost
• Explainability of learned DNN models
• Broadening the scope of DNN applications: from analysis to synthesis

7. Overview of DNN Systems Research
• Computational challenges brought by DNN
– Training (off-line)
• DNN Integrated Development Environment (IDE): reduce the number
of iterations required in a training process
• DNN Training Appliance: reduce the amount of time taken by each
training iteration, which involves multiple passes through the training
data set, e.g., Nvidia’s DGX-1 and Intel/Nervana’s Lake Crest
– Inference (on-line)
• Cloud-based DNN inference engine (high performance), e.g., Google’s
TensorFlow processing unit (TPU)
• Embedded DNN inference engine (low power and real-time) for
smartphone, and automatic driving vehicle, e.g., Nvidia’s TX2, Intel’s
Movidius (Neural Computing Stick), and Mobileye’s EyeQ

8. DNN Training Appliance
• DNN training has emerged as a crucial class of workloads in future data centers
• DNN appliance: a system that integrates (I) DNN IDE, (II) DNN model
optimization, (III) DNN training computation mapping and scheduling, and (IV)
GPU-based compute cluster to minimize the end-to-end training time
• Nvidia’s DGX-1
– Deep learning supercomputer
– NTD $4M + $1M
– P100 GPU + NVlink
– 170 TeraFLOPs of FP16
– Effective performance
equal to 250 Intel x86
CPU-based servers
– HGX-1: Hyperscale
GPU Accelerator

9. ITRI DNN Training Appliance
• Objective: Enable Taiwan to become a major power of DNN training appliances
• Hardware enhancements
– Processor
• Nvidia’s Tesla P100 and V100 (12GB, 4.7TFLOPs, $5899)
• Nvidia’s GeForce GTX 1080Ti (11GB, 11.3TFLOPs of FP32, $699)
• AMD’s Radeon RX-500 and RX Vega
• Intel’s Knights Mill (KNM)
– System Interconnect
• NVlink (Gen-Z)
• Meshed PCIe network
– Cooling
• Software Optimizations
– Minimize the performance impacts of lack of NVlink
– DNN training integrated development environment
• Leverage MOST’s AI computation platform as a reference case
Graphics driver API:
• CUDA
• OpenCL

10. DNN Inference Engine
• Track 1: Sensor data processing platform for autonomous driving
– Distributed architecture: edge processing and leaner network
– Centralized architecture: fatter network and more efficient processing
resource utilization
• Track 2: Customized DNN inference processor design
– Digital hardware approach
• Make data access as efficient as possible
• Decrease the total amount of computation for every possible input
• Reduce the amount of computation for easy or already seen inputs
– Analog hardware approach
• Programmable and persistent resistors
• I (output) = V (input) * 1/R could be used to
implement multiplication
• Wired-OR implements (current) addition

11. 無人駕駛車產業技術

12. SAE (Society of Automotive Engineering)
*L3補充說明：自動駕駛系統在發生故障或者超過使用範圍，需要沒辦法直接退出系
統功能，需要給車主足夠的時間來看，準備好方向盤和剎車，系統不需要達到最小
化風險
SAE
level
方向/加速/減
速動作執行
駕駛環境
的掌控感知
駕駛任務動態
接手*
行車環境
類型
0 人 人 人
1 人+系統 人 人 部分
2 系統 人 人 部分
3 系統 系統 人 部分
4 系統 系統 系統 部分
5 系統 系統 系統 全部
資料來源：
自動駕駛的等級

13. Autonomous Driving Vehicle (ADV)
13
Image
Sensor
LiDAR
Radar
感測分析硬體
深度學習影像辨識
多重感測融合
自動駕駛事件推理
自動駕駛決策
模組
人機介面次
系統
生成/修正目標行駛
路線
動力系統制動
車速控制模組
方向盤操控控
制模組
感知次系統
智慧決策
整合控制
定位與地圖
/交通資訊與動態路
徑規畫模組
聚焦發展感知次系統，為車用
電子能量延伸發展之領域
由車廠主導
由車廠主導

14. Key Strategic Decisions
以次系統為優先開發主軸 特定場域自動接駁服務
開發期系統測試能力 高延展性DNN技術自主
 先投入感知次系統掌
握關鍵
 後結合控制與決策次
系統鞏固市場
 開發特定場域自駕車
 特定場域自駕車接駁
服務
 感知技術以DNN為主，
強化大規模訓練所需的
資料及方法
 聚焦東亞/東南亞地區
行車資料蒐集
 測試、驗證、展示、試
營運
 沙崙場域、工研院中興
院區、新竹高鐵等場域

15. 15
Technology Development Plan
通用移動電訊、趨勢科技、聯
發科、新唐、華創車電
3.自駕車規之
資安及可靠度
1. 泛用車輛環境感
知次系統
• 我國首套自主研發之自動
接駁系統，民眾可透過
APP定點呼叫自駕車(預計
108年可定點接駁)
2. 特定場域自駕
車接駁服務
宏碁、華創、車王電、
利佳、光寶、明泰
• 具備自駕車行駛對周遭任何物件可
即時感知相對位置、速度、近端軌
跡，並具備事件推理能力
• 目標：與Mobileye in 2021並駕齊驅
• 協助我國廠商建立自主
「快速反應、高可靠與
高安全性」自駕車資安
系統
自動駕駛感知次系統技術
深度學習影像辨識
聯發科、明泰、朋程、瑞科、
光寶、研勤、勤崴、為升、鼎
天
多重感測融合
即時事件推理
感測分析硬體
自動
駕駛
感知
次系
統
感知次系統驗證

16. 16
DNN Training Data Collection
for Taiwan’s Road Conditions
 發展策略：
• 以深度學習為核心研發適用於各種道路環境之影像辨識技術，補足我國相關產業與國際車
用大廠(如Mobileye)間之競爭基礎。針對台灣各種道路環境與天候狀況建立亞洲第一套以
自動駕駛訓練所用台灣街景影像資料庫「FORMOSA」，利用既有計畫資源（如車隊裝設
影像設備）迅速蒐集訓練資料降低成本，並累積至少10萬公里的影像訓練資料內容，厚實
國內相關產業發展即時影像辨識技術之基礎。
 執行重點：
• 真實環境視訊物件與事件之蒐集
• 晴天、陰天、雨天環境等
• 車輛物件: 大客車、小客車、貨車、聯結車、露營車等
• 交通號誌物件：速限號誌、紅綠燈、交通標誌、道路路面、落地招牌、道路標線等
• 多種感測器:光達(LiDAR)、雷達(RADAR)等
• 動態事件:行人、四輪車輛、二輪車輛與交通號誌、紅綠燈狀態變化等。
• 視訊資料標記工具
• 台灣街景影像資料庫「 FORMOSA 」之資料格式特徵與物件類別定義，並進行人工
標記作業與開發半自動化影像物件標記工具，以簡化人工標記成本。

17. 17
特定場域情境自駕車 系統開發時程
特定場域自駕車接駁服務
*

18. 無人機應用服務系統

19. 19Copyright 2017 限閱資料、禁止複製、轉載及外流
具台灣地域特色之無人機應用
 台灣特色下無人機可擔任之角色與加值開發方向
資料來源：MIC (2017/08)
台灣空域或
環境特色
台灣特色(簡述) 適合無人機技術發展或輸出方案 (舉例)
人口密度 台灣人口密度，全球排名為九。 現場人數統計、民眾移動流量、移動追蹤，但也為使用無機最大安全
顧忌之一。
橋梁密度 全台10,769座公路橋梁，1,605座鐵道
橋梁
巡檢橋梁、裂縫檢視、河川水位與流量、建物靜載重與變形監視、其
他公共設施與建物巡檢維護。
河川密度 短小、流速快、流量變化大，具116獨
立水系。
污染源巡視、水源地保護、高灘地管理、氾濫或乾涸，
複合災害 颱風多、地震多，帶來複合型的災害。 災區掌握、指揮調度、搜尋搜救、協助災區通訊、復原災區緊急物質
運補。
生態物種 熱帶、亞熱帶、溫帶及寒帶各氣候類
型均有，鳥種密度世界第二。
生態觀測、植被檢視，生態資源盤點、水土保持巡檢、看見台灣空拍、
台灣3D景致觀賞、3D型式觀光服務。
小農密度 農牧戶人口為301萬餘人，占總人口數
12.9%。
精準農業、農產估算、噴藥施肥、巡田檢視、作物生病蟲害觀測。
違章建築 全台違章建築未拆除數量高達67.3萬
件，違章建築未拆除數量續創新高。
違章建築查報、違章建築統計、地政分析監理。
海岸線長 北部為岬角海岸、東部為斷層海岸、
南部為珊瑚礁海岸、西部為沙質海岸。
國土巡邏、海岸監理巡防、遏止偷渡運毒、看見台灣無動力滑翔機/飛
行傘活動或觀光。
高低起伏 地形豐富，高低變化大。短距離切面
即有海岸、河谷、平原、丘陵、台地
與山地。具268座3千公尺以上的高山。
植被生態觀測養護、國家森林資源盤保護、水土保持。
全球無人機產品與應用最佳試飛與驗證場地。
富饒3D景致表現、觀光活動或科研活動、無人機競爭。
無人區 人口集中西半部平原，中央山脈與東
部人稀少，僅少部分平原有人口分布。
礦產或森林資源探勘、生態觀測植被檢視；不用造橋鋪路，無人機能
入無人之區巡視與運補；提供無人機安全航道或試飛空域有更多選項。

20. 20Copyright 2017 限閱資料、禁止複製、轉載及外流
無人機產業鏈
無人機製造/品牌商
酬載系統 (Payload)
雲台 (Gimbal)、
攝影機、熱像儀等
飛控系統
飛控軟體、陀螺儀
、磁力計等
機體元件
馬達、電池、
旋翼、機架等
通訊模組
GPS、RC/
4G通訊元件等
操作系統
遙控器、
地面站軟體
無人機隊應用服務維運商
c
學研關鍵技術
應用軟體商
運研所：交通流量分析
高鐵局：橋梁檢測
水利署：砂石盜採、水庫巡檢
能源局：太陽能/風力/電力檢測
警政署：維安、偏鄉巡邏
消防署：災害救援
內政 經濟 交通
空氣、河川、坡
地及廢棄物監測
環保
農藥噴灑、魚塭
巡檢、精準農業
農業
保險業者無人機系統整合商
資料來源：本計畫整理(2017/08)

21. 21Copyright 2017 限閱資料、禁止複製、轉載及外流
無人機應用核心技術開發/實證計畫
 基於國際無人機發展趨勢、我國資通訊產業優勢及無人機產業缺口，發展高價
值無人機隊應用核心技術與系統服務，帶動軟硬體及系統整合產業發展。
 建立穩定、安全、長時、遠距及智慧化之無人機隊系統，結合國內警政巡邏、
智慧巡檢(橋檢/環保)、救災通訊等應用場域進行驗證，提供民眾有感服務。
資料來源：本計畫整理(2017/08)
無人機核心軟體技術
• 無人機安全飛控技術
• 三維避障防撞技術
執勤時間展延技術
• 自動無人機充電站技術
• 長效動力管理技術
• 節能飛行管理技術
酬載系統技術
• 無人機酬載介面標準化
• 酬載協同操控與顯示技術
無人機隊管理技術
• 多機編隊遠距執勤技術
• 無人機空域管理系統
場域實證推動
• 警用巡邏無人機技術
• 智慧巡檢無人機技術
• 救災通訊無人機技術
警政雲
無縫結合路口監視、M-Police及
無人機影音，建構更安全之生活
環境
提升安全性 介面標準化
服務自動化增加續航力
垂直應用整合
警用巡邏
智慧巡檢 救災通訊

22. 22Copyright 2017 限閱資料、禁止複製、轉載及外流
環保空污偵測
 桃園環保局/觀音工業區：環保空污偵測方案
每2秒sample一次
飛行速度:5m/s
三級不同高度的飛行計畫
Mission 1
Mission 2
Mission 3

23. 23Copyright 2017 限閱資料、禁止複製、轉載及外流
高鐵/國道橋梁巡檢
 高鐵：台中大甲溪/新竹頭前溪段，進行電力桿/支承墊/橋墩等檢測。
 高公局：國道六號橋聳雲天(平均高度58公尺)，進行支承墊/橋面/鋼橋檢測。
台中大甲溪
電力桿
螺栓鬆脫
混泥土剝落
鋼橋檢視國六支承墊檢視
國道六號橋聳雲天
飛行路線&
Way Point Way Point
行為設定 30m
無人機飛行排程 排程路徑拍攝現有橋檢車作業

24. 24Copyright 2017 限閱資料、禁止複製、轉載及外流
太陽能板巡檢
 沙崙場域/工研院南分院：進行太陽能板排程巡檢驗證。
沙崙綠能科學城
無人機測試場域建置規劃

25. 25Copyright 2017 限閱資料、禁止複製、轉載及外流
高壓電塔 礙子清洗
• 台電每年在高壓電塔維護作業花費 > NT$80億。
• 需求如：預防電纜線連接頭因高溫斷裂毀損而斷電、礙子表面汙損時洩漏電流及
清掃、高壓電塔與附近的異狀物的距離觀測、高壓電塔表面銹蝕檢測與補漆。
Manpower Helicopters UAV
Man Hour Costs
12.5 thousand/Hour
(Annual cost about 7.5 billion a
year)
100 thousand / Hour
(Annual cost about 0.5 billion)
10 thousand / Hour
Working Area City / Reachable place
Remote /Rural Areas
(Height limit in recent years)
No Limit
Safety
Electric shock / Person fall
down
Mechanical failure / Turbulence
(There are accidents these
three years)
No Casualty
Cleaning Method
Short Distance
Cloth wiping / Water Column
Long Distance
High pressure water column
cleaning
Short Distance
Dry Ice Cleaning
Real-Time Surveillance System
Operation Power Outage / Live Line Live Line Live Line
Manpower : high pressure
water column cleaningManpower : Cloth wiping UAV dry ice cleaning
Helicopters: high pressure water
column cleaning

26. 自動資安攻防系統

27. Common Cyber Attack Scenarios
• From the Internet
– Scanning public IP addresses
– Fingerprinting the OS and applications
– Applying attacks to exploit known vulnerabilities
– Increasingly difficult with multi-level defense around servers
• From the intranet
– Social engineering via email or social network sites
– Drive-by download
– Stepping stone to attack enterprises
– Increasingly common with dirtier endpoints
27

28. Exploiting a Vulnerability
Network-Facing Application
Malware (3)
Bootstrap Payload (2)
Cause Damage
RUN (4)
Vulnerability (1)
28

29. Programmatic Defense Mechanisms
• Stop injection of shell code (Step 1)
– Eliminate security loopholes in applications
– Buffer overflow prevention: CASH: a fast bounds checking compiler
• Stop injected shell code from performing unauthorized sensitive
operations (Step 2)
– Address space or library randomization
– System call monitoring: PAID: an accurate system call monitoring tool
• Stop when malicious binaries are downloaded (Step 3)
– Scanning of malware during transit in firewalls
• Stop an injected malware at invocation time (Step 4)
– Black-listing: traditional anti-virus SW
– White-listing
– Run-time behavior monitoring29

30. DARPA’s Cyber Grand Challenge
• Problem: Once a bug is announced, a
race is on between bad guys that aim
to exploit the bug, and good guys that
aim to patch the bug, and bad guys
typically win.
• Goal: DARPA seeks to create
automatic defensive systems that are
capable of reasoning about flaws,
formulating patches and deploying
them on a network in real time.
• Result: Seven teams competed in
August 2016, and the champion
team, ForAllSecure, led by David
Brumley from CMU, took home a $2
million award.30

31. Automated Cyber Defense/Offense
Program analysis-based software security
– Input: Adobe flash player has a bug
– Outputs:
• Where is the bug? Is it a vulnerability?
• How to exploit it?
• How to patch it?
• How to develop an intrusion detection signature for it?
– Exploration approaches
• Fuzzing: random mutation, and anomaly- or corner case-driven stressing
• Intelligent fuzzing: exploiting knowledge on inputs
• Symbolic execution : source code/binary code
– Full automation: Bug  Vulnerability  Patch  Signature  Attack
– Targets: DARPA CGC and NSA/CIA’s leaked attack toolkit31

32. Summary
• DNN is expected to become the focal point of ICT research in the
next couple of years
– Myriad applications are possible and promising
• Analysis and Synthesis
• More promising bet is on business/enterprise applications using natural language
understanding, e.g., legal technology, regulatory technology, patent analysis, etc.
– Systems support for DNN training and inference is a promising area
• DNN training appliance and DNN inference processor
• Competition in DNN inference processor is like that in GPU design 15 years ago.
• DNN-based systems
– Perception subsystem for autonomous driving
– Tele-operated drone-based application services
– Automated cyber defense and offense

33. Thank You!
Questions and Comments?
tcc@itri.org.tw

34. Training Data Collection and Labeling
• ImageNet is the most successful example so far
• How to reduce the effort and improve the quality of training data collection
and labeling?
– Crowd sourcing for raw data collection
– Human-based computation: image
labeling as a multi-user game
– Computer-aided labeling: object/label tracking across video frames
– Generation of meaningful new training data from example data set
– Unsupervised learning
• Corner-case training data collection and labeling
– Crowd sourcing-based collection
– Model-driven synthesis (for ADV)
• Augmented reality: videos augmented with graphics objects
• Photo-realistic graphics rendering

35. Analysis Applications of DNN
• Perception subsystem for Autonomous Driving
– The relative coordinate, relative speed and future
trajectory of every driving-related object
within a certain distance
– Real-time video object locationing,
recognition and tracking: YOLO  object detection
and classification in one shot
– Driving event prediction
– Multi-sensor data fusion and analysis: RGBD
• Analytics for New Retail
– Trajectory of every shopper, the set of
merchandises he touches, and his likings

36. Synthesis Applications of DNN
• Movie critic  Movie director
• Generative Adversarial Network
(GAN) : a game theoretic
approach to converting an
analysis model into a synthesis
model