SfMLearner + KF selectionを提案した"Unsupervised Collaborative Learning of Keyframe Detection and Visual Odometry Towards Monocular Deep SLAM [ICCV19]"を論文読み会で紹介した時の資料です.
08448380779 Call Girls In Civil Lines Women Seeking Men
Unsupervised Collaborative Learning of Keyframe Detection and Visual Odometry Towards Monocular Deep SLAMの解説
1. Unsupervised Collaborative Learning of
Keyframe Detection and Visual Odometry
Towards Monocular Deep SLAM [Sheng & Xu+, ICCV’19]
東京大学 相澤研究室
M2 金子 真也
2. 1
本論文
• Unsupervised Collaborative Learning of Keyframe Detection
and Visual Odometry Towards Monocular Deep SLAM
– 著者: L. Sheng, D. Xu, W. Ouyang and X. Wang
– 所属: Beihang University, Oxford, SenseTime
– 採択会議: ICCV2019
3. 2
本論文
• Unsupervised Collaborative Learning of Keyframe Detection
and Visual Odometry Towards Monocular Deep SLAM
– 著者: L. Sheng, D. Xu, W. Ouyang and X. Wang
– 所属: Beihang University, Oxford, SenseTime
– 採択会議: ICCV2019
– Monocular Deep SLAMを実現したいという強い気持ちの論文
– わかりみが深い
4. 3
Introduction
• Visual SLAM
– 3D reconstruction + Camera pose estimation
– 両者の同時最適化 (Bundle Adjustment)
Direct Sparse Odometry [Engel+, TPAMI’18]
5. 4
Introduction
• Deep Learning for Visual SLAM (Deep SLAM)
End-to-end Deep SLAMDL helps SLAM
SfMLearner [Zhou+, CVPR’17]
CNN-SLAM [Tateno+, CVPR’17]
CodeSLAM [Tateno+, CVPR’18]
DeepTAM [Zhou+, ECCV’18]
This figure is from Tombari’s presentation slide @ ICCVW.
6. 5
Introduction
• Deep Learning for Visual SLAM (Deep SLAM)
End-to-end Deep SLAMDL helps SLAM
CNN-SLAM [Tateno+, CVPR’17]
CodeSLAM [Tateno+, CVPR’18]
DeepTAM [Zhou+, ECCV’18]
SfMLearner [Zhou+, CVPR’17]
本論文の目標は,
この領域での最高のDeep SLAMを作ること
This figure is from Tombari’s presentation slide @ ICCVW.
7. 6
Related works
• SfMLearner [Zhou+, CVPR’17]
– 古典的なSfMを応用し, UnsupervisedにDeep SLAMを実現
– Training
• Photometric errorを最小化するように学習
8. 7
Related works
• SfMLearner [Zhou+, CVPR’17]
– 古典的なSfMを応用し, UnsupervisedにDeep SLAMを実現
– Inference
• 入力画像の奥行き画像と, 2視点間のカメラ姿勢をCNNで回帰
9. 8
Related works
• SfMLearner [Zhou+, CVPR’17]
– 古典的なSfMを応用し, UnsupervisedにDeep SLAMを実現
– Inference
• 入力画像の奥行き画像と, 2視点間のカメラ姿勢をCNNで回帰
より従来のVSLAMに近い
Deep SLAMを実現するためには???
29. 28
Proposed method
• Training
– Keyframe update & management
Random KF initialization
for epoch:
for iteration:
Choose training pair {ℐ𝑠𝑠, 𝐈𝐈𝑝𝑝, 𝐈𝐈𝑛𝑛}
Train all the model
if iteration > 200 &
Similarity(I𝑝𝑝, I𝑡𝑡) > th:
Insert tgt frame I𝑡𝑡 as KF
Merge KF
KF pool 𝒫𝒫 𝐾𝐾
Dataset
Model
30. 29
Proposed method
• Training
– Keyframe update & management
Random KF initialization
for epoch:
for iteration:
Choose training pair {𝓘𝓘𝒔𝒔, 𝐈𝐈𝒑𝒑, 𝐈𝐈𝒏𝒏}
Train all the model
if iteration > 200 &
Similarity(I𝑝𝑝, I𝑡𝑡) > th:
Insert tgt frame I𝑡𝑡 as KF
Merge KF
ℐ𝑠𝑠
{𝐈𝐈𝑝𝑝, 𝐈𝐈𝑛𝑛}
𝐈𝐈𝑡𝑡
Model
KF pool 𝒫𝒫 𝐾𝐾
Dataset
31. 30
Proposed method
• Training
– Keyframe update & management
Random KF initialization
for epoch:
for iteration:
Choose training pair {ℐ𝑠𝑠, I𝑝𝑝, I𝑛𝑛}
Train all the model
if iteration > 200 &
Similarity(I𝑝𝑝, I𝑡𝑡) > th:
Insert tgt frame I𝑡𝑡 as KF
Merge KF
KF pool 𝒫𝒫 𝐾𝐾
Dataset
ℐ𝑠𝑠
{𝐈𝐈𝑝𝑝, 𝐈𝐈𝑛𝑛}
𝐈𝐈𝑡𝑡
Model Loss
Train
32. 31
Proposed method
• Training
– Keyframe update & management
Random KF initialization
for epoch:
for iteration:
Choose training pair {ℐ𝑠𝑠, I𝑝𝑝, I𝑛𝑛}
Train all the model
if iteration > 200 &
Similarity(I𝑝𝑝, I𝑡𝑡) > th:
Insert tgt frame I𝑡𝑡 as KF
Merge KF
KF pool 𝒫𝒫 𝐾𝐾
Dataset
ℐ𝑠𝑠
{𝐈𝐈𝑝𝑝, 𝐈𝐈𝑛𝑛}
𝐈𝐈𝑡𝑡
Model Loss
Train
33. 32
Proposed method
• Training
– Keyframe update & management
Random KF initialization
for epoch:
for iteration:
Choose training pair {ℐ𝑠𝑠, 𝐈𝐈𝑝𝑝, 𝐈𝐈𝑛𝑛}
Train all the model
if iteration > 200 &
Similarity(𝐈𝐈𝑝𝑝, 𝐈𝐈𝑡𝑡) > th:
Insert tgt frame 𝐈𝐈𝒕𝒕 as KF
Merge KF
KF pool 𝒫𝒫 𝐾𝐾
Dataset
ℐ𝑠𝑠
𝐈𝐈𝑝𝑝
𝐈𝐈𝑡𝑡
Model Score
34. 33
Proposed method
• Training
– Keyframe update & management
Random KF initialization
for epoch:
for iteration:
Choose training pair {ℐ𝑠𝑠, 𝐈𝐈𝑝𝑝, 𝐈𝐈𝑛𝑛}
Train all the model
if iteration > 200 &
Similarity(𝐈𝐈𝑝𝑝, 𝐈𝐈𝑡𝑡) > th:
Insert tgt frame 𝐈𝐈𝒕𝒕 as KF
Merge KF
KF pool 𝒫𝒫 𝐾𝐾
Dataset
ℐ𝑠𝑠
𝐈𝐈𝑝𝑝
𝐈𝐈𝑡𝑡
Model Score
35. 34
Proposed method
• Training
– Keyframe update & management
Random KF initialization
for epoch:
for iteration:
Choose training pair {ℐ𝑠𝑠, 𝐈𝐈𝑝𝑝, 𝐈𝐈𝑛𝑛}
Train all the model
if iteration > 200 &
Similarity(𝐈𝐈𝑝𝑝, 𝐈𝐈𝑡𝑡) > th:
Insert tgt frame 𝐈𝐈𝒕𝒕 as KF
Merge KF
KF pool 𝒫𝒫 𝐾𝐾
Dataset
ℐ𝑠𝑠
𝐈𝐈𝑝𝑝
𝐈𝐈𝑡𝑡
Model Score
36. 35
Proposed method
• Training
– Keyframe update & management
Random KF initialization
for epoch:
for iteration:
Choose training pair {ℐ𝑠𝑠, 𝐈𝐈𝑝𝑝, 𝐈𝐈𝑛𝑛}
Train all the model
if iteration > 200 &
Similarity(𝐈𝐈𝑝𝑝, 𝐈𝐈𝑡𝑡) > th:
Insert tgt frame 𝐈𝐈𝒕𝒕 as KF
Merge KF
KF pool 𝒫𝒫 𝐾𝐾
Dataset
ℐ𝑠𝑠
𝐈𝐈𝑝𝑝
𝐈𝐈𝑡𝑡
Model Score
37. 36
Proposed method
• Training
– Keyframe update & management
Random KF initialization
for epoch:
for iteration:
Choose training pair {ℐ𝑠𝑠, 𝐈𝐈𝑝𝑝, 𝐈𝐈𝑛𝑛}
Train all the model
if iteration > 200 &
Similarity(I𝑝𝑝, I𝑡𝑡) > th:
Insert tgt frame I𝑡𝑡 as KF
Merge KF
KF pool 𝒫𝒫 𝐾𝐾
Dataset
𝐈𝐈𝑛𝑛𝐈𝐈𝑝𝑝
Model
ℐ𝑠𝑠
𝐈𝐈𝑡𝑡
Scores
38. 37
Proposed method
• Training
– Keyframe update & management
Random KF initialization
for epoch:
for iteration:
Choose training pair {ℐ𝑠𝑠, 𝐈𝐈𝑝𝑝, 𝐈𝐈𝑛𝑛}
Train all the model
if iteration > 200 &
Similarity(I𝑝𝑝, I𝑡𝑡) > th:
Insert tgt frame I𝑡𝑡 as KF
Merge KF
KF pool 𝒫𝒫 𝐾𝐾
Dataset
Model
39. 38
Proposed method
• Training
– Keyframe update & management
Random KF initialization
for epoch:
for iteration:
Choose training pair {ℐ𝑠𝑠, I𝑝𝑝, I𝑛𝑛}
Train all the model
if iteration > 200 &
Similarity(𝐼𝐼𝑝𝑝, 𝐼𝐼𝑡𝑡) > th:
Insert tgt frame 𝐼𝐼𝑡𝑡 as KF
Merge KF
KF pool 𝒫𝒫 𝐾𝐾
Dataset
Model
この操作を繰り返すことで
KF poolの最適化を行う