Optimal State Estimation , Sensor Fusion Study - Real World 1: GPS INS Fusion Speaker : howtowhy@gmail.com

1. GPS Carrier Phase / INS Integrated
Smartphone Pedestrian Dead-Reckoning
Using User Context Classifying Deep Learning
SEO YEON YANG

2. Motivation and Background
(2016) Raw GNSS
Measurement
API Android is opened
After 7. nougat version
(2018) Xiaomi Dual
frequency Smarphone
Software and Hardware upgrade in Smartphone

3. Motivation and Background
GPS / INS integration with
Raw measurement
“Sample-level deep
convolutional neural networks
for music auto-tagging using
raw waveforms,”
Jongpil Lee, Jiyoung Park,
Keunhyoung Luke Kim and Juhan
Nam
“Sample-level CNN
architectures for music auto-
tagging using raw waveforms”
Taejun Kim, Jongpil Lee and Juhan
Nam
Smartphone User context classification
Cycle slip in Urban
environment

4. Contribution
• Detail Analysis of the Smartphone INS sensor property
• GPS carrier phase cycle slip detection and compensation
in smartphone platform
• Raw GPS / INS integrated pedestrian dead-reckoning
system construction
• User Pose Context classification performance analysis
with different preprocessing

5. Android App development
5
GNSS raw
measurement Import
Doze, Safe mode
Block.
Asynchronize task
For logging

6. INS sensor Properties Analysis
x
z
Y
핸드폰 좌표계
양수 방향
+
+
+
Galaxy S8

7. INS sensor Properties Analysis
INS sensor Measurement model
: inertial to the body from the rotation matrix
: scale factor
: constant bias
: random bias
: Gaussian noise
: white noise
: constant bias
: random bias
: scale factor
Static experiment
Static experiment
Gravity, Turntable
experiment
Allan variance analysis

8. Accelerometer constant bias Gyroscope constant bias
INS sensor Properties Analysis
Constant bias, noise experiment
Accel
Gyro

9. INS sensor Properties Analysis
Accel x Accel y Accel z Gyro x Gyro y Gyro z
0.0093 0.0094 0.0127 0.0013 0.0014 0.0012
Accel x Accel y Accel z
Pose 1 Mean -0.1691 -0.3070
Pose 2 Mean -0.1597 0.2352
Mean -0.1644 -0.3070 0.2352
Gyro x Gyro y Gyro z
1 experiment 0.0117 -0.0145 -0.0336
2 experiment 0.0120 -0.0143 -0.0344
Mean 0.01185 -0.0144 -0.034

10. INS sensor Properties Analysis
Scale factors
Velocity controlled
Turn table

11. INS sensor Properties Analysis
Scale factors
Sx Sy Sz
1.0009 1.0157 1.00155 3.5386e-04 4.3278e-04 0.00155
Sx Sy Sz
1.0309 1.0162 1.0222 7.9584e-08 8.1220e-08 5.4688e-05
Accel
Gyro

12. Magnetometer Ellipse Fitting Preprocessing
INS sensor Properties Analysis

13. Allan Variance Analysis, Random Bias
https://drive.google.com/file/d/1MTnLZK5xaAtnVU9H2qPGlj0JB
_9ykD5v/view?usp=sharing
https://drive.google.com/file/d/1B1WjtABBleWax1jmqvZ
MtrnlNXvK9jXw/view?usp=sharing

14. Allan Variance Analysis, Random Bias
Accel
1.0e-03 *0.1242 1.0e+04 *3.6016 1.0e-05 *0.1665
1.0e-03 *0.1412 1.0e+04 *2.0633 1.0e-05 *0.1530
1.0e-03 *0.1260 1.0e+04 *1.1997 1.0e-05 *0.1534
Mag
0.0933 1.0e+05 *1.6202 0.0022
0.0926 1.0e+05 *0.2364 0.0008
0.1132 1.0e+05 *1.3894 0.0015
Gyro
0.0010 1.0e+03 *5.7443 1.0e-03 *0.1320
0.0010 1.0e+03 *3.9452 1.0e-03 *0.1305
0.0012 1.0e+03 *6.3465 1.0e-03 *0.1715

15. Attitude Quaternion Extended Kalman Filter
Sensor model
Bias model
Dynamic model

16. Attitude Quaternion Extended Kalman Filter
Process model
Measurement model

17. Simulation of the Attitude Filter
quaternion q0 q1 q2 q3
RMS residual 0.0013 0.0010 0.0017 0.0009
euler Roll (deg) Pitch (deg) Yaw (deg)
RMS residual 0.1669 0.1132 0.1804
True euler, quaternion
Random bias generation
Bias sum modeling
Simulation result and convergence residual

18. True Sensor Fitting
Quaternion Euler Heading
Euler angle
residual plot
(deg)
Roll
residual
-1.7172
deg
Pitch
residual
-1.7529
deg
Yaw
residual
1.9553
deg
Reference Sensor data (Pixhawk)
acquisition and Fitting Kalman Q,R
Quaternion Attitude Kalman filter, Sensor Fitting

19. Walking Detection, Step Counting.
Walking Detection, Step Counting, Stride Length
Accelerometer
magnitude
Walking Detection Step counting

20. PDR Fitting experiment
PDR parameter fitting experiment
Length
(cm)
Heading
(deg)
Step
number
Track line 1 951 325 17
Track line 2 956 415 18
Track line 3 943 145 17
Track line 4 943 235 17
Times Each 3 turn , 5 times experiment is done
Place Seoul National University President Grace
Threshold is
choosed with
experimental result.
Std th. 0.3

21. GPS raw measurement
GPS Pseudorange , Accumulated Deltarange

22. GPS raw measurement navigation
Position Navigation with pseudorange Velocity Esitmation with doppler / carrier

23. Introduction
: Integrated
Doppler (m)
Cycle slip detection with time
differenced carrier and dopplerCarrier sigma parameter
Cycle slip detection

24. Simulation enviromnet : cycle = 1000 * 5
1. 0~250 random position, 1cycle slip, ½
cycle slip injection
2. With threshold the cycle slip detected
1) cycle slip epoch is catched?
2) more than cycle slip is catched?
False Alam and Miss Detection
Cycle slip detection

25. 1 Cycle Half Cycle
Cycle slip detection

26. GPS Raw measurement Navigation
Name Mean
Error (deg)
Mean
Error (deg)
Mean
Error (m)
Mean
Error (m)
NMEA 2.8095e-05 2.7539e-05 2.7805 2.5397
Psuedorange 6.1566e-05 5.5452e-05 9.3837 5.4730
Name North Vel. Mean
Error (m/sec)
East Vel. Mean
Error (m/sec)
Total Mean
Error (m/sec)
WLS doppler 0.1950 0.1692 0.1821
Not WLS
doppler
0.2048 0.1906 0.1977
carrier 0.1878 0.1674 0.17766
NMEA 0.1916 0.2427 0.21715
Position Determination with PR
Velocity Determination with CP, DP

27. Frame

28. Frame

29. Frame

30. Frame

31. GPS / INS integration
Measurement model
Process model Time Interpolation
Experiment with
Trimble GEO-XR as true track
Pos : NMEA
Vel : carrier vel
Heading : cp heading

32. Position
residual N
Position
residual E
Velocity
residual
N
Velocity
residual
E
NMEA 2.5042
e-05
2.6971
e-05
0.1620 0.2091
2.60065 0.18555
Filter 2.4049
e-05
2.6270
e-05
0.1773 0.1396
2.51595 0.15845
LLA Position determination
with Position Filter
GPS / INS integration

33. Deep Learning Scheme
System diagram
<Data-preprocessing>
Smartphone sensors :
- Accelerometer
- Gyroscope
- Magnetometer
- Altimeter
- GPS
1) Sensor data
acquisition app
2) Calibration
<PDR>
Walking detection
Step counting
Stride length
Heading
Attitude Kalman
Pose Context PDR
<GPS/INS>
Cycle-slip detect
/compensate
Position Kalman
Error estimation
• User’s Walking
context
classification :
• GPS outage error estimation
deep learning:
MATLAB PART
TENSORFLOW PART

34. Dataset : Classification
Hand Held Hand Held Using Shirt Pocket
Trousers Front Pocket Trousers Back Pocket Backpack Handback
Walking Detection and
Step Counting on
Unconstrained
Smartphone :
Agata Brajdic
Open Dataset Class

35. Model 1
Time cutting
+ LSTM
Model 2
STFT
+ LSTM
Model 3
CNN
+ LSTM
Different Preprocessing

36. Type Model Prediction Loss
train Time domain LSTM 0.85 1.2
train STFT LSTM 0.95 1.1
train Time domain CNN
+ LSTM
0.92 1.15
test Time domain CNN
+ LSTM
0.75 1.32
Model 1 Model 2 Model 3
Training Result
Pred
Loss

37. The Galaxy S8 smartphone INS, bias, noise, scale factor, raw GPS
cycle slip property is detaily suggested.
Cycle slip elimination and
the raw GPS / INS integrated pedestrian dead reckoning getting
the high performance than NMEA
In position and velocity.
The smartphone’s walking context classification can know where
the phone is , what pose the user is.
Result

38. [1] Joonseong Gim ,; Kwan-dong Park; Comparison of Positioning Accuracy Using the Pseudorange from Android GPS
Raw Measurements. KONI 2017
[2] https://www.gsa.europa.eu/newsroom/news/world-s-first-dual-frequency-gnss-smartphone-hits-market
[3] Zhang, W.; Li, X. Wei, D.; Ji. X.; Yuan, H. A Foot-Mounted PDR System Based on IMU/EKF + HMM + ZUPT + ZARU +
HDR + Compass Algorithm. In Proceedings of the 2017 Internationl Conference on Indoor Positioning and Navigation(IPIN),
Sapporo, Japan, 18-21 September 2017.
[4] Shu, Y.; Bo, C.; Shen, G.; Zhao, C.; Li, L.; Zhao, F. Magicol : Indoor localization using pervasive magnetic field and
opportunistic WiFi sensing. IEEE J.Sel. Areas Commun. 2015. 2015, 33, 1443-1457
[5]. Li-Ta Hsu; Yanlei Gu; Yuyang Huang ; Shunsuke Kamijo; Urban Pedestrian Navigation Using Smartphone-Based
Dead Reckoning and 3-D Map-Aided GNSS. IEEE Sensors Journal Volume: 16 , Issue: 5 , March1, 2016
[6]. Tahmina Zebin ; Patricia J Scully ; Krikor B. Ozanyan; Human activity recognition with inertial sensors using a deep
learning approach. IEEE SENSORS 09 January 2017
Motion Recognition-Based 3D Pedestrian Navigation System Using Smartphone
Reference

39. [7]. Agata Brajdic ; Robert Harle ; Walking Detection and Step Counting on Unconstrained Smartphones UbiComp '13
Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing
[8]. Beomju Shin ; Chulki Kim ; Jaehun Kim ; Seok Lee; Motion Recognition-Based 3D Pedestrian Navigation System
Using Smartphone IEEE Sensors Council
[9]. Marcus Edel ; Enrico Köppe; An advanced method for pedestrian dead reckoning using BLSTM-RNNs. 2015
International Conference on Indoor Positioning and Indoor Navigation (IPIN)
[10] Ciftcioglu, Oe; Adaptive training of feedforward neural networks by Kalman filtering. GENERAL STUDIES OF
NUCLEAR REACTORS (E2400)
[11] Jiheon Kang; Joonbem Lee; Doo-Seop Eom; Smartphone-Based Traveled Distance Estimation Using Individual
Walking Patterns for Indoor Localization. 2018. August 13. IEEE Sensors
Remained Research