Times Series Feature Extraction Methods of Wearable Signal Data for Deep Learning Computer Vision Applications

Copyright © SAS Institute Inc. All rights reserved.
Human Activity Recognition in Python:
Signal Analysis & Feature Extraction Methods
Utilizing Fourier and Wavelet Transformations
Compiled & Presented by William Nadolski
Based upon Jupyter Notebook created by Ahmet Taspinar

Presentation Agenda
• Human Activity Recognition Dataset
- Introduction to the data and classification task
- Performance of traditional feature extraction approaches for signal analysis
• Fourier Analysis
- Brief introduction to the Fast Fourier Transform (FFT)
- Feature extraction using the FFT
- Disadvantages of the FFT
• Wavelet Analysis
- Introduction to Wavelets
- Spectrogram creation using the Continuous Wavelet Transform (CWT)
• Train a deep learning convolutional neural network model (DLCNN)
on CWT-derived Spectrogram Images Using Keras Python API
• Potential Applications

Human Activity Recognition (HAR) Dataset
Download Data
• Consists of tri-axial accelerometer, gyroscope, and total acceleration data
collected via waist-worn smartphone while performing six different activities
• Data is sampled at 50 Hz and segmented into 10,299 distinct windows of
128 samples (2.56 sec each) and pre-split 70/30 into train/test subsets
• Data has been pre-processed and standardized to remove noise
• Goal is to classify activity using only the nine signal components
Data Citation: Davide Anguita, Alessandro Ghio, Luca Oneto, Xavier Parra and Jorge L. Reyes-Ortiz. A Public Domain Dataset for Human Activity RecognitionUsing Smartphones.
21th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, ESANN 2013. Bruges, Belgium 24-26 April 2013
Image Citation: http://ataspinar.com/wp-content/uploads/2018/01/3d_plot.png
9 Signal Components
128
Samples

Introduction to Fourier Transform
What is it and how can it be leveraged for machine learning?
The Fourier Transform decomposes a stationary time series signal into its
constituent frequencies. Allows us to convert the signal from the time domain
to the frequency domain and extract potentially useful features.
Fast Fourier Transform (FFT) is synonymous w/ Discrete Fourier Transform (DFT)
Fourier Equation: https://betterexplained.com/wp-content/uploads/images/DerivedDFT.png
Fourier Visual: https://www.nti-audio.com/portals/0/pic/news/FFT-Time-Frequency-View-540.png

Fast Fourier Transform in Python
Simulating and Decomposing a Composite Signal
• Create a composite signal from five constituent sine waves
• Decompose into Fourier magnitude and Power Spectral Density (PSD)

Feature Extraction Using FFT in Python
Return to HAR Dataset
• Example of applying the FFT to each signal component
• Interested in identifying the local maxima of the FFT & PSD functions
• Specifically interested in extracting the (f, a) coordinates for each peak

Returning to HAR Dataset
• No longer using traditional descriptive statistics
• Instead identify local maxima based on FFT of the signal component
• Wish to extract both the frequency and amplitude for top N peaks
• One set for FFT coefficient peaks and another set based on PSD values
• Using only top 3 peaks, we should expect the following features per record:
• 9 components X top 3 peaks X 4 coordinates (f,p for FFT and f,p for PSD) = 108 features
• If no peak present, impute zero value
108 features
Image: http://ataspinar.com/wp-content/uploads/2018/01/signal_to_matrix2.png
f = frequency
P = power (amplitude)

• Again, iterate over 10 different classifiers using scikit-learn
FFT provides a 3.4% point
boost in accuracy compared
to basic summary stats approach!

El Niño Data
Limitations of the Fast Fourier Transform
• Example: El Niño quarterly sea surface temperature from 1870-1996

El Niño Data
Limitations of the Fast Fourier Transform
Suggests a seasonality of
2-8 years (0.125 – 0.50 Hz)
Wikipedia: El Niño phases are known to occur close to four years, however,
records demonstrate that the cycles have lasted between two and seven years.
• Example: El Niño quarterly sea surface temperature from 1870-1996

Introduction to Wavelets
What are they and how can it be leveraged for machine learning?
• Wavelets: Localized in time with finite energy and zero mean
• Instead of frequency, uses frequency bands referred to as scales
• Wavelet Transform involves convolving a time series signal with a
“mother” wavelet at different scales (frequencies)
Sine vs. Wavelet: http://ataspinar.com/wp-content/uploads/2018/07/Wavelet-Out1.jpg
Wavelet Transform GIF: http://ataspinar.com/wp-content/uploads/2018/12/wavelet_convolution_3d_2d_final.gif
Image: http://ataspinar.com/wp-content/uploads/2018/08/Comparisonoftransformations.jpg

El Niño Data
Continuous Wavelet Transform (CWT)
• Wavelet transform is used in many digital signal processing tasks
such as audio filtering, data compression, image analysis, etc.
• Allows us to generate a 2D image of the signal called a spectrogram
• Generate spectrogram from El Niño data

Continuous Wavelet Transform
CWT Spectrogram of El Nino Data
• Shows power of a signal across different frequencies at different points in time
• Allows us to visualize changes over time within the frequency domain
• Shaded area corresponds to cone of influence

Continuous Wavelet Transform
• Going to train a CNN on each set of 9 spectrogram images via Keras in Python
• Extract 128x128 pixel spectrogram of each signal component for each record
• RGB images have 3 channels, we’ll just treat each signal component as its own channel

Convolutional Neural Network for HAR Classification
Utilizing Keras API within Python
• Modified LeNet architecture utilized:
- 1. Produce 32 feature maps using a 5x5 kernel with ReLu activation
- 2. Apply max pooling of size 2x2 with a 2x2 stride
- 3. Apply a dropout layer with probability 2.5%
- 4. Produce 64 feature maps using a 5x5 kernel with ReLu activation
- 5. Apply max pooling of size 2x2
- 6. Apply a dropout layer with probability 2.5%
- 7. Flatten the feature maps for each recording
- 8. Pass through a fully connected layer of 1,000 nodes
- 9. Apply dropout layer with probability 50%
- 10. Apply a softmax activation function to predict class label
CNN Architecture: http://ataspinar.com/wp-content/uploads/2018/12/9layer_image_CNN.png
def train_CNN(batch_size=64, epochs=100, lr=0.0001):
model = Sequential()
model.add(Conv2D(32, kernel_size=(5, 5),
strides=(1, 1), activation='relu’, input_shape=(127,127,9)))
model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))
model.add(Dropout(0.025))
model.add(Conv2D(64, (5, 5), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(1000, activation='relu'))
model.add(Dense(num_classes, activation='softmax'))
model.compile(loss=keras.losses.categorical_crossentropy,
optimizer=keras.optimizers.Adam(lr=lr, decay=0.0, amsgrad=False),
metrics=['accuracy'])
model.fit(Xtrain_CWT, Ytrain_CWT, batch_size=batch_size,
epochs=epochs, verbose=1,
validation_data=(Xtest_CWT, Ytest_CWT),
callbacks=[history])

Convolutional Neural Network for HAR Classification
Model Performance Over Time
• Obtained a maximum test accuracy of 96.1% with an average of ~95.5%
• Achieved +6.11% point increase in classification accuracy using CWT over FFT!
Epoch 71/100: 5000/5000 [==============================] train_loss: 0.0026 – train_acc: 0.9998 - val_loss: 0.3205 - val_acc: 0.9565
Comparison of different methods

Potential Applications
• IoT signal analysis
• Wearable medical devices
• Audio processing and analysis
• Image analysis and computer vision tasks
• Defect detection based on manufacturing machine sensors
• Supplement existing time series modelling approaches
• Data compression and dimensionality reduction techniques
• Data smoothing and signal denoising

Summary of Analysis
• Reviewed attributes of the HAR dataset and visualized sensor data
• Extracted traditional summary statistics for use in modeling
• Iterated over numerous classifiers to assess baseline model accuracy
• Fast Fourier Transformation
• Introduced the concept of the Fourier Transform
• Extracted FFT-derived features from the HAR dataset
• Improved model accuracy using only the FFT-derived features
• Illustrated some of the limitations and disadvantages of the FFT
• Wavelet Transformation
• Introduced the concept of the Wavelet Transformation and its benefits
• Used the CWT to create a spectrogram and visualize the El Nino dataset
• Used CWT-derived spectrograms of HAR data to train a CNN with Keras

Questions?

Times Series Feature Extraction Methods of Wearable Signal Data for Deep Learning Computer Vision Applications

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie Times Series Feature Extraction Methods of Wearable Signal Data for Deep Learning Computer Vision Applications

Ähnlich wie Times Series Feature Extraction Methods of Wearable Signal Data for Deep Learning Computer Vision Applications (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Times Series Feature Extraction Methods of Wearable Signal Data for Deep Learning Computer Vision Applications