Paper: http://ceur-ws.org/Vol-2882/paper50.pdf Hai Nguyen-Truong, San Cao, N. A. Khoa Nguyen, Bang-Dang Pham, Hieu Dao, Minh-Quan Le, Hoang-Phuc Nguyen-Dinh, Hai-Dang Nguyen and Minh-Triet Tran : HCMUS at MediaEval 2020: Ensembles of Temporal Deep Neural Networks for Table Tennis Strokes Classification Task. Proc. of MediaEval 2020, 14-15 December 2020, Online. The Sports Video Classification Tasks in the Multimedia Evaluation 2020 Challenge focuses on classifying different types of table tennis strokes in video segments. In this task, we - the HCMUS Team - perform multiple experiments, which includes a combination of models such as SlowFast, Optical Flow, DensePose, R2+1, Channel-Separated Convolutional Networks, to classify 21 types of table tennis strokes from video segments. In total, we submit eight runs corresponding to five different models with different sets of hyper-parameters in each of our models. In addition, we apply some pre-processing techniques on the dataset in order for our model to learn and classify more accurately. According to the evaluation results, one of our team's methods out-performs the other team's. In particular, our best run achieves 31.35\% global accuracy, and all of our methods show potential results in terms of local and global accuracy for action recognition tasks.

1. Hai Nguyen-Truong, San Cao, Khoa N. A. Nguyen, Bang-Dang
Pham, Hieu Dao, Minh-Quan Le, Hoang-Phuc Nguyen-Dinh,
Hai-Dang Nguyen, and Minh-Triet Tran
Ensembles of Temporal Deep Neural Networks
for Table Tennis Strokes Classification Task
MediaEval 2020 Classification of Strokes in Table Tennis
{nthai18, ctsan18, nnakhoa18, pbdang18, dhieu}@apcs.vn
{lmquan, ndhphuc, nhdang}@selab.hcmus.edu.vn, tmtriet@fit.hcmus.edu.vn
Dec.15,2020
1
MediaEval 2020
11th Anniversary Workshop
14-15 December 2020
Sophia Antipolis, France

2. Run 04 -Late Temporal Modeling in 3D-CNN [4]
2
1. Approach
● Late temporal modeling in 3D CNN Architectures with BERT
● Replace the conventional Temporal Global Average Pooling (TGAP) layer with
Bidirectional Encoder Representations from Transformers (BERT)
2. Training Config
● ResNeXt101[7] with 64 frames
3. Result
● 87.9% on validation dataset
● 25.42% on final result
[4] M. Esat Kalfaoglu et al. “Late Temporal Modeling in 3D CNN Architectures with BERT for Action Recognition”
[7] Saining Xie et al. “Aggregated Residual Transformations for Deep Neural Networks”

3. Run 08 - TSTCNN [6] with Multi-labeling and Ensembling
3
We reimplemented TSTCNN based on: https://github.com/P-eMartin/crisp
[6] Pierre-Etienne Martin et al. “Fine grained sport action recognition with Twin spatio-temporal convolutional neural networks: Application to table tennis”

4. Run 06 -Slowfast Networks [5]
4
4
Background Removal Process using Dense Pose [1]
[1]: Rıza Alp Güler et al. “DensePose: Dense Human Pose Estimation In The Wild”
[5] Christoph Feichtenhofer et al. “SlowFast Networks for Video Recognition”
Sample Results

5. Run 06 -Slowfast Networks (cont)
5
SFN 1
SFN 2
● Forehand
● BackHand
● Serve
● Offensive
● Defensive
● Forehand_Offensive_??
● Backhand_Offensive_??
● Forehand_Defensive_??
● Backhand_Defensive_??
● Forehand_Serve_??
● Backhand_Serve_??
concatenate
SFN 3
SFN 4
SFN 5
SFN 6

6. Run 03 -Baseline using CSN [2] method
1. Approach
● Channel-Separated Convolutional Networks (CSN) [2]
● Pointwise 1 x 1 x1 or Depthwise 3x3x3 convolutions.
Reduce computational cost and increase accuracy
2. Training Config
● Resnet3D [3] with Batch Normalization Frozen.
3. Result
● 86.67% on validation dataset.
● 28.81% on the final result.
6
[2]: Du Tran et al. “Video Classification with Channel-Separated Convolutional Networks”
[3]: https://github.com/kenshohara/3D-ResNets-PyTorch
Our code was based on: https://github.com/open-mmlab/mmaction2

7. Run 07-Improve CSN-based method and Ensemble
1. Training config
● Switch to Multi-label Classification.
● BCEWithLogitsLoss
2. Result
● 0.97 mAP = 0.9 top 1 on validation dataset.
● Stable learning with the first two label (Serve/Offensive/Defensive) and
(Forehand/Backhand).
● 25.98% on the final result.
● After post processing phase: 31.35%
7

8. Run 07 (cont)
Post Processing Diagram
8

9. Results
9
Accuracies of our 5 runs

10. Conclusion and Future Work
● CSN is an efficient method.
● Multi-label classification.
10

11. Thank you
Q&A
11