NS-CUK Joint Journal Club: Minwoo Choi, Review on "Spatio-Temporal Wind Speed Forecasting using Graph Networks and Novel Transformer Architectures", Appied Energy 333 2023

1. Min-Woo Choi
Network Science Lab
Dept. of Artificial Intelligence
The Catholic University of Korea
E-mail: choimin1231@catholic.ac.kr
2023. 02. 14 Publish: 2022
Journal: Energy (IF: 8.85)

2. 1
 Introduction
• Limitation
• Purpose
• Main contribution
 Methodology
• Data description
• Experiment setup
 Results
 Conclusions
 Limitation and Future work

3. 2
1. Introduction
Limitation
• CNN-based spatio-temporal prediction models require wind information that nodes at regular
intervals or nodes in a square array.
• But actual wind farms are not necessarily arranged regularly.
Related study
• Fu et al. (2019): Spatiotemporal attention network (STAN)
- Using Multi head self attention to extract the spatial correlation.
• Khodayar et al. (2019): Graph convolution deep learning architecture (GCDLA)
- To capture the deep spatiotemporal information of wind speed.
1. Develop a relatively simple multi-node forecasting model by taking advantage of the various
architectures of the above methods.
2. Proposed Spatial-Temporal Graph Transformer Network (STGTN) model to improve short-
term wind speed prediction performance.
Purpose of study

4. 3
1. Introduction
Main contribution
• External attention mechanism is incorporated into the forecasting model
 It can capture dynamic spatial information and reduce the complexity of the network.
• A transformer model with graph convolution is proposed
 To learn spatial correlation based on the Euclidean distance between wind farms.

5. 4
2. Methodology
Data description
• Location: Danish offshore wind farms
• Period : February 6, 2014 to June 6, 2014
• Number of data: 111 wind turbine node (14,400 points)
• Resolution: 10 min
• Range of data:
• Training/Validation/Test set: 3:1:1
• Using historical data for each 12 points, the wind speed
is predicted for 10 min to 1 h.

6. 5
2. Methodology
Problem formulation
• Position information between wind turbines is represented as a graph
𝐺 = 𝑉, 𝐸, 𝐴
𝑉 = 𝑛𝑜𝑑𝑒𝑠 𝑜𝑓 𝑤𝑖𝑛𝑑 𝑡𝑢𝑏𝑖𝑛𝑒𝑠
𝐸 = 𝑐𝑜𝑛𝑛𝑒𝑐𝑡𝑖𝑜𝑛𝑠 𝑏𝑒𝑡𝑤𝑒𝑒𝑛 𝑡ℎ𝑒 𝑛𝑜𝑑𝑒𝑠
𝐴 = 𝑎𝑑𝑗𝑎𝑐𝑒𝑛𝑐𝑦 𝑚𝑎𝑡𝑟𝑖𝑥 𝑤𝑖𝑡ℎ 𝐸𝑢𝑐𝑙𝑖𝑑𝑒𝑎𝑛 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒 𝑏𝑒𝑡𝑤𝑒𝑒𝑛 𝑛𝑜𝑑𝑒𝑠
Spatialtemporal model of Short-term wind speed forecasting
Experiment setup
• Batch size: 64
• learning rate: 0.01
• Optimization: Adam
• Decay rate of 0.7 after every 10 epochs
𝑉
𝐸 𝐴
1 2 3
2
3
𝑑 𝑑

7. 6
2. Methodology
Graph transformer
Structure of the graph transformer
 The fusion of spatial and
temporal information
 it considered to improve the robustness of
extracting spatial features of wind speed
 higher-level features can be extracted in the
spatial correlation based on Euclidean distance
External attention mechanism

8. 7
2. Methodology
Architecture of the proposed model
Aggregate
spatiotemporal
features
Extract
temporal
feature
Extract
spatial
feature
Form input
through
residual
connection
Fig. 1. Illustration of the proposed spatial-temporal graph transformer network (STGTN).
Main contribution

9. 8
2. Methodology
Benchmark models
• SVR
• DL-STF
• STAN
• STGTN-T (Transformer)
• STGTN (proposed)
Not include spatial information
Based on spatial-temporal information (previous study)
Transformer and MLP

10. 9
3. Results
10 min to 1 h prediction
• The results show that STGTN dominates all
considered methods in terms of the lowest RMSE
values.
• STGTN with MLP outperforms STGTN-T with
transformer.
• Forecasting models based on spatiotemporal
information can track changes of wind speed faster.
• The performance of the STGTN model is more stable
compared with the DL-STF and STAN models.
0 min to 2000 min prediction
Consider the all wind turbine node

11. 10
3. Results
May 20, performance
• These results show that spatial information may degrades the forecasting performance when the
standard deviation of wind speed is small.
May 22, performance Wind speed Condition: fluctuation is small
• SVR is better than DL-STF
• SVR is better than STGTN-T

12. 11
3. Results
June 4, performance
May 21, performance
June 4, performance Wind speed Condition: fluctuation is large
• The MLP and Transformer performs similarly as the feature extractor when large-scale wind speed
fluctuations exist.
 On the whole, STGTN provides more stable and accurate wind speed forecasts compared with
other methods under different wind speed conditions.

13. 12
4. Conclusions
• The results indicate that the proposed model can effectively utilize the spatiotemporal
information to generate more accurate wind speed forecasts.
• Wind speed data of adjacent nodes is sufficiently used to correct the wrong predictions
caused by outliers in the historical data of individual nodes.
• The forecasting results confirms that the proposed model can yield stable wind speed
forecasts regardless of different scales of wind speed fluctuations.

14. 13
5. Limitation and Future work
• Seasonal factors are not considered. (just consider May & June)
• The utilization of wind direction information is not discussed.

15. 14
Thank you!