GraphENS: Neighbor-Aware Ego Network Synthesis for Class-Imbalanced Node Classification

1. GraphENS: Neighbor-Aware Ego Network
Synthesis for Class-Imbalanced Node Classification
Joonhyung Park*, Jaeyun Song*, Eunho Yang
ICLR 2022
Graduate School of AI, KAIST
Machine Learning & Intelligence Laboratory

2. Introduction
● In real-world node classification tasks, graphs could be class-imbalanced inherently
● Overfitting to the neighbor sets of minor class due to message passing would be a
major challenge for class-imbalanced node classification
● We first investigate ‘Neighbor memorization’ problem and propose GraphENS
which effectively alleviates the memorization problem by synthesizing feasible
ego-network for minor class
Graph Neural Networks (GNNs) can be biased to major classes
!"#$%&
* Ego-network : the central node and its one-hop neighbors

3. Problem: Overfitting to Minor Classes
3
● The difference between train and test accuracies is large in minor classes
● Solid lines: Learning curves of minor class accuracy
● Dash lines: Learning curves of overall accuracy
● Compensating minor classes inevitably causes overfitting to minor classes
● Is this overfitting mainly due to memorizing node features? (or neighbor structures?)
Learning curves of imbalance handling approaches

4. Problem: Scrutinize the overfitting for minor classes
4
● Investigate overfitting in two aspects:
● 1) Node replacing experiment
● 2) Neighbor replacing experiment
Seen Nodes surrounded by Seen Neighbors Unseen Nodes surrounded by Seen Neighbors
Node replacing
Seen Nodes surrounded by Seen Neighbors Seen Nodes surrounded by Unseen Neighbors
Neighbor replacing

5. Problem: Scrutinize the overfitting for minor classes
5
● Performance drop of conventional algorithms in the neighbor-replacing
experiment is steeper than in the node-replacing
● Neighbor memorization problem is a critical obstacle in properly handling the
imbalance in node classification
: Accuracy of seen nodes surrounded by seen neighbors
: Accuracy of unseen nodes surrounded by seen neighbors
: Accuracy of seen nodes surrounded by seen neighbors
: Accuracy of seen nodes surrounded by unseen neighbors

6. Method Overview
● GraphENS synthesizes ego networks to construct a balanced graph

7. Method Overview
● Sample minor and target ego networks to synthesize a new ego network
1) Sample minor & target ego networks

8. Method Overview
● Neighbor Sampling assigns neighbors of a new mixed node
2) Neighbor Sampling

9. Method Overview
● Saliency-Based Node Mixing determines node feature of a new mixed node
3) Saliency-Based Node Mixing

10. Method Overview
● Attach the synthesized ego network to the original graph
4) Attach the ego network to the original graph

11. : Accuracy of seen nodes surrounded by seen neighbors
: Accuracy of unseen nodes surrounded by seen neighbors
: Accuracy of seen nodes surrounded by seen neighbors
: Accuracy of seen nodes surrounded by unseen neighbors
Experiment: Node & Neighbor Memorization
● GraphENS that substantially mitigates the both node & neighbor memorization
problem

12. Experiment: Semi-Supervised Learning on Citation Networks
● GraphENS outperforms baselines consistently on 3 citation network graphs

13. Experiment: Imbalance handling on Co-Purchase Graphs
● Our approach outperforms other baselines by significant margin in naturally class-
imbalanced graphs (Highly class-imbalanced)

14. Conclusion
● Our contribution is threefold:
● We demonstrate that in class-imbalanced node classification, GNNs severely overfit to
neighbor sets of minor class nodes
● GraphENS effectively alleviates the neighbor memorization by synthesizing feasible ego
networks based on the similarity between source ego networks
● We also block the injection of harmful features in generating the mixed nodes using
node feature saliency