3. 2
Hierarchical graph structure
• Hierarchical structure is pervasive across complex networks with examples spanning from
neuroscience, economics, social organizations, urban systems, communications,
pharmaceuticals and biology, particularly metabolic and gene networks.
4. 3
Problems
Many embedding methods can be used in the node classification task by converting the graph
structure into sequences by performing random walks on the graph and computing co-
occurrence statistics.
=> unsupervised algorithms and cannot perform node classification tasks in an end-to-end
applications.
5. 4
Problems
Graph convolutional networks (GCNs) generates node embedding by combining information
from neighborhoods.
lack the “graph pooling” mechanism, which restricts the scale of the receptive field.
difficulty in obtaining adequate global information.
adding too many convolutional layers will result in the output features over-smoothed and
make them indistinguishable
6. 5
Problems
Some recent methods try to get the global information through deeper models.
they are either unsupervised models or need many training examples.
they are still not capable of solving the semi-supervised node classification task directly.
7. 6
Contributions
• First work to design a deep hierarchical model for the semi-supervised node classification task
which consists of more layers with larger receptive fields.
obtain more global information through the coarsening and refining procedures.
• Applying deep architectures and the pooling mechanism into classification tasks.
• The proposed model outperforms other state-of-the-art approaches and gains a considerable
improvement over other approaches with very few labeled samples provided for each class.
8. 7
Overview
• For each coarsening layer, the GCN is conducted to learn node representations.
• A coarsening operation is performed to aggregate structurally similar nodes into hyper-nodes.
each hyper-node represents a local structure of the original graph, which can facilitate exploiting global
structures on the graph.
• Following coarsening layers, a symmetric graph refining layers is applied to restore the original graph
structure for node classification tasks.
comprehensively capture nodes’ information from local to global perspectives, leading to better node
representations.
10. 9
Graph Coarsening Layer
Two steps:
• Structural equivalence grouping (SEG). If two nodes
share the same set of neighbors, they are considered to
be structurally equivalent.
assign these two nodes to be a hyper-node.
• Structural similarity grouping (SSG). Then, we
calculate the structural similarity between the
unmarked node pairs.
=> form a new hyper-node and mark the two nodes by
largest structural similarity. The largest structural similarity
is defined by comparing the normalized connection
strength:
11. 10
Graph Coarsening Layer
• The hidden node embedding matrix is determined as:
• Update adjacency matrix:
The hidden representation is fed into the next layer as
input.
The resulting node embedding to generate in each
coarsening layer will then be of lower resolution.
12. 11
Graph Refining Layer
• To restore the original topological structure of the graph and further facilitate node
classification
stacking the same numbers of graph refining layers as coarsening layers.
• Each refining layer contains two steps:
o generating node embedding vectors
o restoring node representations.
• A residual connections between the two corresponding coarsening and refining layers.
13. 12
Node Weight Embedding and Multiple Channels
• Multi-channel mechanisms help explore
features in different subspaces and H-GCN
employs multiple channels on GCN to
obtain rich information jointly at each layer
14. 13
The Output Layer
• Softmax classifier:
• The loss function is defined as the cross-entropy of predictions over the labeled nodes:
16. 15
Experiment
• The proposed method consistently outperforms
other state-of-the-art methods, which verify the
effectiveness of the proposed coarsening and
refining mechanisms.
• DeepWalk cannot model the attribute information,
which heavily restricts its performance.
• The proposed H-GCN manages to capture global
information through different levels of
convolutional layers and achieves the best results
among all four datasets.
17. 16
Impact of Scale of Training Data
• The proposed method outperforms other baselines
in all cases.
• With the number of labeled data decreasing, it
obtains a more considerable margin over these
baseline algorithms.
• The proposed H-GCN with increased receptive
fields is well-suited when training data is extremely
scarce and thereby is of significant practical values.
18. 17
Coarsening, refining layers and embedding weights
• The proposed H-GCN has better performance
compared to H-GCN without coarsening mechanisms
on all datasets.
=> The coarsening and refining mechanisms contribute
to performance improvements since they can obtain
global information with larger receptive fields.
• Model with node weight embeddings performs
better,
the necessity to add this embedding vector in the
node embeddings.
19. 18
Comparing with number of coarsening layers and channels
• Since fewer labeled nodes are supplied on NELL than
others, deeper layers and larger receptive fields are needed.
• Adding too many coarsening layers, the performance drops
due to overfitting.
• The performance improves with the number of channels
increasing until four channels => help capture accurate
node features.
• Too many channels will inevitably introduce redundant
parameters to the model, leading to overfitting as well.
20. 19
Conclusions
• A novel hierarchical graph convolutional networks for the semi-supervised node
classification task.
• The H-GCN model consists of coarsening layers and symmetric refining layers.
• By grouping structurally similar nodes to hyper-nodes, this model can get a larger receptive
field and enable sufficient information propagation.
• Compared with other previous work, H-GCN is deeper and can fully utilize both local and
global information.
• The has achieved substantial gains over them in the case that labeled data is extremely
scarce.