0
$\begingroup$

I have been combining Graph Convolutional Layers and Graph Pooling layers to define a neural network architecture for Graph Classification tasks. Specifically, using the Graph Convolutional Layer proposed by:

  • Kipf, T.N. and Welling, M., 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907.

And the Graph Pooling layer based-upon the Graclus algorithm from:

  • Defferrard, M., Bresson, X. and Vandergheynst, P., 2016. Convolutional neural networks on graphs with fast localized spectral filtering. Advances in neural information processing systems, 29, pp.3844-3852.

My graph contains 100 nodes, with 2 features per node. Where for the first feature we find -10<=x_1<=10 and for the second feature -30<=x_2<=-30. Nodes correspond to actual subregions of high-dimensional spaces, and edges can be introduced to model the neighborhood relationships between these different subregions. Notably, the feature matrix is rather sparse. Based upon labeled sets of feature matrices, I want to classify these matrices accurately, while exploiting knowledge about the neighborhood relationships between the nodes.

Therefore, I decided to setup a neural network architecture as follows:

  • Input Layer
  • Graph Convolution Layer (filters= 25, adjacency_matrix = A[0])
  • ReLU Activation
  • MaxPooling1D (pool_size=2)
  • Graph Convolution Layer (filters= 16, adjacency_matrix = A1)
  • ReLU Activation
  • MaxPooling1D (pool_size=2)
  • Flatten
  • Dense (nodes= 1024)
  • ReLU Activation
  • Dense (nodes= 256)
  • ReLU Activation
  • Dense (num_classes)
  • Softmax Activation

Note that the Graclus algorithm adds fake nodes to the original graph and reduces the size of the adjacency matrix. Such that one needs to pass the new adjacency matrices to every Graph convolution operation. However, I find that no matter how I train this network, the accuracy turns out be wildly fluctuating.

enter image description here

I achieve much better results when just using a MLP and neglecting the adjacency matrix, i.e. the neighborhood relations between subregions/nodes, completely. I also achieved very competitive results to the MLP when using a 2-channel CNN instead (by converting the graph into a 2d euclidean map using a self-organized map beforehand). The Graph Neural Network architecture not working is counter-intuitive and I am wondering whether or not one can identify a reason for this to be the case. I do not suppose feature normalization is a problem, as it has not been for the CNN.

$\endgroup$
1
  • $\begingroup$ The issue has been solved. The weights of the Graph convolution layer have been accidentally reinitialized at every call in my implementation. $\endgroup$ – Steve Jan 12 at 22:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.