Beyond Accurate Graph Neural Networks
Open Access
- Author:
- Tang, Xianfeng
- Graduate Program:
- Information Sciences and Technology
- Degree:
- Doctor of Philosophy
- Document Type:
- Dissertation
- Date of Defense:
- October 12, 2020
- Committee Members:
- Prasenjit Mitra, Dissertation Advisor/Co-Advisor
Prasenjit Mitra, Committee Chair/Co-Chair
Suhang Wang, Committee Chair/Co-Chair
Xiang Zhang, Committee Member
Wang-Chien Lee, Outside Member
Mary Beth Rosson, Program Head/Chair
Suhang Wang, Dissertation Advisor/Co-Advisor - Keywords:
- Machine Learning
Data Mining
Graph Data
Graph Neural Networks
Adversarial Attack
Robustness
Fairness
Explainable Machine Learning
Deep Learning - Abstract:
- Graphs are ubiquitous and are widely used to represent non-Euclidean correlations among entities. A large amount of graph data are collected to study science, human society, and economics, such as academic graphs, social correlations, and transportation networks. The advances in both machine learning algorithms and computer hardware make it possible to train deep neural networks on large-scale datasets. Inspired and motivated by the success of neural networks in computer vision, natural language processing, and graphics, researchers have developed Graph Neural Networks (GNNs) that leverage the power of deep learning for graph-based machine learning tasks. In recent years, various GNN models are proposed to improve the representation learning on both node/edge features and graph topology. Though these models have provided accurate outcomes for different regression/classification tasks, some essential characteristics of GNNs are neglected including robustness, fairness, and explainability, making the models less reliable and less trustful in many applications. For example, adopting graph neural networks on social graphs contributes to credit score analysis and loan prediction. However, if the models are not robust, criminals can raise their credit score and loan approval rate by adversarial attacks; GNN models for job candidates ranking and resume screening applications will lead to Unfair competitions with biased training data regarding gender and race. While the accuracy of GNN approaches is pioneered in most tasks, their black-box designs have obstructed researchers from understanding how GNNs produce outputs and why GNNs make mistakes. It is critical to improving the robustness, fairness, and explainability of accurate GNN models. However, building robust, unbiased, and explainable GNN models is an extremely challenging task, mainly owing to the unique message-passing mechanism of GNNs. Different from other neural network designs, GNN models not only transform feature vectors between layers but also aggregate information from neighborhoods for every node in the graph. The message-passing mechanism could broadcast perturbations injected by attackers and harm the representation learning on nodes. Besides, since many semi-supervised tasks on graphs are trained and tested on the same graph structure, GNNs need to defend against adversarial attacks in their training phase. How to design robust GNN models to jointly address these challenges is not well studied. Similarly, the fairness of GNNs is also different from other neural networks. The reasons are two folds: (1) GNNs require new definitions of fairness that not only cover node features but also cover properties graph topological structure, such as degree distribution and centrality; (2) algorithms and approaches should be developed accordingly to train/learn GNN models that satisfy the above goals. Unfortunately, fair and unbiased GNN models are still under-explored. Finally, the unique message-passing mechanism requires us to design special explanations that can consider the non-Euclidean graph topology, as well as new approaches to deriving the explanations. In this dissertation, I will go beyond accurate GNNs and focus on the robustness, fairness, and explainability of GNN models. I will start by introducing GNN designs, as well as the limitations of GNNs regarding robustness, fairness, and explainability. Then I will show three novel GNN models that enhance robustness, fairness, and explainability respectively. Each GNN model is developed based on real-world observations and theoretical analysis to tackle the most challenging parts of the under-explored limitations. Besides, I validate all proposed models on benchmark datasets and in real-world industrial applications, showing that the proposed methods/models achieve significant improvements over state-of-the-art approaches.