deep neural networks and homotopy continuation methods
Open Access
- Author:
- Zheng, Chunyue
- Graduate Program:
- Mathematics
- Degree:
- Doctor of Philosophy
- Document Type:
- Dissertation
- Date of Defense:
- October 01, 2021
- Committee Members:
- Wenrui Hao, Chair & Dissertation Advisor
Xiantao Li, Major Field Member
Jinchao Xu, Major Field Member
Alexei Novikov, Professor in Charge/Director of Graduate Studies
Sencun Zhu, Outside Unit & Field Member - Keywords:
- deep neural network
linear finite element
bifurcation
homotopy continuation method
adaptive - Abstract:
- The dissertation contains two parts. In the first part of the dissertation, we study the approximation property of deep neural networks(DNNs) and their application in numerical problems. In particular, we dive deep into the connection between DNNs with rectified linear unit (ReLU) function as the activation function and linear finite elements. By exploring the DNN representation of its nodal basis functions, we present a ReLU DNN representation of continuous piecewise linear (CPWL) functions in FEM with an estimation of the number of neurons in DNN that are needed in such a representation. Moreover, we present some numerical results for using ReLU DNNs to solve a two-point boundary problem to demonstrate the potential of applying DNN for the numerical solution of partial differential equations. In addition to the PDE example mentioned, we also apply the DNNs to approximate bifurcations of nonlinear parametric systems. After representing the solution of the nonlinear system in the form of neural networks with the parameters as input, we define an objective function and solve an optimization problem to obtain the approximation of bifurcation. We provide numerical results to demonstrate the feasibility of the method. In the second part of the dissertation, we study the numerical methods for computing bifurcations of nonlinear parametric systems. First, we propose an adaptive step-size homotopy tracking method. We use the Puiseux series interpolation and tangent cone structure to numerically compute bifurcation points and solutions on different branches. While the adaptive homotopy tracking algorithm focuses on computing bifurcations, we sometimes only care about the path's starting and ending points. To avoid the singularities during tracking, we present a stochastic homotopy tracking algorithm that can randomly perturb the original parametric system in each step. We then show that the stochastic solution path introduced by this new method is still theoretically close to the original solution path. Various numerical examples of nonlinear systems are given to illustrate the efficiency of these new approaches. Moreover, as an application of the adaptive homotopy tracking method, we develop a bifurcation analysis for a mathematical model of the plaque formation with a free boundary in the early stage of atherosclerosis. By performing the perturbation analysis to the radially symmetric steady-state solutions, we establish the existence of bifurcation branches for the low-density lipoprotein (LDL)/high-density lipoprotein (HDL)-cholesterol ratio and derive a theoretical condition that a bifurcation occurs for different modes. We also analyze the stability of radially symmetric steady-state solutions and conduct numerical simulations using the adaptive homotopy tracking method to verify all the theoretical results.