Learning to Learn with Structured Knowledge
Open Access
- Author:
- Yao, Huaxiu
- Graduate Program:
- Information Sciences and Technology
- Degree:
- Doctor of Philosophy
- Document Type:
- Dissertation
- Date of Defense:
- January 19, 2021
- Committee Members:
- Zhenhui Li, Dissertation Advisor/Co-Advisor
Zhenhui Li, Committee Chair/Co-Chair
Suhang Wang, Committee Member
Xiang Zhang, Committee Member
Lingzhou Xue, Outside Member
Mary Beth Rosson, Program Head/Chair - Keywords:
- Machine Learning
Data Mining
Deep Learning
Meta-learning
Relation Learning
Few-shot Learning - Abstract:
- The prevalence of AI platforms (e.g., Watson for Oncology) in many real-world applications demonstrates their power in learning from historical data. However, a ready-to-serve platform requires massive data to achieve good performance, leaving it infeasible to apply on newly-emerged tasks limited by insufficient data. By contrast, human intelligence is capable of quickly learning a task with limited data by referring to their relevant historical experiences. That leads us to a question: what if we can summarize models' learning process across training tasks as experience and make adaption on new tasks? We posit that this would shorten the period of data collection and maximize the data utilization by sharing information across tasks, thereby reducing the data amount requirement and facilitating the learning process. To leverage these learning experiences in intelligent agents, state-of-the-art learning to learn systems generalize and transfer globally shared knowledge from previous tasks. However, the past experiences are highly versatile (e.g., tasks are sampled from different distributions, tasks have heterogeneous input and output spaces), limiting the generalization ability of learning agents with global experiences. For example, the learning agents have learned several logic, athletic, and linguistic (e.g., English, Chinese) skills. To learn a new computer language (e.g., Python), the learning experiences from logic are significantly more important than other skills. The "messy" globally shared experiences fail to effectively capture and process information like humans and further benefit the new task learning. Hence, the "messy" nature of past experiences poses significant challenges to creating tools for learning from past experiences. Motivated by the psychology findings of human knowledge organization and the principle of compositionality, my research harnesses the power of "structure" and focuses on how to transform the past experience from "messy" to "structured". I have been developing a series of algorithms to provide a structured view of all previous experiences, enable efficient, interpretable, and holistic ways of generalization, and finally achieve quick adaptation of learning agents to versatile new tasks. This thesis will majorly discuss two directions: structured learning to learn algorithms and its applications on graph and time-series data. For the structured learning to learn algorithms, I propose a novel paradigm to automatically and continually extract the structure encrypted in massive related tasks and further utilizes the structured prior to benefit the newly encounter tasks. Second, I will discuss how to involve apply these algorithms and deploy them into several real-world E-commerce and smart city applications.