Predicting Concept Evolution during Naturalistic Reading with Simultaneous Eye-tracking and fMRI

Restricted (Penn State Only)
Author:
Schloss, Benjamin J
Graduate Program:
Psychology
Degree:
Master of Arts
Document Type:
Master Thesis
Date of Defense:
March 15, 2017
Committee Members:
  • Ping Li, Thesis Advisor
Keywords:
  • Multiband fMRI
  • eye-tracking
  • fixation related fMRI
  • conceptual representation
  • multivoxel pattern anlaysis
  • distributed semantics
  • word2vec
  • BEAGLE
  • semantic representation
  • naturalistic reading
Abstract:
The ability to predict how concepts are represented at a neural level has been the impetus for a surge in research attempting to understand how the human brain encodes meaning in language (Mitchell et al, 2008). One recent study demonstrated that we can even observe how these patterns of activity in the brain change as a function of learning (Bauer & Just, 2015). However, most of these studies are based on extremely small sample sizes, non-representative sets of stimuli, and tasks that do not have clear correspondences to ecologically valid learning scenarios. Therefore, the generalizability of this research is a major issue that we address in the current study, by extending the study of the neural representation of semantics and the functions which govern representational change to naturalistic reading. We combine multiband fMRI with simultaneous eye-tracking, collecting one full brain volume every 400ms and one eye measurement every millisecond, in order to allow maximal precision in aligning the evolving BOLD activity with the participants’ fixations in individual words. We demonstrate for the first time that computational models of word learning are able to predict not only how concepts are represented, but how they change online during reading.