An Eyes and Hands Model: Extending Visual and Motor Modules for Cognitive ArchitecturesTehranchi, Farnaz User modelingVisual attentionMotor skillsCognitive architecturesCognitive computingHuman-computer interactionArtificial intelligenceError detectionError correctionhuman errorsEyes and Hands modelComputer Science and EngineeringPHDA form of Artificial Intelligence simulates human intelligence and behavior. These simulations are not always complete and not always interactive. Adding a new type of memory and extending the visual and motor modules to existing cognitive architecture offers a motivating approach for simulating human behavior. This dissertation presents an Eyes and Hands model, a new approach to facilitate cognitive models to interact with the world. For this approach, the Java Segmentation and Manipulation (JSegMan) tool is built. JSegMan builds upon Java packages to segment and manipulate the screen. JSegMan also generates operating system commands to implement actions with interfaces. Cognitive architectures provide a unified theory of cognition for developing and simulating cognition and human behavior. The Eyes and Hands model extends two cognitive architecture modules, along with JSegMan, to facilitate interaction. Eyes and hands models can be used to explore the role of interaction in human behavior. In this dissertation, three Eyes and Hands models were developed: (a) the Dismal model that completed a spreadsheet task in the Dismal mode of Emacs, (b) the Biased-coin model based on an existing two-choice experiment, and (c) the Excel model that completed the spreadsheet task in the Excel task environment. I conducted two studies to investigate the model’s visual attention and response time. In the first study, learners’ eye movements data were recorded to predict learning. The results showed that with eye movement data, the learners’ performance could be predicted correctly 76% of the time. Therefore, where users are looking is important and should be considered in the simulation. In the second study, participants’ response time and eye movements were recorded. The Excel model was built upon this study. A simple Eyes and Hands Error model was built to demonstrate how the model’s time is allocated to error detection, error correction, and different types of knowledge. The results suggested that further analysis is required to investigate human errors.Rebecca Jane Passonneau, Dissertation Advisor/Co-AdvisorRebecca Jane Passonneau, Committee Chair/Co-ChairReginald Adams, Jr., Outside MemberRobert Collins, Committee MemberJesse Louis Barlow, Committee MemberFrank Edward Ritter, Dissertation Advisor/Co-AdvisorChitaranjan Das, Program Head/ChairFrank Edward Ritter, Committee Chair/Co-Chairopen_access2020-12-04T21:05:22Zhttps://etda.libraries.psu.edu/catalog/18328fjt5064