The Design and Scalable Implementation of Collaborative Mixed-Reallity-Aided LEGO Creation
Open Access
- Author:
- Yao, Xinyi
- Graduate Program:
- Electrical Engineering
- Degree:
- Master of Science
- Document Type:
- Master Thesis
- Date of Defense:
- November 02, 2022
- Committee Members:
- Bin Li, Thesis Advisor/Co-Advisor
Mahanth Gowda, Committee Member
Thomas F La Porta, Program Head/Chair - Keywords:
- Mixed Reality
Collaboration
Validation - Abstract:
- Mixed reality (MR) is a new paradigm that merges both real and virtual worlds to create new environments and visualizations. This together with the rapid growth of wireless virtual/augmented reality devices (such as smartphones and Microsoft HoloLens) spurs collaborative MR applications that provide an interactive and immersive experience for a group of people. In this thesis, we first develop a scalable MR-based platform for remote collaborative LEGO creation that guides each user to build LEGO units while allowing them to see each other's progress virtually. Towards this end, we introduce a central server to facilitate user synchronization via exchanging small messages. Each user reports to the server with its LEGO design progress, which is then distributed to all other users by the server; all other users render the corresponding virtual LEGO models in their own design space. We demonstrate via real-world implementations and evaluations that: 1) our system performance (e.g., synchronization delay, frame rate) does not degrade with the increase of the number of users; 2) our developed system not only yields a motion-to-display delay of 11 ms (i.e., 90 frames per second) but also achieves a screen resolution of each user's mobile device (e.g., $2400\times1080$ pixels for Google Pixel 6). To support real-time feedback and accurate validation, 1) the computation should be completed within 1 second, and 2) detection results should achieve as high accuracy as possible. Then, we develop an auto-validation algorithm to determine whether each user's LEGO creation exactly follows the MR guidance. We normalize the color to minimize the color differences and downscale the image to reduce the computation latency. The evaluation results demonstrate that our proposed algorithm 1) provides real-time feedback within 20ms, and 2) achieves more than 90$\%$ detection accuracy.