System Design and Implementation of Collaborative Mixed-Reality-Based Firefighter Training
Open Access
- Author:
- Dong, Zhanchen
- Graduate Program:
- Electrical Engineering
- Degree:
- Master of Science
- Document Type:
- Master Thesis
- Date of Defense:
- October 17, 2022
- Committee Members:
- Thomas F La Porta, Program Head/Chair
Bin Li, Thesis Advisor/Co-Advisor
Mahanth Gowda, Committee Member - Keywords:
- Mixed Reality
Collaboration
Firefighter Training - Abstract:
- Mixed reality (MR) is an approach to the hybrid of the physical world and the digital world. It expands the possibilities of the people’s realistic surroundings rather than simply supplementing information or adding virtual items as an overlay. In this work, we develop an MR system that facilities firefighter training. It uses a classic server-client architecture, which is widely adopted in multi-user video games. For the accurate displaying of MR contents, we build a virtual laboratory, a 3D model with the same scale as the real one. This virtual laboratory is deployed on a server with a simulated fire accident (e.g., virtual flames and smoke), where each client corresponds with an avatar inside to represent its movement and action. This avatar is used to determine visible contents that will be sent to the corresponding client. Users can see these virtual flames and smoke in the proper position through mobile mixed-reality-compatible devices. Besides, users can put out flames through a virtual fire extinguisher and see the change in temperature through a temperature visualization bar through the user interface. To further improve the synchronization between all connected clients (i.e., all clients should see the same status of the virtual fire accident), we propose a latency compensation algorithm. This algorithm can reduce the time difference of receiving the same message from the server for clients under different network conditions, in other words, decreasing the range of latency between the server and all clients. This algorithm can also reduce the extremes of latency so that more frames in the client can receive new information to update rendering. The latency compensation algorithm can decrease the maximum latency difference by 14.3% to 26%. However, the overall latency between clients and the server increases by about 32.5%.