Optimizing Realistic 3D Facial Models for VR Avatars through Mesh Simplification

Sammanfattning: The use of realistic 3D avatars in Virtual Reality (VR) has gained significant traction in applications such as telecommunication and gaming, offering immersive experiences and face-to-face interactions. However, standalone VR devices often face limitations in computational resources and real-time rendering requirements, necessitating the optimization of 3D models through mesh simplification to enhance performance and ensure a smooth user experience. This thesis presents a pipeline that utilizes a Convolutional Neural Network to reconstruct realistic 3D human facial models in a static form from single RGB head images. The reconstructed models are then subjected to the Quadric Error Metrics simplification algorithm, enabling different levels of simplification to be achieved. An evaluation was conducted, utilizing 30 photos from the NoW dataset, to examine the trade-offs associated with employing mesh simplification on the generated facial models within the VR environment. The evaluation results demonstrate that reducing the polygon count improves frame rates and reduces GPU usage in VR, thereby enhancing overall performance. However, this improvement comes at the cost of increased simplification execution time and geometric errors, and decreased perceptual quality. This research contributes to the understanding of mesh simplification’s impact on human facial models within the VR context, providing insights into balancing model complexity and real-time rendering performance, particularly in resource-constrained environments such as mobile devices or cloud-based applications, as well as for models located farther away from the cameras.