Innovation Owner
Mr. PHATTARA SRILACHOT
Student
Details
The Thai Sign Language Generation System creates a 3D animation platform that translates Thai text into accurate sign language gestures. It utilizes VQVAE and LLM technologies to bridge the communication gap for the Thai deaf community.
The Thai Sign Language Generation System aims to create a comprehensive 3D modeling and animation platform that translates Thai sentences into dynamic and accurate representations of Thai Sign Language (TSL) gestures. This project enhances communication for the Thai deaf community by leveraging a landmark-based approach using a Vector Quantized Variational Autoencoder (VQVAE) and a Large Language Model (LLM) for sign language generation.
- Model Training: The system trains a VQVAE encoder using landmark data to learn compact latent representations of TSL gestures.
- Data Augmentation: Encoded representations are used to generate additional landmark-based sign sequences, expanding the BigSign ThaiPBS dataset.
- Animation: An LLM is trained to output accurate landmark sequences from Thai text, which are then used to animate a 3D model in Blender.
- Implementation: The project uses Python, incorporating MediaPipe for landmark extraction, OpenCV for real-time image processing, and Blender’s Python API for 3D animation.
Objective
The project aims to develop a 3D animation system that translates Thai text into accurate Thai Sign Language, adhering to linguistic rules to improve accessibility and communication for the deaf community.
- Design and develop a 3D animation system capable of accurately and naturally translating written Thai text into Thai Sign Language (TSL) gestures.
- Ensure the system adheres to the linguistic rules of TSL, covering essential elements such as gesture speed and hand shapes.
- Utilize advanced techniques in Natural Language Processing (NLP), Machine Learning, and 3D modeling to translate Thai input and generate real-time 3D TSL animations.
- Provide an accessible communication tool to reduce barriers between hearing individuals and the deaf community or non-verbal individuals in Thailand.
- Create a flexible and user-friendly interface that allows users to input Thai text and receive accurate real-time TSL animations, promoting equality and improved communication.