Sony Patents In-Game Sign Language Translator

Sony's groundbreaking patent aims to revolutionize accessibility for deaf gamers. This innovative technology translates sign languages within video games, breaking down communication barriers and fostering inclusivity.

Sony Patents ASL to JSL Translator for Video Games

Leveraging VR and Cloud Gaming for Seamless Translation

Sony's patent, titled "TRANSLATION OF SIGN LANGUAGE IN A VIRTUAL ENVIRONMENT," details a real-time sign language translation system for video games. Imagine playing with someone using American Sign Language (ASL), while a Japanese-speaking player sees the conversation seamlessly translated into Japanese Sign Language (JSL). This technology uses on-screen virtual indicators or avatars to communicate sign language instantaneously. The process involves a three-step translation: sign language is first converted to text, then translated to the target language, and finally re-rendered as the corresponding sign language.

As Sony explains in the patent: "Implementations of the present disclosure relate to methods and systems for capturing sign language of one user (e.g., Japanese), and translating the sign language to another user (e.g., English). Because sign languages vary depending on geographical origins, sign language is not universal. This provides a need for appropriately capturing the sign language of one user, understanding the native language, and generating new sign language as output for another user in their native sign language."

Sony envisions this system working with VR headsets or head-mounted displays (HMDs). These HMDs would connect to a user's device (PC, game console, etc.), providing an immersive gaming experience. The patent suggests a system where user devices communicate with each other and a game server over a network. This server maintains the game's state, ensuring synchronization between players. Furthermore, Sony proposes integration with cloud gaming, allowing for seamless streaming and interaction between players regardless of location.

This shared network or server allows players to interact within the same virtual environment. Sony suggests that the game server could be part of a cloud gaming system, rendering and streaming video to each user's device, enhancing the overall gaming experience and accessibility for deaf gamers worldwide.