Posted on Nov 23, 2017 | Rating
   
  

LipSync Generator

LipSync Generator generates lip-synchronized speech animation for the game characters given text as input.

Short non-technical description: LipSync Generator is implemented as a Unity 3D plug-in. The component works together with a Text-to-Speech (TTS) system. Since Unity 3D is multi-platform, the component can be deployed on different platforms including Web and mobile applications.

Technical description: The component takes as input text, sends it to the text-to-speech (TTS) system, parses the phonemes, maps them to visemes (visual counterparts of phonemes) and finally blends them for smooth speech animation. It currently uses an off-the-shelf text-to-speech (CereVoice) library. It is also possible to link it with other TTS systems. The component provides a state-of-the-art co-articulation algorithm to generate natural lip-sync animation for game characters.

Language: English

Download: LipSync-Generator.zip

Game development environment: Unity

Target platform: Windows

Programming language: C#

Version: 0.1

Version notes: Initial version

Development status: Completed

Commit URL: https://github.com/uu-rage/Lipsync-Generator

Type: Apache 2.0 (Apache License 2.0)

URL: https://opensource.org/licenses/Apache-2.0

Interpersonal skills Communication
Computers in other domains Computer games
Component Avatar Systems

Related Articles

Component
BML Realizer
Utrecht University Virtual Human Controller, Rage project, Utrecht University

Component
Dialogue Scenario Editor
Utrecht University, Rage project, Johan Jeuring

Component
Dialogue Scenario Reasoner
Utrecht University, Rage project, Johan Jeuring

Component
Performance Statistics
Open University of the Netherlands, Rage project, Rage project

Component
Client Side Game Storage Asset
Rage project, Rage project, Open University of the Netherlands

Component
Player-centric rule-and-pattern-based adaptation asset
Dessislava Vassileva, Rage project, Sofia University

Component
Real-Time Arousal Detection Using Galvanic Skin Response
Dessislava Vassileva, Boyan Bontchev, Rage project, Sofia University

Component
RAGE Tutorial Demo
Rage project, Rage project, Dominic Heutelbeck

Component
Competence Component v2
Matthias Maurer, Alexander Nussbaumer, Christina Steiner, Rage project, Graz University of Technology

Component
Virtual Labs Authoring tool
ENVISAGE project, Rage project, Dimitrios Ververidis
×