A Novel Human Interaction Game-like Application to Learn, Perform and Evaluate Modern Contemporary Singing: "Human Beat Box"

The paper presents an interactive game-like application to learn, perform and evaluate modern contemporary singing. The Human Beat Box (HBB) is being used as a case study. The game consists of two main modules. A sensor module that consists of a portable helmet based system containing an ultrasonic (US) transducer to capture tongue movements, a video camera for the lips, Kinect camera for face gestures, and a microphone for sound. A 3D environment game module is used to visualize a 3D recording studio as game world with all of its unique elements like guitars, mixer, amplifier, speakers and a microphone in front of the 3D avatar to simulate the recording ambience. The game also features a 2D virtual tutor to help the learner by giving oral and written feedback during the game. He also gives feedbacks during the practice session to improve the student's performance. The game is still at its early stages of development and it is been tested using simple HBB plosive sounds for percussion such as "PTK".