(Click on the image for the demo video.)

A prototype of human-to-machine 3D interaction. In contrast to conventional 2D interaction practices, this framework thereby enables an advanced interaction routine:

1. It exercises pure direct 3D input capability.
2. It allows total touch free controlling capability.
3. It permits simultaneous multi-user interaction capability which opens access to multiple users for interacting with the system concurrently.
4. Equipped with a wireless module which can be used to control other electronic equipment.

The proposed interaction framework provides the users an innovative 3D workstation which comprises an array of three dimensional virtual dashboards. Each dashboard carries individually programmable virtual buttons. The user is able to switch through those 3D virtual dashboards by exercising the horizontal left and right swipe gestures. Furthermore, the framework enables 360° 3D rotation over 3D models. The users are able to perform all aforementioned functions by means of mid-air hand gestures without physically touching any equipment or without wearing any makers. One of the ambitions of the framework is to project those virtual entities into real world by using a sophisticated hologram projector. Nonetheless, since current hologram projectors are not advanced enough to project three dimensional entities to a sufficient distance, the framework renders all the virtual entities including the users themselves in real-time via a point cloud which assists users for interactions for the time being. Once the hologram technology is sophisticated enough to project three dimensional entities into a large distance, the point cloud visualizer of the framework can be replaced with a sophisticated hologram projector. The framework can be used as a user input module or as an interaction module which can be integrated into other user applications or systems as well as a standalone three dimensional interaction application. Consequently, the framework can be used to develop applications in many different fields, such as health care, three dimensional designing and education.

This Project is a prototype only. Please use only for reference purpose.

1. Extremely optimized real-time point cloud registration algorithm using CUDA.
2. Mid-air hand gestures algorithms
   1. Engage and disengage gestures
   2. Click gesture
   3. Rotation gesture
   4. Swipe gesture