Egocentric Ranging/Pointing System Based on Smart Glasses

ProfessorJen-Hui Chuang
ApplicationCloud computing and the application of internet of things(IoTs), Entertainment, Medicine and health care, the applications of education, and Virtual reality and augmented reality
FunctionAR (Augmented reality), Information of depth sensor, and user interface
Technical BenefitImprove system integration and Improve system intelligence
Technology StatusCan be transferred

We propose an egocentric (first person vision) ranging/pointing system based on smart glasses which is equipped with an optical head-mounted display (OHMD) and a near-eye camera. For a pointed target, the ranging/pointing algorithm is based on a distance-pixel curve established from the camera-eye (epipolar) geometry. Additional pointing algorithms for estimating gazing point on a planar surface are also developed by establishing another distance-pixel curve along the same epipolar line.

When user is pointing a target (Side recording), the system can capture an image with the camera (View of camera), and find the gazing point along epipolar line in the image. Finally, the AR information of target can be shown on a display or OHMD (Output display).