Canesta announced its collaboration with Hitachi to create a television that can be controlled by open-air hand gestures, rather than by the familiar hand-held remote control. The prototype, which uses a tiny Canesta 3D sensor for gesture recognition and a Hitachi TV can be controlled at distances up to 3 meters (9.8 feet) by simple hand gestures. For example, a rapid wave will power up the set, and a circular motion will change either the video source or the channel. The technology can also discriminate between single- and two-hand gestures, providing additional command options. Because of the underlying Canesta 3D sensor's immunity to extremes in room lighting or décor, the interface can work both in and out doors.

"The touchless, gesture-based interface is one of the most exciting projects that the Hitachi Consumer Business Group has worked on in some time," said Hiroyuki Mizukami, chief technology officer of that group and general manager of the Hitachi, Ltd. Consumer Electronics Laboratory. "Consumers have shown a growing preference for the multi-touch, gestural interface pioneered by the iPhone, but that is only appropriate for small devices in your hand or embedded in a surface. For control of entertainment devices across the room, such as a television or multi-media center, the next logical step is gestures in open air."

Mizukami said that consumers have been increasingly preconditioned to gestural interfaces since they were first widely imagined in Steven Spielberg's 2002 film Minority Report, in which John Anderton operated a complex forensic computer display simply by moving his hands in thin air. Since that time, such interfaces have been increasingly showcased in weekly television dramas, such as the Emmy award-winning CSI: Miami, to the point where many consumers assume they are commonplace. "We believe that it is only a matter of time until gestural interfaces are found in virtually every living room," says Mizukami.

The challenges in creating an open-air gestural interface revolve around reliably separating out an individual from the surrounding environment - continuously, and in real time. Methods that rely upon pure imaging, such as those using one or more video camera chips, all fail in one way or another, depending upon the specific technology.

By contrast, the Canesta sensor is a 3D sensor; it produces a real-time, continuously-varying "depth map" of the local environment, with a distance measurement - within the resolution of the device - to each individual feature in the scene.

In the case of this Hitachi prototype, an independent Canesta sensor sends a stream of 3D information at 30 "frames" per second to the TV's microcontroller. There, gesture-recognition software translates the moving depth maps into gestures, and those gestures into commands for the set.

Canesta's president Jim Spare sees Hitachi's new television as trendsetting. "Once consumers have experienced the fun and ease of use of the touchless, gestural interface, it will become a must-have for the home media center," he says. Spare believes that media-centric PCs, and "intelligent TVs" that transparently support broadcast, cable, and Internet video will be the early adopters of the technology.

Spare also reported that the chip is seeing widespread development activity in other areas, such as automotive safety applications, where a vehicle can benefit from a 3D awareness of its surroundings.

Canesta's single chip 3D sensor is built with standard CMOS chip technology.