Capturing 3D Images with Time-of-Flight Camera Technology

Cameras are on the verge of becoming completely ubiquitous devices in modern society, incorporated into everything from smartphones and tablets through to vending machines, home automation systems, and even cars. Yet most of these cameras are still missing a key element: the ability to render images using the third dimension. Enter time-of-flight.

Time-of-flight (ToF) is a relatively new method by which it is possible to gather detailed 3D information. By illuminating a scene, usually using infrared or near-infrared light, a ToF camera can measure the distance between itself and objects within that scene. Compared to other techniques for acquiring 3D information - such as use of scanning or stereoscopic vision - ToF cameras are capable of exhibiting greater accuracy, while being extremely fast and affordable. This opens up the advantages of 3D imaging to a much greater variety of applications than was feasible previously, covering gaming, medicine, manufacturing, etc.

How Does ToF Work?

ToF camera systems consist of an image sensor, image processing chip and modulated light source. In simple terms, these systems work by illuminating a scene with a modulated light source, and then measuring the phase shift of the wave that is reflected back. Since light has a constant speed, ToF cameras are able to calculate the distance to each point in the scene based on the time it took for that light to return to the camera. Rather than scanning an image line by line, a ToF camera system will illuminate the entire scene all at once and then measures phase shift in the light reflected back to the image sensor. This raw data can be captured quickly and the calculation required to derive the distance is relatively straightforward, with ToF cameras thus achieving extremely high frame rates (even beyond what human vision can discern). This means that unlike many other 3D vision arrangements, ToF allows 3D depth information to be extracted from a scene in real-time using embedded processors.

Figure 1. A ToF imaging system using Melexis hardware.

While it is possible to source the components needed for a ToF camera system individually, several manufacturers supply compact, ready-made solutions that are generally more convenient. Targeted at the automotive sector, Melexis’ solution comprises the MLX75x23 image sensor paired with the MLX75123 companion chip. The MLX75x23 is a sunlight-robust image sensor with QVGA resolution, while the MLX75123 controls the sensor, modulates the light source and communicates with the host processor. Evaluation boards that bring these items of hardware together, and incorporate a light source, are also available. Besides Melexis, STMicroelectronics’ VL6180 provides a compact, integrated ToF solution. This is aimed at smartphone designs and enables gesture recognition functionality to be benefitted from. The OPT8241-CDK-EVM evaluation hardware from Texas Instruments is based on the company’s OPT8241 320×240 resolution ToF imaging device, which supports up to 150fps operation.

Figure 2. A functional block diagram of a VL6180 ToF system from STMicroelectronics.

Smarter Machine Vision

ToF camera technology is allowing machines to see beyond simple 2D images and explore the third dimension, thereby enabling depth perception and better object recognition. Compared to other 3D machine vision techniques, ToF is much faster, and its ability to generate real-time depth information means that a wide variety of applications can be served. These include:

Augmented Reality - ToF-enabled 3D vision is allowing exciting new augmented reality (AR) applications to be explored, and existing ones to work better. The point clouds generated by a ToF camera enable AR software to map out its surroundings for an enhanced 3D understanding of the environment around it. This lets it place in-software objects more accurately and facilitates more dynamic interaction between virtual and actual elements of the environment. ToF can also detect the user's movements and posture, so that they are able to interact with virtual elements using their body directly, without having to rely on handheld controllers or gloves.

Industrial Robots - For the industrial segment, the capacity to recognize objects and produce real-time 3D depth maps will prove invaluable to robotics. Manufacturing robots involved in automated quality inspection will be able to quickly and accurately produce a 3D scan of an object. ToF may also be used in collaborative robot designs, to prevent collisions with humans nearby or to provide interactive gesture control. For logistics, it will allow robots to grab and place objects more accurately.

Medical, Scientific, Engineering - In the medical field there can often be a need to interface with electronics, but the risk of cross-contamination means that touch-based interaction is undesirable. Gesture-based control using ToF cameras will allow doctors and nurses to manipulate images or utilise software without having physical contact with the device. For scientific investigation, ToF cameras would enable gesture-based manipulation of 3D images - such as DNA strands or protein molecules. In engineering, being able to quickly and affordably 3D scan items will be helpful for hardware prototyping and design activities.

Drones and Vehicles - ToF cameras could also bring greater intelligence to drones and unmanned ground vehicles. Drones using ToF would have a better awareness of their 3D environment and be able to create 3D maps or perform automated obstacle avoidance. Similarly, unmanned ground vehicles could use ToF cameras to provide obstacle sensing capabilities, allowing for autonomous navigation.

Real-Time Environmental Understanding

While there are other methods for achieving 3D machine vision, ToF features the most compelling combination of affordability, compactness, speed and accuracy. It furnishes real-time data on the surrounding location that can be compiled using embedded processors. This will bring 3D machine vision into a more expansive variety of industries and enable many new applications to emerge.