ToF (Time-of-Flight) — A Depth Resolution Camera Technique
The latest flagship smartphone cameras incorporate a feature called ToF (Time-of-Flight). It has been marketed by Huawei as a “Super Sensing Camera” because of enhanced depth resolution that has been described as 4 times better than other cameras. Depth resolution is another way of differentiating the subject or object from the nearest point (foreground) to the farthest point (background) i.e. “Distance-to-Subject”. This gives an enhanced DoF (Depth of Field), which isolates the subject in the image with sharper focus and clarity. It is another form of HD camera that uses increased depth resolution techniques to create stunning images.
How ToF Works
ToF uses a type of sensor that measures an emitted infrared (IR) light signal’s return for depth measurement. The infrared light reflects off the subject or object which the sensor measures. It will take longer for the infrared signal sent to the farthest point (background) to return, while the nearest points and the subject come into focus. The camera software then computes the returned values to create an image using a detailed 3D depth map. In a way the ToF is like a 3D sensor that can help not only with 3D imaging but with AR (Augmented Reality) applications.
The sensor is a type of CMOS array that uses an active modulated light source. The IR signal is emitted and reflects off the subject to determine distance to create depth. What is called the phase shift, is the measure of the distance between the illumination and the reflection of the IR signal. The signal is emitted by either an LED or solid-state laser source at a frequency of ~850 nm (infrared range). This is similar to the frequency emitted by a TV remote control.
One of the methods used to measure distance is called the pulse method. In this method, the light source illuminates a brief period of time (∆t), with the reflected light sampled at every pixel (in parallel) using an out-of-phase window C, with the same ∆t. This accumulates electrical charges, Q1 and Q2, used in measuring the distance (d) formula:
At a more basic level, distance is the product of rate (r) and time (t).
d = r x t
What the sensor is determining is how far away something is by emitting a signal at a certain rate and time. In range finding terminology it is measuring the distance from the observer to the object. The information is then used by the image processor to create a 3D depth map.
Advantages
The advantage of depth sensing is not only for creating stunning imagery. It is also able to help the camera in low light situations since the sensor uses infrared signals, not conventional light pulses. This would allow the camera better focus even at night, when there is less light. Low light photography has become a competitive feature among smartphone makers as to whose camera has the best capability for taking photos at night. As there is less light available, focusing becomes much more difficult. It has been about the ability of the sensor to focus to capture the image. Huawei claims their ToF camera in the P30 Pro can still capture a vast array of colors even in extremely dark lighting conditions.
According to Huawei their ToF camera sensor:
“…. measures the depth of objects with accuracy and AIS Long Exposure Shot, your portrait will be highlighted and the sharpness will be exuded to perfection. The background is then progressively blurred with different creamy effects to give a professional-level bokeh.”
AIS refers to Artificial Image Stabilization, a type of image stabilization technique that uses on-chip AI processing algorithms to counter blur and/or shake in order to produce sharper high quality images.
ToF cameras will also have a considerable advantage over conventional camera systems because of its more accurate measurement of distance. This allows the camera to capture more realistic looking images with the information it has about the depth details. This means the separation of the subject or object is more clearly defined with sharp details. For portraits that means crisp clear images with a pronounced blurred background (e.g. bokeh effect). Features like this did not used to be available in smartphone cameras, only in DSLR and mirrorless cameras with high-end lenses. Smartphone cameras for the most part use computational imaging algorithms along with what was captured from the ToF sensor to produce high quality images.
Bringing 3D Imaging To Retail
Although these features are new in retail cameras, they have actually been around for quite some time now. They are used mostly in range imaging cameras used for more specific type of photography. It is only now that it is finding its way to consumer products thanks to some circuit integration processes that made the components cheaper to manufacture. Besides smartphone cameras, they are also being used by drones. The technology is also being developed for self-driving cars, AR and face recognition systems. They are currently being used for machine automation in factories. 3D photos were once only possible after the image is taken from the camera and has been edited with software. Now it is all possible in real time with just a single click.
Final Thoughts
At the moment several smartphone vendors have incorporated ToF features. Besides Huawei, there is also Samsung, Honor and Oppo to mention a few. Apple’s next version of the iPhone is reportedly going to use ToF, with Sony ramping up production due to the news. As we see more smartphones using ToF, 3D will be as widespread and available as selfie filters. This is one way to invigorate the smartphone industry and computational photography market as a driver for consumer demand.
