Ultrafast Camera for Self-Driving Automobiles

Unlike typical optical cameras, which can be linked by bright light and unable to make out details in the dark, NTU’s novel smart camera can record the smallest movements and objects in real time.

The novel camera records the alterations in light intensity between scenes at nanosecond intervals, much faster than traditional video, and it accumulates the images in a data format that is many times smaller as well. With a unique in-built circuit, the camera can do an instant analysis of the captured scenes, highlighting vital objects and details.

Introduced by Assistance Lecturer Chen Shoushun from the NTU’s School of Electrical and Electronic Engineering, the novel camera named Celex is now in its eventual prototype phase. “Our novel camera can be a great security tool for autonomous automobiles, sine it can witness very far ahead like optical cameras but without the time lag required to analyse and process the video feed,” explains assistant Lecturer Chen

“With its regular tracking feature and instant analysis of a scene, it complements present optical and laser cameras and can aid self-driving automobiles and drones avoid unexpected collisions that usually happen within seconds.”

Assistant Lecturer Chen disclosed the prototype of Celex last month at the 2017 IS and T International Symposium on Electronic Imaging held in the United States. It received positive feedback from the conference attendees, most of which are academic and top industry players.

A simple camera sensor has numerous millions pixels, which are sensor sites that record light information and are employed to form a resulting picture. High-speed video cameras that record up to 120 frames or photos per second generate gigabytes of video data, which are then processed by a computer in order to self-drive automobiles to see and analyse their environment.

The more intricate the environment, the slower the processing of video information, resulting in lag times between seeing the environment and the corresponding actions that the self-driving automobiles has to take.

To allow an instant processing of visual data, NTU’s patient pending camera records the changes between light intensity of individual pixels at its sensor, which reduces the information output. This avoids the needs to capture the entire scene like a photograph, hence increasing the camera’s processing speed.

The camera sensor also possesses an in-built processor that can analyse the movement of information instantly to differentiate between the foreground objects and the background, also known as optical flow computation. Such innovation enables self-driving automobiles more time to react to any oncoming obstacles or vehicles. The research into the sensor technology instigated in 2009 and it has obtained $500,000 in funding from the Ministry of Education Tier 1 research grant and the Singapore-MIT Alliance for Technology and Research Proof-of-Concept grant. With deep interest from the industry, Assistant Lecturer Chen and his scientists have spun off a start-up company names Hillhouse Tech to commercialize the novel camera technology