The $93 Image Sensor for Robots
Will SONY’s New AI-Image Sensor Make Robots Smarter?
Intelligent sensing: World’s first image sensor with integrated AI combines processing power and memory for vision tasks without extra hardware
Always a new horizon
With an industry-leading 60 percent share of the image processing market—shipping 1.6 billion image sensors in 2019 for Apple’s iPhone and dedicated photo and video cameras from the likes of Nikon and others—SONY says that it’s now looking to expand into automotive and industrial automation with its new IMX500, the world’s first image sensor with built-in AI.
“I believe the next megatrend 
[after mobile phones] will be mobility,” said SONY Chairman and President Kenichiro Yoshida, looking squarely at the automotive industry where SONY has barely a 9 percent market share.
Any machine that moves autonomously, whether along city streets or around a warehouse or factory, has to have the ability to intelligently understand its surroundings, where it’s going, and what’s in its path. SONY sees its IMX500 fitting neatly into that mobility space.
Danger, a human’s nearby
A cobot wearing one of SONY’s $93 IMX500 vision chips could detect a human and stop in 3.1 milliseconds.
SONY vice president of business and innovation Mark Hanson says the IMX500 is much faster for these sorts of tasks than many other AI cameras, with the ability to apply a standard image recognition algorithm (MobileNet V1) to a single video frame in just 3.1 milliseconds. By comparison, other chips can take hundreds of milliseconds — even seconds — to process.
SONY claims its IMX500 to be the world’s first image sensor with integrated AI intelligence. “The new sensor incorporates both processing power and memory, allowing it to perform machine learning-powered computer vision tasks without extra hardware. The result,” says SONY, “will be faster, cheaper, and more secure AI cameras.”
Like Huawei and Alphabet Inc.’s Google, already building dedicated AI silicon, SONY’s new sensors are like “tiny self-contained computers, incorporating a logic processor and memory, and are capable of image recognition without generating any images,” allowing them, according to a report by Bloomberg, “to do AI tasks like identifying, analyzing or counting objects without off-loading any information to a separate chip… making it possible to do near-instant analysis and object tracking.”
Other applications send images or video to the cloud for analysis, which can be slow and insecure. SONY claims that its image sensor provides a more streamlined solution. “If the AI chip is stuck directly onto the back of the image sensor then object detection can be done on-device.” Instead of sending off data to another processor or to the cloud, the image sensor itself performs whatever “AI analysis is necessary and simply produces the metadata instead.”
“Now we’ve eliminated what would normally be a 60 frames per second, 4K video stream to just that one ‘hey, I recognize this object,’” says Hanson. “That can reduce data traffic [and] it also helps things like privacy.”
“The new AI-augmented sensors,” reports Bloomberg, “are capable of capturing a regular 12-megapixel image, 4K video at up to 60 frames per second or neither, providing only metadata about what the sensor has seen.”
Sony has sent samples of the new sensor to potential customers, who are mostly in the BtoB segment, including factory automation.
SONY’s hunt for new money
A COVID economy with its lockdowns and interruptions to manufacturing have not treated SONY well, especially cellphones, said the company’s CFO, Hiroki Totoki. “Along with a decline in net profit, SONY also forecasts operating profit to decline 27 percent to $5.9 billion for the year ending March 2021, with revenue remaining flat at, a 0.5 percent increase from the previous year. “SONY had previously said it anticipated operating profit to fall around 30 percent this fiscal year.
SONY’s revenue from movies has slumped to historic levels, and the company has pushed into next year nearly every movie title produced.
SONY’s big winner, its games segment has seen an extraordinary increase. “We are seeing a surge in demand as more people stay at home,” said Totoki. “However, the massive sales growth in its gaming business will not be able to offset the decline in Sony’s other segments. In addition, the initial costs of rolling out the PS5 means there will only be a slight increase in profit for the game business.”
New sales for new money would be very much appreciated, and expanding image sensors into the mobility segment seems like a win.
Machines that see
SONY has stiff competition breaking into mobility, especially the automotive market, led by U.S. manufacturers like Arizona-based ON Semiconductor, which has been producing sensors for over 50 years and controls 45 percent of the market.
However, mobile machines—autonomous or otherwise— need to see, and right now three types of sensors dominate that seeing; image sensors, millimeter-wave radar, and lidar are mobility’s eyes, and each one has its strengths and weaknesses.
“Automotive sensors must be able to record scenes with sharp contrast while drastically reducing flicker from light-emitting diode lamps of the kind found in nearly all new cars and modern traffic signals. Fulfilling one or the other of these requirements is easy, but to do both is a challenge.”
ON Semiconductor’s solution involved changing the structure of pixels used in the sensors. Its products have become a mainstay for automakers like Subaru, which adopted them for its EyeSight safety system.
The two other types of sensors are millimeter-wave radar and lidar three-dimensional sensors. Both measure distances between cars and objects, however, one measures using radar waves while the other uses infrared laser light, which has a shorter wavelength, allowing it to detect smaller objects.
Image sensor sales are forecast by Markets and Markets to rise quickly through 2025: “The Image Sensor Market size is projected to reach USD 28.0 billion by 2025 from an estimated USD 18.5 billion in 2020, at a CAGR of 8.6% from 2020 to 2025.”
Market drivers are reported to be high-resolution cameras equipped with image sensors in mobile devices, and growth in adoption of image cameras for automotive applications. In short, as Yoshida aptly foresees, mobility.
And by the looks of the sharp increase in market need since 2017, SONY’s AI-image sensor may indeed have found fertile ground in which to grow quickly (see chart below).
Great expectations
Although the cellphone business has rebounded as of late, SONY said “profitability of the business could deteriorate this year as the coronavirus outbreak has clouded the outlook of the already saturated cellphone market.”
The company therefore aims to boost the percentage of its sensing solutions business, including “the latest AI-embedded chip, to 30 percent of the chip division by the year ending March 2026.”
Quite simply, vision-guided robots are more precise and productive. They bring the promise of automation for more challenging, complex tasks. To achieve it, they rely on one thing more than any other: Machine vision. SONY, with its superior sensor tech, is positioning itself to take an ever-increasing share of influence and sales.