New chip can process nearly two billion frames per second


Engineers at Penn State University have created a chip capable of processing and classifying nearly two billion images per second.

Penn Engineering today reports that a team of researchers – including Farshid Ashtiani, Alexander J. Geers and Firooz Aflatouni – have developed the chip which is smaller than one square centimeter.

The chip can perform an entire image classification in about half a nanosecond and all without the need for a separate processor or memory unit.

In traditional artificial intelligence (AI) systems used for image recognition, the image of the target object is first formed on an image sensor, such as a smartphone’s digital camera .

Next, the image sensor converts the light into electrical signals, and ultimately into binary data. Only then can the system “understand” the image enough to process, analyze, store and classify it using computer chips.

While digital chips today can perform billions of calculations per second, more sophisticated image classification, such as moving object identification or 3D object identification, pushes the boundaries of the technology. the most powerful.

The current speed limit of these technologies is set by the clock-based timing of computational steps in a computer processor, where computations occur one after another in a linear schedule.

Penn State engineers have created the first scalable chip that classifies and recognizes images almost instantly – by designing a workaround that removes the most time-consuming aspects of traditional on-chip AI image processing.

Their custom 9.3 square millimeter processor directly processes light received from an “object of interest” using what they call an “optical deep neural network”.

The researchers’ processor effectively uses “optical neurons” interconnected using optical wires, called waveguides, to form a multi-layered deep network.

Information passes through the layers, with each step helping to classify the input image into one of its learned categories.

Using the chip’s ability to calculate when light travels through it to directly read and process optical signals, the researchers say the chip doesn’t need to store information and can perform image classification. complete in about half a nanosecond.

“We are not the first to offer technology that directly reads optical signals, but we are the first to create the complete system in a chip that is both compatible with existing technology and scalable to work with more complex data. “, says Geers.

The team expects the work to have apps for automatically detecting text in photos, helping self-driving cars recognize obstacles, and other computer vision tasks.

AI has changed the world of camera technology in recent months. Earlier this year, scientists developed an AI camera capable of color filming in total darkness.

This week, Camero-Tech announced its latest AI-powered detection system, the Xaver 1000, which allows soldiers to see through walls before attacking.


Picture credits: Header photo licensed via Depositphotos.

Previous Xiaomi 12 Lite reveals itself in advance: images, specifications and video
Next How to watch NASA release the first images from the James Webb Space Telescope