Scientists Develop Camera System That Captures “5D” Images


An international team of scientists developed an experimental camera system capable of simultaneously capturing five dimensions of information from a single snapshot. The concept works by combining two types of sensors to gather a multitude of information at the same time.

Researchers say that to better observe and understand non-repeatable or irreversible dynamic scenes, it is essential that ultra-fast optical imaging can capture an object’s structure, its time course, and its spectral composition. Those who developed the new system, however, hit a wall, because prior to their development, no camera was capable of simultaneously capturing spatial, temporal and spectral information in five dimensions of dynamic scenes.

“To break the limitation of existing techniques in imaging dimensions, we are developing a spectral-volumetric compressed ultrafast photography (SV-CUP) technique,” the researchers state in the summary of their research paper.

Configuration and principle of the SV-CUP. (a) SV-CUP system configuration: M1 and M2, mirrors; ED, engineering broadcaster; DS, dynamic scene; CL, camera lens; BS1 and BS2, beam splitters (reflection / transmission: 50/50); F1 and F2, filters; G, diffraction grating; L1 and L2, lenses; DMD, digital micromirror device; CMOS, complementary metal oxide semiconductor camera; and SC1 and SC2, scanning cameras. (b) SV-CUP operating principle: C, spatial coding operator; T, temporal shear operator; K, operator of spatio-temporal integration; S, spectral shear operator; and M, operator of spatio-temporal-spectral integration.

It’s not particularly easy to summarize the team’s research in easily understandable language. But basically the researchers paired a super fast time-of-flight camera (called CUP or Compressed UltraFast Photography) with a CUP camera that captures the wavelength (the color emitted by an object) rather than the z dimension. By combining these elements, the team was able to capture three dimensions in space (x, y, z) plus the time dimension

The SV-CUP camera appears to be an evolution of that PetaPixel’s DL Cade covered in May 2020 which is capable of filming 70 trillion frames per second. In this case, the researchers combine two CUP cameras, working in unison, to capture information about space, time and wavelength and merge the two outputs into a final 5D output.

“Our SV-CUP brings unprecedented detection capabilities to dynamic scenes, which has important prospects for application in basic research and applied sciences,” the team writes.

As noted by To spy, systems like this with the ability to record dynamic scenes over a picosecond (a trillionth of a second) – or even a femtosecond (a quadrillionth of a second) – timescales give physicists, chemists and biologists a way to obtain ultrafast multidimensional optics of images allowing ultrafast phenomena to be detected. While this particular system can’t record that quickly – it seems to run at around two picoseconds – the number of dimensions it can capture at one time at that speed is likely to be extremely useful to some scientists.

SV-CUP 5D Imaging: (a) Experimental setup for imaging the photoluminescent dynamics of a 3D mannequin coated with CdSe quantum dots; (b) reconstructed data cube of the 3D mannequin; (c) selected reconstructed images of the 3D mannequin at representative times and wavelengths; (d) photoluminescent dynamics calculated from (c) (blue line) and measured by a scanning camera (red line); (e) fluorescence spectrum calculated from (c) (blue line) and measured by a spectrometer (red line); (f) time resolved spectroscopy taken from (c); and (g) calculated fluorescence lifetimes at certain selected spectral components.

The full article titled “Single-shot spectral-volumetric compressed ultrafast photography” can be read at Spied on digital library.

Image credits: The header image represents a snapshot of five-dimensional imagery with spatio-temporal-spectral resolutions. | Credit: S. Zhang, East China Normal University.


Previous Microscopy technique produces high-resolution images of deeper tissue, faster
Next Will Ferrell is ghost of past Christmas in new Spirited BTS footage