ALLPCB
From smartphones to surveillance cameras, artificial intelligence is increasingly effective at converting blurry, noisy photos into higher-resolution images.
The improvements are driven by two related approaches, commonly called AI super-resolution and AI upscaling. In both, AI models are trained on large datasets to improve the quality of blurry images or increase their resolution.
Yale Fox, an IEEE member, said: "Thanks to advances in several key technologies, AI upscaling has improved significantly in recent years. One advance is transfer learning, which allows developers to reuse pretrained models for new tasks, greatly improving the speed and accuracy of upscaling. Another key development is the availability of large image datasets, which are essential for training deep learning algorithms to recognize and generate high-quality images."
Fox also noted that progress in graphics processing units (GPUs) has helped this field. "The combination of GPUs and deep learning is critical for making AI-based upscaling faster and more efficient. GPUs can process large amounts of data in parallel, enabling faster training and inference."
Many smartphones use AI upscaling to improve photo quality and resolution, but the technique is also applied in medical imaging, satellite imaging, and the entertainment industry.
Satellite imaging
The number of satellites launched into orbit has increased in recent years. Launching smaller, lighter satellites is easier and cheaper, but smaller satellites often cannot carry expensive high-resolution cameras. As noted in an IEEE Xplore article, AI super-resolution enables smaller satellites to capture better images with smaller cameras.
Weather forecasting
Some meteorological satellites monitor large portions of Earth, producing images that may show detail only on the scale of a few square miles. An IEEE Access article describes a system that teaches low-resolution weather satellites to produce better images by using historical weather and climate data to enhance image quality for these wide-area satellites.
Video conferencing
Real-time video transmitted from laptops or phones consumes significant internet bandwidth. For decades, these services have relied on audio and video codecs to compress and decompress streams. Many of those codecs were developed when internet speeds were much slower.
Fox said: "State-of-the-art algorithms can now take a single high-resolution photo of a person and then track facial muscle movements to create a deepfake-style image. This can reduce bandwidth and file sizes by nearly 800 times, which is crucial for applications such as video calls. This represents a large efficiency gain."
Video games and streaming
Image upscaling has also had a major impact on video games. An IEEE Spectrum report indicates that for some graphically intensive games, seven out of eight pixels are generated by AI, resulting in speedups of about 530 percent. There is also active research on applying super-resolution techniques to on-demand video streaming.
Medical imaging
Super-resolution techniques have wide applications in medicine. Many medical images, such as X-rays and MRI scans, have limited resolution because of scan duration and equipment constraints. For example, improving the resolution of tiny vessels around a tumor can aid in cancer detection and treatment (https://ieeexplore.ieee.org/document/8854062). Super-resolution can also improve fetal imaging, since fetal movement often makes imaging difficult.
So what makes these techniques better?
Fabrizzio Soares, an IEEE senior member, highlighted better training data. "These algorithms rely on prior knowledge embedded in images, so the more samples provided, the more accurate the model becomes. However, assembling the most complete image database is challenging. Moreover, more samples typically mean larger models, so training and maintaining these models requires greater effort."
