AI and Video Compression in the Era of Internet of Video Things

Presentation Menu

Abstract

We are at the very beginning of the era of Internet of Video Things (IoVT), where many cameras collect a huge amount of visual data to be analyzed. IoVT will become more critical as the number of cameras and applications grows exponentially in the coming years. Humans cannot process all the videos, and it is critical to use artificial intelligence (AI) to process the data. Many challenges arise fulfilling the era of IoVT, e.g., accuracy, energy efficiency, and processing speed. To design efficient IoVT systems by co-optimizing video compression and computer vision algorithms, this talk will discuss the following six topics:

  1. AI-aware compression
  2. AI-assisted compression
  3. AI-based compression
  4. Compression-aware AI
  5. Compression-assisted AI
  6. Compression-based AI

In AI-aware compression, the goal of video compression is for AI to consume the video (instead for humans). This is because many video, e.g., security and surveillance video, are now analyzed by machines. Therefore, optimizing for better video analytics results is more important than optimizing for human perceptual comfort. This is also known as video compression for machines.

In AI-assisted compression, when the image/video are compressed by the commonly used standards, the decisions in compression tools are assisted by the AI algorithms. For example, video-on-demand vs. live broadcasting has different requirements. AI algorithms can better optimize decisions than human-engineered rules.

In AI-based compression, instead of commonly used standards, we exploit the opportunity to compress the image/video using AI (often deep learning) algorithms. This is also known as learned image and video codecs.

In compression-aware AI, the computer vision (CV) algorithms should know that lossy compression may create artifacts. Differentiating signals from compression noises can improve the accuracy of the video analysis system.

In compression-assisted AI, we exploit some information from the compressed bit-streams (e.g., motion vectors) to help the CV algorithm.

In compression-based AI, we directly apply CV on compressed domain data (e.g., DCT coefficients). This can reduce the required decompression time, which is needed in IoT edge computing. However, this challenges us to develop new CV algorithms.