Image and Video Processing

Thanks to Can Temel

Image and video coding is a sub class of data compression problem which involves encoding information using fewer bits than the original representation. Since the raw data is mostly coded at the source side, data compression is also commonly referred as source coding. Data compression aims reducing the resource usage such as storage size and transmission capacity. In the case of images and videos, we need to capture the scene, encode the data and store or transmit the data.

Image/video coding is a way to convert analog data into digital video format. Analog data is composed of color (chroma) and luma information. Human visual system is more sensitive to luma compared to chroma so these channels can be represented with different resolutions which is the basis of Chroma Subsampling. Image/video codecs try to compress the raw data as much as they can and maintain the quality of experience at the same time.

There is a tradeoff between the video quality and the bitrate. Content providers should satisfy the subjective quality requirements of the users and they should not exceed the bandwidth limitations at the same time. For a high end user, tradeoff is mostly between the storage size and the quality of the video. Engineers and mathematicians have been working on this problem to balance the main factors in quality-bandwidth tradeoff. The goal is to refine compression capability versus complexity and to enable parallel signal processing operations.

Video compression standards can be classified into two as lossless and lossy compression. Lossless methods can achieve a compression rate over factor 3. However, lossy compression methods such as MPEG-4 result in compression ratios from 20 to 200 depending on the video stream. Even MPEG-4 is the most commonly used format in video coding, constant macro block size limits the efficiency of the state of the art method. High Efficiency Video Coding (HEVC) is the new video compression standard that requires half of the bitrate used by H.264/MPEG4 for high definition videos while maintaining the visual quality.

Even quality-bandwidth tradeoff is the main concern of the image/video coders, developers also need to take into account the complexity of the encoding and decoding algorithms, robustness to data losses and errors, ease of editing, random access and end-to-end delay. Researchers and developers who work on image and video coding focus on determining video coding standards that aims enhanced compression efficiency, network friendly video representation for interactive (video telephony) and non-interactive applications (broadcast, streaming, storage, video on demand).