Codecs

Video Compression Methods (Codecs)

The recent surge of multimedia applications for desktop computers has led to the development of a variety of compression/decompression (codec) methods, many of them proprietary, for use by desktop digital video implementations and it is quite challenging to understand the tradeoffs involved in each method. Because video processing can load even current high-end processors, a variety of hardware accessory boards are available.

Both Quicktime for Windows and Video for Windows are designed with open codec architectures. The form of the algorithm used to compress a video sequence is specified in the digital video data file format. On identifying the codec, the video software loads the appropriate driver to decompress the data. This can include codecs which use hardware boards to speed processing as well as algorithms implemented solely in software.

For CD-ROM applications, codecs that can decompress video without any special hardware are attractive because the potential audience is large. That is why most video on CD-ROM is compressed using either SuperMac's Cinepak or Intel's Indeo. These codecs can achieve a full 30 frame per second rate for 160 X 120-pixel 16-bit color pictures and 15 frames per second at 320 X 240.

But choice of a codec is more than a battle between Cinepak and Indeo. The industry standard for compressing movies, MPEG is now moving into the marketplace. MPEG I, the codec designed for CD-ROMs can achieve 30 fps on 320 X 240 windows when played back on boards that cost less than $500.

The key question is whether developers should move to MPEG I or stick with software. Most people agree that MPEG I playback looks better and it has the added advantage of compressing audio. However, faster CPUs and buses on multimedia PCs and low-cost general-purpose video acceleration cards may save software codecs from being replaced by MPEG. On fast Pentium and Power PCs, Cinepak and Indeo can achieve 30 fps at 320 X 240. They can be blown up to 640 X 480 using new graphics accelerators. The quality is not quite as good as MPEG but close and they are a lot cheaper for encoding and playback.

To make matters more complicated, there are a number of mature, established codecs such as Motion-JPEG and DVI as well as some new up-and-coming technologies such as fractals and wavelets to choose from.

There are primarily three criteria involved in selecting a codec:

Compression Level,
Quality of the Compressed Video
Compression/Decompression Speed

Generally each of these criteria must be traded off against one another. For example, a codec may produce high levels of compression but the quality of the resulting video is poor. Another codec may produce high quality video at high levels of compression, but it requires a lot of computing power which means that either it can not be implemented in real-time or an expensive video capture board is required. The following procedure should be used when selecting a video codec:

Define the Distribution Medium and Playback Platform
- Know the bit stream a storage device can sustain and the total storage available. To be played from a CD-ROM, a video must have a bit stream of 90K or less for single speed drives and 170K or less for double speed drives. CD-ROMs offer about 650 MB of storage.
- Know the codec's availability on the multiple platforms you plan to distribute your software
- Know the codec's ability to adapt the synchronized playback speed to the available hardware without user interference.
Know the Content of the Video. Some codec are better suited for outdoor colors, some codecs offer sharper edges, there are 8, 24 and 32 bit codecs.
Weigh the developer issues
- Determine if it is okay to use an asymmetrical codec, that is one which takes a lot longer to compress video than to decompress it.
- Know the source of the video and whether it has previously been compressed.

The following chart gives a summary of your video codec choices:

Codecs
Company
Method
Platform S/W Playback Playback Size/Rate
Captain Crunch Media Vision Wavelet PC No 320 X 240 @ 30fps
Cinepak SuperMac VQ Mac, PC Yes 320 X 240 @ 15fps
DVI-RTL Intel VQ PC No 256 X 240 @ 15fps
DVI-RTL Intel VQ PC No 640 X 480 @ 30fps
Indeo Intel VQ Mac, PC Yes 320 X 240 @ 15fps
Motion -JPEG n/a DCT Mac, PC No 640 X 480 @ 60fps
MotiVE Media Vision VQ PC Yes 160 X 120 @ 12fps
MPEG I n/a DCT PC No 320 X 240 @ 30fps
MPEG II n/a DCT PC No 704 X 480 @ 60fps
Px64 n/a DCT Mac, PC No 352 X 288 @ 15fps
Pro-Frac TMM RLE PC Yes 320 X 200 @ 30fps
SoftVideo TMM RLE PC Yes 640 X 480 @ 15fps
Ultimotion IBM n/a OS/2 Yes 320 X 240 @ 30fps
Video Apple VQ Mac Yes 160 X 120 @ 15fps
Video Cube IMIX/Aware Wavelet Proprietary No 640 X 480 @ 60fps

More detailed descriptions of codecs that are in use in Quicktime and Video for Windows follows:

Motion JPEG

JPEG is a well-established still-image compression codec that removes the redundancies in individual frames. Motion-JPEG is an adaptation of the still-image standard to video. Motion-JPEG uses the same algorithms as JPEG to create I-frames (compressed intraframes), and then successive frames are compressed by holding the compression parameters constant, to keep up with the video data stream in real-time. There are various, non-compatible versions of Motion JPEG from different manufacturers. Using fast codec accelaration hardware available from several manufacturers, I-frames are coded and decoded symmetrically in less than one-thirtieth of a second.

MPEG

MPEG was designed for digital video. MPEG uses the same algorithms as JPEG to create one I-frame, then removes the redundancy from successive frames by predicting them from the I-frame and encoding only the difference from its predictions. This is called interframe compression. The MPEG committee created two standards, MPEG I that can play back from a single speed CD-ROM (150 KB per second) at 320 X 240 at 30 fps and MPEG II with enough data (1.2 MB/second) to encode studio-quality video at 704 X 480 at 30 frames per second.

Cinepak - SuperMac Technology

Cinepak is a vector quantization based codec developed specifically to deliver 24-bit video in quarter screen (320 X 240 pixel) windows from files restricted to single-spin CD-ROM data rates. Vector quantization stores information about differences between frames of video by quantifying the magnitude and direction of a pixel's movement. Cinepak's decompressor then uses a CLUT (color lookup table) to recreate the color of each pixel in a frame.

Cinepak is a highly asymmetric codec, with a compression process that takes 300 times longer than decompression. However, Cinepak has a highly efficient decompression and playback scheme and is integrated into Apple's Quicktime 1.5. SuperMac has expanded Cinepak to other major platforms by licensing the codec for use in such products as Microsoft's Video for Windows 1.1, Creative Lab's VideoSpigot for the PC and Sega's Genesis CD-based systems.

The general opinion is that Cinepak is better than Indeo for action sequences and is typically faster at 16 and 24-bit playback. Cinepak's quality compared to Indeo slips for "talking head" and other low-motion sequences.

Indeo - Intel

Intel's Indeo, a subset of that company's DVI software. Indeo (currently upgrading from Indeo R2 to R3) defines the format for video capture used by Intel's Intelligent Smart Video Recorder (ISVR) hardware, which permits it to compress in real time. It is currently one of the more highly-regarded video codecs for desktop computer applications. Indeo produces the highest-quality "talking-head" sequences at very low data rates but it is worse than Cinepak for action sequences. It is slightly faster than Cinepak at compression but is still slow.

Indeo is a proprietary blend of color sampling, vector quantization, and run-length encoding. Indeo, which is the past, depended on the use of an i750 chip set for decompression, now plays back on 486 platforms without any additional hardware. One of Indeo's valuable characteristics is its scalability. Indeo will deliver a movie clip a higher frame rates when more processing power is available.

Indeo has a strong following on multiple platforms through licensing with Apple, IBM and Microsoft. In addition, more than 20 independent software vendors support Indeo video in applications such as editors, presentation programs, morphing software, screen savers and utilities.

Video1 and MotiVE - Media Vision

MotiVE is a discrete cosine based codec which was licensed by Microsoft as the cornerstone of Microsoft Video 1 which was the primary codec in Video for Windows 1.0.

TrueMotion - Duck Corporation

TrueMotion is a proprietary vector quantization scheme. It produces "studio-quality" video at a 600 KB/second playback rate suitable for quad-speed CD-ROM players. Its compression is highly asymmetric but decompression runs quickly on desktop computers..

RLE - Microsoft

This codec was released with Video for Windows 1.0. It is a simple codec which is optimized for animations or cartoons, but gives relatively poor performance for real-life images.

VideoCube - ImMIX/Aware

In this compression method, a wavelet codec transforms the image into a set of different spatial representations, some of which contain high frequencies and some of which contain low frequency information. ImMix licensed Aware's discrete wavelet codec technology for use in the VideoCube codec.

Captain Crunch - Media Vision

Captain Crunch is a patented blend of a discrete wavelet transform, a tree-based encoder, a quantizer and a Huffman encoder. Captain Crunch is a rival to Cinepak for delivering high visual quality at low data rates. When the decoder is installed in the operating system of a 386 PC or MAC IIci, 24-bit Captain Crunch will play quarter screen video at 30 fps from a CD-ROM. In contrast to Cinepak, Captain Crunch's encoder is nearly symmetric. Captain Crunch is also capable of scaling not only a video's playback frame rate, but also adjusting the size and bit depth as well.

SoftVideo - Iterated Systems and TMM

Softvideo is a video codec based on Iterated System's fractal compression algorithm. This is an extremely asymmetric codec since fractal compression is a very time consuming process, as much as 15 hours are required to compress one minute of video. However, decompression and playback are fast. One key feature of fractal compression is scalability, that is the video's resolution is independent of the size of the window. Because fractal images are encoded with equations, they have no inherent size, they look as good on a 13 inch monitor as a 21 inch monitor. Preliminary results show that this fractal-based method produces very good results.

DVI - Intel

Intel's Digital Video Interactive (DVI) standard uses interframe video compression techniques similar to that of MPEG and ADPCM audio compression compatible with CDROM XA standards. DVI provides different levels of quality for video and audio data. For high resolution, full-color digital video with stereo quality approaching that of audio CDs, compression becomes so time-consuming, that special supercomputers must handle the conversion. A compression service bureau can be used to perform DVI compression.

UltiMotion - IBM

UltiMotion is a VQ codec which is available on OS/2 systems running IBM's presentation manager.

Px64

P times 64 is a codec used in videoconferencing which is similar to JPEG and MPEG. This codec is comprised of the H.261 compression and signal, and the H.320, audiovisual support, protocols. They use interframe compression techniques which are a subset of MPEG.