Perception-motivated High Dynamic Range Video Encoding Rafal Mantiuk, Grzegorz Krawczyk, Karol Myszkowski, Hans-Peter Seidel MPI Informatik, Saarbrücken, Germany

Abstract Due to rapid technological progress in high dynamic range (HDR) video capture and display, the efficient storage and transmission of such data is crucial for the completeness of any HDR imaging pipeline. We propose a new approach for inter-frame encoding of HDR video, which is embedded in the well-established MPEG-4 video compression standard. The key component of our technique is luminance quantization that is optimized for the contrast threshold perception in the human visual system. The quantization scheme requires only 10--11 bits to encode 12 orders of magnitude of visible luminance range and does not lead to perceivable contouring artifacts. Besides video encoding, the proposed quantization provides perceptually-optimized luminance sampling for fast implementation of any global tone mapping operator using a lookup table. To improve the quality of synthetic video sequences, we introduce a coding scheme for discrete cosine transform (DCT) blocks with high contrast. We demonstrate the capabilities of HDR video in a player, which enables decoding, tone mapping, and applying post-processing effects in real-time. The tone mapping algorithm as well as its parameters can be changed interactively while the video is playing. We can simulate post-processing effects such as glare, night vision, and motion blur, which appear very realistic due to the usage of HDR data.

High Dynamic Range Video Encoding

The traditional video compression (MPEG-1,2,4) can encode just enough information to display video on LDR or CRT monitors. High Dynamic Range video, on the other hand, can store complete color information that is visible for the human eye - the only limitation is the human perception. Such extended information is necessary to play video on the next generation of displays or to better render (tone-map) video on existing monitors.

High Dynamic Range Video Player

Our video player for HDR Video can extract more information from the video stream that can be displayed on typical CRT or LCD monitors. The screen-shots below show a dynamic range exploration window (blue frame), which can reveal additional luminance information encoded in the HDR stream. An HDR display can take full advantage of the HDR stream and show video as it appears in reality. Click on an image to see it in its full resolution.

Blooming effect	HDR Rendering	LDR / HDR Motion blur
HDR Panorama 1	HDR Panorama 2	HDR Panorama 3
Gray-scale HDR camera 1	Gray-scale HDR camera 2	Gray-scale HDR camera 3
Night vision sequence 1	Night vision sequence 2	Night vision sequence 3
Night vision sequence 4	Night vision sequence 5	Night vision sequence 6

Click here to download a high-quality QuickTime movie, which demonstrates our HDR video encoding. Note that the movie is over 60MB.

Resources

• High Dynamic Range Resources
• GNU Octave code for finding perceptual quantized luminance space (download)
• Power point slides from the SIGGRAPH 2004 presentation
• IMS-CHIPS - developer of the HDR video camera we used to record test sequences
• High dynamic range display (project page)
• High dynamic range encoding of static images (OpenEXR, LogLuv TIFF, Radiance RGBE)
• pfstools for reading, writing and modifying HDR images and video sequences.

Literature

• Rafal Mantiuk, Grzegorz Krawczyk, Karol Myszkowski, Hans-Peter Seidel. Perception-motivated High Dynamic Range Video Encoding. Proc. of SIGGRAPH '04 (Special issue of ACM Transactions on Graphics), 2004.
  pdf bib