Chair of
Multimedia Communications and Signal Processing
Prof. Dr.-Ing. André Kaup
The content of the English page is outdated, please use the updated German version of our page at the moment: lms.tf.fau.de

Energy Efficient Video Coding

Field of activity: Video Signal Processing and Transmission
Research topic: Video Coding and Transmission
Staff: Dr.-Ing. Christian Herglotz

The focus of this field of activity lies on the energy efficiency of video coding. State-of-the-art codecs like the H.264/AVC or the HEVC suffer from very high complexity. As portable devices such as smartphones or cameras provide limited power resources, efficient algorithms that save energy are highly useful to the user. In order to achieve this goal, the decoder as well as the encoder is investigated.

Decoder: In order to learn about the energetic behaviour of the decoder, we build a model capable of estimating the energy consumption for the decoding process of a given video bit stream on a predefined hardware. This model is intended to provide the following benefits:

A model being capable of estimating the decoding energy for P- and B-frames (without considering in-loop filters) on a special platform has been presented in 2014-29 (see below). Most recent work has shown that the model is also applicable for an arbitrary HEVC decoding system. A paper presenting these results has been submitted to IEEE Circuits & Systems for Video Technology (CSVT) and is currently in the reviewing process. To show the capabilities of this model, we provide an online demonstrator:

Decoding Energy Estimation Tool 

   

Moreover, the source code of this tool can be downloaded for personal use following this link:

Download HEVC Decoding Energy Estimator

The code can be compiled and executed on Linux and Windows systems. The sources are freely available under the GNU-GPL license for further use. The tool is about one third less complex than the standard decoder.

 

The model comprises a predefined set bit stream features, where each feature has two main properties:

To give an example, such a feature can be the transformation of the residual coefficients, which is performed if a coded block flag (CBF) is set. Another example can be the decoding of a nonzero coefficient which is indicated by the significant coefficient flag. This information can not only be used to perform an early stage decoding energy estimation, but also to exploit it in the RDO-process to encode energy-saving video bit streams.

Another approach investigates if the energy can be estimated using CPU instruction counts of a process (see paper 2014-16). We could show that valid estimations can be made for simplified processors and we plan to extend the model to incorporate important features like FPUs and MMUs.

A software that is capable of constructing decoding energy saving bit streams can be downloaded here.

Encoder: State-of-the-art encoders often use brute-force search algorithms to find the best possible solution for rate-distortion optimized encoding of a given video signal. This search can be circumvented if information about the scene exists beforehand. As an example, computer generated scenes in video games contain inherent information about the motion of objects. It is possible to exploit this knowledge to directly encode the frame using the given motion vector without having to perform an exhaustive motion vector search. First measurements have shown that up to 50% of the encoding complexity can be saved accepting small deteriorations in rate-distortion performance.

This field of activity is part of the research training group “Heterogene Bildsysteme” and is funded by the Deutsche Forschungsgemeinschaft (DFG).

Publications

2016-49
CRIS
C. Herglotz, Y. Wen, B. Dai, M. Kränzler, A. Kaup
   [bib]

A Bit Stream Feature Based Model for Video Decoding Energy Estimation
Picture Coding Symposium (PCS), Nuremberg, Germany, Dec. 2016
2016-48
CRIS
C. Herglotz, R. Rosales, M. Glass, J. Teich, A. Kaup
   [bib]

Multi-Objective Design Space Exploration for the Optimization of the HEVC Mode Decision Process
Picture Coding Symposium (PCS), Nuremberg, Germany, Dec. 2016
2016-40 C. Herglotz, D. Springer, M. Reichenbach, B. Stabernack, A. Kaup
   [doi]   [bib]

Modeling the Energy Consumption of the HEVC Decoding Process
IEEE Transactions on Circuits and Systems for Video Technology (IEEE TCSVT) Pages: to appear, 2016
2016-21
CRIS
C. Herglotz, A. Kaup
   [bib]

Joint Optimization of Rate, Distortion, and Decoding Energy for HEVC Intraframe Coding
IEEE Int. Conf. on Image Processing (ICIP), Pages: 544 - 548, Phoenix, Arizona, USA, Sep. 2016
2016-6
CRIS
R. Rosales, C. Herglotz, M. Glaß, J. Teich, A. Kaup
   [bib]

Analysis and Exploitation of CTU-Level Parallelism in the HEVC Mode Decision Process Using Actor-based Modeling
Architecture of Computing Systems (ARCS), Pages: 263-276, Nürnberg, Germany, Apr. 2016
2015-26
CRIS
C. Herglotz, A. Kaup
   [bib]

Estimating The HEVC Decoding Energy Using High-Level Video Features
European Signal Processing Conf. (EUSIPCO), Pages: 1591-1595, Nice, France, Aug. 2015
2015-7
CRIS
C. Herglotz, A. Hendricks, M. Reichenbach, J. Seiler, D. Fey, A. Kaup
   [bib]

Estimation of Non-Functional Properties for Embedded Hardware with Application to Image Processing
22nd Reconfigurable Architectures Workshop (RAW) on the 29th Annual International Parallel & Distributed Processing Symposium (IPDPS), Pages: 190-195, Hyderabad, India, May 2015
2015-3
CRIS
C. Herglotz, E. Walencik, A. Kaup
   [bib]

Estimating the HEVC Decoding Energy Using the Decoder Processing Time
IEEE Int. Symp. on Circuits and Systems (ISCAS), Pages: 513-516, Lisbon, Portugal, May 2015
2014-29
CRIS
C. Herglotz, D. Springer, A. Kaup
   [bib]

Modeling the Energy Consumption of HEVC P- and B-Frame Decoding
IEEE Int. Conf. on Image Processing (ICIP), Pages: 3661-3665, Paris, France, Oct. 2014
2014-16
CRIS
S. Berschneider, C. Herglotz, M. Reichenbach, D. Fey, A. Kaup
   [bib]

Estimating Video Decoding Energies And Processing Times Utilizing Virtual Hardware
Proc. 3PMCES Workshop. Design, Automation & Test in Europe (DATE), Pages: 1-2, Dresden, Germany, Mar. 2014
2013-40
CRIS
C. Herglotz, D. Springer, A. Eichenseer, A. Kaup
   [bib]

Modeling the Energy Consumption of HEVC Intra Decoding
IEEE International Conference on Systems, Signals and Image Processing (IWSSIP 2013), Pages: 91-94, Bucharest, Romania, Jul. 2013