Chair of
Multimedia Communications and Signal Processing
Prof. Dr.-Ing. André Kaup

Reconstruction of Non-Regularly Sampled Data

Field of activity: Video Signal Processing and Transmission
Research topic: Video Analysis and Video Processing
Staff: Dr.-Ing. Jürgen Seiler
M.Sc. Markus Jonscher

The pixels of a low resolution image sensor are masked to obtain a non-regular sampling pattern. By assuming sparsity and using Frequency Selective Extrapolation (FSE), which already gives good results in error concealment, a high resolution image can be reconstructed.

Rightarrow

Low resolution sensor

 

      Masked sensor     

 

High resolution image

A camera with high resolution does not need a high resolution sensor anymore and therefore power consumption, bandwidth and complexity can be reduced during acquisition. Although only 1/4 of the pixels are recorded a similar visual quality as using a high resolution sensor can be achieved which can be seen in the following example.

Example for reconstruction of non-regular sampled image data

Low resolution image
Sampled image Reconstructed image
kodim19_low kodim19_sampled kodim19_rec
Best to be viewed enlarged due to additional aliasing that may be caused by scaling!

Publications

2016-47
CRIS
M. Jonscher, K. Jaskolka, J. Seiler, A. Kaup
   [doi]   [bib]

Recursive Frequency Selective Reconstruction of Non-Regularly Sampled Video Data
Picture Coding Symposium (PCS), Pages: 1-5, Nuremberg, Germany, Dec. 2016
2016-46
CRIS
M. Jonscher, J. Seiler, A. Kaup
   [doi]   [bib]

Texture-Dependent Frequency Selective Reconstruction of Non-Regularly Sampled Images
Picture Coding Symposium (PCS), Pages: 1-5, Nuremberg, Germany, Dec. 2016
2015-38
CRIS
M. Jonscher, J. Seiler, M. Bätz, T. Richter, W. Schnurrer, A. Kaup
   [link]   [doi]   [bib]

Reconstruction of Videos Taken by a Non-Regular Sampling Sensor
IEEE International Conference on Visual Communications and Image Processing (VCIP), Pages: 1-4, Singapore, Dec. 2015
2015-34
CRIS
J. Seiler, M. Jonscher, M. Schöberl, A. Kaup
   [link]   [doi]   [bib]

Resampling Images to a Regular Grid from a Non-Regular Subset of Pixel Positions Using Frequency Selective Reconstruction
IEEE Transactions on Image Processing (IEEE TIP) Vol. 24, Num. 11, Pages: 4540-4555, Nov. 2015
2014-47
CRIS
M. Bätz, A. Eichenseer, M. Jonscher, J. Seiler, A. Kaup
   [bib]

Accelerated Hybrid Image Reconstruction for Non-Regular Sampling Color Sensors
IEEE Int. Conf. on Visual Communications and Image Processing (VCIP), Pages: 217-220, Valletta, Malta, Dec. 2014
2014-46
CRIS
M. Jonscher, J. Seiler, T. Richter, A. Kaup
   [link]   [doi]   [bib]

Reducing Randomness of Non-Regular Sampling Masks for Image Reconstruction
IEEE Int. Conf. on Visual Communications and Image Processing (VCIP), Pages: 266-269, Valletta, Malta, Dec. 2014
2014-24
CRIS
M. Jonscher, J. Seiler, T. Richter, M. Bätz, A. Kaup
   [link]   [doi]   [bib]

Reconstruction of Images Taken by a Pair of Non-Regular Sampling Sensors Using Correlation Based Matching
IEEE Int. Conf. on Image Processing (ICIP), Pages: 2879-2882, Paris, France, Oct. 2014
2013-1
CRIS
M. Schöberl, J. Keinert, M. Ziegler, J. Seiler, M. Niehaus, G. Schuller, A. Kaup
   [link]   [doi]   [bib]

Evaluation of a high dynamic range video camera with non-regular sensor
SPIE Electronic Imaging - Digital Photography IX, Feb. 2013
2012-54 M. Schöberl, J. Seiler, A. Kaup
   [link]   [bib]

Bildsensor, Bildaufnahmesystem und Verfahren zum Aufnehmen eines Bildes
WO 2012/098117 A2, Jul. 2012
2012-41 M. Schöberl, J. Seiler, A. Kaup
   [link]   [bib]

Bildsensor, Bildaufnahmesystem und Verfahren zum Aufnehmen eines Bildes
DE 10 2011 002 824 A1, Jul. 2012
2012-22
CRIS
M. Schöberl, A. Belz, J. Seiler, S. Foessel, A. Kaup
   [link]   [doi]   [bib]

High Dynamic Range Video by Spatially Non-Regular Optical Filtering
IEEE International Conference on Image Processing (ICIP), Pages: 2757-2760, Sep. 2012
2012-21
CRIS
M. Schöberl, A. Belz, A. Nowak, J. Seiler, A. Kaup, S. Foessel
   [link]   [doi]   [bib]

Building a High Dynamic Range Video Sensor with Spatially Non-Regular Optical Filtering
SPIE Optics + Photonics, Applications of Digital Image Processing XXXV, Aug. 2012
2012-19
CRIS
M. Schöberl, J. Seiler, A. Kaup, A. Nowak
   [link]   [bib]

Auflösungserhöhung durch nichtregelmäßige Abtastung
Fernseh- und Kinotechnik Vol. 4, Apr. 2012
2011-43
CRIS
J. Seiler, A. Kaup
   [bib]

Optimized and Parallelized Processing Order for Improved Frequency Selective Signal Extrapolation
European Signal Processing Conf. (EUSIPCO), Pages: 269-273, Barcelona, Spain, Aug. 2011
2011-32
CRIS
M. Schöberl, J. Seiler, S. Foessel, A. Kaup
   [link]   [doi]   [bib]

Increasing Imaging Resolution by Covering Your Sensor
IEEE International Conference on Image Processing (ICIP), Pages: 1937-1940, Brussels, Belgium, Sep. 2011
2011-9
CRIS
M. Schöberl, J. Seiler, B. Kasper, S. Foessel, A. Kaup
   [link]   [doi]   [bib]

Sparsity-based Defect Pixel Compensation for Arbitrary Camera Raw Images
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Pages: 1257-1260, Prague, Czech Republic, May 2011
2010-56
CRIS
J. Seiler, A. Kaup
   [doi]   [bib]

Complex-Valued Frequency Selective Extrapolation for Fast Image and Video Signal Extrapolation
IEEE Signal Processing Letters (IEEE SPL) Vol. 17, Num. 11, Pages: 949 - 952, Nov. 2010