Super-resolution aims at estimating a high-resolution image from a single low-resolution input (left). Traditional methods tend to produce over-smoothed images that lack high frequency textures and do not look natural (second picture). One focus of our research was the development of algorithms that are able to create realistic textures (third picture) rather than a pixel-accurate reproduction of ground truth (right picture).
Handheld video cameras now being available in every smartphone, images and videos have become ubiquitous. The amount of visual content on the internet has been ever increasing and digital images and videos have become the main carrier of information over the last few decades.
In our computational imaging group we are interested in a range of signal and image processing problems both in computational photography and scientific imaging. Our focus is on digital image restoration that aims at computationally enhancing the quality of images and recovering probable original images by undoing the adverse effects of image degradation such as noise and blur. Advances in convolutional neural networks have revolutionized computer vision and the field of digital image restoration has been no exception to this rule.
An important problem in digital image restoration is super-resolution, aiming at recovering a high-resolution image from low-resolution input. Many traditionally used performance measures such as peak-signal-to-noise ratio (PSNR) correlate poorly with the human perception of image quality. Algorithms minimizing these metrics thus tend to produce over-smoothed images that lack high-frequency textures and do not look natural despite yielding high PSNR values. By focussing on realistic textures rather than pixel-wise accuracy we achieved a significant boost in image quality even at high magnification ratios [ ] (see figure above). Enforcing time consistency leads to novel approaches for the challenging problem of video super-resolution [ ] and video prediction [ ].
Another focus area has been the problem of blind deblurring. Images often exhibit blur due to unwanted camera shake or moving objects in the scene. Removing the blur is hard as neither the sharp image nor the motion blur kernel is known. We have developed efficient recurrent network architectures that propagate information between multiple consecutive blurry observations which greatly helps to restore the desired sharp image or video [ ]. The modulation transfer function (MTF) plays an important role for lens quality assessment and non-blind deblurring. We have developed a framework to directly estimate it from natural images [ ].
In another line of work we applied image processing to MR images [ ]. Moreover, we have worked on MRI sequence generation [ ] hardware optimization [ ] and acceleration of aquisition processes [ ].
Recently we have entered the field of acoustic computer-generated holography (CGH). Acoustic CGH is concerned with the computation of 3D-printed plastic devices that modulate the phase of passing ultrasound waves in order to generate desired intensity profiles with possible applications in the medical sciences and one-shot manufacturing. Casting the computation of holograms as an inverse problem allows us to employ tools from image processing in this context.
2021
Loktyushin, A., Herz, K., Dang, N., Glang, F., Deshmane, A., Weinmüller, S., Doerfler, A., Schölkopf, B., Scheffler, K., Zaiss, M.
MRzero - Automated discovery of MRI sequences using supervised learning
Magnetic Resonance in Medicine, 86(2):709-724, 2021 (article)
2020
Aghaeifar, A., Zhou, J., Heule, R., Tabibian, B., Schölkopf, B., Jia, F., Zaitsev, M., Scheffler, K.
A 32-channel multi-coil setup optimized for human brain shimming at 9.4T
Magnetic Resonance in Medicine, 83(2):749-764, 2020 (article)
2019
Scheffler, K., Loktyushin, A., Bause, J., Aghaeifar, A., Steffen, T., Schölkopf, B.
Spread-spectrum magnetic resonance imaging
Magnetic Resonance in Medicine, 82(3):877-885, 2019 (article)
Groh*, F., Wieschollek*, P., Lensch, H. P. A.
Flex-Convolution
Computer Vision - ACCV 2018 - 14th Asian Conference on Computer Vision, 11361, pages: 105-122, Lecture Notes in Computer Science, (Editors: Jawahar, C. V. and Li, Hongdong and Mori, Greg and Schindler, Konrad), Springer International Publishing, December 2019, *equal contribution (conference)
2018
Wieschollek, P., Gallo, O., Gu, J., Kautz, J.
Separating Reflection and Transmission Images in the Wild
European Conference on Computer Vision (ECCV), Part XIII, 11217, pages: 90-105, Lecture Notes in Computer Science, (Editors: Vittorio Ferrari and Martial Hebert and Cristian Sminchisescu and Yair Weiss), Springer, September 2018 (conference)
Meding, K., Hirsch, M., Wichmann, F. A.
Retinal image quality of the human eye across the visual field
14th Biannual Conference of the German Society for Cognitive Science (KOGWIS 2018), 2018 (poster)
Gondal, M. W., Schölkopf, B., Hirsch, M.
The Unreasonable Effectiveness of Texture Transfer for Single Image Super-resolution
Workshop and Challenge on Perceptual Image Restoration and Manipulation (PIRM) at the 15th European Conference on Computer Vision (ECCV), September 2018 (conference)
Pérez-Pellitero, E., Sajjadi, M. S. M., Hirsch, M., Schölkopf, B.
Photorealistic Video Super Resolution
Workshop and Challenge on Perceptual Image Restoration and Manipulation (PIRM) at the 15th European Conference on Computer Vision (ECCV), 2018 (poster)
Kim, T. H., Sajjadi, M. S. M., Hirsch, M., Schölkopf, B.
Spatio-temporal Transformer Network for Video Restoration
15th European Conference on Computer Vision (ECCV), Part III, 11207, pages: 111-127, Lecture Notes in Computer Science, (Editors: Vittorio Ferrari, Martial Hebert,Cristian Sminchisescu and Yair Weiss), Springer, September 2018 (conference)
Loktyushin, A., Ehses, P., Schölkopf, B., Scheffler, K.
Learning-based solution to phase error correction in T2*-weighted GRE scans
1st International conference on Medical Imaging with Deep Learning (MIDL), July 2018 (conference)
Xiao, L., Heide, F., Heidrich, W., Schölkopf, B., Hirsch, M.
Discriminative Transfer Learning for General Image Restoration
IEEE Transactions on Image Processing, 27(8):4091-4104, 2018 (article)
Bauer, M., Volchkov, V., Hirsch, M., Schölkopf, B.
Automatic Estimation of Modulation Transfer Functions
IEEE International Conference on Computational Photography (ICCP), pages: 1-12, May 2018 (conference)
Sajjadi, M. S. M., Parascandolo, G., Mehrjou, A., Schölkopf, B.
Tempered Adversarial Networks
Proceedings of the 35th International Conference on Machine Learning (ICML), 80, pages: 4448-4456, Proceedings of Machine Learning Research, (Editors: Dy, Jennifer and Krause, Andreas), PMLR, July 2018 (conference)
Loktyushin, A., Ehses, P., Schölkopf, B., Scheffler, K.
Autofocusing-based phase correction
Magnetic Resonance in Medicine, 80(3):958-968, 2018 (article)
Sajjadi, M. S. M., Vemulapalli, R., Brown, M.
Frame-Recurrent Video Super-Resolution
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages: 6626-6634, June 2018 (conference)
2017
Aghaeifar, A., Loktyushin, A., Eschelbach, M., Scheffler, K.
Improving performance of linear field generation with multi-coil setup by optimizing coils position
Magnetic Resonance Materials in Physics, Biology and Medicine, 30(Supplement 1):S259, 34th Annual Scientific Meeting of the European Society for Magnetic Resonance in Medicine and Biology (ESMRMB), October 2017 (poster)
Wieschollek, P., Hirsch, M., Schölkopf, B., Lensch, H.
Learning Blind Motion Deblurring
Proceedings IEEE International Conference on Computer Vision (ICCV), pages: 231-240, IEEE, Piscataway, NJ, USA, IEEE International Conference on Computer Vision (ICCV), October 2017 (conference)
Kim, T. H., Lee, K. M., Schölkopf, B., Hirsch, M.
Online Video Deblurring via Dynamic Temporal Blending Network
Proceedings IEEE International Conference on Computer Vision (ICCV), pages: 4038-4047, IEEE, Piscataway, NJ, USA, IEEE International Conference on Computer Vision (ICCV), October 2017 (conference)
Gondal, M. W., Köhler, J. M., Grzeszick, R., Fink, G., Hirsch, M.
Weakly-Supervised Localization of Diabetic Retinopathy Lesions in Retinal Fundus Images
IEEE International Conference on Image Processing (ICIP), pages: 2069-2073, September 2017 (conference)
Katiyar, P., Divine, M. R., Kohlhofer, U., Quintanilla-Martinez, L., Schölkopf, B., Pichler, B. J., Disselhorst, J. A.
Spectral Clustering predicts tumor tissue heterogeneity using dynamic 18F-FDG PET: a complement to the standard compartmental modeling approach
Journal of Nuclear Medicine, 58(4):651-657, 2017 (article)
Katiyar, P., Divine, M. R., Kohlhofer, U., Quintanilla-Martinez, L., Schölkopf, B., Pichler, B. J., Disselhorst, J. A.
A Novel Unsupervised Segmentation Approach Quantifies Tumor Tissue Populations Using Multiparametric MRI: First Results with Histological Validation
Molecular Imaging and Biology, 19(3):391-397, 2017 (article)
Schober, M., Adam, A., Yair, O., Mazor, S., Nowozin, S.
Dynamic Time-of-Flight
Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017, pages: 170-179, IEEE, Piscataway, NJ, USA, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017 (conference)
Lu, C., Hirsch, M., Schölkopf, B.
Flexible Spatio-Temporal Networks for Video Prediction
Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017, pages: 2137-2145, IEEE, Piscataway, NJ, USA, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017 (conference)
Sajjadi, M. S. M., Schölkopf, B., Hirsch, M.
EnhanceNet: Single Image Super-Resolution through Automated Texture Synthesis
Proceedings IEEE International Conference on Computer Vision (ICCV), pages: 4501-4510, IEEE, Piscataway, NJ, USA, IEEE International Conference on Computer Vision (ICCV), October 2017 (conference)
Khatami, M., Schmidt-Wilcke, T., Sundgren, P. C., Abbasloo, A., Schölkopf, B., Schultz, T.
BundleMAP: Anatomically Localized Classification, Regression, and Hypothesis Testing in Diffusion MRI
Pattern Recognition, 63, pages: 593-600, 2017 (article)
Wieschollek, P., Schölkopf, B., Lensch, H. P. A., Hirsch, M.
End-to-End Learning for Image Burst Deblurring
Computer Vision - ACCV 2016 - 13th Asian Conference on Computer Vision, 10114, pages: 35-51, Image Processing, Computer Vision, Pattern Recognition, and Graphics, (Editors: Lai, S.-H., Lepetit, V., Nishino, K., and Sato, Y. ), Springer, November 2017 (conference)
2016
Wieschollek, P., Wang, O., Sorkine-Hornung, A., Lensch, H. P. A.
Efficient Large-scale Approximate Nearest Neighbor Search on the GPU
29th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages: 2027 - 2035, IEEE, June 2016 (conference)
Xiao, L., Wang, J., Heidrich, W., Hirsch, M.
Learning High-Order Filters for Efficient Blind Deconvolution of Document Photographs
Computer Vision - ECCV 2016, Lecture Notes in Computer Science, LNCS 9907, Part III, pages: 734-749, (Editors: Bastian Leibe, Jiri Matas, Nicu Sebe and Max Welling), Springer, October 2016 (conference)
Sajjadi, M. S. M., Köhler, R., Schölkopf, B., Hirsch, M.
Depth Estimation Through a Generative Model of Light Field Synthesis
Pattern Recognition - 38th German Conference (GCPR), 9796, pages: 426-438, Lecture Notes in Computer Science, (Editors: Rosenhahn, B. and Andres, B.), Springer International Publishing, September 2016 (conference)
Divine, M. R., Katiyar, P., Kohlhofer, U., Quintanilla-Martinez, L., Disselhorst, J. A., Pichler, B. J.
A Population Based Gaussian Mixture Model Incorporating 18F-FDG-PET and DW-MRI Quantifies Tumor Tissue Classes
Journal of Nuclear Medicine, 57(3):473-479, 2016 (article)
Schuler, C. J., Hirsch, M., Harmeling, S., Schölkopf, B.
Learning to Deblur
IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(7):1439-1451, IEEE, 2016 (article)