Note: This is a virtual presentation. Here is the link for where the presentation will be taking place.
Title: Supervised and semi-supervised approaches for restoring images degraded by adverse weather conditions
Abstract: Single image de-raining is an extremely challenging problem since the rainy images contain rain streaks which often vary in size, direction, and density. This varying characteristic of rain streaks affects different parts of the image differently. Previous approaches have attempted to address this problem by leveraging some prior information to remove rain streaks from a single image. The major limitations of these approaches are: (i) they do not consider the location information of raindrops in the image while retrieving the rain-free clean image, (ii) these methods use synthetic datasets for training which results in sub-optimal performance on the real-world images, typically because of the distributional-shift between synthetic and rainy images. In the first part of this proposal, I will present an Uncertainty guided Multi-scale Residual Learning (UMRL) network that attempts to address the aforementioned issue by learning the rain content at different scales and using them to estimate the final rain-free clean output image. In addition, we introduce a technique that guides the network to learn the network weights based on the confidence measure about the estimate. Furthermore, we introduce a new training and testing procedure based on the notion of cycle spinning to improve the final deraining performance. In the second part of this proposal, we address the distributional-shift between synthetic and rainy images, where we propose a Gaussian Process-based semi-supervised learning framework called Syn2Real which enables the network in learning to derain using synthetic dataset while generalizing better using unlabeled real-world images. In our proposed Syn2Real technique we formulate joint Gaussian distribution between labeled (synthetic rain) images and unlabeled (real rain) images in the latent space using Gaussian Process. Using this joint distribution we obtain pseudo-GT (pseudo ground-truth) for unlabeled images and use these obtained pseudo-GTs to learn the network weights. We evaluate the effectiveness of these methods on multiple publicly available datasets and show that these approaches achieve better performance compared to previous state-of-the-art methods.
Committee Members
- Vishal Patel, Department of Electrical and Computer Engineering
- Rama Chellappa, Department of Electrical and Computer Engineering
- Carlos D. Castillo, Department of Electrical and Computer Engineering
