My Publications

This study addresses the data gap in the deep learning-based pan sharpening of hyperspectral images, a technique used to improve the spatial resolution of an image using a high-resolution panchromatic image while preserving spectral information. Using the ASI PRISMA sensor, a dataset of 262,200 km^2 was collected, making it the largest dataset in terms of statistical relevance and scene diversity, which are essential for robust model generalization. Reduced resolution (RR) and full resolution (FR) experiments were also conducted to compare several deep learning pan sharpening algorithms with various non-machine learning methods. The investigation shows that data-driven neural networks significantly outperform traditional methods in terms of spectral and spatial fidelity. An in-depth analysis of both aspects is presented in this work.

Rendering night photography pictures is a challenging task that requires advanced processing techniques. Although deep learning-based Image Signal Processing (ISP) pipelines have shown promising results, current limitations are set by the lack of proper nighttime image datasets, their high computational requirements, and low explainability. In this paper, we propose a traditional ISP pipeline for rendering visually pleasing photographs of night scenes. Our pipeline is comprised of various algorithms addressing the different challenges presented by night images, and it is characterized by a shallow structure, explainable steps, and a low parameter count, resulting in computationally efficient processing. Moreover, it does not require training data. Experiments show that our pipeline can produce more pleasing results compared to other deep learning-based ISP pipelines, as it won first place in people’s choice track and third place in photographer’s choice track in the NTIRE 2023 Night Photography Rendering Challenge.

Enhancing night photography images is a challenging task that requires advanced processing techniques. While CNN-based methods have shown promising results, their high computational requirements and limited interpretability can pose challenges. To address these limitations, we propose a camera pipeline for rendering visually pleasing photographs in low-light conditions. Our approach is characterized by a shallow structure, explainable steps, and a low parameter count, resulting in computationally efficient processing. We compared the proposed pipeline with recent CNN-based state-of-the-art approaches for low-light image enhancement, showing that our approach produces more aesthetically pleasing results. The psycho-visual comparisons conducted in this work show how our proposed solution is preferred with respect to the other methods (in about 44% of the cases our solution has been chosen, compared to only about 15% of the cases for the state-of-the-art best method).

In this paper, we analyze how the color jitter traditionally used in data augmentation negatively impacts the quality of the color features in learned feature representations. To address this problem, we propose a more realistic, physics-based color data augmentation – which we call Planckian Jitter – that creates realistic variations in chromaticity and produces a model robust to illumination changes that can be commonly observed in real life, while maintaining the ability to discriminate image content based on color information. Experiments confirm that such a representation is complementary to the representations learned with the currently-used color jitter augmentation and that a simple concatenation leads to significant performance gains on a wide range of downstream datasets.

We present a general-purpose framework for the optimization of parametric contrast enhancement algorithms. We first define a regression module for image acceptability, which is based on deep neural features and which is trained on a large dataset of user-expressed preferences. This regression module is then used as the objective function of a Bayesian optimization process, guiding the search for the optimal parameters of a given contrast enhancement algorithm. In our experiments we optimize three different contrast enhancement algorithms of varying levels of complexity. The effectiveness of our optimization framework is experimentally confirmed by evaluating the output of the optimized contrast enhancement algorithms with respect to reference enhanced images.

In this paper we overview the most common datasets for Automatic White Balance, including those for single as well as multiple illuminant estimation, providing a critical analysis on their characteristics. Furthermore, we identify a number of existing methods for single illuminant estimation, as a representative pool of approaches to the problem with various levels of complexity. We investigate how the performance of these correlate to the image content of common datasets.

We present an efficient combination strategy for color constancy algorithms. We define a compact neural network architecture to process and combine the illuminant estimations of individual algorithms, that may be based on different assumptions over the input scene content. Our solution can be specialized to the image domain, thus expecting a single frame input, and to the video domain, exploiting a Long Short-Term Memory module (LSTM) to handle varying-length sequences. We experiment on the standard Shi-Gehler and NUS datasets for still images, and on the recent Burst Color Constancy dataset for videos. Experimental results show that our method outperforms other combination strategies, and reaches an illuminant estimation accuracy comparable to more sophisticated and computationally-demanding solutions when the standard dataset split is used.

In this paper, we design and implement a new encoder-decoder neural network for supervised raindrop removal. Our network, given a rainy input image, produces as output the Laplacian pyramid of a rain-free version of the input, making it possible to handle the variety of appearances of rain droplets by processing different frequency bands independently. To this end, we define and experimentally prove the effectiveness of a custom loss function that combines the errors of the different Laplacian frequency bands.

We investigate the negative effects of rain streaks over the performance of a neural network for real time semantic segmentation of street scenes. This is done by synthetically augmenting the CityScapes dataset with artificial rain. We present a per-class analysis in order to provide deeper insights over the impact of rain on semantic segmentation.

In this paper, we present a Convolutional Neural Network for rain streaks removal from images, with specific interest in evaluating the results of the processing operation with respect to the Optical Character Recognition (OCR) task.

A deep learning approach to JPEG artifact reduction without any knowledge on previous compression operation parameters.

This paper proposes a framework to train an artifact-free thin cloud removal model using Generative Adversarial Nets (GANs) with thick cloud masks.