Summary

Added explicit deletion of temporary variables to reduce the memory consumption of the demosaicing methods.

Description

In particular for the demosaicing method by Menon, I have noticed a huge memory consumption. In one particular case I was not able to demosaic a large image (around 7000 x 5000) with 14 GB of available memory. By introducing the explicit deletion, I was able to tone it down to 5 GB in that particular case. I've added the garbage collection to the other demosaicing methods as well, even though the gain in memory usage is not as big.

Description

Hey folks,

I am a frequent user of your library and made it even part of my own project PlenoptiCam, where I recently encountered a problem, which I believe is due to the colour-demosaicing software library. Raw images passed to PlenoptiCam are relativey large in terms of their spatial resolution, thus requiring a large amount of memory. I observed that occasionally my software quit at some point without throwing an error or exception. When debugging, I noticed that this behaviour occurs at a point of my pipeline where the rather costly demosaicing_CFA_Bayer_Menon2007 function is called. I attempted to reproduce the problem with a synthetic image having a very large image resolution and observed the same problem: program abort without any message. Below you can find a short code example causing this issue on my machine.

While it is obvious why this error occurs, I would like to have a little control about the case when it happens. Would it be possible to throw an exception just before the function call crashes? This would allow users to keep programs running and let them know about the cause of the problem.

Code for Reproduction

import numpy as np
from colour_demosaicing import demosaicing_CFA_Bayer_Menon2007

mockup_image = np.ones([int(1e4), int(1e4)])
res = demosaicing_CFA_Bayer_Menon2007(CFA=mockup_image, pattern="RGGB", refining_step=True)

Exception Message

An Exception would be desired, but unfortunately there is no.

Environment Information

===============================================================================
*                                                                             *
*   Interpreter :                                                             *
*       python : 3.8.10 (default, Mar 15 2022, 12:22:08)                      *
*                [GCC 9.4.0]                                                  *
*                                                                             *
*   colour-science.org :                                                      *
*       colour : 0.4.1                                                        *
*       colour-demosaicing : 0.2.1                                            *
*                                                                             *
*   Runtime :                                                                 *
*       imageio : 2.16.1                                                      *
*       matplotlib : 3.5.1                                                    *
*       networkx : 2.7.1                                                      *
*       numpy : 1.22.3                                                        *
*       scipy : 1.8.0                                                         *
*                                                                             *
===============================================================================

I would like to use colour_demosaicing in my python script as follows: from colour_demosaicing import demosaicing_CFA_Bayer_bilinear as demosaic I installed colour-science with conda install -c conda-forge colour-science but I get the following error:

Traceback (most recent call last):
  File "robotcar.py", line 12, in <module>
    from colour_demosaicing import demosaicing_CFA_Bayer_bilinear as demosaic
ImportError: No module named colour_demosaicing

Is there any other package library I should install in conda env to import colour_demosaicing ?

Hi, Trying to use your library for demosaic RAW16 -> RGB images and for some reason getting 'The specified module could not be found', despite the fact that installed all prerequisites listed and VC code can see colour package as locally installed, any help will be appreciated.

My apologies ahead of time for not posting images as I am uncertain whether I have the appropriate rights.

I just tried demosaicing_CFA_Bayer_Malvar2004() and demosaicing_CFA_Bayer_Menon2007() on the standard MATLAB test image "mandi.tif" (see https://www.mathworks.com/help/images/ref/demosaic.html).

With the Malvar method, I see distinctive speckles of pure red and blue in the output near highlights. Besides that, the results are basically identical to that of MATLAB.

With the Menon method, there are pure red, green, and blue everywhere.

I suspect these are some bugs in the way overflow and underflow are handled.

Hi, When using the Malvar algorithm I get an output image which is much brighter than the original input image. This change is different for different images. What is the reason this happens?

Hi, @KelSolaar , thanks a lot for the lib.
I noticed that the demosaicing function does not return the output in [0,1] range, how can I make it in [0,1] range?

def demosaicing_CFA_Bayer_bilinear(CFA, pattern='RGGB'):
Notes
    -----
    -   The definition output is not clipped in range [0, 1] : this allows for
        direct HDRI / radiance image generation on *Bayer* CFA data and post
        demosaicing of the high dynamic range data as showcased in this
        `Jupyter Notebook <https://github.com/colour-science/colour-hdri/\
blob/develop/colour_hdri/examples/\
examples_merge_from_raw_files_with_post_demosaicing.ipynb>`__.

The existing convolutions caused pixels with a value of zero to underflow. Problem is solved by performing floating point convolutions, and clipping negative pixel values to zero. nb. scipy.signal dependency is added

Demosaicing a normalized bayer image with float values between 0 and 1 using the demosaicing_CFA_Bayer_Malvar2004 method does not return a clipped image. This should either be mentioned in the documentation or demosaicing methods should finish with an appropriate clipping step since without noticing this behaviour the resulting PSNRs are significantly worse.

I am using this with a point grey camera, and I get some weird artifacts while using the demosaicing_CFA_Bayer_DDFAPD, demosaicing_CFA_Bayer_Malvar2004, and demosaicing_CFA_Bayer_Menon2007 methods. The method demosaicing_CFA_Bayer_bilinear doesn't seem to have that problem.

They look like these blue dots. I am thinking that the colors are overflowing the byte.

Bumps bleach from 3.2.1 to 3.3.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

The default image extension mode used in scipy.ndimage.convolve is 'reflect', which reflects the image about the edge of the pixels. This introduces artefacts in the edge of the resulting RGB layers when demosaicing:

# Constant ones as image
orig = np.ones((2,2)) 
R, G, B = demosaicing_CFA_Bayer_bilinear(orig)

# R, G and B should also be constant 1 (at least in the measured pixels), but instead
# R: [[2.25, 0.75], G: [[0.5, 1.5], B: [[0.25, 0.75],
#     [0.75, 0.25]]     [1.5, 0.5]]     [0.75, 2.25]]

Passing mode='mirror' - which reflects about pixel centers (doesn't repeat the edge values) - to convolve in demosaicing_CFA_Bayer_bilinear returns the expected values. I figure this should be the default for demosaicing?