• [x] Support mixed-precision of Tensorflow 2.4+, (i.e., NOT support experimental API).
• [x] Change a terminology that Loss to Score
• [x] Refactoring Testcases, ActivationMaximization and some utilities
• [x] Some bugfixes
• [x] Write docstrings
• [x] Update example notebook
• [x] README.md
• [x] Add tests
• [x] Update setup.py
• [x] Write Release note

Closes #24, #43 , #45, #47 and #51

Hello. I'm new to ML and newer still to this library. I want to use the gradcam on a vgg16 architecture with custom top layers that I created. If my model in this case consists of two separate models, what would I have to change, say, in the example case to make this work?

Hi @keisen, thanks for putting this together. Is there a way to generate a guided backpropagation saliency map with a custom function in gradient-modifier?

This example fails when interpolating data deep down in scipy:

import numpy as np
import tensorflow.keras as keras
from tf_keras_vis.scorecam import ScoreCAM

policy = keras.mixed_precision.experimental.Policy("mixed_float16")
keras.mixed_precision.experimental.set_policy(policy)

ScoreCAM(model := keras.applications.MobileNet())(
    lambda output: [o[0] for o in output], np.empty(model.input.shape[1:])
)

This short example shows that GradCam(PlusPlus) fails when the network is trained with mixed precision:

import numpy as np
import tensorflow.keras as keras
from tensorflow.keras.mixed_precision import experimental as mixed_precision
from tf_keras_vis.gradcam import Gradcam, GradcamPlusPlus
from tf_keras_vis.saliency import Saliency

policy = mixed_precision.Policy("mixed_float16")
mixed_precision.set_policy(policy)

model = keras.applications.MobileNet()
input_data = np.empty(model.input.shape[1:])


def model_modifier(model):
    model.layers[-1].activation = keras.activations.linear
    return model


def loss(output):
    return [o[0] for o in output]


saliency = Saliency(model, model_modifier, clone=False)

# works
saliency(loss, input_data)

# works
saliency(loss, input_data, smooth_samples=20)

# does not work: "array type dtype('float16') not supported"
gradcam = Gradcam(model, model_modifier, clone=False)
gradcam(loss, input_data, penultimate_layer=-1)

# does not work: "array type dtype('float16') not supported"
gradcampp = GradcamPlusPlus(model, model_modifier, clone=False)
gradcampp(loss, input_data, penultimate_layer=-1)

It works fine when you comment out mixed_precision.set_policy(policy).

Hello Keisen,

First I'd like to thank you for sharing this project. I've already used the former keras-vis and I found your project exactly when I was developing a new code for TF 2.1.

I built a multi task model with 3 output layers and I'm not sure how to select each of this outputs before running the saliency and normalize methods.

Here is the outputs of my model (obtained via model.outputs command). The first and third outputs use binary_crossentropy loss while the second uses MSE loss.

[<tf.Tensor 'sex_3/Identity:0' shape=(None, 1) dtype=float32>,
 <tf.Tensor 'age_3/Identity:0' shape=(None, 1) dtype=float32>,
 <tf.Tensor 'autism_3/Identity:0' shape=(None, 1) dtype=float32>]

I had success running the code bellow (from your examples). But I failed when I needed to select a different output and loss than the last one.

def model_modifier(m):
    m.layers[-1].activation = tf.keras.activations.linear
saliency = Saliency(model, model_modifier)
loss = lambda output: K.mean(output[:, 0])

Thanks in advance!

[email protected]:~/pip_patches> python --version
Python 3.8.2
[email protected]:~/pip_patches> pip install tf-keras-vis==0.2.2
ERROR: Could not find a version that satisfies the requirement tf-keras-vis==0.2.2 (from versions: 0.0.1, 0.1.0)
ERROR: No matching distribution found for tf-keras-vis==0.2.2

First, great Thanks to this masterpiece! It really saved my life when I found this toolkit. But I got some errors while I was trying to visualize my model via saliency map. I think the problem is from the format of seed_input, I followed exactly the same way as the tutorial did, using one image, using img_to_array to process it, and passing the array to the seed_input, but some how seed_inputs were transformed into a tensor such that it has no attribute 'ndim', I tried to transform X into ndarray by K.eval(X), but so far it didn't work well. Many thanks for all the help!!

---> 94 seed_inputs = list(seed_inputs)
     95 total = (np.zeros_like(X[0]) for X in seed_inputs)
     96 for i in range(smooth_samples):

File ~\Anaconda3\lib\site-packages\tf_keras_vis\saliency.py:93, in <genexpr>(.0)
     89 seed_inputs = ((X, smooth_noise * (tf.math.reduce_max(X, axis=axis, keepdims=True) -
     90                                    tf.math.reduce_min(X, axis=axis, keepdims=True)))
     91                for X, axis in seed_inputs)
     92 print(seed_inputs)
---> 93 seed_inputs = (X + np.random.normal(0., sigma,X.shape) for X, sigma in seed_inputs)
     94 seed_inputs = list(seed_inputs)
     95 total = (np.zeros_like(X[0]) for X in seed_inputs)

File ~\Anaconda3\lib\site-packages\tf_keras_vis\saliency.py:89, in <genexpr>(.0)
     86 seed_inputs = (tf.reshape(X, (smooth_samples, -1) + tuple(X.shape[1:]))
     87                for X in seed_inputs)
     88 seed_inputs = ((X, tuple(range(K.eval(X).ndim)[2:])) for X in seed_inputs)
---> 89 seed_inputs = ((X, smooth_noise * (tf.math.reduce_max(X, axis=axis, keepdims=True) -
     90                                    tf.math.reduce_min(X, axis=axis, keepdims=True)))
     91                for X, axis in seed_inputs)
     92 print(seed_inputs)
     93 seed_inputs = (X + np.random.normal(0., sigma, X.shape) for X, sigma in seed_inputs)

File ~\Anaconda3\lib\site-packages\tf_keras_vis\saliency.py:88, in <genexpr>(.0)
     84 seed_inputs = (tf.tile(X, (smooth_samples, ) + tuple(np.ones(X.ndim - 1, np.int)))
     85                for X in seed_inputs)
     86 seed_inputs = (tf.reshape(X, (smooth_samples, -1) + tuple(X.shape[1:]))
     87                for X in seed_inputs)
---> 88 seed_inputs = ((X, tuple(range(X.ndim)[2:])) for X in seed_inputs)
     89 seed_inputs = ((X, smooth_noise * (tf.math.reduce_max(X, axis=axis, keepdims=True) -
     90                                    tf.math.reduce_min(X, axis=axis, keepdims=True)))
     91                for X, axis in seed_inputs)
     92 print(seed_inputs)

File ~\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py:446, in Tensor.__getattr__(self, name)
    437 if name in {"T", "astype", "ravel", "transpose", "reshape", "clip", "size",
    438             "tolist", "data"}:
    439   # TODO(wangpeng): Export the enable_numpy_behavior knob
    440   raise AttributeError(
    441       f"{type(self).__name__} object has no attribute '{name}'. " + """
    442     If you are looking for numpy-related methods, please run the following:
    443     from tensorflow.python.ops.numpy_ops import np_config
    444     np_config.enable_numpy_behavior()
    445   """)
--> 446 self.__getattribute__(name)

AttributeError: 'Tensor' object has no attribute 'ndim'

I tried to run on a custom model and also ran the examples and it seems that score-cam is broken - takes too much time to compute, most of the time it fails due to memory issues, and when it does finish, the output is just zeros.

Can you confirm that the problem is just not with me?

Cheers.

Hi, I'm getting an error when I try to get the grad-cam visualizations for a custom model as given below.

Then when I try to call to GradCam I get: ValueError: Graph disconnected: cannot obtain value for tensor Tensor("input_1:0", shape=(None, 224, 224, 3), dtype=float32) at layer "rescaling". The following previous layers were accessed without issue: []

Hi, @keisen , Our paper "LayerCAM: Exploring Hierarchical Class Activation Maps for Localization" is accepted by TIP recently, which can visualize the class activation maps from any CNN layer of an off-the-shelf network. Could you add our method to your popular repository for more people to try this method? Our method is a simple modification of Grad-CAM. It should easy to implement. Here is the paper and code. Hope for your reply.

Thank you for this useful visualization package!

Right now I have a two-input, one-output model as follows

I am using GradCAM, with code snippet below. I have stated my penultimate layer to be 'ria-conv', which happens after concatenation of both networks.

However, the dimension of cam is 2 - one for each network (15 * 256 * 320 is my image count * image dimensions) , as shown in this screenshot

As far as my understanding goes, since the actual last convolutional layer is after concatenation, cam should be 1 * 15 * 256 * 320? This happens both when I explicitly state the penultimate layer and when I state it as -1.

Hi! Thanks for this XAI visualization package!

I am trying to use the saliency method applied to Autoencoders and reconstruction errors. Also, I do not have images, but a feature array for each observation. My goal is to check the "importance" of each input feature to the overall reconstruction, thus was trying to use smoothgrad method.

In this case, I do not understand what should I pass to 'score'. The output of the model is just the reconstructed input (n_samples x n_features).

I also couldn't pass a custom function to the score attribute as you suggest in the documentation. 'From the example in the repository:

Instead of using CategoricalScore object, you can also define the function from scratch as follows:

def score_function(output): # The output variable refers to the output of the model, # so, in this case, output shape is (3, 1000) i.e., (samples, classes). return (output[0][1], output[1][294], output[2][413]) , But then, how can you pass the function to the method? It needs to be callable, so it gives "ValueError: Score object must be callable! " Could you add an example explicitly using score_function passed to saliency() (or any other) instance, instead of the instance score from one of the defined score classes (BinaryScore, CategoricalScore) ?

Thank you in advance!

Hi there!

Currently the last Conv layer is being automatically used for GradCAM if not specified differently. We noticed that for some models like MobileNetV3 this results in the wrong layer being used (as already mentioned in #61).

Specifically to MobileNetV3 this has some problems:

1. The logits layer is implemented using a Conv2D layer causing this layer with shape (None, 1, 1, 1024) to be selected as the penultimate layer. Obviously this will result in incorrect/useless GradCAM images.
2. Selecting the previous Conv layer manually however does not include some important activations which occasionally causes inverted GradCAM images (see #61).

We propose to use a different search condition which searches for the last layer with four dimensions and a width and height of more than 1. This will help both problems mentioned above:

1. The selected layer will have dimensions greater than 1 by 1
2. Important activations will still be included since the penultimate layer doesn't necessarily have to be a Conv layer anymore resulting in non-inverted GradCAM images.

A similar implementation is being used in sicara/tf-explain. Their implementation would however still be affected by the first problem.

I added some quick tests which could be extended in the future.

Let me know what you think!

Hello all,

wondered if anyone has encounted the error:

TypeError: '<=' not supported between instances of 'int' and 'str'

`%%time
from tensorflow.keras import backend as K
from tf_keras_vis.saliency import Saliency
# from tf_keras_vis.utils import normalize

# Create Saliency object.
saliency = Saliency(model,
                    model_modifier=replace2linear,
                    clone=True)

# Generate saliency map
saliency_map = saliency(score, X)

## Since v0.6.0, calling `normalize()` is NOT necessary.
#saliency_map = normalize(saliency_map)

# Render
f, ax = plt.subplots(nrows=1, ncols=3, figsize=(12, 4))
for i, title in enumerate(image_titles):
    ax[i].set_title(title, fontsize=16)
    ax[i].imshow(saliency_map[i], cmap='jet')
    ax[i].axis('off')
plt.tight_layout()
plt.show()

TypeError                                 Traceback (most recent call last)
<timed exec> in <module>

/usr/local/lib/python3.8/dist-packages/tf_keras_vis/saliency.py in __call__(self, score, seed_input, smooth_samples, smooth_noise, keepdims, gradient_modifier, training, normalize_map, unconnected_gradients)
     98             grads = [g / smooth_samples for g in total]
     99         else:
--> 100             grads = self._get_gradients(seed_inputs, scores, gradient_modifier, training,
    101                                         unconnected_gradients)
    102         # Visualizing

/usr/local/lib/python3.8/dist-packages/tf_keras_vis/saliency.py in _get_gradients(self, seed_inputs, scores, gradient_modifier, training, unconnected_gradients)
    115             outputs = self.model(seed_inputs, training=training)
    116             outputs = listify(outputs)
--> 117             score_values = self._calculate_scores(outputs, scores)
    118         grads = tape.gradient(score_values,
    119                               seed_inputs,

/usr/local/lib/python3.8/dist-packages/tf_keras_vis/__init__.py in _calculate_scores(self, outputs, score_functions)
     86         score_values = (func(output) for output, func in zip(outputs, score_functions))
     87         score_values = (self._mean_score_value(score) for score in score_values)
---> 88         score_values = list(score_values)
     89         return score_values
     90 

/usr/local/lib/python3.8/dist-packages/tf_keras_vis/__init__.py in <genexpr>(.0)
     85     def _calculate_scores(self, outputs, score_functions):
     86         score_values = (func(output) for output, func in zip(outputs, score_functions))
---> 87         score_values = (self._mean_score_value(score) for score in score_values)
     88         score_values = list(score_values)
     89         return score_values

/usr/local/lib/python3.8/dist-packages/tf_keras_vis/__init__.py in <genexpr>(.0)
     84 
     85     def _calculate_scores(self, outputs, score_functions):
---> 86         score_values = (func(output) for output, func in zip(outputs, score_functions))
     87         score_values = (self._mean_score_value(score) for score in score_values)
     88         score_values = list(score_values)

/usr/local/lib/python3.8/dist-packages/tf_keras_vis/utils/scores.py in __call__(self, output)
     99             raise ValueError("`output` ndim must be 2 or more (batch_size, ..., channels), "
    100                              f"but was {output.ndim}")
--> 101         if output.shape[-1] <= max(self.indices):
    102             raise ValueError(
    103                 f"Invalid index value. indices: {self.indices}, output.shape: {output.shape}")

TypeError: '<=' not supported between instances of 'int' and 'str'
`