CNN Attention Cards / Activations

What are the general techniques for determining which portions of images are most conducive to image classification across convolutional neural networks ?

In general, let's assume we have 2d matrices with float values ​​from 0 to 1. Each matrix is ​​associated with a label (single-labeled, multi-class), and the goal is to perform classification via (Keras) 2D CNN.

I am trying to find methods for highlighting matching subsequences of rows / columns that are most conducive to classification.

Two examples:

https://github.com/jacobgil/keras-cam

https://github.com/tdeboissiere/VGG16CAM-keras

Other examples / resources with an eye on Keras would be much appreciated.

Please note, my datasets are not actual images, so using methods with ImageDataGenerator might not apply directly in this case.