Perturbation based explainability of Grad-CAM

This project was part of a master lecture on the topic of Visual Analytics. The task was to develop and evaluate an interactive approach to the education of an Explainable AI method. Target users were students or professionals who wanted to understand the principle, inner workings, advantages and possibly disadvantages of a particular XAI approach or instructors who teach XAI. My fellow students and I had decided to take a closer look at Grad-CAM.

To make Grad-CAM more understandable, we insisted on using data that any person can generally relate to. Therefore, we used an EfficientNet (B0), which has already been trained on the ImageNet dataset and provides very good prediction accuracies despite a very efficient approach. In addition, we wanted to integrate perturbation to provide an interactive approach whereby a user can directly see how the prediction as well as the relevant areas change by graying out certain areas. To our knowledge, there was no other publication in which Grad-CAM was used in combination with perturbation in such a web app until the publication of our project.

# Technologies

We used Python for this project. For the CNN as well as the preprocessing of the images we used Tensorflow, Pandas and Pillow. For the implementation of Grad-CAM, we followed existing approaches. We were able to achieve perturbation via extending Plotly functionalities and using Dash callbacks. To build the web app, we used Plotly's Dash framework, which also allows for easy integration with Plotly. We host the web app via Heroku.

# Usage

A user can either upload a picture himself or use the preloaded picture of a cat and a dog. Our web app generates the image classification prediction in the background based on the pre-trained CNN and visualizes the prediction accuracies of the five most likely classes. Likewise, the most relevant areas based on Grad-CAM are visualized in the form of a heat map. The user can now interactively gray out certain areas. The web app then generates an image with the grayed out areas and predicts it again, updating the five most likely classes as well as the most relevant areas. The user can also use a slider to select which feature map should be used to calculate the most relevant areas. This should show that early layers tend to generate low level features. If the user is not familiar with Grad-CAM, he can also get an introduction to the topic at the end of the page.