How can we build AI systems that not only perform well but can also explain themselves? Explainability is not only a matter of gaining users' trust, needed especially in high-stakes environments like medicine, but also a legal requirement under the EU AI Act. The question of establishing explainability is addressed by the field of Explainable AI (xAI), which provides techniques to open up the black box of many AI models, like deep neural networks. However, most of these techniques only provide approximations of the model's reasoning, which leaves the faithfulness of the explanations questionable. The rather novel field of intrinsically explainable AI mitigates this issue by building explainability directly into the model, approaching the interpretability of a simple logistic regression model.
This intrinsic explainability is realized by introducing a bottleneck layer into the neural network, which consists of a small, human-understandable set of features that sits between the deep feature-extracting layers and the final classification layer. In prototype-based networks, for example, the AI learns so-called prototypes: image patches from the training samples that are prototypical for the respective classes. In a bird classifier, a prototype might be the specific beak that is typical for a blackbird. Since these prototypes are a layer within the model, we get a built-in interpretation simply by inspecting the prototype weights for each class.
In my Master's thesis, I developed an end-to-end pipeline for the classification of 3D brain MRI images using PIPNet3D, a prototype-based learning architecture. As part of this pipeline, I implemented a web-based visualization tool that lets clinicians inspect the generated explanations. Additionally, I used brain segmentation to split the brain into regions of interest, which makes full-resolution processing feasible and improves interpretability. The overall pipeline was evaluated on the Alzheimer's disease classification problem.
The evaluation revealed that, by using brain segmentation, the pipeline achieves results comparable to whole-brain classification while relying on only three brain regions (hippocampus, amygdala, temporal lobe) and thus far less data. Furthermore, discretizing the prototype values not only improved performance but also let clinicians reason about whether a disease pattern is absent, partially present, or clearly present, rather than interpreting an arbitrary continuous score. I assessed the quality of the explanations using the Co-12 properties framework: while many quality properties are guaranteed by design, a small user-based evaluation with clinicians indicated room for improvement regarding the localization of the prototypes. Finally, the design of the pipeline allows for the easy integration of multi-modal data, which provides an exciting ground for future work.
The full thesis can be found here: An intrinsically explainable pipeline for MRI classification.
