Faithfulness, Stability, Agreement, and Complexity of Post-hoc Explanations for Card Fraud Alerts: A Comparative Evaluation of SHAP, LIME, Permutation Importance, and Counterfactuals

Yunfei Gao; Chenhui Hao

Faithfulness, Stability, Agreement, and Complexity of Post-hoc Explanations for Card Fraud Alerts: A Comparative Evaluation of SHAP, LIME, Permutation Importance, and Counterfactuals

Authors

Yunfei Gao Electrical Engineering, Northwestern University, IL, USA
Chenhui Hao Civil Engineering, University of California, CA, USA

Keywords:

post-hoc explanation, fraud detection, explanation faithfulness, explanation stability

Abstract

Fraud investigators increasingly rely on post-hoc explanations to justify why a transaction was flagged, yet the reliability of these explanations on actual alert queues is rarely measured. This paper compares three alert-level explanation approaches—SHAP, LIME, and counterfactual explanations—and uses permutation feature importance (PFI) as a retrospective global-importance reference for alerts raised by logistic regression, random forest, and XGBoost classifiers on two public card-transaction datasets: a simulated dataset with documented fraud mechanisms and a real PCA-anonymized dataset. For 200 alerts per classifier, the evaluation covers faithfulness, stability, inter-method agreement, and complexity. Exact SHAP attained the highest PGI-based faithfulness in four of the six settings and showed consistently strong within-setting faithfulness correlations, while maintaining low PGU values.A global PFI ranking matched SHAP for logistic regression but lost alert-level evidence for XGBoost, where its mechanism alignment fell to 0.442. LIME was stable under input perturbations, but its top-three overlap across seeds dropped to 0.453 on the PCA data, and counterfactual importance was the least stable, with run-to-run Jaccard between 0.212 and 0.459. Agreement between explainers was partial and classifier-dependent. The results support reporting explainer configurations and seed stability whenever explanations document fraud alerts.

References

Dal Pozzolo, A., Boracchi, G., Caelen, O., Alippi, C., & Bontempi, G. (2018). Credit card fraud detection: A realistic modeling and a novel learning strategy. IEEE Transactions on Neural Networks and Learning Systems, 29(8), 3784–3797. https://doi.org/10.1109/TNNLS.2017.2736643

Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5), 206–215. https://doi.org/10.1038/s42256-019-0048-x

Amarasinghe, K., Rodolfa, K. T., Jesus, S., Chen, V., Balayan, V., Saleiro, P., Bizarro, P., Talwalkar, A., & Ghani, R. (2024). On the importance of application-grounded experimental design for evaluating explainable ML methods. Proceedings of the AAAI Conference on Artificial Intelligence, 38(19), 20921–20929. https://doi.org/10.1609/aaai.v38i19.30082

Nauta, M., Trienes, J., Pathak, S., Nguyen, E., Peters, M., Schmitt, Y., Schlötterer, J., van Keulen, M., & Seifert, C. (2023). From anecdotal evidence to quantitative evaluation methods: A systematic review on evaluating explainable AI. ACM Computing Surveys, 55(13s), 1–42. https://doi.org/10.1145/3583558

Krishna, S., Han, T., Gu, A., Wu, S., Jabbari, S., & Lakkaraju, H. (2024). The disagreement problem in explainable machine learning: A practitioner's perspective. Transactions on Machine Learning Research. https://openreview.net/forum?id=jESY2WTZCe

Slack, D., Hilgard, S., Singh, S., & Lakkaraju, H. (2021). Reliable post hoc explanations: Modeling uncertainty in explainability. Advances in Neural Information Processing Systems, 34. https://papers.nips.cc/paper_files/paper/2021/hash/4e246a381baf2ce038b3b0f82c7d6fb4-Abstract.html

Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30. https://papers.nips.cc/paper_files/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html

Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). "Why should I trust you?": Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 1135–1144). Association for Computing Machinery. https://doi.org/10.1145/2939672.2939778

Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32. https://doi.org/10.1023/A:1010933404324

Covert, I., Lundberg, S., & Lee, S.-I. (2021). Explaining by removing: A unified framework for model explanation. Journal of Machine Learning Research, 22(209), 1–90. https://jmlr.org/papers/v22/20-1316.html

Tolomei, G., Silvestri, F., Haines, A., & Lalmas, M. (2017). Interpretable predictions of tree-based ensembles via actionable feature tweaking. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 465–474). Association for Computing Machinery. https://doi.org/10.1145/3097983.3098039

Panagiotou, E., Heurich, M., Landgraf, T., & Ntoutsi, E. (2024). TABCF: Counterfactual explanations for tabular data using a transformer-based VAE. In Proceedings of the 5th ACM International Conference on AI in Finance (pp. 274–282). Association for Computing Machinery. https://doi.org/10.1145/3677052.3698673

Yeh, C.-K., Hsieh, C.-Y., Suggala, A. S., Inouye, D. I., & Ravikumar, P. (2019). On the (in)fidelity and sensitivity of explanations. Advances in Neural Information Processing Systems, 32. https://papers.nips.cc/paper_files/paper/2019/hash/a7471fdc77b3435276507cc8f2dc2569-Abstract.html

Chen, D., Chen, J., & Ye, W. (2024). Why groups matter: Necessity of group structures in attributions. In Proceedings of the 5th ACM International Conference on AI in Finance (pp. 63–71). Association for Computing Machinery. https://doi.org/10.1145/3677052.3698667

Müller, R., Schreyer, M., Sattarov, T., & Borth, D. (2022). RESHAPE: Explaining accounting anomalies in financial statement audits by enhancing SHapley Additive exPlanations. In Proceedings of the Third ACM International Conference on AI in Finance (pp. 174–182). Association for Computing Machinery. https://doi.org/10.1145/3533271.3561667

Kamalaruban, P., Pi, Y., Burrell, S., Drage, E., Skalski, P., Wong, J., & Sutton, D. (2024). Evaluating fairness in transaction fraud models: Fairness metrics, bias audits, and challenges. In Proceedings of the 5th ACM International Conference on AI in Finance (pp. 555–563). Association for Computing Machinery. https://doi.org/10.1145/3677052.3698666

Le Borgne, Y.-A., Siblini, W., Lebichot, B., & Bontempi, G. (2022). Reproducible machine learning for credit card fraud detection: Practical handbook. Université Libre de Bruxelles. https://github.com/Fraud-Detection-Handbook/fraud-detection-handbook

Dal Pozzolo, A., Caelen, O., Johnson, R. A., & Bontempi, G. (2015). Calibrating probability with undersampling for unbalanced classification. In 2015 IEEE Symposium Series on Computational Intelligence (pp. 159–166). IEEE. https://doi.org/10.1109/SSCI.2015.33

Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 785–794). Association for Computing Machinery. https://doi.org/10.1145/2939672.2939785

Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020). From local explanations to global understanding with explainable AI for trees. Nature Machine Intelligence, 2(1), 56–67. https://doi.org/10.1038/s42256-019-0138-9

Mothilal, R. K., Sharma, A., & Tan, C. (2020). Explaining machine learning classifiers through diverse counterfactual explanations. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (pp. 607–617). Association for Computing Machinery. https://doi.org/10.1145/3351095.3372850

Agarwal, C., Krishna, S., Saxena, E., Pawelczyk, M., Johnson, N., Puri, I., Zitnik, M., & Lakkaraju, H. (2022). OpenXAI: Towards a transparent evaluation of post hoc model explanations. Advances in Neural Information Processing Systems, 35. https://papers.nips.cc/paper_files/paper/2022/hash/65398a0eba88c9b4a1c38ae405b125ef-Abstract-Datasets_and_Benchmarks.html

Bhatt, U., Weller, A., & Moura, J. M. F. (2020). Evaluating and aggregating feature-based model explanations. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (pp. 3016–3022). International Joint Conferences on Artificial Intelligence Organization. https://doi.org/10.24963/ijcai.2020/417

Guidotti, R. (2024). Counterfactual explanations and how to find them: Literature review and benchmarking. Data Mining and Knowledge Discovery, 38(5), 2770–2824. https://doi.org/10.1007/s10618-022-00831-6

Alvarez-Melis, D., & Jaakkola, T. S. (2018). Towards robust interpretability with self-explaining neural networks. Advances in Neural Information Processing Systems, 31. https://papers.nips.cc/paper_files/paper/2018/hash/3e9f0fc9b2f89e043bc6233994dfcf76-Abstract.html

Grinsztajn, L., Oyallon, E., & Varoquaux, G. (2022). Why do tree-based models still outperform deep learning on typical tabular data? Advances in Neural Information Processing Systems, 35. https://papers.nips.cc/paper_files/paper/2022/hash/0378c7692da36807bdec87ab043cdadc-Abstract-Datasets_and_Benchmarks.html

Fisher, A., Rudin, C., & Dominici, F. (2019). All models are wrong, but many are useful: Learning a variable's importance by studying an entire class of prediction models simultaneously. Journal of Machine Learning Research, 20(177), 1–81. https://jmlr.org/papers/v20/18-760.html

Laugel, T., Lesot, M.-J., Marsala, C., Renard, X., & Detyniecki, M. (2019). The dangers of post-hoc interpretability: Unjustified counterfactual explanations. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (pp. 2801–2807). International Joint Conferences on Artificial Intelligence Organization. https://doi.org/10.24963/ijcai.2019/388

Lucic, A., Oosterhuis, H., Haned, H., & de Rijke, M. (2022). FOCUS: Flexible optimizable counterfactual explanations for tree ensembles. Proceedings of the AAAI Conference on Artificial Intelligence, 36(5), 5313–5322. https://doi.org/10.1609/aaai.v36i5.20468

Zhou, Z., Hooker, G., & Wang, F. (2021). S-LIME: Stabilized-LIME for model explanation. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (pp. 2429–2438). Association for Computing Machinery. https://doi.org/10.1145/3447548.3467274

Han, T., Srinivas, S., & Lakkaraju, H. (2022). Which explanation should I choose? A function approximation perspective to characterizing post hoc explanations. Advances in Neural Information Processing Systems, 35. https://papers.nips.cc/paper_files/paper/2022/hash/22b111819c74453837899689166c4cf9-Abstract-Conference.html

Aas, K., Jullum, M., & Løland, A. (2021). Explaining individual predictions when features are dependent: More accurate approximations to Shapley values. Artificial Intelligence, 298, Article 103502. https://doi.org/10.1016/j.artint.2021.103502

Faithfulness, Stability, Agreement, and Complexity of Post-hoc Explanations for Card Fraud Alerts: A Comparative Evaluation of SHAP, LIME, Permutation Importance, and Counterfactuals