Discrimination versus Stability: An Empirical Comparison of Filter, Embedded, and Wrapper Feature Selection Methods for Consumer Credit Pre-Approval

Qiwen Zhao; Yue Xi

Discrimination versus Stability: An Empirical Comparison of Filter, Embedded, and Wrapper Feature Selection Methods for Consumer Credit Pre-Approval

Authors

Qiwen Zhao Computer Science, University of California San Diego, CA, USA
Yue Xi Information Systems, Northeastern University, WA, USA

Keywords:

feature selection, credit pre-approval, population stability, empirical comparison

Abstract

Consumer credit pre-approval requires selecting, from hundreds of candidate variables, a compact set of features that are both discriminative and stable across shifting applicant populations. This study compares three families of feature selection methods — filter criteria (information value, mutual information with minimum redundancy, and chi-square), embedded techniques (L1-regularized logistic regression, random-forest importance, and gradient-boosting gain), and a wrapper based on recursive feature elimination — on five public consumer-credit datasets. Each method is evaluated along three axes: discrimination, measured by the area under the ROC curve and the Kolmogorov–Smirnov statistic; stability, measured by a population stability index over temporal and resampled splits together with a selection stability index; and dimensionality-reduction efficiency. A fixed downstream classifier isolates the effect of selection from that of the learner. Across datasets, embedded methods retain discrimination within the run-to-run variation of the full feature pool, at a mean area under the curve of 0.768 against 0.767, while removing roughly three quarters of the variables and yielding the most stable selections. The wrapper attains competitive discrimination but the least stable selections at a markedly higher cost, and filter methods are fastest yet trail slightly. The choice among method families in pre-approval is governed less by discrimination than by stability and cost.

References

[1] Lessmann, S., Baesens, B., Seow, H.-V., & Thomas, L. C. (2015). Benchmarking state-of-the-art classification algorithms for credit scoring: An update of research. European Journal of Operational Research, 247(1), 124–136.

[2] Luo, Z. (2026). Fairness-Aware Credit Evaluation: Bias Detection and Mitigation Techniques for Inclusive Lending Practices. Journal of Sustainability, Policy, and Practice, 2(2), 78–89.

[3] Guyon, I., & Elisseeff, A. (2003). An introduction to variable and feature selection. Journal of Machine Learning Research, 3, 1157–1182.

[4] Jiang, Z., & Wang, M. (2025). Evaluation and Analysis of Chart Reasoning Accuracy in Multimodal Large Language Models: An Empirical Study on Influencing Factors. Pinnacle Academic Press Proceedings Series, 3, 43–58.

[5] Kozodoi, N., Lessmann, S., Papakonstantinou, K., Gatsoulis, Y., & Baesens, B. (2019). A multi-objective approach for profit-driven feature selection in credit scoring. Decision Support Systems, 120, 106–117.

[6] Li, Y., Zhao, F., & Hu, J. (2026). Identifying Cross-Market Risk Contagion Amplifiers via Graph Attention Networks: Empirical Evidence from US Financial Stress Periods. Journal of Computing Innovations and Applications, 4(1), 164–175.

[7] Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., & Bouchachia, A. (2014). A survey on concept drift adaptation. ACM Computing Surveys, 46(4), 1–37.

[8] Wang, M., Xiao, P., & Yu, M. (2026). Sparse, Dense, or Hybrid? Comparing Retrieval Strategies for Biomedical Question Answering with Retrieval-Augmented Generation. Journal of Science, Innovation & Social Impact, 2(2), 141–152.

[9] Peng, H., Long, F., & Ding, C. (2005). Feature selection based on mutual information: Criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8), 1226–1238.

[10] Tang, T., & Yu, M. (2024). A Comparative Empirical Study of Semantic Signal Enhancement Methods for User Interest Features in CTR Prediction: Applicability of TF-IDF Weighting, Sentence-BERT Embeddings, and LDA Topic Fusion. Journal of Computing Innovations and Applications, 2(1), 165–174.

[11] Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1), 267–288.

[12] Ming, Y., Fu, H., Jiang, Y., & Yu, H. (2020). Variational Bayesian Group-Level Sparsification for Knowledge Distillation. IEEE Access, 8, 126628–126636.

[13] Ming, Y., & Fu, H. (2020, July). Variational Bayesian sparsification for distillation compression. In 2020 IEEE International Conference on Multimedia and Expo (ICME) (pp. 1–6). IEEE.

[14] Fu, H., Ming, Y., Jiang, Y., & Fan, C. (2018, August). Effective SVD-based deep network compression for automatic speech recognition. In International Workshop on Face and Facial Expression Recognition from Real World Videos (pp. 35–47). Cham: Springer International Publishing.

[15] Kohavi, R., & John, G. H. (1997). Wrappers for feature subset selection. Artificial Intelligence, 97(1–2), 273–324.

[16] Guyon, I., Weston, J., Barnhill, S., & Vapnik, V. (2002). Gene selection for cancer classification using support vector machines. Machine Learning, 46(1–3), 389–422.

[17] Zhang, R., Fan, C., Ming, Y., Fu, H., & Meng, X. (2020, October). An Effective Hierarchical Resolution Learning Method for Low-Resolution Targets Tracking. In 2020 IEEE International Conference on Image Processing (ICIP) (pp. 2076–2080). IEEE.

[18] Han, M., Jin, Z., & Zou, D. (2026). Comparative Evaluation of Self-Supervised Pretraining Strategies for Few-Shot Medical Image Analysis. Artificial Intelligence and Machine Learning Review, 7(2), 23–42.

[19] Maldonado, S., Bravo, C., López, J., & Pérez, J. (2017). Integrated framework for profit-based feature selection and SVM classification in credit scoring. Decision Support Systems, 104, 113–121.

[20] Cao, H., Hu, J., & Luo, C. (2025). Behavioural Feature Analysis for Anomalous Click Detection in Mobile Advertising Environments: Toward In-App Browser-Specific Detection. Journal of Computing Innovations and Applications, 3(2), 96–105.

[21] Hu, J., & Long, X. (2024). Graph Learning-Based Behavioral Detection for Software Supply Chain Attacks. Journal of Advanced Computing Systems, 4(4), 49–60.

[22] Chen, Z., & Wang, M. (2025). Evaluating the Quality of Large Language Model-Generated Explanations in Recommendation Tasks: A Multi-Dimensional Comparative Analysis. Journal of Global Engineering Review, 3(1), 54–63.

[23] Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.

[24] Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 785–794). Association for Computing Machinery.

[25] Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30, 4765–4774.

[26] Tang, T., Fu, X., & Luo, C. (2026). An Empirical Comparison of High-Order Feature Interaction Operators for Conversion Rate Prediction in Sparse, High-Cardinality Message-Ads Traffic: Accuracy, Efficiency, and Offline–Online Consistency. Journal of Science, Innovation & Social Impact, 2(3), 12–22.

[27] Tang, T., & Yu, M. (2024). A Comparative Evaluation of LLM-Generated Semantic Tags versus Classical Text Features (TF-IDF, LDA, BERT Embeddings) for User-Interest Enrichment in Short-Video Recommendation. Artificial Intelligence and Machine Learning Review, 5(1), 129–140.

[28] Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16, 321–357.

[29] Hu, J., Wang, X., & Lai, J. (2026). Benchmarking Learned Cardinality Estimation Techniques for Analytical Query Processing in Data Warehouses. Journal of Computer Technology and Applied Mathematics, 3(3), 1–8.

[30] Nogueira, S., Sechidis, K., & Brown, G. (2018). On the stability of feature selection algorithms. Journal of Machine Learning Research, 18(174), 1–54.

[31] Guo, Y., Li, Y., Chen, S., Wu, Y., Poll, O., Ren, Z., ... & Baran, P. S. (2025). Scalable total synthesis of saxitoxin and related natural products. Nature, 646(8084), 351–357.

[32] Guo, Y., Li, Y., Chen, S., Poll, O., Wu, Y., Prier, C., ... & Baran, P. (2025). Scalable, convergent total synthesis of (+)-saxitoxin and related natural products.

[33] Li, Y., Buchberger, B., Liu, Y., Capone, C., Adak, T., Ojha, S., ... & Baran, P. S. (2026). General Access to C-Glycosides via Redox-Neutral Radical Cross-Coupling of Glycohydrazides.

[34] Brown, G., Pocock, A., Zhao, M.-J., & Luján, M. (2012). Conditional likelihood maximisation: A unifying framework for information theoretic feature selection. Journal of Machine Learning Research, 13, 27–66.

[35] Louppe, G., Wehenkel, L., Sutera, A., & Geurts, P. (2013). Understanding variable importances in forests of randomized trees. Advances in Neural Information Processing Systems, 26, 431–439.

[36] Weng, H., Yang, Q., Fu, H., Pan, H., & Lv, X. (2026). When Retrieval Hurts Code Completion: A Diagnostic Study of Stale Repository Context. arXiv preprint arXiv:2605.14478.

[37] Wang, M., & Zhu, L. (2024). Linguistic analysis of verb tense usage patterns in computer science paper abstracts. Academia Nexus Journal, 3(3).

[38] Wang, M., Jiang, Z., & Zhou, S. (2024). Cross-cultural semantic differences in emoji usage on social media platforms. Journal of Advanced Computing Systems, 4(5), 55–66.

[39] Zhao, F., & Tang, T. (2026). AI-Based Sentiment Analysis for Stock Market Prediction: A Systematic Literature Review. Journal of Sustainability, Policy, and Practice, 2(3), 115–124.

[40] Chen, Y., & Tang, T. (2026). Evaluating Prompt Engineering Strategies for Few-Shot Cyber Threat Intelligence Entity and Relation Extraction from Multi-Source Reports. Journal of Science, Innovation & Social Impact, 2(2), 153–164.

[41] Fu, X., Tang, T., & Luo, C. (2026). An Empirical Comparison of ReAct, Reflexion, Plan-and-Solve, and Tree-of-Thought Planning Strategies on Financial Question Answering and Numerical Reasoning Tasks. Journal of Science, Innovation & Social Impact, 2(3), 23–34.

[42] Maldonado, S., Pérez, J., & Bravo, C. (2017). Cost-based feature selection for support vector machines: An application in credit scoring. European Journal of Operational Research, 261(2), 656–665.

[43] Luo, C., & Wang, X. (2026). Latency–Throughput Tradeoffs of ONNX Runtime, TensorRT-LLM, vLLM, and Triton: An Empirical Comparison on 1B–3B Parameter LLM Inference. Spectrum of Research, 6(1).

[44] Long, L., & Hu, J. (2026). Multi-Objective Particle Swarm Optimization for Site Selection and Policy Subsidy Maximization of Foreign Renewable Energy Enterprises in the United States. Artificial Intelligence and Machine Learning Review, 7(2), 54–69.

[45] Long, X., Hu, J., & Ling, Z. (2026). A Comparative Analysis of Telemetry-Driven Anomaly Detection Approaches for Dual-Purpose Operational and Security Optimization in Edge Computing Infrastructure. Journal of Computing Innovations and Applications, 4(1), 79–88.

[46] Chen, Y., & Hu, J. (2026). Graph Neural Network-Based Cascading Disruption Path Identification in Multi-Tier Rare Earth Processing Networks. Journal of Global Engineering Review, 4(1), 99–112.

Discrimination versus Stability: An Empirical Comparison of Filter, Embedded, and Wrapper Feature Selection Methods for Consumer Credit Pre-Approval