Hardev Ranglani, EXL Service Inc, USA

Traditional stacking ensembles in Machine Learning aggregate predictions from multiple models to improve accuracy, but they often fail to address the residual errors left by base models. This paper introduces ResidualAware Stacking (RAS), a novel approach that trains additional models to predict residuals (errors) of base models, creating a second layer of predictions. These, combined with the original base model predictions, are used to train a meta-model for the final output. This meta-model leverages both the original predictions and residual corrections to produce the final output. We demonstrate the improved accuracy and robustness of this technique by applying it on various regression datasets and comparing their performance against traditional models. This method highlights the potential of residual modeling in enhancing ensemble learning.

Stacking, Ensemble Methods, Residual Models, Meta Learners, Regression Modeling.