You can statistically assess the predictive accuracies of two classification models using holdout sample predictions or repeated cross validation.
Software Author(s): The Mathworks; Inc. Tags: MATLAB compiler runtime math mathematics library. Edit Package. This discussion is only about MATLAB Compiler Runtime R2015a and the MATLAB Compiler Runtime R2015a package. If you have feedback for Chocolatey, please contact the Google Group.
- The
testcholdout
accepts holdout sample predicted labels from both classification models and the true labels. This function implements the asymptotic, exact, or mid-p version of McNemar’s test. If you specify misclassification costs,testcholdout
compares the models using a likelihood ratio or a chi-square test. - The
compareHoldout
object function accepts any two trained classification model objects in Statistics and Machine Learning Toolbox™, sets of holdout predictor data for both models, and corresponding true labels. Liketestcholdout
, this object function implements the asymptotic, exact, or mid-p version of McNemar’s test. If you specify misclassification costs,compareHoldout
compares the models using a likelihood ratio or a chi-square test. - The
testckfold
function accepts any two trained classification model objects or templates in Statistics and Machine Learning Toolbox, and repeatedly applies k-fold cross validation using two sets of out-of-sample predictor data and true labels. Then,testckfold
assesses the resulting accuracies using a t or an F test.