Easy and comprehensive assessment of predictive power, with support for neuroimaging features

Related tags

Deep Learningmachine-learningscikit-learncross-validationreporteasy-to-useneuroimagingpattern-recognitionnilearntractographystructural-imaginganatomical-mrifunctional-connectivitytract-based-statistics
Overview

docs/logo_neuropredict.png

https://landscape.io/github/raamana/neuropredict/master/landscape.svg?style=flat https://api.codacy.com/project/badge/Grade/501e560b8a424562a1b8f7cd2f3cadfe https://travis-ci.org/raamana/neuropredict.svg?branch=master

Documentation: https://raamana.github.io/neuropredict/

News

  • As of v0.6, neuropredict now supports regression applications i.e. predicting continuous targets (in addition to categorical classes), as well as allow you to regress out covariates / confounds within the nested-CV (following all the best practices). Utilizing this feature requires the input datasets be specified in the pyradigm data structures: code @ https://github.com/raamana/pyradigm, docs @ https://raamana.github.io/pyradigm/. Check the changelog below for more details.

Older news

  • neuropredict can handle missing data now (that are encoded with numpy.NaN). This is done respecting the cross-validation splits without any data leakage.

Overview

On a high level,

roleofneuropredict

On a more detailed level,

roleofneuropredict

Long term goals

neuropredict, the tool, is part of a broader initiative described below to develop easy, comprehensive and standardized predictive analysis:

longtermgoals

Citation

If neuropredict helped you in your research in one way or another, please consider citing one or more of the following, which were essential building blocks of neuropredict: - Pradeep Reddy Raamana. (2017, November 18). neuropredict: easy machine learning and standardized predictive analysis of biomarkers (Version 0.4.5). Zenodo. http://doi.org/10.5281/zenodo.1058993 - Raamana et al, (2017), Python class defining a machine learning dataset ensuring key-based correspondence and maintaining integrity, Journal of Open Source Software, 2(17), 382, doi:10.21105/joss.00382

Change Log - version 0.6

  • Major feature: Ability to predict continuous variables (regression)
  • Major feature: Ability to handle confounds (regress them out, augmenting etc)
  • Redesigned the internal structure for easier extensibility
  • New CVResults class for easier management of a wealth of outputs generated in the Classification and Regression workflows
  • API access is refreshed and easier

Change Log - version 0.5.2

  • Imputation of missing values
  • Additional classifiers such as XGBoost, Decision Trees
  • Better internal code structure
  • Lot more tests
  • More precise tests, as we vary number of classes wildly in test suites
  • several bug fixes and enhancements
  • More cmd line options such as --print_options from a previous run
Comments
  • ImportError: cannot import name 'MultiDatasetClassify'

    ImportError: cannot import name 'MultiDatasetClassify'

    Hi,

    When I try to run neuropredict on the command line using neuropredict -m ~/NeuroPredict/meta_data.csv -o ~/NeuroPredict/results/ --user_feature_paths ~/NeuroPredict/face/ ~/NeuroPredict/cnlg/ ~/NeuroPredict/glmn/ ~/NeuroPredict/cnsr/ ~/NeuroPredict/bimndir/ ~/NeuroPredict/bimnemo/ ~/NeuroPredict/msep/ -t 0.8 -n 20 -g 'light' I get the error that ImportError: cannot import name 'MultiDatasetClassify' from 'pyradigm.multiple' It appears that the multiple.py file that I've downloaded does not match the current version that is in your pyradigm repository. I installed it just now though, using pip3 install neuropredict. Do you know why I am getting this problem?

    Thanks

    opened by cpappas18 13
  • Add new classifier: need of probability output?

    Add new classifier: need of probability output?

    Hi @raamana ,

    I would like to add LinearSVC classifier based on liblinear implementation. Does the current implementation of neuropredict need that predictions are based on probability values? Because, LinearSVC doesn't allow prediction of probabilities.

    opened by mattvan83 12
  • Implement option to let user select number of features to select

    Implement option to let user select number of features to select

    To do:

    • [x] add an argument: --num_features
    • [x] describe the argument thoroughly in help text
    • [x] checks on the input types
    • [x] checks on the input ranges
    • [ ] checks on whether all the feature sets have the same number of features
    • [ ] make the new argument interface with all the other methods, if need be.
    • [ ] add option to pass on subset of feature names to visualize.feature_importance_map

    We can add/remove as we realize them.

    enhancement 
    opened by raamana 12
  • How to add multiple features?

    How to add multiple features?

    Hi Pradeep,

    I can see the file structure for adding a single feature is:

    subject1/features.txt subject2/features.txt subject3/features.txt subject4/features.txt

    Where the feature is a vector in each case. To clarify, if I wanted to add a second feature, would this be a second column in the features.txt file of each participant?

    Thanks,

    John

    question 
    opened by johnaeanderson 10
  • Classification blocked with multiple 1-dimensional features

    Classification blocked with multiple 1-dimensional features

    Hello,

    I have problems when using multiple 1-D features with the following command:

          neuropredict -m meta_data.csv \
      -d mask_WMpet_av45_early.ero1.25mm.csv mask_WMpet_av45_early.ero1.5mm.csv \
      -o outdir -t 0.8 -n 250 -k 'all' \
     --gs_level 'exhaustive' --classifier "LinearSVC"

    The CV trial were well launched for the first subgroup CN,MCI for both 1-D features but it seemed that at the end of CV trials the process got stuck at this stage:

    >

>  Python 3.7.3
> > SGE recognized, job set up with 40 slots.
> > Positive class inferred for AUC calculation: CN
> > Running neuropredict 0.5+34.g220af55.dirty
> > 
> > Requested features for analysis:
> > get_data_matrix from /netapp/vol2_agewell/pro/IMAP/imap_mvh/CAT12/pet/Analyses/ML/All/CN_vs_MCI+AD/av45_early/pons/start0_dur4/mask_WMpet_av45_early.ero1.25mm.csv
> > get_data_matrix from /netapp/vol2_agewell/pro/IMAP/imap_mvh/CAT12/pet/Analyses/ML/All/CN_vs_MCI+AD/av45_early/pons/start0_dur4/mask_WMpet_av45_early.ero1.5mm.csv
> > Ignoring imputation strategy chosen, as no missing data were found!
> > 
> > Data import is done.
> > 
> > 
> > Requested processing for the following subgroups:
> > DIS,CN
> > 
> > --------------------------------------------------------------------------------
> > Processing subgroup : DIS,CN (1/1)
> > --------------------------------------------------------------------------------
> > SGE recognized, job set up with 40 slots.
> > Training percentage      : 0.8
> > Number of CV repetitions : 250
> > Classifier chosen        : linearsvc
> > Feature selection chosen : variancethreshold
> > Level of grid search     : exhaustive
> > Number of processors     : 40
> > Saving the results to 
> >   /netapp/vol2_agewell/pro/IMAP/imap_mvh/CAT12/pet/Analyses/ML/All/CN_vs_MCI+AD/av45_early/pons/start0_dur4/linearsvc/binary_WMmasks/CN_DIS
> > 
> > -------------------------
> > All datasets contain:
> >  
> > 101 samples, 2 classes, 1 features
> > Class  CN : 71 samples
> > Class DIS : 30 samples
> > -------------------------
> > 
> > Estimated chance accuracy : 0.500
> > 
> > Different classes in the training set are stratified to match the smallest class!
> > Parallelizing the repetitions of CV with 40 processes ...
> > CV trial 52     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6525 	 weighted AUC: 0.7482
> > CV trial 78     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6631 	 weighted AUC: 0.8227
> > CV trial 50     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5993 	 weighted AUC: 0.6596
> > CV trial 72     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6082 	 weighted AUC: 0.6596
> > CV trial 46     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7021
> > CV trial 58     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8475
> > CV trial 38     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 	 weighted AUC: 0.7518
> > CV trial 16     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 	 weighted AUC: 0.7801
> > CV trial 12     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8794
> > CV trial 0      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6738 	 weighted AUC: 0.6950
> > CV trial 68     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7057 	 weighted AUC: 0.8369
> > CV trial 14     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8794
> > CV trial 22     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8333
> > CV trial 6      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6631 	 weighted AUC: 0.7234
> > CV trial 48     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8298
> > CV trial 10     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7624
> > CV trial 64     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8191
> > CV trial 20     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8688
> > CV trial 4      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8688
> > CV trial 56     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 	 weighted AUC: 0.8262
> > CV trial 70     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8440
> > CV trial 74     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7553
> > CV trial 36     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8050
> > CV trial 2      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.9043
> > CV trial 54     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8830
> > CV trial 24     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 	 weighted AUC: 0.9043
> > CV trial 76     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7340 	 weighted AUC: 0.9220
> > CV trial 42     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 	 weighted AUC: 0.8759
> > CV trial 26     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.9078
> > CV trial 62     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8103
> > CV trial 66     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6418 	 weighted AUC: 0.7624
> > CV trial 60     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8511 	 weighted AUC: 0.8972
> > CV trial 72     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6294 	 weighted AUC: 0.6418
> > CV trial 40     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.9149
> > CV trial 52     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7340
> > CV trial 16     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7411
> > CV trial 46     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6720 	 weighted AUC: 0.6489
> > CV trial 38     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6507 	 weighted AUC: 0.7376
> > CV trial 78     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 	 weighted AUC: 0.8014
> > CV trial 50     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6099 	 weighted AUC: 0.6454
> > CV trial 58     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8298
> > CV trial 6      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.6879
> > CV trial 14     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8688
> > CV trial 36     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6844 	 weighted AUC: 0.8156
> > CV trial 18     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8014
> > CV trial 0      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5904 	 weighted AUC: 0.6454
> > CV trial 20     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8582
> > CV trial 68     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7677 	 weighted AUC: 0.7979
> > CV trial 4      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8511 	 weighted AUC: 0.8440
> > CV trial 74     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.7358
> > CV trial 56     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8050
> > CV trial 34     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5816 	 weighted AUC: 0.7518
> > CV trial 12     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8723
> > CV trial 66     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7376
> > CV trial 70     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6844 	 weighted AUC: 0.8227
> > CV trial 64     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 	 weighted AUC: 0.8121
> > CV trial 24     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8794
> > CV trial 76     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7234 	 weighted AUC: 0.9007
> > CV trial 79     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5904 	 weighted AUC: 0.7021
> > CV trial 53     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7766
> > CV trial 2      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8936
> > CV trial 73     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.9255
> > CV trial 54     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8759
> > CV trial 62     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8014
> > CV trial 17     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 	 weighted AUC: 0.7624
> > CV trial 22     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8191
> > CV trial 39     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6631 	 weighted AUC: 0.7695
> > CV trial 60     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8191 	 weighted AUC: 0.9007
> > CV trial 51     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8404 	 weighted AUC: 0.8227
> > CV trial 26     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.9007
> > CV trial 42     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8546
> > CV trial 48     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8121
> > CV trial 59     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8014
> > CV trial 69     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7695
> > CV trial 47     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7695
> > CV trial 21     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 	 weighted AUC: 0.9220
> > CV trial 28     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8316 	 weighted AUC: 0.9007
> > CV trial 37     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8475
> > CV trial 57     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8582
> > CV trial 1      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8440
> > CV trial 40     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.9149
> > CV trial 30     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8582
> > CV trial 18     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 	 weighted AUC: 0.7589
> > CV trial 65     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7837
> > CV trial 75     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8617
> > CV trial 25     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7943
> > CV trial 13     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8121
> > CV trial 79     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6525 	 weighted AUC: 0.6915
> > CV trial 61     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 	 weighted AUC: 0.7766
> > CV trial 71     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8156
> > CV trial 32     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8316 	 weighted AUC: 0.9291
> > CV trial 3      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8440
> > CV trial 53     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7411
> > CV trial 55     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8298
> > CV trial 15     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8582
> > CV trial 23     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7872
> > CV trial 34     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5284 	 weighted AUC: 0.7376
> > CV trial 59     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.7713
> > CV trial 43     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.7766
> > CV trial 77     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7305
> > CV trial 49     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 	 weighted AUC: 0.8528
> > CV trial 27     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8191
> > CV trial 73     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7340 	 weighted AUC: 0.8936
> > CV trial 39     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5798 	 weighted AUC: 0.7305
> > CV trial 47     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7234
> > CV trial 69     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7199
> > CV trial 21     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7021 	 weighted AUC: 0.9149
> > CV trial 44     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8369
> > CV trial 65     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.7943
> > CV trial 41     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 	 weighted AUC: 0.7092
> > CV trial 80     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6011 	 weighted AUC: 0.7305
> > CV trial 51     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6738 	 weighted AUC: 0.8121
> > CV trial 57     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7128 	 weighted AUC: 0.8369
> > CV trial 75     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8156
> > CV trial 1      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8191
> > CV trial 63     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7340 	 weighted AUC: 0.8582
> > CV trial 82     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8475
> > CV trial 3      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8050
> > CV trial 23     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7270
> > CV trial 61     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.7553
> > CV trial 55     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7340 	 weighted AUC: 0.8014
> > CV trial 43     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6418 	 weighted AUC: 0.7270
> > CV trial 77     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7234
> > CV trial 25     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6613 	 weighted AUC: 0.7872
> > CV trial 84     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.9362
> > CV trial 30     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 	 weighted AUC: 0.8333
> > CV trial 19     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8298
> > CV trial 37     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8298
> > CV trial 88     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8121
> > CV trial 96     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6507 	 weighted AUC: 0.7270
> > CV trial 86     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8759
> > CV trial 13     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.7837
> > CV trial 41     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6206 	 weighted AUC: 0.6809
> > CV trial 5      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8617 	 weighted AUC: 0.9043
> > CV trial 102    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8333
> > CV trial 80     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6011 	 weighted AUC: 0.7163
> > CV trial 92     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.9220
> > CV trial 15     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8227
> > CV trial 104    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6436 	 weighted AUC: 0.7660
> > CV trial 35     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8333
> > CV trial 106    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5585 	 weighted AUC: 0.7163
> > CV trial 108    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6418 	 weighted AUC: 0.7163
> > CV trial 110    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.6844
> > CV trial 94     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8617 	 weighted AUC: 0.9255
> > CV trial 98     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8652
> > CV trial 82     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8546
> > CV trial 100    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7890 	 weighted AUC: 0.8652
> > CV trial 49     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8440
> > CV trial 63     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7234 	 weighted AUC: 0.8475
> > CV trial 112    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8475
> > CV trial 90     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8511 	 weighted AUC: 0.8511
> > CV trial 118    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6525 	 weighted AUC: 0.7411
> > CV trial 27     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.7943
> > CV trial 44     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8014
> > CV trial 124    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8085
> > CV trial 84     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7553 	 weighted AUC: 0.9255
> > CV trial 110    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.4645 	 weighted AUC: 0.6454
> > CV trial 31     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.9149
> > CV trial 96     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6401 	 weighted AUC: 0.7021
> > CV trial 108    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.6738
> > CV trial 126    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8582
> > CV trial 81     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6631 	 weighted AUC: 0.7589
> > CV trial 116    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.9184
> > CV trial 104    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7163
> > CV trial 106    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5585 	 weighted AUC: 0.7163
> > CV trial 67     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7996 	 weighted AUC: 0.9007
> > CV trial 92     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.9043
> > CV trial 88     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7695
> > CV trial 120    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 	 weighted AUC: 0.7695
> > CV trial 122    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 	 weighted AUC: 0.8511
> > CV trial 130    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 	 weighted AUC: 0.7376
> > CV trial 19     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8085
> > CV trial 102    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8262
> > CV trial 98     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8688
> > CV trial 112    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7234 	 weighted AUC: 0.8298
> > CV trial 114    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8652
> > CV trial 86     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8475
> > CV trial 35     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8121
> > CV trial 100    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7057 	 weighted AUC: 0.8475
> > CV trial 71     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7996
> > CV trial 118    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6525 	 weighted AUC: 0.6986
> > CV trial 94     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8901
> > CV trial 83     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6738 	 weighted AUC: 0.8475
> > CV trial 128    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8369
> > CV trial 85     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 	 weighted AUC: 0.7695
> > CV trial 17     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6631 	 weighted AUC: 0.7447
> > CV trial 90     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8404 	 weighted AUC: 0.8511
> > CV trial 45     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8617 	 weighted AUC: 0.8191
> > CV trial 111    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6188 	 weighted AUC: 0.8333
> > CV trial 124    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.7713
> > CV trial 97     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8582
> > CV trial 109    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8652
> > CV trial 105    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 	 weighted AUC: 0.8369
> > CV trial 120    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.7340
> > CV trial 113    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 	 weighted AUC: 0.6702
> > CV trial 87     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.9362
> > CV trial 81     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6011 	 weighted AUC: 0.7305
> > CV trial 136    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5904 	 weighted AUC: 0.8050
> > CV trial 132    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6738 	 weighted AUC: 0.7589
> > CV trial 116    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8901
> > CV trial 99     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8972
> > CV trial 119    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7021 	 weighted AUC: 0.8759
> > CV trial 103    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.9113
> > CV trial 95     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.6879
> > CV trial 134    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 	 weighted AUC: 0.8014
> > CV trial 130    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.6986
> > CV trial 93     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8121
> > CV trial 125    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 	 weighted AUC: 0.7482
> > CV trial 101    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8652
> > CV trial 91     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7021 	 weighted AUC: 0.9255
> > CV trial 85     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7482
> > CV trial 128    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8262
> > CV trial 107    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 	 weighted AUC: 0.7801
> > CV trial 111    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6188 	 weighted AUC: 0.7979
> > CV trial 5      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8865
> > CV trial 31     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.9220
> > CV trial 138    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8901
> > CV trial 114    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8333
> > CV trial 121    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8546
> > CV trial 113    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.6312
> > CV trial 97     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8546
> > CV trial 28     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8528 	 weighted AUC: 0.8617
> > CV trial 89     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6418 	 weighted AUC: 0.6702
> > CV trial 109    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8688
> > CV trial 122    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8422
> > CV trial 87     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.9149
> > CV trial 105    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8191
> > CV trial 83     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8475
> > CV trial 95     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6826 	 weighted AUC: 0.6879
> > CV trial 140    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8546
> > CV trial 7      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8830
> > CV trial 125    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.6844
> > CV trial 142    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8723
> > CV trial 136    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 	 weighted AUC: 0.7908
> > CV trial 103    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8759
> > CV trial 119    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7021 	 weighted AUC: 0.8262
> > CV trial 99     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8759
> > CV trial 132    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6436 	 weighted AUC: 0.7234
> > CV trial 129    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7021
> > CV trial 91     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6915 	 weighted AUC: 0.9184
> > CV trial 126    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8493
> > CV trial 150    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.7376
> > CV trial 93     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6011 	 weighted AUC: 0.8014
> > CV trial 144    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.7872
> > CV trial 148    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.7323
> > CV trial 146    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.7624
> > CV trial 107    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6294 	 weighted AUC: 0.7411
> > CV trial 152    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8723
> > CV trial 101    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6950 	 weighted AUC: 0.8617
> > CV trial 115    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7234 	 weighted AUC: 0.9007
> > CV trial 154    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8759
> > CV trial 134    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6401 	 weighted AUC: 0.8014
> > CV trial 29     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8582
> > CV trial 158    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7128 	 weighted AUC: 0.8121
> > CV trial 166    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7340 	 weighted AUC: 0.7872
> > CV trial 137    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7234
> > CV trial 168    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7057
> > CV trial 160    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8440
> > CV trial 123    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8121
> > CV trial 117    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8652
> > CV trial 164    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8369
> > CV trial 45     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6950 	 weighted AUC: 0.7979
> > CV trial 170    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6011 	 weighted AUC: 0.7766
> > CV trial 150    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.7057
> > CV trial 7      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8723
> > CV trial 121    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8050
> > CV trial 174    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8298
> > CV trial 176    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8369
> > CV trial 162    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 	 weighted AUC: 0.7660
> > CV trial 156    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.7908
> > CV trial 127    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8014
> > CV trial 8      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8723 	 weighted AUC: 0.9326
> > CV trial 172    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6117 	 weighted AUC: 0.7589
> > CV trial 144    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7553 	 weighted AUC: 0.7589
> > CV trial 148    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5691 	 weighted AUC: 0.7021
> > CV trial 129    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.4965 	 weighted AUC: 0.6809
> > CV trial 180    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.8121
> > CV trial 140    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8191
> > CV trial 146    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.7624
> > CV trial 142    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8652
> > CV trial 154    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8475
> > CV trial 115    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6915 	 weighted AUC: 0.8865
> > CV trial 29     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7340 	 weighted AUC: 0.8227
> > CV trial 166    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7021 	 weighted AUC: 0.7553
> > CV trial 89     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6418 	 weighted AUC: 0.6348
> > CV trial 133    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8298
> > CV trial 168    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.6702
> > CV trial 184    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5780 	 weighted AUC: 0.7589
> > CV trial 137    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6418 	 weighted AUC: 0.6950
> > CV trial 158    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.7837
> > CV trial 152    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7234 	 weighted AUC: 0.8262
> > CV trial 151    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5479 	 weighted AUC: 0.7305
> > CV trial 131    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.9255
> > CV trial 160    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8227
> > CV trial 123    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 	 weighted AUC: 0.7766
> > CV trial 32     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.0709
> > CV trial 135    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.7713
> > CV trial 170    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6011 	 weighted AUC: 0.7270
> > CV trial 188    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.6454
> > CV trial 145    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7234 	 weighted AUC: 0.7553
> > CV trial 117    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7677 	 weighted AUC: 0.8369
> > CV trial 156    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.7837
> > CV trial 127    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7890
> > CV trial 186    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 	 weighted AUC: 0.7979
> > CV trial 178    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.7624
> > CV trial 141    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7340 	 weighted AUC: 0.8511
> > CV trial 147    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 	 weighted AUC: 0.8759
> > CV trial 180    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6418 	 weighted AUC: 0.7979
> > CV trial 174    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8085
> > CV trial 149    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8582
> > CV trial 162    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6507 	 weighted AUC: 0.7163
> > CV trial 172    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5904 	 weighted AUC: 0.7270
> > CV trial 184    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5674 	 weighted AUC: 0.7340
> > CV trial 151    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6826 	 weighted AUC: 0.7163
> > CV trial 176    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7677 	 weighted AUC: 0.7872
> > CV trial 194    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6525 	 weighted AUC: 0.7092
> > CV trial 188    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.4645 	 weighted AUC: 0.5993
> > CV trial 169    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 	 weighted AUC: 0.8688
> > CV trial 155    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 	 weighted AUC: 0.7872
> > CV trial 171    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 	 weighted AUC: 0.7766
> > CV trial 159    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6223 	 weighted AUC: 0.7482
> > CV trial 167    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8511
> > CV trial 153    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8652
> > CV trial 133    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7465 	 weighted AUC: 0.7766
> > CV trial 143    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8617
> > CV trial 198    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8333
> > CV trial 161    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8475
> > CV trial 202    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.6950
> > CV trial 182    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.8333
> > CV trial 164    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6844 	 weighted AUC: 0.8191
> > CV trial 186    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6401 	 weighted AUC: 0.7411
> > CV trial 8      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8830 	 weighted AUC: 0.0674
> > CV trial 157    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 	 weighted AUC: 0.9574
> > CV trial 141    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7021 	 weighted AUC: 0.8617
> > CV trial 135    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6631 	 weighted AUC: 0.7695
> > CV trial 145    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7340 	 weighted AUC: 0.7199
> > CV trial 190    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7553
> > CV trial 181    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.7979
> > CV trial 196    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7766
> > CV trial 33     feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8103 	 weighted AUC: 0.8936
> > CV trial 192    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8422 	 weighted AUC: 0.8546
> > CV trial 138    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7784 	 weighted AUC: 0.8901
> > CV trial 200    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.9468
> > CV trial 149    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8546
> > CV trial 173    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8475
> > CV trial 177    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8014
> > CV trial 178    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7465 	 weighted AUC: 0.7340
> > CV trial 204    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8404
> > CV trial 206    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.7872
> > CV trial 163    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8085
> > CV trial 185    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8298
> > CV trial 171    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7801
> > CV trial 202    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6206 	 weighted AUC: 0.6950
> > CV trial 169    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8440
> > CV trial 167    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8369
> > CV trial 153    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8369
> > CV trial 131    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 	 weighted AUC: 0.9326
> > CV trial 159    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6950 	 weighted AUC: 0.7340
> > CV trial 194    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6418 	 weighted AUC: 0.6879
> > CV trial 187    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 	 weighted AUC: 0.7553
> > CV trial 155    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6738 	 weighted AUC: 0.7660
> > CV trial 212    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.8546
> > CV trial 198    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6826 	 weighted AUC: 0.8191
> > CV trial 189    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8794
> > CV trial 165    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8652
> > CV trial 157    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7340 	 weighted AUC: 0.9397
> > CV trial 208    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8546
> > CV trial 139    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7855
> > CV trial 190    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7305
> > CV trial 214    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5780 	 weighted AUC: 0.6879
> > CV trial 181    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7128 	 weighted AUC: 0.7660
> > CV trial 210    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8121
> > CV trial 143    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7553 	 weighted AUC: 0.8404
> > CV trial 9      feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 	 weighted AUC: 0.8511
> > CV trial 196    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6738 	 weighted AUC: 0.7518
> > CV trial 182    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8227
> > CV trial 147    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.8582
> > CV trial 175    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.9113
> > CV trial 216    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7943
> > CV trial 173    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8440
> > CV trial 177    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.7979
> > CV trial 206    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 	 weighted AUC: 0.7518
> > CV trial 179    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8121
> > CV trial 204    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8440
> > CV trial 218    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8759
> > CV trial 185    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8014
> > CV trial 226    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6312 	 weighted AUC: 0.8227
> > CV trial 224    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8475
> > CV trial 220    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.7801
> > CV trial 195    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7872
> > CV trial 200    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8511 	 weighted AUC: 0.9291
> > CV trial 163    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7465 	 weighted AUC: 0.7908
> > CV trial 161    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8156
> > CV trial 230    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6401 	 weighted AUC: 0.8227
> > CV trial 199    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8050
> > CV trial 212    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6613 	 weighted AUC: 0.8262
> > CV trial 187    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7447
> > CV trial 232    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6844 	 weighted AUC: 0.7837
> > CV trial 207    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.4752 	 weighted AUC: 0.6064
> > CV trial 33     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.0975
> > CV trial 228    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.8440
> > CV trial 203    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 	 weighted AUC: 0.7766
> > CV trial 139    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6525 	 weighted AUC: 0.7624
> > CV trial 183    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 	 weighted AUC: 0.8262
> > CV trial 208    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8404
> > CV trial 165    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8191
> > CV trial 234    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.9468
> > CV trial 214    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6206 	 weighted AUC: 0.6631
> > CV trial 216    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7589
> > CV trial 191    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8511
> > CV trial 240    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8440
> > CV trial 236    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.8262
> > CV trial 210    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.7943
> > CV trial 242    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8582
> > CV trial 224    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7553 	 weighted AUC: 0.7943
> > CV trial 189    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 	 weighted AUC: 0.8387
> > CV trial 220    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7163
> > CV trial 213    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6950 	 weighted AUC: 0.8156
> > CV trial 238    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6844 	 weighted AUC: 0.8085
> > CV trial 218    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8582
> > CV trial 230    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6401 	 weighted AUC: 0.7730
> > CV trial 226    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6312 	 weighted AUC: 0.8014
> > CV trial 179    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8014
> > CV trial 201    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6507 	 weighted AUC: 0.8298
> > CV trial 207    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.4645 	 weighted AUC: 0.5816
> > CV trial 246    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 	 weighted AUC: 0.8121
> > CV trial 248    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6117 	 weighted AUC: 0.7163
> > CV trial 195    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.7518
> > CV trial 199    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.7837
> > CV trial 183    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6525 	 weighted AUC: 0.8298
> > CV trial 244    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 	 weighted AUC: 0.8298
> > CV trial 203    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.7270
> > CV trial 209    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7340 	 weighted AUC: 0.8511
> > CV trial 205    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5691 	 weighted AUC: 0.7801
> > CV trial 217    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.9149
> > CV trial 240    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.8191
> > CV trial 221    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 	 weighted AUC: 0.9291
> > CV trial 225    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 	 weighted AUC: 0.8262
> > CV trial 232    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6436 	 weighted AUC: 0.7660
> > CV trial 211    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8262
> > CV trial 234    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 	 weighted AUC: 0.9326
> > CV trial 236    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7943
> > CV trial 246    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7730
> > CV trial 213    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8404 	 weighted AUC: 0.8050
> > CV trial 209    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 	 weighted AUC: 0.8316
> > CV trial 219    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6631 	 weighted AUC: 0.7305
> > CV trial 248    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5071 	 weighted AUC: 0.7092
> > CV trial 201    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6613 	 weighted AUC: 0.7979
> > CV trial 67     feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7163 	 weighted AUC: 0.8617
> > CV trial 228    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8121
> > CV trial 9      feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8262
> > CV trial 217    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8830
> > CV trial 215    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 	 weighted AUC: 0.9255
> > CV trial 244    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8121
> > CV trial 175    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8688
> > CV trial 231    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7979
> > CV trial 227    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8369
> > CV trial 241    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 	 weighted AUC: 0.8652
> > CV trial 211    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 	 weighted AUC: 0.8014
> > CV trial 205    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 	 weighted AUC: 0.7660
> > CV trial 235    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 	 weighted AUC: 0.8156
> > CV trial 191    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8475
> > CV trial 249    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 	 weighted AUC: 0.6773
> > CV trial 233    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6206 	 weighted AUC: 0.7234
> > CV trial 225    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 	 weighted AUC: 0.7908
> > CV trial 247    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7021 	 weighted AUC: 0.8865
> > CV trial 237    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8652
> > CV trial 242    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8333
> > CV trial 229    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7872
> > CV trial 219    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5904 	 weighted AUC: 0.7199
> > CV trial 221    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8901
> > CV trial 238    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6011 	 weighted AUC: 0.7943
> > CV trial 245    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 	 weighted AUC: 0.8528
> > CV trial 222    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8794
> > CV trial 215    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.9078
> > CV trial 249    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.6312
> > CV trial 231    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 	 weighted AUC: 0.7447
> > CV trial 235    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6720 	 weighted AUC: 0.7943
> > CV trial 241    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8191 	 weighted AUC: 0.8582
> > CV trial 247    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6915 	 weighted AUC: 0.8511
> > CV trial 229    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 	 weighted AUC: 0.7624
> > CV trial 233    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6206 	 weighted AUC: 0.6915
> > CV trial 237    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 	 weighted AUC: 0.8546
> > CV trial 227    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 	 weighted AUC: 0.8262
> > CV trial 239    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8901
> > CV trial 245    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8404 	 weighted AUC: 0.8121
> > CV trial 243    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7996 	 weighted AUC: 0.8688
> > CV trial 222    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 	 weighted AUC: 0.8723
> > CV trial 239    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 	 weighted AUC: 0.8901
> > CV trial 192    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8316 	 weighted AUC: 0.8298
> > CV trial 197    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.9539
> > CV trial 197    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 	 weighted AUC: 0.9433
> > CV trial 223    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5709 	 weighted AUC: 0.7908
> > CV trial 243    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7890 	 weighted AUC: 0.8617
> > CV trial 193    feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 	 weighted AUC: 0.9255
> > CV trial 223    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.4982 	 weighted AUC: 0.7943
> > CV trial 193    feature 1  mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8191 	 weighted AUC: 0.9397
>

    If I launch the previous command line with only one 1-D feature as follows, everything is going well:

          neuropredict -m meta_data.csv \
      -d mask_WMpet_av45_early.ero1.25mm.csv \
      -o outdir -t 0.8 -n 250 -k 'all' \
      --gs_level 'exhaustive' --classifier "LinearSVC"

    Do you have any idea?

    Best,

    Matthieu

    Sources.zip

    opened by mattvan83 6
  • IO error, unable to load features

    IO error, unable to load features

    Hi Pradeep,

    I finally got neuropredict installed with the dependencies using python 2.7.13! I'm having some issues with running the command, see the error message below,

    Cheers,

    John

    (py2713) Johns-MacBook-Pro-2:KNN John$ neuropredict -m /Users/John/Desktop/KNN/METADATAFILE.csv -o /Users/John/Desktop/KNN/features/results -u /Users/John/Desktop/KNN/features

    Requested features for analysis: get_dir_of_dirs from /Users/John/Desktop/KNN/features Traceback (most recent call last): File "/Users/John/anaconda3/envs/py2713/lib/python2.7/site-packages/neuropredict/neuropredict.py", line 333, in getfeatures data, feat_names = getmethod(featdir, subjid) File "/Users/John/anaconda3/envs/py2713/lib/python2.7/site-packages/neuropredict/neuropredict.py", line 274, in get_dir_of_dirs raise IOError('Unable to load features from \n{}'.format(featfile))

    opened by johnaeanderson 5
  • Unable to launch multi-class classifier

    Unable to launch multi-class classifier

    Hi Pradeep,

    When trying to launch multi-class classifier (whatever the classifier chosen) on a 3-class (CN, MCI, AD) problem, I got the following problem:

    Traceback (most recent call last):
  File "/homes_unix/mvanhoutte/Soft/anaconda3/envs/neuropredict/bin/neuropredict", line 8, in <module>
    sys.exit(main())
  File "/homes_unix/mvanhoutte/Soft/anaconda3/envs/neuropredict/lib/python3.7/site-packages/neuropredict/__main__.py", line 11, in main
    run_workflow.cli()
  File "/homes_unix/mvanhoutte/Soft/anaconda3/envs/neuropredict/lib/python3.7/site-packages/neuropredict/run_workflow.py", line 969, in cli
    grid_search_level, classifier, feat_select_method = parse_args()
  File "/homes_unix/mvanhoutte/Soft/anaconda3/envs/neuropredict/lib/python3.7/site-packages/neuropredict/run_workflow.py", line 525, in parse_args
    class_set, subgroups, positive_class = validate_class_set(classes, user_args.sub_groups, user_args.positive_class)
  File "/homes_unix/mvanhoutte/Soft/anaconda3/envs/neuropredict/lib/python3.7/site-packages/neuropredict/run_workflow.py", line 659, in validate_class_set
    ''.format(comb))
ValueError: Subgroup all does not contain 2 unique classes! Each subgroup must contain atleast two classes for classification experiments.

    I don't understand since I have well defined 3-class in the metadata.csv file and used the following command line:

      neuropredict -m meta_data.csv \
  -d metaROI.csv metaROI_split.csv HCP_parcellation.csv \
  -t 0.8 -n 250 -k 'all' --sub_groups 'all' --gs_level 'exhaustive' \
  --feat_select_method 'variancethreshold' --classifier ${classifier} --make_vis ./visu \
  --num_procs ${NCPU} --print_options ./options

    Could you help me? Best, Matthieu

    opened by mattvan83 4
  • Regression output visualizations: residual plots

    Regression output visualizations: residual plots

    A good output in the regression version would be residual plots

    Good example is from yellowbrick: http://www.scikit-yb.org/en/latest/api/regressor/residuals.html

    opened by raamana 2
  • CLI flag to print chosen options from an existing run

    CLI flag to print chosen options from an existing run

    normal run would print only the most important options such as which classifier, cross-validation details etc, whereas verbose option would print which subjects and more info into feature sets etc.

    opened by raamana 1
  • Add documentation to help interpret the results and visualizations

    Add documentation to help interpret the results and visualizations

    Have a separate markdown file

    • [ ] describe each output figure comprehensively
    • [ ] describe each output CSV file
    • [ ] And what else they can do with results
    enhancement help wanted 
    opened by raamana 1
  • feature request: covariates and ability to regress them

    feature request: covariates and ability to regress them

    few ideas:

    CSV input

    • [ ] have column names, specify which columns are class, id, covariates and feature anmes!
    • [ ] get the class ID from a specified colum (e.g. first, or specified)
    • [ ] get the subject ID from a specified column (or the second or specified)
    sbuject_id,class,feature1,feature2,age,sex
acbde_01,healthy,1.2,2.3,3.4,74,female
fghijk_01,healthy,1.2,2.3,3.4,76,male
...
lmnop_01,alzheimer,1.2,2.3,3.4,70,female
...
qrstu_01,depression,1.2,2.3,3.4,71,male

    Directory of directories input

    • [ ] covariates.txt in addition to features.txt within each subject folder
    enhancement Hackathon project 
    opened by raamana 1
  • Enabling plug-in user-chosen models or hyper-parameters

    Enabling plug-in user-chosen models or hyper-parameters

    Sometimes some advanced users would like to choose their model or hyper-parameters for many good reasons including model not existing in sklearn etc or needing deep control over them etc, then having the API or CLI allowing users to do it would be great!

    opened by raamana 0
  • Differences betw. frequency misclassified vs frequently correctly classified

    Differences betw. frequency misclassified vs frequently correctly classified

    besides the misclassification frequency plots (which can be a helpful diagnostic), it may be useful to offer an option to enable the user to compare and study the differences characteristics between frequency misclassified vs. frequently correctly classified..

    Characteristics could be the input features themselves, or the other associated meta data and attributes (e.g. provided as part of pyradigm)

    examples include: those frequently misclassified were mostly the older patients , and those frequently correctly classified has "property X" etc.

    opened by raamana 0
  • Add docs for all API, esp. CVResults

    Add docs for all API, esp. CVResults

    CVResults might be individually even more useful than others in the API. So document it thoroughly align with the rest with a tutorial or two for use by dev and reuse by the replicator or meta-analytic researcher.

    Hackathon project 
    opened by raamana 0
  • update all docs and graphics to indicate users can try different models also

    update all docs and graphics to indicate users can try different models also

    not just compare different feature sets with a fixed builtin model, but users can also input a model of their own choice. it does not limit exploration of new models or pipelines - they can use implementation of best practices while evaluating the such new models on features of their choice

    opened by raamana 1
  • Attribute-contrained performance estimates

    Attribute-contrained performance estimates

    Often, performance is estimated in aggregate on the entire test-set (in a split), regardless of their covariate- or attribute characteristics.. hence looking into performance estimates segregated by certain characteristics of covariates would reveal additional insights - such as difference in AUC in men vs women, or across different age bins, or between sites etc.

    Hackathon project 
    opened by raamana 2
Releases(0.6)

  • 0.6(Jun 8, 2020)

    • Major feature: Ability to predict continuous variables (regression)
    • Major feature: Ability to handle confounds (regress them out, augmenting etc)
    • Redesigned the internal structure for easier extensibility
    • New CVResults class for easier management of a wealth of outputs generated in the Classification and Regression workflows
    • API access is refreshed and easier
    Source code(tar.gz)
    Source code(zip)

  • 0.5(Jan 31, 2019)

    • Good news: neuropredict can handle missing data now (that are encoded with numpy.NaN). This is done respecting the cross-validation splits without any data leakage.
    • Few other useful utilities
    Source code(tar.gz)
    Source code(zip)

  • 0.4.5(Nov 18, 2017)

    • new classifier : SVM
    • new flag to choose a feature selection method
    • user chosen options now saved to disk, to better handle complex interactions between options
    • code clean up and faster tests
    Source code(tar.gz)
    Source code(zip)

  • 0.4.1(Nov 6, 2017)

    • Parallelizing the main the CV loop, leading to great reduction in total time for report generation!
    • More options, including choice of different classifiers (Random Forest and Extra Trees classifiers)
    • Support for dataset in Weka's ARFF format
    • Better visualizations (handling small/nan values in feature importance, layouts and design)
    • auto versioning!
    • Ability to read meta data from pyradigms or ARFF files, without having to specify that separately.
    • Dropping support for Python 2.7 :(
    Source code(tar.gz)
    Source code(zip)

  • v0.3-DOI(Sep 26, 2017)

Owner
Pradeep Reddy Raamana
Neuroscientist trying to bridge the gap between clinic & computer science. Interests: Machine learning, Neuroimaging, Brain disorders, Informatics, Open science
Pradeep Reddy Raamana
GitHub Repository https://raamana.github.io/neuropredict/
CAMPARI: Camera-Aware Decomposed Generative Neural Radiance Fields

CAMPARI: Camera-Aware Decomposed Generative Neural Radiance Fields Paper | Supplementary | Video | Poster If you find our code or paper useful, please

26 Nov 29, 2022
Meaningful titles for tabs and PDF downloads! Also supports tab search.

arxiv-utils If you are a researcher that reads a lot on ArXiv, you'll benefit a lot from this web extension. Renames the title of PDF page to the pape

Johnson 174 Dec 20, 2022
Learning View Priors for Single-view 3D Reconstruction (CVPR 2019)

Learning View Priors for Single-view 3D Reconstruction (CVPR 2019) This is code for a paper Learning View Priors for Single-view 3D Reconstruction by

Hiroharu Kato 38 Aug 17, 2022
An all-in-one application to visualize multiple different local path planning algorithms

Table of Contents Table of Contents Local Planner Visualization Project (LPVP) Features Installation/Usage Local Planners Probabilistic Roadmap (PRM)

Abdur Javaid 47 Dec 30, 2022
Python lib to talk to pylontech lithium batteries (US2000, US3000, ...) using RS485

python-pylontech Python lib to talk to pylontech lithium batteries (US2000, US3000, ...) using RS485 What is this lib ? This lib is meant to talk to P

Frank 26 Dec 28, 2022
Anchor-free Oriented Proposal Generator for Object Detection

Anchor-free Oriented Proposal Generator for Object Detection Gong Cheng, Jiabao Wang, Ke Li, Xingxing Xie, Chunbo Lang, Yanqing Yao, Junwei Han, Intro

jbwang1997 56 Nov 15, 2022
NL-Augmenter 🦎 → 🐍 A Collaborative Repository of Natural Language Transformations

NL-Augmenter 🦎 → 🐍 The NL-Augmenter is a collaborative effort intended to add transformations of datasets dealing with natural language. Transformat

684 Jan 09, 2023
Ray tracing of a Schwarzschild black hole written entirely in TensorFlow.

TensorGeodesic Ray tracing of a Schwarzschild black hole written entirely in TensorFlow. Dependencies: Python 3 TensorFlow 2.x numpy matplotlib About

5 Jan 15, 2022
Implementation of the method proposed in the paper "Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation"

Neural Descriptor Fields (NDF) PyTorch implementation for training continuous 3D neural fields to represent dense correspondence across objects, and u

167 Jan 06, 2023
App customer segmentation cohort rfm clustering

CUSTOMER SEGMENTATION COHORT RFM CLUSTERING TỔNG QUAN VỀ HỆ THỐNG DỮ LIỆU Nên chuyển qua theme màu dark thì sẽ nhìn đẹp hơn https://customer-segmentat

hieulmsc 3 Dec 18, 2021
Codebase for Amodal Segmentation through Out-of-Task andOut-of-Distribution Generalization with a Bayesian Model

Codebase for Amodal Segmentation through Out-of-Task andOut-of-Distribution Generalization with a Bayesian Model

Yihong Sun 12 Nov 15, 2022
A Python Package For System Identification Using NARMAX Models

SysIdentPy is a Python module for System Identification using NARMAX models built on top of numpy and is distributed under the 3-Clause BSD license. N

Wilson Rocha 175 Dec 25, 2022
A set of tools for converting a darknet dataset to COCO format working with YOLOX

darknet格式数据→COCO darknet训练数据目录结构（详情参见dataset/darknet）： darknet ├── class.names ├── gen_config.data ├── gen_train.txt ├── gen_valid.txt └── images

RapidAI-NG 148 Jan 03, 2023
Deploy recommendation engines with Edge Computing

RecoEdge: Bringing Recommendations to the Edge A one stop solution to build your recommendation models, train them and, deploy them in a privacy prese

NimbleEdge 131 Jan 02, 2023
A python library for face detection and features extraction based on mediapipe library

FaceAnalyzer A python library for face detection and features extraction based on mediapipe library Introduction FaceAnalyzer is a library based on me

Saifeddine ALOUI 14 Dec 30, 2022
IRON Kaggle project done while doing IRONHACK Bootcamp where we had to analyze and use a Machine Learning Project to predict future sales

IRON Kaggle project done while doing IRONHACK Bootcamp where we had to analyze and use a Machine Learning Project to predict future sales. In this case, we ended up using XGBoost because it was the o

1 Jan 04, 2022
Gesture recognition on Event Data

Event based Gesture Recognition Gesture recognition on Event Data usually involv

2 Feb 14, 2022
Text Extraction Formulation + Feedback Loop for state-of-the-art WSD (EMNLP 2021)

ConSeC is a novel approach to Word Sense Disambiguation (WSD), accepted at EMNLP 2021. It frames WSD as a text extraction task and features a feedback loop strategy that allows the disambiguation of

Sapienza NLP group 36 Dec 13, 2022
A Runtime method overload decorator which should behave like a compiled language

strongtyping-pyoverload A Runtime method overload decorator which should behave like a compiled language there is a override decorator from typing whi

20 Oct 31, 2022
Method for facial emotion recognition compitition of Xunfei and Datawhale .

人脸情绪识别挑战赛-第3名-W03KFgNOc-源代码、模型以及说明文档 队名：W03KFgNOc 排名：3 正确率: 0.75564 队员：yyMoming,xkwang,RichardoMu。 比赛链接：人脸情绪识别挑战赛 文章地址:link emotion 该项目分别训练八个模型并生成csv文

6 Oct 17, 2022