Matlab:trainingoptions() explain training options in detail

trainingOptions It is used to set the training strategy and super parameters of neural network

usage ：options = trainingOptions(solverName,Name,Value)
           solverName Is an optimization function ,Name-Value Key value pairs , Return to one TrainingOptions object

It's more intuitive to see an example ,

>> options = trainingOptions('sgdm', ...
    'MiniBatchSize',512, ...
    'MaxEpochs',2, ...
    'InitialLearnRate',1e-3, ...
    'Shuffle','every-epoch', ...
    'ValidationData',imdsValidation, ...
    'ValidationFrequency',3, ...
    'Verbose',false, ...
    'Plots','training-progress');
>> options

options = 

  TrainingOptionsSGDM -  attribute :

                    Momentum: 0.9000
            InitialLearnRate: 1.0000e-03
           LearnRateSchedule: 'none'
         LearnRateDropFactor: 0.1000
         LearnRateDropPeriod: 10
            L2Regularization: 1.0000e-04
     GradientThresholdMethod: 'l2norm'
           GradientThreshold: Inf
                   MaxEpochs: 2
               MiniBatchSize: 512
                     Verbose: 0
            VerboseFrequency: 50
              ValidationData: [1×1 matlab.io.datastore.ImageDatastore]
         ValidationFrequency: 3
          ValidationPatience: Inf
                     Shuffle: 'every-epoch'
              CheckpointPath: ''
        ExecutionEnvironment: 'auto'
                  WorkerLoad: []
                   OutputFcn: []
                       Plots: 'training-progress'
              SequenceLength: 'longest'
        SequencePaddingValue: 0
    SequencePaddingDirection: 'right'
        DispatchInBackground: 0
     ResetInputNormalization: 1

You can see , There are many adjustable parameters , The following is a detailed explanation

• solverName： Optimization function , Optional ’sgdm’,‘rmsprop’,‘adam’
• Momentum： momentum ,[0,1] Between
• InitialLearnRate： Initial learning rate
• LearnRateSchedule： Learning rate strategy ,‘none’ perhaps ’piecewise’,'none’ It means that the learning rate remains unchanged ,'piecewise’ Is the segmented learning rate
• LearnRateDropFactor： Learning rate decline factor ,[0,1] Between , The learning rate after reduction is ： Current learning rate * Drop factor
• LearnRateDropPeriod： Learning rate decline cycle , A few epoch Reduce the learning rate once
• L2Regularization：L2 Regularization factor
• GradientThresholdMethod： Used to crop gradients that exceed the threshold , Optional ’l2norm’,‘global-l2norm’,‘absolute-value’
• GradientThreshold： Gradient threshold , If the gradient is greater than the threshold , Then press GradientThresholdMethod Set the method to deal with
• MaxEpochs： Maximum training rounds , Positive integer , The default is 20
• MiniBatchSize： Namely batchsize, The amount of data used in each iteration , Positive integer
• Verbose： Whether to display the real-time training process in the command line window ,0 or 1, if 1, Then the command line shows what is currently doing , The default is true
• VerboseFrequency：Verbose The frequency of printing on the command line , The default is 100
• ValidationData： Validation set data , It's a ImageDatastore object
• ValidationFrequency： Verification frequency , How many? batchsize Once after verification , No epoch
• ValidationPatience： Early stop condition ,Validation On loss Greater than or equal to the minimum loss How many? epoch Stop training after , such as , At present loss Is the minimum value and 0.01, After a few more rounds, it is not lower than 0.01, Just stop training
• Shuffle： Data disruption strategy , Optional ’once’,‘never’,‘every-epoch’
                ‘once’： Disrupt before training
                ‘never’： Don't disturb
                ‘every-epoch’： Every epoch Disrupt once
  The default is ’once’, Suggested choice ‘every-epoch’, because MATLAB When training the network , If there is not enough data batchsize Will throw away ,‘every-epoch’ It can avoid discarding the same batch of data
• CheckpointPath： Network save path , The default is ’’, That is, it does not save by default , If you set a path , Then each epoch It will be saved once without overwriting
• ExecutionEnvironment： Hardware environment , use GPU still CPU, Optional ’auto’,‘cpu’,‘gpu’,‘multi-gpu’,'auto’ To have gpu Then use , It doesn't work cpu
• WorkerLoad： I didn't understand it very well , It's probably about computing resources , It's not used in general
• OutputFcn： By default
• Plots： Whether to draw the real-time training process , Optional ’none’ perhaps ’training-progress’, The default is ’none’
• SequenceLength： For processing sequence data
• SequencePaddingValue： ditto
• SequencePaddingDirection： ditto
• DispatchInBackground： Use background scheduling ( Asynchronous prefetch queue ) Read training data from data storage , Appoint false or true, Background scheduling requires parallel computing toolbox , The default is 0

Actually , In practice , There are only a few that need to be adjusted , Most of them just default , The above code is commonly used .

The last attached Official document link , A more detailed