NetInitialize
NetInitialize[net]
gives a net in which all uninitialized learnable parameters in net have been given initial values.
NetInitialize[net,All]
gives a net in which all learnable parameters have been given initial values.
Details and Options
- NetInitialize[net,All] overwrites any existing training or preset learnable parameters in net.
- NetInitialize typically assigns random values to parameters representing weights and zero to parameters representing biases.
- The following optional parameters can be included:
-
Method Automatic which initialization method to use - Possible settings for Method include:
-
"Xavier" choose weights to preserve variance of random tensors when propagated through affine layers "Orthogonal" choose weights to be orthogonal matrices "Random" choose weights from a given univariate distribution "Identity" choose weights so as to preserve components of tensors when propogated through affine layers - Suboptions for specific methods can be specified using Method{"method",opt1val1,…}.
- For the method "Xavier", the following suboptions are supported:
-
"FactorType" "Mean" one of "In", "Out", or "Mean" "Distribution" "Normal" either "Normal" or "Uniform" - For the method "Random", the following suboptions are supported:
-
"Weights" NormalDistribution[0,1] random distribution to use to initialize weight matrices "Biases" None random distribution to use to initialize bias vectors - For the method "Identity", the following suboption is supported:
-
"Distribution" NormalDistribution[0,0.01] random distribution used to add noise to the initial identity matrices in order to break symmetries - For any suboption that expects a distribution, a numeric value stddev can be specified and is taken to mean NormalDistribution[0,stddev].
- By default, all methods initialize bias vectors to zero.
Examples
open allclose all
See Also
Related Guides
Introduced in 2016
(11.0)