CrossEntropyLossLayer

CrossEntropyLossLayer["Index"]

represents a net layer that computes the cross-entropy loss by comparing input class probability vectors with indices representing the target class.

CrossEntropyLossLayer["Probabilities"]

represents a net layer that computes the cross-entropy loss by comparing input class probability vectors with target class probability vectors.

CrossEntropyLossLayer["Binary"]

represents a net layer that computes the binary cross-entropy loss by comparing input probability scalars with target probability scalars, where each probability represents a binary choice.

Details and Options

  • CrossEntropyLossLayer exposes the following ports for use in NetGraph etc.:
  • "Input"numerical tensor of rank n
    "Target"numerical tensor of rank n or class index tensor of rank n-1
    "Loss"real number
  • When operating on general tensor inputs, CrossEntropyLossLayer effectively threads over any extra tensor dimensions to produce a tensor of losses and returns the mean of these losses.
  • For CrossEntropyLossLayer["Binary"], the input and target should be scalar values between 0 and 1, or tensors of these.
  • For CrossEntropyLossLayer["Index"], the input should be a vector of probabilities {p1,…,pc} that sums to 1, or a tensor of such vectors. The target should be an integer between 1 and c, or a tensor of such integers.
  • For CrossEntropyLossLayer["Probabilities"], the input and target should be a vector of probabilities that sums to 1, or a tensor of such vectors.
  • For the "Index" and "Probabilities" forms, where the input tensor has dimensions {d1,d2,…,dn}, the final dimension dn is used to index the class. The output loss is taken to be the mean over the remaining dimensions {d1,…,dn-1}.
  • CrossEntropyLossLayer[…][<|"Input"->in,"Target"target|>] explicitly computes the output from applying the layer.
  • CrossEntropyLossLayer[…][<|"Input"->{in1,in2,…},"Target"->{target1,target2,…}|>] explicitly computes outputs for each of the ini and targeti.
  • CrossEntropyLossLayer is typically used inside NetGraph to construct a training network.
  • A CrossEntropyLossLayer[…] can be provided as the third argument to NetTrain when training a specific network.
  • When appropriate, CrossEntropyLossLayer is automatically used by NetTrain if an explicit loss specification is not provided. One of "Binary", "Probabilities", or "Index" will be chosen based on the final activation used for the output port and the form of any attached NetDecoder.
  • CrossEntropyLossLayer[form,"port"->shape] allows the shape of the input or target port to be specified. Possible forms for shape are:
  • "Real"a single real number
    "Integer"a single integer
    na vector of length n
    {n1,n2,…}a tensor of dimensions n1×n2×…
    {"Varying",n2,n3,…}a variable-length sequence of tensors of dimensions n2×n3×…
    NetEncoder[…]an encoder
    {n,encoder}an encoder mapped over a sequence of length n

Examples

open allclose all

Basic Examples  (2)

Create a CrossEntropyLossLayer object that takes a probability vector and an index:

In[1]:=
Click for copyable input
Out[1]=

Create a CrossEntropyLossLayer where the input is a probability vector and the target is an index:

In[1]:=
Click for copyable input
Out[1]=

Apply it to an input and a target:

In[2]:=
Click for copyable input
Out[2]=

Scope  (5)

Applications  (2)

Properties & Relations  (4)

See Also

MeanAbsoluteLossLayer  MeanSquaredLossLayer  NetChain  NetGraph  NetTrain

Introduced in 2016
(11.0)
| Updated in 2017
(11.1)