EmbeddingLayer

EmbeddingLayer[size,n]

represents a trainable net layer that embeds integers between 1 and n into a continuous vector space of dimension size.

EmbeddingLayer[size]

leaves the n to be inferred from context.

Details and Options

  • EmbeddingLayer operates on a single integer or tensors of integers, producing a vector or tensor, respectively.
  • Specifying EmbeddingLayer[size] will produce a net that infers the n when a NetEncoder[…] that produces integers is connected to its input.
  • The embedding performed by EmbeddingLayer is learned during training.
  • The following optional parameter can be included:
  • "Weights"Automaticinitial matrix of weights of dimensions n×size
  • With Automatic settings, the weights are added automatically when NetInitialize or NetTrain is used.
  • If weights have been added, EmbeddingLayer[…][input] explicitly computes the output from applying the layer.
  • EmbeddingLayer[…][{input1,input2,…}] explicitly computes outputs for each of the inputi.
  • NetExtract can be used to extract weights from an EmbeddingLayer object.
  • EmbeddingLayer is typically used inside NetChain, NetGraph, etc.
  • EmbeddingLayer exposes the following ports for use in NetGraph etc.:
  • "Input"an integer or tensor of integers
    "Output"a vector or tensor of numerical vectors
  • EmbeddingLayer[size,n,"Input"->shape] allows the shape of the input to be specified. Possible forms for shape are:
  • NetEncoder[…]encoder producing an integer or tensor of integers
    "Integer"a single integer
    da vector of integers of length d
    {d1,d2,…}a tensor of integers of dimensions d1×d2×…
    "Varying"a variable-length vector of integers
    {"Varying",d2,d3,…}a variable-length sequence of tensors of dimensions d2×d3×…

Examples

open allclose all

Basic Examples  (2)

Create an EmbeddingLayer that will take integers 1, 2 and 3 and produces vectors of size 4:

In[1]:=
Click for copyable input
Out[1]=

Create a randomly initialized EmbeddingLayer that will take integers 1, 2 and 3 and produces vectors of size 2:

In[1]:=
Click for copyable input
Out[1]=

Apply the layer to an integer:

In[2]:=
Click for copyable input
Out[2]=

Apply the layer to a vector of integers:

In[3]:=
Click for copyable input
Out[3]=

Scope  (4)

Options  (1)

Applications  (2)

Properties & Relations  (1)

See Also

UnitVectorLayer  NetChain  NetGraph  NetInitialize  NetTrain  NetExtract

Introduced in 2016
(11.0)