Autoencoder Manifold Learning

struct LayerIterator
    width     :: Array{Int}
    dropout   :: Set{Int}
    normalize :: Set{Int}
    σᵢ        :: Function
    σₒ        :: Function
    σ         :: Function
end

An iterator used to generate dense latent layers within a neural network. width denotes the widths of each layer; the length of this array immediately determines the depth. dropout denotes the layers, as given by width that are followed by a dropout layer. normalize denotes the layers, as given by width that are followed by a batch normalization layer. σᵢ, σₒ, σ is the activation energy on the first, last, and intermediate layers respectively.

batch(data, n)

Randomly partition data into groups of size n.

model(dᵢ, dₒ; Ws=Int[], normalizes=Int[], dropouts=Int[], σ=elu)

Initialize an autoencoding neural network with input dimension dᵢ and latent layers dₒ. Ws specifies both the width and depth of the encoder layer - the width of each layer is given as an entry in the array while the length specifies the depth. normalizes and dropouts denote which layers are followed by batch normalization and dropout specifically. The decoder layer is given the mirror symmetric architecture.

train!(model, data, index, loss; B=64, η=1e-3, N=100, log=noop)

Trains autoencoder model on data by minimizing loss. index stores the underlying indices of data used for training. Will mutate the underlying parameters of model. Optional parameters include:

1. B denotes the batch size to be used.
2. N denotes the number of epochs.
3. η denotes the learning rate.

update_dimension(model, dₒ; ϵ = 1e-6)

Add a colection of new neurons in the encoding layer to encode in the encoding layer to increase dimensions to dₒ. Model weights for the initial dimensions are kept the same.

validate(data, len)

Reserve len samples from data during training process to allow for model validation.