www.pudn.com > deepmat-master-for-spectral.zip > README.md, change:2014-07-28,size:1349b


deepmat
====
WARNING: this is not my main code, and there is no warranty attached!

= Generative Stochastic Network =
 - A simple implementation of GSN according to (Bengio et al., 2013)

= Convolutional Neural Network =
 - A naive implementation (purely using Matlab)
 - Pooling: max (Jonathan Masci's code) and average
 - Not for serious use!

= Restricted Boltzmann Machine & Deep Belief Networks =
 - Binary/Gaussian Visible Units + Binary Hidden Units
 - Enhanced Gradient, Adaptive Learning Rate
 - Adadelta for RBM
 - Contrastive Divergence
 - (Fast) Persistent Contrastive Divergence
 - Parallel Tempering
 - DBN: Up-down Learning Algorithm

= Deep Boltzmann Machine =
 - Binary/Gaussian Visible Units + Binary Hidden Units
 - (Persistent) Contrastive Divergence
 - Enhanced Gradient, Adaptive Learning Rate
 - Two-stage Pretraining Algorithm (example)
 - Centering Trick (fixed center variables only)

= Denoising Autoencoder (Tied Weights) =
 - Binary/Gaussian Visible Units + Binary(Sigmoid)/Gaussian Hidden Units
 - tanh/sigm/relu nonlinearities
 - Shallow: sparsity, contractive, soft-sparsity (log-cosh) regularization
 - Deep: stochastic backprop
 - Adagrad, Adadelta

= Multi-layer Perceptron =
 - Stochastic Backpropagation, Dropout
 - tanh/sigm/relu nonlinearities
 - Adagrad, Adadelta
 - Balanced minibatches using crossvalind()