Andrew Ng Tutorial: https://web.stanford.edu/class/cs294a/sparseAutoencoder_2011new.pdf UFLDL Exercise: http://ufldl.stanford.edu/wiki/index.php/Exercise:Sparse_Autoencoder A Chinese blog: http://www.cnblogs.com/tornadomeet/archive/2013/03/20/2970724.html Stacked Denoising Autoencoder (paper): http://jmlr.csail.mit.edu/papers/volume11/vincent10a/vincent10a.pdf
Category Archives: Algorithm
Convolutional Neural Network Simple Tutorial
I am going to demonstrate a simple version of convolutional neural network, a type of deep learning structure in this post. Motivation Knowing normal multi-layer neural network (probably the one with input layer, output layer and one hidden layer) is helpful before you proceed reading this post. A good tutorial of MLN can be found …
Continue reading “Convolutional Neural Network Simple Tutorial”
How does gradient vanish in Multi-Layer Neural Network?
Background This post reviews how we update weights using the back propagation approach in a neural network. The goal of the review is to illustrate a notorious phenomenon in training MLNN, called “gradient vanish”. Start Let’s suppose that we have a very simple NN structure, with only one unit in each hidden layer, input layer …
Continue reading “How does gradient vanish in Multi-Layer Neural Network?”
Logical deduction in NP-completeness proof
Maybe it sounds simple to computer scientists, I just want to backup some logical deduction of myself in NP-completeness proofs in case I will deal with such proofs in the future. In NP-completeness proof, we always want to find a polynomial-time reduction (transformation) from problem A to problem B, denoted as: We say that there …
Continue reading “Logical deduction in NP-completeness proof”
2-SAT is in P
Proof that BFS finds shortest distance
Intuitively, Breadth-First Search (BFS) can find shortest distances from a source $latex s&s=4$ to any other reachable vertex. However, proving this intuition needs a bit hard work. After some searches, I found that there are two ideas to prove it. The two ideas both use induction to prove it. Both ideas require a bit long proof. …
How is Logistic Regression designed？
背景 Logistic Regression是ML中再熟悉不过的Model了，它能基于数据X，得出生成binary label的概率： （在上式中，X仅有一个feature） 假设你出生在Logistic Regression被发明之前且在Normal Linear Regression被发明之后，现在让你设计一个Model来预测Binary的label——Y，使得这个Model能够基于观测数据X得出Y。你会怎么设计呢？ (注：接下来我们都假设X仅有一个feature）