On chain rule, computational graphs, and backpropagation

September 05, 2015
Here I'm going to revisit backpropagation theory by thinking about neural networks as computational graphs upon which we can easily visualize the chain rule to compute partial derivatives.
Read More

Simple(r) RNN in Julia

August 10, 2015
A beginner tutorial for an even simpler recurrent neural network implementation in Julia.
Read More

Simple Recurrent Neural Network

August 05, 2015
A beginner tutorial on building a simple (Elman) recurrent neural network in Python. We'll build an RNN than can compute XOR on sequential data and another that can predict the next character in a sequence.
Read More

Gradient Descent with Backpropagation

July 31, 2015
A beginner tutorial to understand the theoretical and implementation details of gradient descent by backpropagation using Python.
Read More

Simple Genetic Algorithm In 15 Lines Of Python

July 15, 2015
Beginner tutorial on writing a genetic algorithm to train a simple 2-layer feed-forward neural network. (Ok, I admit it, the "15 lines" is a gimmick)
Read More