The EM Algorithm


In this post, we review the Expectation-Maximization (EM) algorithm and its use for maximum-likelihood problems.


Stochastic gradient descent

17/01/2017 update: While searching for something else, I came across with my old blogpost on stochastic gradient descent (SGD) dated back to 23/05/2013. I found it a bit low-level and little informative (this, in fact, is true for most posts from that year). Despite there have been many great posts published on SGD since then, I still wanted to update the version in this blog. So I decided to rewrite it from scratch.


Gaussianity, Least squares, Pseudoinverse


In this post, we show the relationship between Gaussian observation model, Least-squares and pseudoinverse. We start with a Gaussian observation model and then move to the least-squares estimation. Then we show that the solution of the least-squares corresponds to the pseudoinverse operation.


The use of Ito-Doeblin formula to solve SDEs


These notes are mostly based on the book Stochastic Calculus for Finance vol. II, Chapter 4. I give a few propositions and focus on exercises of Shreve by make use of the Ito-Doeblin formula. The use of Ito-Doeblin formula is almost purely practical to solve continuous-time stochastic models. My treatment is slightly different from the Shreve since I emphasize on the differential forms of the formulas.