## Introduction.

In this post, we review the Expectation-Maximization (EM) algorithm and its use for maximum-likelihood problems.

## 2013/05/23

### Stochastic gradient descent

17/01/2017 update: While searching for something else, I came across with my old blogpost on stochastic gradient descent (SGD) dated back to 23/05/2013. I found it a bit low-level and little informative (this, in fact, is true for most posts from that year). Despite there have been many great posts published on SGD since then, I still wanted to update the version in this blog. So I decided to rewrite it from scratch.

## Introduction.

In this post, we show the relationship between Gaussian observation model, Least-squares and pseudoinverse. We start with a Gaussian observation model and then move to the least-squares estimation. Then we show that the solution of the least-squares corresponds to the pseudoinverse operation.

## Introduction

These notes are mostly based on the book Stochastic Calculus for Finance vol. II, Chapter 4. I give a few propositions and focus on exercises of Shreve by make use of the Ito-Doeblin formula. The use of Ito-Doeblin formula is almost purely practical to solve continuous-time stochastic models. My treatment is slightly different from the Shreve since I emphasize on the differential forms of the formulas.