Probabilistic models of nonnegative matrix factorisation


I wrote this post last year. I thought it is good to publish this here. Here, we give a brief review of probabilistic models of nonnegative matrix factorisation (NMF). We mainly list the papers which are important to gain intuition and sketch the main ideas without too much mathematical detail.


Fisher's Identity

Fisher's identity is useful to use in maximum-likelihood parameter estimation problems. In this post, I give its proof. The main reference is Douc, Moulines, Stoffer; Nonlinear time series theory, methods and applications.


Batch MLE for the GARCH(1,1) model


In this post, we derive the batch MLE procedure for the GARCH model in a more principled way than the last GARCH post. The derivation presented here is simple and concise.


Convergence of gradient descent algorithms


In this post, I review the convergence proofs of gradient algorithms. Our main reference is: Leon Bottou, Online learning and stochastic approximations. I rewrite the proofs described in Bottou's paper but with more details about the points which are subtle to me. I tried to write the proofs as clear as possible so as to make them accessible to everyone.