2015/03/17

Trapezoid rule as Bayesian inference

I learned a very interesting relation between numerical integration and Bayesian inference from Diaconis, which I shortly review in what follows.

Say we have a function $f:[x_1,x_2] \to \bR$. Say we don't know it, but we have only two values $f(x_1)$ and $f(x_2)$, in both ends. Suppose our aim is to estimate the integral $\int_{x_1}^{x_2} f(x) dx$ and we would like to use trapezoid rule for this purpose. In this extreme condition, what is the rule? You draw a line from $f(x_1)$ to $f(x_2)$ and calculate the area below it. Let's find the trivial formula of the line with respect to $x$. The slope is given by, \begin{align*} m = \frac{f(x_2) - f(x_1)}{x_2 - x_1} \end{align*} and the whole equation is $\hat{f}(x) - f(x_1) = m (x - x_1)$ hence, \begin{align*} \hat{f}(x) = \frac{f(x_2) - f(x_1)}{x_2 - x_1} (x - x_1) + f(x_1) \end{align*} We denote the function with $\hat{f}$ because it is an approximation of unknown $f$ via its two points. If we integrate $\hat{f}$, we will get an approximation of integral of $f$.

Instead now assume $f(x)$ is a Brownian motion defined on $[x_1,x_2]$. Given its both ends, what is the mean of this process? The process is exactly what is called as Brownian bridge. For the mean, we are interested in the following quantity: $\bE[f(x) | f(x_1),f(x_2)]$ for any $x \in (x_1,x_2)$. This is given by $\hat{f}$ precisely, you can check it from the Wikipedia page of Brownian bridge. They are the same thing and it is possible to generalise this idea for $n$ points.

The interpretation is the following. Suppose you put a prior distribution on the function space of continuous functions. If you put a Brownian motion prior, then the posterior distribution over the function is characterised by a distribution whose mean piecewise linear (between observed points), and with locally quadratic variance function. So using Trapezoid rule is Bayesian inference under the prior that your function is a Wiener process. This gives way to a different interpretation of numerical integration methods.

I will elaborate these ideas in a follow up post, this is just a warm-up.

0 comments:

Post a Comment