Stochastic Processes and Gauss-Markov Models Part I

Outline

• The following topics will be covered in this lecture:
  • Extending the concept of random vectors – stochastic processes
  • Continiuity in probability
  • Separability
  • Gaussian stochastic processes
  • Wiener Processes

Stochastic processes

• We are now familiar with the Gaussian model for a random state, and its centrality due to the central limit theorem.

  • Likewise, we have developed a variety of tools to extend the Gaussian model, and to efficiently simulate it numerically.

• We will now make a fundamental leap that will allow us to model time-varying random processes – i.e., we will introduce stochastic processes.

• A stochastic process is an infinite collection of random variables.

• We will ultimately use this object to model uncertain signals based on mechanistic processes;

  • however, we want to emphasize from the beginning that a stochastic process is just a generalization of a random variable / vector.

• However, unlike a random vector \( \pmb{x}\in\mathbb{R}^{N_x} \), a stochastic process requires considerably more care.

  • This is due to the fact that a stochastic process \( \pmb{x}_t \) is the generalization of a random vector into infinite dimensions.

Stochastic Processes
A stochastic process is a family of random variables, \( X(\omega, t) \), indexed by a real parameter \( t \in T \) and defined on a common probability space \( (\Omega, A, \mathcal{P} ) \).

• The “real parameter \( t \)” in the above is almost always taken a time variable.

• Thus, a random process can be thought of as a function that takes both the sample point and time as arguments.

• Hence, we will often use the notation \( X(\omega, t) \), or \( X_t \) for short, to denote a random variable depending on time.

Stochastic processes

• Note, in many cases we will actually consider time to be discrete in nature;

  • this is a consequence, in part, of the fact that we often have measurements discretely and not continuously in time.

• If the parameter set \( T = \mathbb{N} \), the natural numbers, then \( X_t \) is often called a random sequence to reflect the discrete nature of the time parameter.

  • In this case, we will often use a notation such as \( X_k \) corresponding to a sequence of times \( \{t_0, t_1, \cdots, t_k, \cdots\} \) to denote the discrete nature of the time variable.

• On the other hand, if \( T = \mathbb{R} \), then \( X_t \) is called either a random function or process.

• We should also note that the values of \( X_t \), for a fixed value of \( t \), can also be a discrete or continuous random variable.

• If we look at random processes in a certain way, we can see that they are a natural extension of the idea of random vectors to infinite collections:

\[ \begin{align} &\begin{matrix}\text{Random variable} & X \end{matrix} & \begin{matrix}\text{Random vector} & & \pmb{x}^\top:= \begin{pmatrix} X_1 & \cdots & X_{N_x}\end{pmatrix}\end{matrix}\\\\ &\begin{matrix}\text{Random sequence} & \begin{pmatrix} \cdots & X_k & \cdots \end{pmatrix}\end{matrix} & \begin{matrix} \text{Random process} & \begin{pmatrix} \cdots & X_t & \cdots \end{pmatrix}\end{matrix} \end{align} \]

• The key notion to take away is that a random process is a set of time functions, each of which is one possible outcome, \( \omega \), out of the set of all possible outcomes, \( \Omega \).
• A random variable returns a real number, and a random process returns a continuously indexed collection of real numbers or a real-valued function.

Stochastic processes

• As a quick example, consider a sine wave with an amplitude given as a random variable taking on values from −1 to 1 with a uniform distribution:

  \[ \begin{align} X(\omega, t) := A(\omega) \sin(t): \Omega \times \mathbb{R} \rightarrow \mathbb{R} & & A(\omega) \sim U[-1, 1] \end{align} \]

• Again, we emphasize that the random variable \( X \) takes \( \omega \) and returns a function.

  • In particular, if we know which sample point, \( \omega \), we have selected, we know the plot of the function \( X_t \) for all \( t\in \mathbb{R} \).

• We can consider this in Python as follows, where A below is a uniform random variable on \( [-1,1] \)

import matplotlib.pyplot as plt
import numpy as np
np.random.seed(123)
A = np.random.uniform(low=-1, high=1)
A

0.3929383711957233

• Having determined the particular outcome for \( \omega \), we now define a function in time given by

  \[ \begin{align} 0.3929383711957233 * \sin(t). \end{align} \]

Stochastic processes

• We just showed how fixing \( \omega \) yields an outcome for a random function defined for all time.

  • This particular function, defined over the time variable \( t \), is known as a sample path or a realization of the random process.

• On the other hand, if we fix \( t \) and let \( X \) vary over \( \omega \), i.e.,

  \[ \begin{align} X_{t_0}(\omega) := X(\omega,t)|_{t=t_0}, \end{align} \]

• we get a random variable \( X_{t_0} \) defined on \( (\Omega, \mathcal{A}, \mathcal{P}) \) with

  \[ \begin{align} \{\omega : X_{t_0} (\omega) \leq x \} \in \mathcal{A}, \end{align} \] i.e., these form the generating sets of the observable events for the probability space.

• This is what is meant that the parameters \( t \) indexes a family of random variables.

Stochastic processes

• We can consider random processes as infinite-dimensional extensions of the idea of random vectors.

• However, when we extend our ideas to infinite dimensions, new complications arise.

• With random vectors, we can completely define the probability of the vector as a collection of random variables by the joint probability,

  \[ \begin{align} P(x_1, \cdots, x_{N_x}) := \mathcal{P}\left( X_1(\omega) \leq x_1 , \cdots, X_{N_x}(\omega) \leq x_{N_x}\right) \end{align} \]

• However, when \( T \) is an infinite set, we have to be more careful.

• If \( T \) is a countably infinite set, then we can characterize the process with sets of the form

  \[ \begin{align} \{\omega: X(\omega,t_k) \leq a, \quad k= 1,\cdots\} = \cap_{k=1}^\infty \{\omega : X_{t_k} \leq a\}; \end{align} \]

  • in the measure-theoretic approach, this type of countably-infinite intersection still generates the observable events of the process.

• However, trouble arises with the uncountable infinity of the continuum for such constructions, where

  \[ \begin{align} \{\omega: X(\omega,t_k) \leq a, \quad k= 1,\cdots\} = \cap_{t\in\mathbb{R}} \{\omega : X_{t_k} \leq a\} \end{align} \] an intersection over a continuum can lead to problems with our construction of observable events.

  • In particular, if we follow the above alone, the probability of \( \mathcal{P}\left(X_t \geq 0 : \forall t\in[0,1]\right) \) may not actually be defined.

Separability

• Our second condition is known as separability, which is described as follows.

Separability
A stochastic process \( X(\omega, t) \) is said to be separable if there exists a countable, dense set \( S \subset T \) such that for any closed set \( K \subset [-\infty, \infty] \) the two sets \[ \begin{align} A_1 = \{\omega : X(\omega, t) \in K, \forall t \in T \} & & A_2 =\{\omega: X(\omega, t)\in K, \forall s \in S\} \end{align} \] differ by a set \( A_0 \) such that \( \mathcal{P}\left(A_0\right)=0 \).

• Separability means that for a continuous-parameter random process, it is possible to analyze it like a discrete-parameter one.

• When we say that \( S \) is dense in \( T \), we mean that \( S \) must contain enough points of \( T \) so as to provide essentially a complete representation of \( T \).

• That is, if we plotted out the set \( \{X(\omega, t) : t \in S\} \) versus \( t \), it would look indistinguishable from \( \{X(\omega, t), t \in T \} \).

• Lets suppose that \( T = \mathbb{R} \), the real line, or perhaps the positive real line \( \mathbb{R}^+ \).

• The simplest way to get a separating set of the continuous-time interval \( T \) is to simply choose \( S \) to be the set of rational numbers \( \mathbb{Q} \cap T \).

• Mathematically, what this means is that for any point \( t \in T \), there exists a discrete sequence \( t_k \in S \) such that \( t_k \rightarrow t \).

• Thus, if \( X(t, \omega) \) is a separable process, it is such that \( X(t_k , \omega) \rightarrow X(t, \omega) \) except on the zero probability set mentioned in the definition.

• We, therefore, have well-defined discretized limits in time with probability one.

Gaussian processes

• Just like we extended the Gaussian to multiple variables, we can now extend the (multivariate) Gaussian to a family of Gaussian random (vectors) variables parameterized by time.

• We will need to start with identifying the notion of independence properly for a stochastic process.

Independent increments
Let \( X \) be a random process defined on the time interval, \( T \). Let \( t_0 < t_1 < \cdots < t_n \) be a partition of the time interval, \( T \). If the increments, \( X(t_k) − X(t_{k−1}) \), are mutually independent random variables for any partition of \( T \), then \( X \) is said to be a process with independent increments.

• In the above, this is to say that if we take arbitrary, discrete time points \( t_{k-1} < t_{k} < t_{k+1} \in T \), the following holds

  \[ \begin{align} \mathcal{P}\left( X_{k+1} - X_{k} \leq a | X_{k} - X_{k-1} = b\right) = \mathcal{P}\left( X_{k+1} - X_{k} \leq a\right), \end{align} \] and vice-versa in the time indices.

• In particular, the size of any past difference in realizations doesn't affect the probability of any future difference in realizations.

Gaussian processes

The Weiner process

Courtesy of Lookang et al., CC BY-SA 3.0, via Wikimedia Commons