I have been attending a reading course on stochastic analysis led by Professor Ioannis Karatzas, where the students take turns in presenting a topic of their choice. I recently presented on Choquet's theory of capacities and its applications to measure theory and in the general theory of processes. This blog post is based on this presentation. I have freely copied^[1] from many sources, but my primary reference are the (unfortunately unpublished) lecture notes by Prof. Karatzas.

Introduction

Kolmogorov laid the modern axiomatic foundations of probability theory with the German monograph Grundbegriffe der Wahrscheinlichkeitsrechnung which appeared in Ergebnisse Der Mathematik in 1933. This was a period of intense discussions on the foundations of probability, and a majority of probabilists at the time considered measure theory not only a waste of time, but also an offense to "probabilistic intuition" (Meyer, 2009). But by 1950, with the work of Doob in particular, these discussions of foundations had been settled.

Continuous-time processes, on the other hand, were difficult to tame even with measure theory: if a particle is subject to random evolution, to show that its trajectory is continuous, or bounded, requires that all time values be considered, whereas classical measure theory can only handle a countable infinity of time values. Thus, not only does probability depend on measure theory, but it also requires more of measure theory than the rest of analysis (Meyer, 2009).

The missing pieces of the puzzle, which will be the highlight of this and the next blog post, are the debut, section and projection theorems. These theorems are also indispensable in many applications, for instance in dynamic programming and stochastic control (Karoui and Tan, 2013).

To get a taste of these theorems, let's recall a famous error made by Lebesgue in the paper Sur les fonctions représentables analytiquement published in 1905. Consider the measurable space $(\mathbb R^2, \mathcal{B}(\mathbb R^2))$ and the projection map $\pi$ given by $\mathbb R^2 \ni (x,y) \mapsto \pi(x,y) = y \in \mathbb R.$ It is easy to see that for any open set $O$ in $\mathbb R^2$, the set $\pi(O)$ is also open in $\mathbb R$: Recall that the standard topology on $\mathbb R^2$ is same as the product topology on $\mathbb R^2.$ By the definition of the product topology on $\mathbb R^2$, an open set $O$ in $\mathbb R^2$ is of the form $O = \bigcup_{i \in I} \bigcap_{j \in J_i} U_{ij} \times V_{ij}$ for open $U_{ij}, V_{ij}$ in $\mathbb R$, $I$ arbitrary and $J_i$ finite. A simple argument gives $\pi(O) = \bigcup_{i \in I} \bigcap_{j \in J_i} U_{ij}$ which is open in $\mathbb R.$ In fact, more generally, projection from any product space (with product topology) is an open map. Now it seems reasonable to expect that for any Borel set $B \in \mathcal{B}(\mathbb R^2)$ its projection is also a Borel set in $\mathcal{B}(\mathbb R)$, and Lebesgue assumed this in his paper. But, in fact, this is FALSE! The error was spotted in around 1917 by Mikhail Suslin, who realised that the projection map need not be Borel, and this lead to his investigation of analytic sets and to begin the study of what is now known as descriptive set theory (Almost Sure blog).

The problem is projection doesn't commute with countable decreasing intersection. For example, consider the decreasing sequence of sets $S_n = (0, 1/n) \times \mathbb R.$ Then $\pi(S_n) = \mathbb R$ for all $n$, giving $\bigcap_{n \in \mathbb N} \pi(S_n) = \mathbb R$, but $\bigcap_{n \in \mathbb N} S_n = \varnothing$, giving $\pi \left( \bigcap_{n \in \mathbb N} S_n \right) = \varnothing.$ The measurable projection theorem stated next will be one of the highlights of this post.

Why is proving such results difficult? As mentioned above it's because projection doesn't behave nicely with intersections. Nevertheless, let us try to see how one might try to prove the above theorem. A standard approach in measure theory is to construct a collection like

which contains the sets satisfying the desired property, and show that it is a $\sigma$-algebra containing a simple collection, say $\mathscr{A}$, such that it easy to show that elements of $\mathscr{A}$ satisfy the desired property and $\mathscr{A}$ generates $\mathcal{B}(K) \otimes \mathcal{F}$, because then we will have $\mathcal{B}(K) \otimes \mathcal{F} \subseteq \mathscr{E}.$ To this end, let Then it is easy to see that $\mathscr{A} \subseteq \mathscr{E}$, $\mathscr{A}$ generates $\mathcal{B}(K) \otimes \mathcal{F}$ and that $\mathscr{A}$ is an algebra. If we could show that $\mathscr{E}$ is a monotone class, then we would be done on account of monotone class theorem. Increasing sequences are easily handled, in fact if $\{S_n\} _ {n \in \mathbb N} \subseteq K \times \Omega$ is any sequence, then But if $S_1 \supseteq S_2 \supseteq \cdots$, then in general we cannot say Enter Choquet's theory of capacities. It provides the language to prove results like these. We know that every Borel measure $\mu$ on $\mathbb R^d$ (in fact, any Polish space) has the interior regularity (also known as tightness) property where $\mathscr{K}(\mathbb R^d)$ is the collection of all compact sets in $\mathbb R^d.$ The Choquet's theory of capacities generalizes this approximation-from-below property, and distills those properties of measure that allow for such approximation to hold in very general settings. As we will see, monotonicity and continuity from above and below properties are at play here, and notions corresponding to complements or differences will be absent.

The next few sections will be very abstract and it is easy to lose sight of our goal. Some people enjoy this mental gymnastics, but even if you find this dry, the reward at the end will be worth the initial struggle. We start with Choquet's theory of capacities. The highlight of this part will be Choquet's capacitability theorem. To prove this major result we will need to define a lot of new terminology and prove some major results like Sierpiński's theorem and Sion's theorem. Armed with Choquet's capacitability theorem we will prove Measurable Section theorem which in turn will form the backbone of various other results in measure theory. These results in measure theory will then help us prove results in general theory of processes, but we will discuss this part in the next blog post.

Pavings and Mosaics

The concept of paving generalizes the concept of algebra. As an example, if $E$ is a topological space, then the collection of closed subsets of $E$ forms a paving. As another example, if $E$ is a Hausdorff space, then $(E, \mathscr K(E))$ is a paved space, where as before $\mathscr{K}(E)$ denotes the collection of all compact sets in $E.$

It is easy to check that an arbitrary intersection of pavings is also a paving, and that the collection $\mathfrak{P}(E)$ of all subsets of $E$ is a paving. Thus for any collection $\mathcal{A}$ of subsets of $E$, we can define the notion of paving generated by $\mathcal{A}$ as the smallest paving of subsets of $E$ that contains $\mathcal{A}$ by simply defining it to be the intersection of all pavings of $E$ containing $\mathcal{A}.$

Using the fact that $(A_1 \times B_1) \cap (A_2 \times B_2) = (A_1 \cap A_2) \times (B_1 \cap B_2)$ we see that $\mathcal{R}$ is stable under finite intersections. Therefore, any element of $\mathcal{E} \otimes_p \mathcal{F}$ is of the form $\bigcup_{i=1}^n A_i \times B_i$, where $A_i \in \mathcal{E}$ and $B_i \in \mathcal{F}$ for all $i=1,\ldots,n.$

The concept of mosaic generalizes the concept of $\sigma$-algebra. Just like paving, it is easy to define the notion of mosaic generated by a collection. They will always occur in the context of a paving $\mathcal{E}$ on $E.$ We denote by $\widehat{\mathcal{E}}$ the mosaic generated by $\mathcal{E}.$ $\mathcal{E}\ \widehat{\otimes}_p \mathcal{F}$ will denote the mosaic generated by the product paving $\mathcal{E} \otimes_p \mathcal{F}.$

Just like with monotone class arguments in measure theory, the paving $\mathcal E$ will be a simple collection of subsets of $E$ for which it is easy to prove a property $P.$ From here we will show that the elements of the mosaic $\widehat{\mathcal E}$ also satisfy $P.$ Often $\widehat{\mathcal E}$ will be a $\sigma$-algebra.

Henceforth the notation $\mathcal{E}$ will be used for a paving on $E.$

Properties of pavings and mosaics

Just like the results connecting algebra and $\sigma$-algebra, we have results connecting pavings and mosaics.

As an example, if $E$ is a separable, locally compact metric space, and $\mathcal{E} = \mathscr{K}(E)$ is the collection of compact subsets of $E$, then this property holds. In fact, $E$ can be a second-countable locally compact Hausdorff space. Then it is $\sigma$-compact and every compact subset is closed. This implies that the open set given by the complement of a compact set is a countable union of compact sets. Combine it with the fact that an open subset of a locally compact and $\sigma$-compact is itself $\sigma$-compact to get that if $A \in \mathcal E$, then $A^\mathsf{c} \in \widehat{\mathcal E}.$

Compact pavings

Recall a standard result from topology: a topological space $X$ is compact if and only if for every collection $\mathcal C$ of closed subsets of $X$ having the finite intersection property (i.e., every finite sub-collection of $\mathcal C$ has nonempty intersection), the intersection $\bigcap_{C \in \mathcal C} C$ is nonempty. For our use case, we consider countable sub-collections.

It is easy to verify that a compact paving is a compact collection. As an example, if $E$ is a separable metric space, the collection, $\mathscr{K}(E)$, of all compact subsets of $E$ is a compact paving. This follows immediately from the finite intersection characterization of a compact topological space stated above.

We define

and note that if $\mathcal{E}$ is a compact paving then so is $\mathcal{E}_\delta.$ Indeed, $\mathcal{E}_\delta$ is a paving because for $A = \bigcap_{n \in \mathbb N} A_n$ and $B = \bigcap_{m \in \mathbb N} B_m$ in $\mathcal{E}_\delta$, and $\mathcal{E}_\delta$ is a compact paving because intersection of a sequence of countable intersections is again a countable intersection.

The next lemma tells us when it acceptable to commute projection with countable intersections.

Envelopes

Examples:

1. Let $E$ be a separable metric space, and $\mathcal{E}$ be the paving consisting of all closed subsets of $E.$ Then a subset $A$ of $E$ is an $\mathcal{E}$-envelope of a given decreasing sequence $\{A_k\} _ {k \in \mathbb N} \subseteq \mathfrak{P}(E)$ if, and only if, $A$ contains $\bigcap_{k \in \mathbb N} \overline{A}_k$, the intersection of the closures of the sets $A_k, k \in \mathbb N$ in the sequence. Indeed, that $A$ is an $\mathcal{E}$-envelope of $\{A_k\} _ {k \in \mathbb N}$ (with the existence of a sequence $\{C_k\}_{k \in \mathbb N}$ as in Definition 5 above) implies $A$ contains $\bigcap_{k \in \mathbb N} \overline{A}_k$ follows from the observation that $\overline{A}_k \subseteq C_k$ for each $k \in \mathbb N$, while the other side is easy to see if we let $C_k = \overline{A}_k$ for each $k \in \mathbb N.$

2. An abstract version of the example above: Let $(E, \mathcal{E})$ be a paved space; for every subset $A$ of $E$, introduce the collection of sets $\mathcal{A} := \{B \in \mathcal{E} \cup \{E\} \,:\, A \subseteq B\}$ and assume that the intersection $\overline{A} := \bigcap_{B \in \mathcal{A}} B$, called the adherent of $A$ in the paving $\mathcal{E}$, belongs to $\mathcal{E}_\delta \cup \{E\}$, i.e., $\overline{A}$ is a countable intersection of sets in $\mathcal{E} \cup \{E\}.$ We claim, and show next, that $A$ is an $\mathcal{E}$-envelope of a given decreasing sequence $\{A_k\}_{k \in \mathbb N} \subseteq \mathfrak{P}(E)$ if, and only if, $A$ contains $\bigcap_{k \in \mathbb N} \overline{A}_k.$

Properties of envelopes

The next lemma lists some properties of envelopes which we will be using frequently.

Capacitance

Intuitively, a capacitance is a collection of “big” sets: the power set $\mathfrak{P}(E)$ is a capacitance, and so are the collections of nonempty and of uncountable subsets of $E.$ The notion of pre-capacity, defined next, gives a more useful example.

If $I \colon \mathfrak{P}(E) \to \overline{\mathbb R}$ is a pre-capacity, then for every given real number $t$ the collection

is a capacitance. Conversely, given a capacitance $\mathcal{C}$, one can associate to it a pre-capacity by defining This then leads to the identification $\mathcal{C} = \{A \in \mathfrak{P}(E) \,:\, I(A) > 0\}.$

Henceforth assume that there is an underlying paved space $(E, \mathcal{E})$ and a capacitance $\mathcal{C}$ of subsets of $E.$

Scrapers

The first property expresses the intuitive notion that $f_n(B_1, B_2, \ldots, B_n)$ “scrapes” $B_n$ and the second property ensures that “the scraping does not remove too big a chunk” from $B_n.$ In French, scraper is called rabotage which can be translated also as planing. A simple example of a scraper is the identity scraper: $f_n(B_1, \ldots B_n) = B_n$ for all $n \in \mathbb N$ and for all sets $B_1, \ldots, B_n \in \mathfrak{P}(E).$

(Dellacherie, 1972) calls $\{P_n\} _ {n \in \mathbb N}$ above as the $\mathfrak f$-scraped sequence deduced from $B.$

If $A \notin \mathcal C$, then no subset of $A$ can be in $\mathcal C$ either by the definition of a capacitance. This implies that there does not exist a $\mathfrak f$-scraped sequence $\{B_n\}_{n \in \mathbb N}$ satisfying $B_1 \subset A.$ On the other hand, with the identity scraper $\mathfrak f$, $A$ envelopes every $\mathfrak f$-scraped sequence $\{B_n\}_{n \in \mathbb N}$ because $B_1$ must equal $A.$ Thus $A$ is smooth.

If $A \in \mathcal C$ and is smooth, then there always exists a sequence $\{B_n\}_{n \in \mathbb N}$ of which $A$ is an envelope. Indeed, by assumption there exists $\mathfrak f$, a scraper, compatible with $A.$ Let $\{B_n\}_{n \in \mathbb N}$ be the $\mathfrak f$-scraped orbit of $A.$ Then since $A$ is smooth, $A$ envelops $\{B_n\}_{n \in \mathbb N}.$

Sierpiński’s Theorem

The next result is central in this theory.

We will come back to its proof later. Let's prove some of its useful consequences first.

This theorem is very useful in proving some important results. We will discuss two of them. The first one is the metric space version of Choquet's capacitability theorem. The proof of it will be very similar to the general Choquet's capacitability theorem, but we will need Sion's theorem for the general version, which will be the second result.

We now come back to the proof of Theorem 1. But first we will need the following clever operation on scrapers, and a couple of results.

Mixing of Scrapers

An immediate corollary of this theorem: If $\{A_n\}_{n \in \mathbb N}$ is a sequence of smooth subsets of $E$, there exists a scraper $\mathfrak{f}$ which is compatible with all the sets $A_n, {n \in \mathbb N}.$

Proof of Sierpiński’s Theorem

We state Theorem 1 again:

Choquet Capacities

Examples:

1. Consider a paved space $(E, \mathcal{E})$, where $\mathcal E$ is a compact paving, and define $I(A) = 0$ if $A = \varnothing$, $I(A) = 1$ if $A \neq \varnothing.$ Then $I$ is a Choquet $\mathcal{E}$-capacity. The property 3. in the definition of a capacity reflects now the assumption that the paving $\mathcal{E}$ is compact.

2. Consider a probability space $(\Omega, \mathcal{F}, \mathbb{P})$, then the outer measure

   $$\begin{aligned} \mathbb{P}^*(A) := \inf_{\substack{B \in \mathcal{F} \\ A \subseteq B}} \mathbb{P}(B),\end{aligned}$$
   which is well-known to be a monotone, countably sub-additive set function taking the value $0$ on the empty set $\varnothing$, is a Choquet $\mathcal{F}$-capacity. Proof of this is a standard exercise in measure theory, albeit with a different terminology of continuity from below.

3. Consider a locally compact, separable metric space $K$, and the paving $\mathscr{K}$ of its compact subsets. If $\pi$ denotes the projection of $K \times \Omega$ onto $\Omega$ and

   $$\begin{aligned} I(A) := \mathbb{P}^*(\pi(A)) \text{ for all } A \in \mathfrak{P}(K \times \Omega),\end{aligned}$$
   then $I$ is a Choquet $(\mathscr{K} \otimes_p \mathcal{F})$-capacity: Monotone increasing property is trivial. To see the ascending property, note that increasing sequence $\{A_n\} _ {n \in \mathbb N} \subseteq \mathfrak{P}(K \times \Omega)$ we have
   $$\begin{aligned} x \in \pi \left( \bigcup_n A_n \right) \iff \exists \; y \in K \text{ such that } (y,x) \in \bigcup_n A_n \iff x \in \bigcup_n \pi(A_n),\end{aligned}$$
   showing $\pi \left( \bigcup_n A_n \right) = \bigcup_n \pi(A_n).$ Thus,
   $$\begin{aligned} I\left( \bigcup_n A_n \right) = \mathbb{P}^*\left( \bigcup_n \pi(A_n) \right) = \sup_n \mathbb{P}^*(\pi(A_n)) = \sup_n I(A_n).\end{aligned}$$
   Finally, for the descending on pavings property, compactness implies that for all $\omega \in \Omega$, $(\mathscr{K} \otimes_p \mathcal{F})(\omega)$ is a compact paving, and thus we can apply Lemma 3 to get
   $$\begin{aligned} \pi\left(\bigcap _ {n \in \mathbb N} E_n\right) = \bigcap _ {n \in \mathbb N} \pi(E_n)\end{aligned}$$
   for every decreasing sequence $\{E_n\} _ {n \in \mathbb N} \subseteq \mathscr{K} \otimes_p \mathcal{F}.$ Therefore,
   $$\begin{aligned} I\left(\bigcap _ {n \in \mathbb N} E_n\right) = \mathbb{P}^* \left(\bigcap _ {n \in \mathbb N} \pi(E_n)\right).\end{aligned}$$
   Since $E_n$ is a finite union of rectangles, $\pi(E_n) \in \mathcal{F}$ and therefore,
   $$\begin{aligned} \mathbb{P}^* \left(\bigcap _ {n \in \mathbb N} \pi(E_n)\right) = \inf_n \mathbb{P}^* (\pi(E_n)) = \inf_n I(E_n).\end{aligned}$$

We are now ready to prove some major results in measure theory.

Measurable Projection

It is not easy to construct an example of a Borel set in the product space whose projection is not Borel. It requires study of analytic sets. Check out Corollary 8.2.17 in (Cohn, 2013) for more details.

Measurable Graph

The next theorem is a very visual theorem, especially for the case where $K$ and $\Omega$ are $\mathbb R.$

Debut

Now let $K = [0, \infty)$, the case important in stochastic processes.

Recall the convention $\inf \varnothing = \infty.$ It is easy to see that $\{D_A < \infty\} = \pi(A).$

Measurable Section

Let $(\Omega, \mathcal{F}, \mathbb{P})$ be a complete probability space, and consider a set $A \subseteq [0, \infty) \times \Omega.$ Then for every $\omega \in \pi(A),$ there exists a $t \in [0, \infty)$ such that $(t, \omega) \in A.$ In other words, we can define a mapping $Z \colon \pi(A) \to [0, \infty).$ It is convenient to extend $Z$ to the whole of $\Omega$ by setting $Z = \infty$ on $\Omega \setminus \pi(A).$ When is it possible to choose $Z$ such that it is measurable? The measurable section theorem (also known as measurable selection theorem) says that it is possible to define $Z$ to be measurable if $A \in \mathcal{B}([0, \infty)) \otimes \mathcal{F}.$

Recall that for $Z \colon \Omega \to [0, \infty]$ its graph is the product set

The condition $(Z(\omega), \omega) \in A$ whenever $Z < \infty$ can then be expressed by saying $\llbracket Z \rrbracket \subseteq A.$

Other than stochastic processes, measurable section theorems have applications in optimal control and game theory.

Epilogue

With this we are done laying the foundations. In the next blog post, we will discuss applications of these results in the general theory of processes.

References

• Cohn, Donald L.. Measure Theory, second edition, Birkhäuser, 2013.

• Dellacherie, Claude. Capacités et processus stochastiques, Springer-Verlag, 1972.

• Dellacherie, Claude and Meyer, Paul-André. Probabilities and Potential, North-Holland Publishing Company, 1978.

• Dellacherie, Claude. Capacities and analytic sets, Part of the Lecture Notes in Mathematics book series (LNM, volume 839), Springer-Verlag, 1981.

• Karoui, Nicole El and Tan, Xiaolu. Capacities, Measurable Selection and Dynamic Programming Part I: Abstract Framework, 2013.

• Lowther, George. Almost Sure blog.

• Meyer, Paul-André. Stochastic Processes from 1950 to the Present (Translated from French by Jeanine Sedjro), 2009.

[1]	If you steal from one author, it’s plagiarism; if you steal from many, it’s research.