Legendre transformation

From Example Problems
Jump to navigation Jump to search

In mathematics, two differentiable functions f and g are said to be Legendre transforms of each other if their first derivatives are inverse functions of each other:

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle Df = \left( Dg \right)^{-1}}

f and g are then said to be related by a Legendre transformation. Legendre transformations are named after Adrien-Marie Legendre. They are unique up to an additive constant which is usually fixed by the additional requirement that

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f(x) + g(y) = \left\langle x,y\right\rangle.}

The Legendre transformation is its own inverse, and is related to integration by parts.

Applications

Legendre transformations are used in thermodynamics to transform between the different thermodynamic potentials, and in classical mechanics to derive Hamiltonian mechanics from Lagrangian mechanics, as well as the other way around.

Examples

The exponential function ex has x ln xx as a Legendre transform since the respective first derivatives ex and ln x are inverse to each other. This example shows that the respective domains of a function and its Legendre transform need not agree.

Similarly, the quadratic form

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle u(x) = \frac{1}{2} \, x^t \, A \, x }

with A a symmetric invertible n-by-n-matrix has

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle v(y) = \frac{1}{2} \, y^t \, A^{-1} \, y }

as a Legendre transform.

Legendre transformation in one dimension

In one dimension, a Legendre transform to a function f : R → R with an invertible first derivative may be found using the formula

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle g(y) = y \, x - f(x), \, x = f^{\prime-1}(y) }

This can be seen by integrating both sides of the defining condition restricted to one-dimension

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f^\prime(x) = g^{\prime-1}(x) }

from x0 to x1, making use of the fundamental theorem of calculus on the left hand side and substituting

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle y = g^{\prime-1}(x) }

on the right hand side to find

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f(x_1) - f(x_0) = \int_{y_0}^{y_1} y \, g^{\prime\prime}(y) \, dy }

with g′(y0) = x0, g′(y1) = x1. Using integration by parts the last integral simplifies to

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle y_1 \, g^\prime(y_1) - y_0 \, g^\prime(y_0) - \int_{y_0}^{y_1} g^\prime(y) \, dy = y_1 \, x_1 - y_0 \, x_0 - g(y_1) + g(y_0) }

Therefore,

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f(x_1) + g(y_1) - y_1 \, x_1 = f(x_0) + g(y_0) - y_0 \, x_0 }

Since the left hand side of this equation does only depend on x1 and the right hand side only on x0, they have to evaluate to the same constant.

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f(x) + g(y) - y \, x = C,\, x = g^\prime(y) = f^{\prime-1}(y) }

Solving for g and choosing C to be zero results in the above-mentioned formula.

Geometric interpretation

For a strictly convex function the Legendre-transformation can be interpreted as a mapping between the graph of the function and the family of tangents of the graph. (The tangents are well-defined at all but at most countably many points since a convex function is differentiable at all but at most countably many points.)

The equation of a line with slope m and y-intercept b is given by

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle y = mx + b}

For this line to be tangent to the graph of a function f at the point (x0, f(x0)) requires

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f\left(x_0\right) = m x_0 + b}

and

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle m = f^{\prime}\left(x_0\right)}

f′ is strictly monotone as the derivative of a strictly convex function, and the second equation can be solved for x0, allowing to eliminate x0 from the first giving the y-intercept b of the tangent as a function of its slope m:

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle b = f\left(f^{\prime-1}\left(m\right)\right) - m \cdot f^{\prime-1}\left(m\right) }

Legendre transformation in more than one dimension

For a differentiable real-valued function on an open subset U of Rn the Legendre conjugate of the pair (U, f) is defined to be the pair (V, g), where V is the image of U under the gradient mapping Df, and g is the function on V given by the formula

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle g(y) = \left\langle y, x \right\rangle - f\left(x\right), \, x = \left(Df\right)^{-1}(y) }

where

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \left\langle u,v\right\rangle = \sum_{k=1}^{n}u_{k} \cdot v_{k}}

is the scalar product on Rn.

Alternatively, if X is a real vector space and Y is its dual vector space, then for each point x of X and y of Y, there is a natural identification of the cotangent spaces T*Xx with Y and T*Yy with X. If f is a real differentiable function over X, then ∇f is a section of the cotangent bundle T*X and as such, we can construct a map from X to Y. Similarly, if g is a real differentiable function over Y, ∇g defines a map from Y to X. If both maps happen to be inverses of each other, we say we have a Legendre transform.

Convex conjugates

Definition

For a function

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f:\mathbb{R}^n\rightarrow\mathbb{R}\cup\{+\infty\}}

taking values on the extended real number line the Legendre transformation can be generalized to the Legendre-Fenchel transformation or convex conjugate of f by

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f^\star:\mathbb{R}^n\rightarrow\mathbb{R}\cup\{+\infty\}}
Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f^{\star}\left(p\right) = \sup\left\{\left\langle x,p\right\rangle - f\left(x\right) : x \in \mathbb{R}^n \right\} = - \inf\left\{f\left(x\right) - \left\langle x,p\right\rangle : x \in \mathbb{R}^n \right\} }

Examples of convex conjugates

The convex conjugate of an affine function

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f(x) = \left\langle a,x \right\rangle - b,\, a \in \mathbb{R}^n, b \in \mathbb{R} }

is

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f^\star\left(x^\star\right) = \begin{cases} b, & x^\star = a \\ \infty, & x^\star \ne a \end{cases} }

The convex conjugate of the absolute value function

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f(x) = \left| x \right|}

is

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f^\star\left(x^\star\right) = \begin{cases} 0, & \left|x^\star\right| \le 1 \\ \infty, & \left|x^\star\right| > 1 \end{cases} }

The convex conjugate of the exponential function is

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \exp^\star\left(x^\star\right) = \begin{cases} x^\star \ln x^\star - x^\star, & x^\star > 0 \\ 0 , & x^\star = 0 \\ \infty , & x^\star < 0 \end{cases} }

Properties of convex conjugation

Convex-conjugation is order-reversing: if fg then f*g*. The convex conjugate of a closed convex function is again a closed convex function. The convex conjugate of a polyhedral convex function (a convex function with polyhedral epigraph) is again a polyhedral convex function. For any proper convex function f and its convex conjugate f* Fenchel's inequality (also known as the Fenchel-Young inequality) holds:

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \left\langle p,x \right\rangle \le f(x) + f^\star(p) }

The convex conjugate of a function is always lower semi-continuous. The biconjugate f** (the convex conjugate of the convex conjugate) is also the closed convex hull, i.e. the largest lower semi-continuous convex function smaller than f. Therefore, f = f** iff f is convex and lower semi-continuous.

Further properties

Scaling properties

The Legendre transformation has the following scaling properties:

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f(x) = a \cdot g(x) \Rightarrow f^\star(p) = a \cdot g^\star\left(\frac{p}{a}\right) }
Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f(x) = g(a \cdot x) \Rightarrow f^\star(p) = g^\star\left(\frac{p}{a}\right) }

It follows that if a function is homogeneous of degree r then its image under the Legendre transformation is a homogeneous function of degree s, where 1/r + 1/s = 1.

Behavior under translation

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f(x) = g(x) + b \Rightarrow f^\star(p) = g^\star(p) - b }
Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f(x) = g(x + y) \Rightarrow f^\star(p) = g^\star(p) - p \cdot y }

Behavior under inversion

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f(x) = g^{-1}(x) \Rightarrow f^\star(p) = - p \cdot g^\star\left(\frac{1}{p}\right) }

Behavior under linear transformations

Let A be a linear transformation from Rn to Rm. For any convex function f on Rn, one has

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \left(A f\right)^\star = f^\star A^\star }

where A* is the adjoint operator of A defined by

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \left \langle Ax, y^\star \right \rangle = \left \langle x, A^\star y^\star \right \rangle }

A closed convex function f is symmetric with respect to a given set G of orthogonal linear transformations,

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f\left(A x\right) = f(x), \; \forall x, \; \forall A \in G }

if and only if f* is symmetric with respect to G.

Infimal convolution

The infimal convolution of two functions f and g is defined as

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \left(f \star_\inf g\right)(x) = \inf \left \{ f(x-y) + g(y) \, | \, y \in \mathbb{R}^n \right \} }

Let f1, …, fm be proper convex functions on Rn. Then

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \left( f_1 \star_\inf \cdots \star_\inf f_m \right)^\star = f_1^\star + \cdots + f_m^\star }

References

de:Legendre-Transformation ko:르장드르 변환 it:Trasformata di Legendre sl:Legendrova transformacija