Learn Linear Algebra

The Spectral Theorem for Symmetric Matrices

An n×n n \times n symmetric matrix A A has the following properties:

a. A A has n n real eigenvalues, counting multiplicities.
b. The dimension of the eigenspace for each eigenvalue λ \lambda equals the multiplicity of λ \lambda as a root of the characteristic equation.
c. The eigenspaces are mutually orthogonal, in the sense that eigenvectors corresponding to different eigenvalues are orthogonal.
d. A A is orthogonally diagonalizable.

Proof:

Let A A be an n×n n \times n symmetric matrix.

(a) A A has n n real eigenvalues, counting multiplicities:
Since A A is symmetric, the characteristic polynomial of A A has real coefficients. By the Fundamental Theorem of Algebra, the eigenvalues of A A must all be real because symmetric matrices are self-adjoint, and the eigenvalues of self-adjoint matrices are real.

(b) The dimension of the eigenspace for each eigenvalue λ \lambda equals the multiplicity of λ \lambda as a root of the characteristic equation:
Let λ \lambda be an eigenvalue of A A . Since A A is symmetric, it is diagonalizable. Therefore, the geometric multiplicity (dimension of the eigenspace) of λ \lambda equals its algebraic multiplicity (multiplicity as a root of the characteristic equation).

(c) The eigenspaces are mutually orthogonal:
Let λ1 \lambda_1 and λ2 \lambda_2 be distinct eigenvalues of A A , with corresponding eigenspaces E1 E_1 and E2 E_2 . Let u⃗∈E1 \vec{u} \in E_1 and v⃗∈E2 \vec{v} \in E_2 . Since A A is symmetric: (Au⃗)⋅v⃗=(Av⃗)⋅u⃗. (A \vec{u}) \cdot \vec{v} = (A \vec{v}) \cdot \vec{u}. Substituting Au⃗=λ1u⃗ A \vec{u} = \lambda_1 \vec{u} and Av⃗=λ2v⃗ A \vec{v} = \lambda_2 \vec{v} , this becomes: λ1(u⃗⋅v⃗)=λ2(u⃗⋅v⃗). \lambda_1 (\vec{u} \cdot \vec{v}) = \lambda_2 (\vec{u} \cdot \vec{v}). Since λ1≠λ2 \lambda_1 \neq \lambda_2 , it follows that u⃗⋅v⃗=0 \vec{u} \cdot \vec{v} = 0 , proving that E1 E_1 and E2 E_2 are orthogonal.

(d) A A is orthogonally diagonalizable:
Since A A is symmetric, the eigenvalues are real, the eigenspaces are mutually orthogonal, and the geometric multiplicity equals the algebraic multiplicity for all eigenvalues. Therefore, there exists an orthogonal matrix P P (columns are orthonormal eigenvectors of A A ) such that: PTAP=Λ, P^TAP = \Lambda, where Λ \Lambda is a diagonal matrix of eigenvalues. Thus, A A is orthogonally diagonalizable.

Theorem

If A A is symmetric, then any two eigenvectors from different eigenspaces are orthogonal.

Proof:

Let A A be a symmetric n×n n \times n matrix. Assume u⃗ \vec{u} and v⃗ \vec{v} are eigenvectors of A A corresponding to distinct eigenvalues λ1 \lambda_1 and λ2 \lambda_2 , respectively. We aim to prove that u⃗ \vec{u} and v⃗ \vec{v} are orthogonal.

Since u⃗ \vec{u} is an eigenvector of A A with eigenvalue λ1 \lambda_1 , we have: Au⃗=λ1u⃗. A \vec{u} = \lambda_1 \vec{u}. Similarly, since v⃗ \vec{v} is an eigenvector of A A with eigenvalue λ2 \lambda_2 , we have: Av⃗=λ2v⃗. A \vec{v} = \lambda_2 \vec{v}.

Taking the dot product of Au⃗ A \vec{u} with v⃗ \vec{v} , we get: (Au⃗)⋅v⃗=(λ1u⃗)⋅v⃗. (A \vec{u}) \cdot \vec{v} = (\lambda_1 \vec{u}) \cdot \vec{v}. Expanding, this becomes: λ1(u⃗⋅v⃗). \lambda_1 (\vec{u} \cdot \vec{v}).

Now take the dot product of u⃗ \vec{u} with Av⃗ A \vec{v} : u⃗⋅(Av⃗)=u⃗⋅(λ2v⃗). \vec{u} \cdot (A \vec{v}) = \vec{u} \cdot (\lambda_2 \vec{v}). Expanding, this becomes: λ2(u⃗⋅v⃗). \lambda_2 (\vec{u} \cdot \vec{v}).

Since A A is symmetric, we know that: (Au⃗)⋅v⃗=u⃗⋅(Av⃗). (A \vec{u}) \cdot \vec{v} = \vec{u} \cdot (A \vec{v}). Substituting the earlier results, we have: λ1(u⃗⋅v⃗)=λ2(u⃗⋅v⃗). \lambda_1 (\vec{u} \cdot \vec{v}) = \lambda_2 (\vec{u} \cdot \vec{v}).

Rearranging: (λ1−λ2)(u⃗⋅v⃗)=0. (\lambda_1 - \lambda_2) (\vec{u} \cdot \vec{v}) = 0.

Since λ1≠λ2 \lambda_1 \neq \lambda_2 by assumption, it follows that: u⃗⋅v⃗=0. \vec{u} \cdot \vec{v} = 0.

Thus, u⃗ \vec{u} and v⃗ \vec{v} are orthogonal.

Theorem

An n×n n \times n matrix A A is orthogonally diagonalizable if and only if A A is a symmetric matrix.

Proof:

Let A A be an n×n n \times n matrix.

(If direction):
Assume A A is symmetric. By the spectral theorem, all eigenvalues of A A are real, and the eigenspaces corresponding to distinct eigenvalues are mutually orthogonal. Furthermore, the geometric multiplicity of each eigenvalue equals its algebraic multiplicity, allowing A A to be diagonalized. Since the eigenspaces are mutually orthogonal, it is possible to construct an orthogonal matrix P P such that: PTAP=Λ, P^T A P = \Lambda, where Λ \Lambda is a diagonal matrix containing the eigenvalues of A A . Thus, A A is orthogonally diagonalizable.

(Only if direction):
Assume A A is orthogonally diagonalizable. Then there exists an orthogonal matrix P P such that: PTAP=Λ, P^T A P = \Lambda, where Λ \Lambda is a diagonal matrix. Rewriting this, we have: A=PΛPT. A = P \Lambda P^T. Since P P is orthogonal, PT=P−1 P^T = P^{-1} , so: A=PΛP−1. A = P \Lambda P^{-1}. The product PΛPT P \Lambda P^T is symmetric because PT P^T is the transpose of P P and Λ \Lambda is diagonal (hence symmetric). Thus, A A is symmetric.