ZoomNotes for Linear Algebra

Part 6: Eigenvalues and Eigenvectors: $Ax = \lambda x$ and $A^n x = \lambda^n x$

6.1 Eigenvalues $\lambda$ and Eigenvectors $x$: $Ax = \lambda x$

An eigenvector $x$ of a square matrix $A$ is a nonzero vector that, when multiplied by $A$, only scales (does not rotate):

$$Ax = \lambda x$$

The scalar $\lambda$ is the corresponding eigenvalue.

Finding eigenvalues: Solve the characteristic equation:

$$\det(A - \lambda I) = 0$$

This is a polynomial of degree $n$ in $\lambda$.

Finding eigenvectors: For each eigenvalue $\lambda_i$, solve:

$$(A - \lambda_i I) x = 0$$

The eigenvectors span the eigenspace corresponding to $\lambda_i$.

Key Properties:

• Trace: $\text{trace}(A) = \sum_{i=1}^n a_{ii} = \sum_{i=1}^n \lambda_i$
• Determinant: $\det A = \prod_{i=1}^n \lambda_i$
• Inverse: $A^{-1} x = (1/\lambda) x$ (if $\lambda \neq 0$)
• Powers: $A^n x = \lambda^n x$
• Shift: $A - cI$ has eigenvalues $\lambda_i - c$

Special Cases:

• Triangular matrices: eigenvalues are the diagonal entries
• Singular matrix: $\lambda = 0$ is an eigenvalue (since $\det A = 0$)
• Identity matrix: $\lambda = 1$ (multiplicity $n$)

---

6.2 Diagonalizing a Matrix: $X^{-1} A X = \Lambda$

If $A$ has $n$ independent eigenvectors (forming a matrix $X$), then:

$$A = X \Lambda X^{-1}$$

where $\Lambda = \text{diag}(\lambda_1, \dots, \lambda_n)$.

Verification: $AX = A[x_1 \; \cdots \; x_n] = [\lambda_1 x_1 \; \cdots \; \lambda_n x_n] = X\Lambda$.

Powers are easy when diagonalized:

$$A^k = X \Lambda^k X^{-1}$$

Each eigenvalue is raised to the $k$-th power independently.

Similar Matrices: $B = M^{-1} A M$ has the same eigenvalues as $A$. The eigenvectors are transformed: if $Ax = \lambda x$, then $B(M^{-1}x) = \lambda (M^{-1}x)$.

When is a matrix NOT diagonalizable? When it has repeated eigenvalues but not enough independent eigenvectors. For example, a $2 \times 2$ Jordan block $\begin{bmatrix} \lambda & 1 \\ 0 & \lambda \end{bmatrix}$ has only one eigenvector.

---

6.3 Symmetric Positive Definite Matrices: Five Tests

Symmetric Matrices $S = S^T$:

• All eigenvalues are real
• Eigenvectors for distinct eigenvalues are orthogonal
• Spectral Theorem: $S = Q \Lambda Q^T$ where $Q$ is orthogonal

Positive Definite Matrices: A symmetric matrix $S$ is positive definite if any of these equivalent conditions holds:

1. All eigenvalues $\lambda_i > 0$
2. Energy $x^T S x > 0$ for all nonzero $x$
3. All pivots are positive (from $LDL^T$ factorization)
4. All leading principal minors (determinants of upper-left submatrices) are positive
5. $S = A^T A$ for some matrix $A$ with independent columns

Positive Semidefinite: Conditions above with $\geq 0$ (equality allowed).

Example: The second difference matrix $K$ (tridiagonal with 2 on diagonal, -1 on off-diagonals) is positive definite. For $n=3$:

$$K = \begin{bmatrix} 2 & -1 & 0 \\ -1 & 2 & -1 \\ 0 & -1 & 2 \end{bmatrix}$$

Its eigenvalues are $\lambda_k = 2 - 2\cos(k\pi/(n+1)) = 4\sin^2(k\pi/(2(n+1)))$, all positive.

---

6.4 Linear Differential Equations $du/dt = Au$

Consider a system of linear ODEs:

$$\frac{du}{dt} = A u, \quad u(0) = u_0$$

Solution via Eigenvectors: Assuming $A$ is diagonalizable with eigenvectors $x_1, \dots, x_n$, express $u_0$ as:

$$u_0 = c_1 x_1 + c_2 x_2 + \cdots + c_n x_n$$

Then:

$$u(t) = c_1 e^{\lambda_1 t} x_1 + c_2 e^{\lambda_2 t} x_2 + \cdots + c_n e^{\lambda_n t} x_n$$

Matrix Exponential:

$$e^{At} = I + At + \frac{(At)^2}{2!} + \frac{(At)^3}{3!} + \cdots$$

When $A = X\Lambda X^{-1}$:

$$e^{At} = X e^{\Lambda t} X^{-1} = X \begin{bmatrix} e^{\lambda_1 t} & & \\ & \ddots & \\ & & e^{\lambda_n t} \end{bmatrix} X^{-1}$$

Stability:

• Asymptotically stable: $u(t) \to 0$ as $t \to \infty$ if all $\text{Re}(\lambda_i) < 0$
• Weakly stable: $u(t)$ remains bounded if $\text{Re}(\lambda_i) \leq 0$ and any $\lambda_i = 0$ is semisimple
• Unstable: Any $\text{Re}(\lambda_i) > 0$

---

6.5 Matrices in Engineering: Second Differences

Second Difference Matrix $K$ (discretizes $-d^2u/dx^2$):

$$K = \begin{bmatrix} 2 & -1 & & \\ -1 & 2 & -1 & \\ & \ddots & \ddots & \ddots \\ & & -1 & 2 \end{bmatrix}$$

Energy: $x^T K x = \sum_{i=1}^{n-1} (x_i - x_{i+1})^2 > 0$ for nonzero $x$, proving $K$ is positive definite.

Spring-Mass Systems:

• Steady state: $Ku = g$ (spring displacements due to gravity)
• Oscillation: $-K u = m \, d^2u/dt^2$ (free vibrations)

The eigenvalues of $K$ determine the natural frequencies of vibration: $\omega_k = \sqrt{\lambda_k}$.

Orthogonal eigenvectors of $K$: The eigenvectors of this tridiagonal matrix are discrete sines:

$$(v_k)_j = \sin\left(\frac{jk\pi}{n+1}\right)$$

These are exactly the eigenvectors that diagonalize the second difference operator -- the discrete analogue of Fourier sine series.