Eigenvectors and eigenvalues

Section 6.4 Eigenvectors and eigenvalues

Subsection 6.4.1 Eigenvectors

Let $T\colon V\rightarrow V$ be linear. We have seen that some choices of bases of $V$ are better than others for representing $T$ as a matrix.

The ideal situation is when $V$ happens to have a basis $B=\{\boldv_1,\dots, \boldv_n\}$ where $T(\boldv_i)=c_i\boldv_i$ for all $i\text{.}$ The matrix $[T]_B$ in this case is diagonal!

\begin{equation*} [T]_B=\begin{bmatrix}c_1\amp 0\amp \dots\\ 0\amp c_2\amp 0\amp \dots\\ \vdots \amp \\ 0\amp 0\amp \dots\amp c_n \end{bmatrix} \end{equation*}

This motivates us to find as many vectors as we can satisfying $T(\boldv)=c\boldv$ for some $c\text{.}$ Such vectors are called eigenvectors.

Definition 6.4.1.

Let $T\colon V\rightarrow V$ be a linear transformation. A (real) eigenvalue of $T$ is a scalar $\lambda\in \R$ such that there is a nonzero vector $\boldv\in V$ with $T(\boldv)=\boldv\text{.}$ The vector $\boldv$ in this case is called an eigenvector of $T$ with eigenvalue $\lambda\text{.}$

Subsection 6.4.1.1 Comment:

why the switch to $\lambda$ all of a sudden? Tradition!

Subsection 6.4.2 Examples by inspection

Define $T\colon M_{22}\rightarrow M_{22}$ as $T(A)=A^T+A\text{.}$

We have seen that

\begin{align*} T\left(\begin{bmatrix}0\amp 1\\ -1\amp 0 \end{bmatrix} \right)\amp =\amp \begin{bmatrix}0\amp 0\\ 0\amp 0 \end{bmatrix} =0\begin{bmatrix}0\amp 1\\ -1\amp 0 \end{bmatrix}\\ T\left(\begin{bmatrix}0\amp 1\\ 1\amp 0 \end{bmatrix} \right)\amp =\amp \begin{bmatrix}0\amp 2\\ 2\amp 0 \end{bmatrix} =2\begin{bmatrix}0\amp 1\\ 1\amp 0 \end{bmatrix} \text{.} \end{align*}

Thus $\begin{bmatrix}0\amp 1\\ -1\amp 0 \end{bmatrix}$ is an eigenvector with eigenvalue 0, and $\begin{bmatrix}0\amp 1\\ 1\amp 0 \end{bmatrix}$ is an eigenvector with eigenvalue 2.

Subsection 6.4.3 Examples by inspection

Let $T_\theta\colon\R^2\rightarrow\R^2$ be rotation counterclockwise by $\theta\text{,}$ $0\lt \theta\lt 2\pi\text{.}$

What are the eigenvectors/eigenvalues of $T_\theta\text{.}$ (Think geometrically, do not use a matrix representation of $T_\theta\text{!}$) \begin{bsolution} Is there any vector $\boldv$ such that when we rotate it by $\theta$ we get a scalar multiple of $\boldv\text{?}$ Our answer actually depends on exactly what $\theta$ is.

If $\theta=\pi\text{,}$ then $T_\pi$ is rotation by 180$^\circ\text{,}$ and so $T_\pi(\boldv)=-\boldv$ for all $\boldv\text{.}$ Thus everything is an eigenvector of $T_\pi$ with eigenvalue $-1\text{.}$ If $\theta\ne\pi\text{,}$ then we see easily that $T_\theta$ has no eigenvectors. \end{bsolution}

Subsection 6.4.4 Examples by inspection

Fix $\alpha$ with $0\leq \alpha\lt \pi\text{,}$ let $\ell_\alpha$ be the line making an angle $\alpha$ with the positive $x$-axis, and let $T_\theta\colon\R^2\rightarrow\R^2$ be reflection through $\ell_\alpha\text{.}$

What are the eigenvectors/eigenvalues of $T_\alpha\text{.}$ (Think geometrically, do not use a matrix representation of $T_\alpha\text{!}$) \begin{bsolution} If $\boldv$ lies along $\ell_\alpha$ to begin with, then its reflection through $\ell_\alpha$ is itself. This means $T_\alpha(\boldv)=\boldv\text{.}$ Thus all vectors along $\ell_\alpha$ are eigenvectors with eigenvalue 1. If $\boldv$ lies along the line perpendicular to $\ell_\alpha\text{,}$ then its reflection is just $-\boldv$ (draw a picture). Thus all vectors along this line are eigenvectors with eigenvalue $-1\text{.}$ Vectors pointing along any other line will not be eigenvectors. Draw a picture!

\end{bsolution}

Subsection 6.4.5 Finding eigenvalues of $T$ systematically

Let $T\colon V\rightarrow V$ be linear, $\dim(V)=n\text{.}$

Our first step for finding eigenvalues and eigenvectors of $T$ is to pick a basis $B$ for $V$ and represent $T$ with the matrix $A=[T]_B\text{.}$

You get to choose the basis $B$ yourself, though typically we start with the standard basis of $V\text{.}$ It is usually only after our eigenvector analysis that we can provide a cleverer choice of basis.

Once we have chosen an $A$ to represent $T\text{,}$ we perform our analysis on $A\text{.}$ As usual, anything we discover about $A$ will be true of $T\text{.}$ For example:

$A$ and $T$ have precisely the same eigenvalues.
Eigenvectors of $A$ can be translated back to $V$ to get eigenvectors of $T\text{.}$

Subsection 6.4.6 Finding eigenvalues of $A$

We now focus on finding eigenvalues of an $n\times n$ matrix $A\text{.}$ Follow this chain of equivalences:

\begin{align*} \text{ $\lambda\in\R$ is an eigenvalue of $A$ } \amp \Leftrightarrow\amp A\boldx=\lambda\boldx \text{ for some $\boldx\ne 0$ } \ \text{ (by def.) }\\ \amp \Leftrightarrow\amp \lambda\boldx-A\boldx=\boldzero\\ \amp \Leftrightarrow\amp (\lambda I_n)\boldx-A\boldx=\boldzero \ \text{ (since $\lambda\boldx=(\lambda I_n)\boldx$) }\\ \amp \Leftrightarrow\amp (\lambda I_n-A)\boldx=\boldzero \text{ for some $\boldx\ne \boldzero$ }\\ \amp \Leftrightarrow\amp \lambda I_n-A \text{ is noninvertible }\\ \amp \Leftrightarrow\amp \det(\lambda I_n-A)=0 \ \text{ (invertibility theorem) } \end{align*}

Thus we see that $\lambda$ is an eigenvalue if and only if $\det(\lambda I_n-A)=0\text{.}$

Set $p(t)=\det(tI_n-A)\text{.}$ This is polynomial function in $t\text{,}$ as you will see shortly. We have just shown that the eigenvalues of $A$ are precisely the real roots of $p(t)\text{.}$ Better give this important object a name!

Definition 6.4.2.

Let $A$ be $n\times n\text{.}$ The characteristic polynomial of $A$ is the polynomial

\begin{equation*} p(t)=\det(tI_n-A)\text{.} \end{equation*}

Subsection 6.4.7 Example

Find all eigenvalues of $A=\begin{bmatrix}1\amp 1\\ 2\amp -1 \end{bmatrix}\text{.}$ \begin{bsolution} Compute:

$p(t)=\det(tI-A)=\val{\begin{array}{cc}t-1\amp -1\\ -2\amp t+1 \end{array} }=t^2-3\text{.}$

The eigenvalues are the roots of $p(t)\text{:}$ $\lambda_1=\sqrt{3}, \lambda_2=-\sqrt{3}\text{.}$ \end{bsolution}

Subsection 6.4.8 Example

Find all eigenvalues of $A=\begin{bmatrix}2\amp -1\amp -1\\ -1\amp 2\amp -1\\ -1\amp -1\amp 2 \end{bmatrix}\text{.}$ \begin{bsolution} Compute:

$p(t)=\det(tI-A)=\val{\begin{array}{ccc}t-2\amp 1\amp 1\\ 1\amp t-2\amp 1\\ 1\amp 1\amp t-2 \end{array} }=t^3-6t^2+9t=t(t-3)^2\text{.}$

The eigenvalues are the roots of $p(t)\text{:}$ $\lambda_1=0\text{,}$ $\lambda_2=3\text{.}$ \end{bsolution}

Subsection 6.4.9 Facts about the characteristic polynomial

Let $A$ be $n\times n$ and let $p(t)=\det(tI-A)$ be the characteristic polynomial.

$p(t)=t^n+a_{n-1}t^{n-1}+\cdots +a_1t+a_0\text{;}$ in particular, $p(t)$ is monic (leading coefficient is 1), of degree $n\text{.}$
$p(t)$ has at most $n$ distinct roots. Some of these roots may be complex. Thus it is possible to have less than $n$ eigenvalues. Indeed, there may be no eigenvalues. What about the possible complex roots? Let's call these complex eigenvalues. For now they do not count as eigenvalues for us, as it is impossible to find a nonzero $\boldx\in\R^n$ with a complex eigenvalue.

Subsection 6.4.10 Fancy facts about $p(t)$

Let $p(t)=\det(tI-A)=t^n+a_{n-1}t^{n-1}+\cdots +a_1t+a_0$ be the characteristic polynomial.

Over $\C$ we can factor $p(t)=(t-\lambda_1)(t-\lambda_2)\cdots (t-\lambda_n)\text{,}$ where some of the $\lambda_i$ may be complex numbers.

Then we have:

\begin{align*} -(\lambda_1+\lambda_2+\cdots +\lambda_n)\amp =\amp a_{n-1}=-\tr(A)\\ (-1)^n\lambda_1\lambda_2\cdots\lambda_n\amp =\amp a_0=(-1)^n\det(A) \end{align*}

An easy consequence of these equalities is the following:

\begin{align*} \tr(A)\amp =\amp \lambda_1+\lambda_2+\cdots +\lambda_n\\ \det(A)\amp =\amp \lambda_1\lambda_2\cdots\lambda_n \end{align*}

In plain English: $\tr(A)$ is the sum of the eigenvalues; $\det(A)$ is the product of the eigenvalues.

When $A$ is $2\times 2\text{,}$ the above implies $p(t)=t^2-\tr(A)t+\det(A)\text{:}$ a useful shortcut when having to compute eigenvalues of $2\times 2$ matrices!

Let's make our list of facts official.

Theorem 6.4.3. Characteristic polynomial theorem.

Let $A\in M_{nn}\text{,}$ and let $p(t)=\det(tI_n-A)$ be its characteristic polynomial.

$p(t)=t^n+a_{n-1}t^{n-1}+\cdots +a_1t+a_0\text{;}$ in particular, $p(t)$ is monic (leading coefficient is 1), of degree $n\text{.}$
The real roots of $p(t)$ are the (real) eigenvalues of $A\text{.}$ Since $\deg p(t)=n\text{,}$ there are at most $n$ distinct eigenvalues of $A\text{.}$
Let $\lambda_1, \lambda_2, \dots, \lambda_n$ be the roots of $p(t)\text{.}$ Then

\begin{align*} \tr(A)\amp =\sum_{i=1}^n\lambda_i=-a_{n-1}\\ \det(A)\amp =\prod_{i=1}^n\lambda_i=(-1)^na_0 \end{align*}

Subsection 6.4.11 Computing eigenspaces of a matrix

Now suppose we have found an eigenvalue $\lambda$ of $A\text{.}$

Our chain of equivalences from earlier told us that the eigenvectors of $A$ with eigenvalue $\lambda$ are precisely the nonzero solutions to the matrix equation $(\lambda I_n-A)\boldx=\boldzero\text{.}$

Thus to find all eigenvectors with eigenvalue $\lambda$ we simply compute $\NS(\lambda I_n-A)$ using Gaussian elimination.

Definition 6.4.4.

Let $\lambda$ be an eigenvalue of $A\text{.}$ We define the $\lambda$-eigenspace of $A$ to be the space

\begin{equation*} W_\lambda=\NS(\lambda I_n-A)\text{.} \end{equation*}

The nonzero elements of $W_\lambda$ are precisely the eigenvectors of $A$ with eigenvalue $\lambda\text{.}$

Procedure 6.4.5. Computing eigenspaces of a matrix.

Let $A$ be an $n\times n$ matrix. To find all real eigenvalues of $A$ and compute bases for their eigenspaces, proceed as follows.

Compute $p(t)=\det(tI-A)\text{.}$ Let $\lambda_1, \lambda_2, \dots, \lambda_r$ be the distinct real roots of $p(t)\text{.}$ These are the real eigenvalues of $A\text{.}$
Given real eigenvalue $\lambda$ its corresponding eigenspace is

\begin{equation*} W_{\lambda}=\NS(\lambda I-A)\text{.} \end{equation*}

Use the null space algorithm (4.6.7) to compute a basis for $W_\lambda$

Let's compute the eigenspaces of our previous examples.

Subsection 6.4.12 Example

$A=\begin{bmatrix}1\amp 1\\ 2\amp -1 \end{bmatrix}\text{,}$ $\lambda_1=\sqrt{3}\text{,}$ $\lambda_2=-\sqrt{3}\text{.}$ \begin{bsolution}

Compute:

$W_{\sqrt{3}}=\NS(\sqrt{3}I-A)=\NS\left(\begin{bmatrix}\sqrt{3}-1\amp -1\\ -2\amp \sqrt{3}+1 \end{bmatrix} \right) =\Span(\left\{ \begin{bmatrix}\sqrt{3}+1\\2 \end{bmatrix} \right\})$

$W_{-\sqrt{3}}=\NS(-\sqrt{3}I-A)=\NS\left(\begin{bmatrix}-\sqrt{3}-1\amp -1\\ -2\amp -\sqrt{3}+1 \end{bmatrix} \right) =\Span(\left\{ \begin{bmatrix}-\sqrt{3}+1\\ 2 \end{bmatrix} \right\})$

Thus any multiple of $\boldv_1=(\sqrt{3}+1,2)$ is an eigenvector of $A$ with eigenvalue $\sqrt{3}\text{,}$ and any multiple of $\boldv_2=(-\sqrt{3}+1,2)$ is an eigenvector of $A$ with eigenvalue $-\sqrt{3}\text{.}$

Check: $A\begin{bmatrix}\sqrt{3}+1\\ 2 \end{bmatrix} =\begin{bmatrix}3\sqrt{3}+3\\ 2\sqrt{3} \end{bmatrix} =\sqrt{3}\begin{bmatrix}\sqrt{3}+1\\ 2 \end{bmatrix} \checkmark\text{.}$ $A\begin{bmatrix}-\sqrt{3}+1\\ 2 \end{bmatrix} =\begin{bmatrix}-3\sqrt{3}+3\\ -2\sqrt{3} \end{bmatrix} =-\sqrt{3}\begin{bmatrix}-\sqrt{3}+1\\ 2 \end{bmatrix} \checkmark\text{.}$

\end{bsolution}

Let's compute the eigenspaces of our previous examples.

Subsection 6.4.13 Example

$A=\begin{bmatrix}2\amp -1\amp -1\\ -1\amp 2\amp -1\\ -1\amp -1\amp 2 \end{bmatrix}\text{,}$ $\lambda_1=0\text{,}$ $\lambda_2=3\text{.}$ \begin{bsolution} Compute:

$W_0=\NS(0I-A)=\NS\left(\begin{bmatrix}2\amp -1\amp -1\\ -1\amp 2\amp -1\\ -1\amp -1\amp 2 \end{bmatrix} \right)=\Span\left(\left\{\boldv_1=\begin{bmatrix}1 \\ 1 \\ 1 \end{bmatrix} \right\}\right)\text{.}$

$W_3=\NS(3I-A)=\NS(\left( \begin{bmatrix}1\amp 1\amp 1\\ 1\amp 1\amp 1\\ 1\amp 1\amp 1 \end{bmatrix} \right) =\Span\left(\left\{\boldv_2= \begin{bmatrix}1 \\ -1 \\ 0 \end{bmatrix} , \boldv_3=\begin{bmatrix}1 \\ 0 \\ -1 \end{bmatrix} \right\}\right)\text{.}$

Thus the eigenvectors of $A$ with eigenvalue 0 are vectors of the form $c\boldv_1\text{,}$ and the eigenvectors of $A$ with eigenvalue 3 are vectors of the form $c\boldv_2+d\boldv_3\text{.}$

Subsection 6.4.14 Comment

Note that in general $W_0=\NS(0I-A)=\NS(-A)=\NS(A)\text{.}$ (Think about why the last equality is true.)

Thus the eigenvectors of $A$ with eigenvalue 0 are precisely the nonzero vectors of $\NS(A)\text{,}$ should there be any at all! \end{bsolution}