Change of basis

Section 6.3 Change of basis

We have seen how the coordinate vector map \([\hspace{10pt}]_B\) and matrix representations \([T]_B^{B'}\) are two invaluable computational tools for dealing with abstract vector spaces.

As the notation indicates, both of these operations depend essentially on your choice of basis or bases. This gives rise to the following questions:

Given \(V\) and two choices of basis, \(B\) and \(B'\text{,}\) what is the relation between \([\boldv]_B\) and \([\boldv]_{B'}\text{?}\)
Given \(T\colon V\rightarrow W\) and two choices of pairs of bases, \((B, B')\) and \((B'', B''')\text{,}\) what is the relation between \([T]_{B}^{B'}\) and \([T]_{B''}^{B'''}\text{?}\)

We will tackle each question in turn. Both answers rely on something called a change of basis matrix \(\underset{B\rightarrow B'}{P}\text{.}\)

Subsection 6.3.1 Change of basis matrices

Definition 6.3.1. Change of basis matrix.

Let \(B=(\boldv_1, \boldv_2, \dots, \boldv_n)\) and \(B'\) be two ordered bases for the vector space \(V\text{.}\) The change of basis from \(B\) to \(B'\) is the \(n\times n\) matrix \(\underset{B\rightarrow B'}{P}\) defined as

\begin{equation*} \underset{B\rightarrow B'}{P}= \begin{bmatrix} \vert \amp \vert \amp \amp \vert \\ [\boldv_1]_{B'} \amp [\boldv_2]_{B'}\amp \dots \amp [\boldv_n]_{B} \end{bmatrix}\text{.} \end{equation*}

In other words, the \(j\)-th column of \(\underset{B\rightarrow B'}{P}\) is obtained by computing the coordinate vector of the \(j\)-th element of the original basis \(B\) with respect to the new basis \(B'\text{.}\)

Theorem 6.3.2. Change of basis for coordinate vectors.

Let \(B\) and \(B'\) be two ordered bases of the \(n\)-dimensional vector space \(V\text{.}\) For all \(\boldv\in V\) we have

\begin{equation*} \underset{B\rightarrow B'}{P}[\boldv]_B=[\boldv]_{B'}\text{.} \end{equation*}

In other words, to convert the \(B\)-coordinates of a vector \(\boldv\in V\) to \(B'\)-coordinates, simply multiply on the left by \(\underset{B\rightarrow B'}{P}\text{.}\)

Proof.

Let \(I_V\colon V\rightarrow V\) be the identity transformation: i.e., \(I_V(\boldv)=\boldv\) for all \(\boldv\in V\text{.}\) By Theorem 6.2.2 the matrix \([I_V]_B^{B'}\) is the unique matrix satisfying

\begin{equation*} [I_V]_B^{B'}[\boldv]_B=[I_V(\boldv)]_{B'}=[\boldv]_{B'}\text{.} \end{equation*}

Comparing the formulas for \([I_V]_{B}^{B'}\) and \(\) we see directly that

\begin{equation*} [I_V]_B^{B'} = \underset{B\rightarrow B'}{P}\text{.} \end{equation*}

Subsection 6.3.2 Example

Let \(V=\R^2\text{.}\) Compute \(\underset{B\rightarrow B'}{P}\) where \(B=\{\boldv_1=(1,1),\boldv_2=(1,-1)\}\) and \(B'=\{(1,2),(2,1)\}\text{.}\)

Test that the matrix converts correctly using the vector \(\boldv=1(1,1)+3(1,-1)=(4,-2)\text{.}\) \begin{bsolution} The recipe tells us that

\begin{align*} \underset{B\rightarrow B'}{P}\amp =\amp \begin{bmatrix}\vert \amp \vert \\ \hspace{7pt}[\boldv_1]_{B'} \amp \hspace{7pt}[\boldv_2]_{B'}\\ \vert \amp \vert \end{bmatrix}\\ \amp =\amp \begin{bmatrix}1/3\amp -1\\ 1/3\amp 1 \end{bmatrix} \hspace{8pt}\text{ (after some computation) } \end{align*}

For \(\boldv=1(1,1)+3(1,-1)=(4,-2)\text{,}\) we have \((\boldv)_B=(1,3)\text{.}\) Thus we should have

\begin{equation*} [\boldv]_{B'}=\underset{B\rightarrow B'}{P}[\boldv]_B=\begin{bmatrix}1/3\amp -1\\ 1/3\amp 1 \end{bmatrix} \begin{bmatrix}1\\ 3 \end{bmatrix} =\begin{bmatrix}-8/3\\ 10/3 \end{bmatrix}\text{.} \end{equation*}

Indeed, one easily verifies that \((4,-2)=-8/3(1,2)+10/3(2,1)\text{.}\) \end{bsolution}

Subsection 6.3.3 Example

Take \(V=P_2\text{,}\) \(B=\{1,x,x^2\}\) and \(B'=\{1,(x-2), (x-2)^2\}\text{.}\) Compute \(\underset{B\rightarrow B'}{P}\text{.}\)

Follow the recipe: let \(\boldp_j\) be the \(j\)-th column of \(\underset{B\rightarrow B'}{P}\text{.}\) We have (after some computation)

\begin{equation*} \boldp_1=[1]_{B'}=(1,0,0), \ \boldp_2=[x]_{B'}=(2,1,0), \ \boldp_3=[x^2]_{B'}=(4,4,1)\text{.} \end{equation*}

Thus \(\underset{B\rightarrow B'}{P}=\begin{bmatrix}1\amp 2\amp 4\\ 0\amp 1\amp 4\\ 0\amp 0\amp 1 \end{bmatrix}\)

Let's check with the test vector \(p(x)=1+x+x^2\text{.}\) We have \((p)_B=(1,1,1)\text{.}\) Thus we should have \([p]_{B'}=\underset{B\rightarrow B'}{P}[p]_B=\begin{bmatrix}1\amp 2\amp 4\\ 0\amp 1\amp 4\\ 0\amp 0\amp 1 \end{bmatrix} \begin{bmatrix}1 \\ 1 \\ 1 \end{bmatrix} =\begin{bmatrix}7\\ 5\\ 1 \end{bmatrix}\text{.}\) Equivalently, this means that \(p(x)=7+5(x-2)+(x-2)^2\text{,}\) as one easily verifies.

Subsection 6.3.3.1 Cool fact

We could have derived the last equality using the theory of Taylor series. Namely any polynomial can be “expanded around \(x=a\)” as \(p(x)=\sum_{i=0}^n\frac{p^{(i)}(a)}{a!}(x-a)^i\text{.}\)

More generally, this means

\begin{equation*} (p(x))_{B'}=(p(a), p'(a), p''(a)/2, \dots , p^{(n)}(a)/n!) \end{equation*}

where \(B'=\{1,x-a, (x-a)^2,\dots , (x-a)^n\}\text{.}\)

Theorem 6.3.3. Change of basis matrix properties.

Let \(B, B', B''\) be ordered bases for the \(n\)-dimensional vector space \(V\text{.}\)

We have

\begin{equation*} \underset{B\rightarrow B}{P}=I\text{.} \end{equation*}
The matrix \(\underset{B\rightarrow B'}{P}\) is invertible. In fact, we have

\begin{equation*} \underset{B\rightarrow B'}{P}^{-1}=\underset{B'\rightarrow B}{P} \end{equation*}
We have

\begin{equation*} \underset{B\rightarrow B''}{P}=\underset{B'\rightarrow B''}{P}\, \underset{B\rightarrow B'}{P}\text{.} \end{equation*}

Subsection 6.3.4 Example: \(V=\R^n\) and \(B\) is standard

Consider the simple example where \(V=\R^n\text{,}\) \(B\) is the standard basis, and \(B'=\{\boldv_1,\dots,\boldv_n\}\) is some nonstandard basis. I claim the matrix \(P=\begin{bmatrix}\vert\amp \dots \amp \vert \\ \boldv_1\amp \cdots\amp \boldv_n\\ \vert\amp \dots \amp \vert \end{bmatrix}\) whose columns are the elements of \(B'\) is the change of basis matrix \(\underset{B'\rightarrow B} P\text{.}\) This follows from our recipe since \(\boldv_j=[\boldv_j]_B\text{.}\) (Recall: when \(B\) is the standard basis \([(a_1,a_2,\dots, a_n)]_B=(a_1,a_2,\dots, a_n)\) for all \((a_1,a_2,\dots, a_n)\in\R^n\text{.}\) )

Since \(\underset{B\rightarrow B'}{P}=(\underset{B'\rightarrow B}{P})^{-1}\text{,}\) we see that in this special case we can compute \(\underset{B'\rightarrow B}{P}\) by placing the elements of \(B'\) as columns of a matrix, and then compute \(\underset{B\rightarrow B'}{P}\) by taking the inverse of this matrix!

Subsection 6.3.4.1 Example

Let \(V=\R^2\text{,}\) \(B\) the standard basis for \(\R^2\text{,}\) and \(B'=\{(1,\sqrt{3}),(-\sqrt{3},1)\}\text{.}\)

Find \(P_{B\rightarrow B'}\text{.}\) \begin{bsolution} The recipe above tells us that \(\underset{B'\rightarrow B}{P}=\begin{bmatrix}1\amp -\sqrt{3}\\ \sqrt{3}\amp 1 \end{bmatrix}\) and hence that

\begin{equation*} \underset{B\rightarrow B'}{P}=(\underset{B'\rightarrow B}{P})^{-1}=\left(\begin{bmatrix}1\amp -\sqrt{3}\\ \sqrt{3}\amp 1 \end{bmatrix} \right)^{-1}=\frac{1}{4}\begin{bmatrix}1\amp \sqrt{3}\\ -\sqrt{3}\amp 1 \end{bmatrix}\text{.} \end{equation*}

\end{bsolution}

Subsection 6.3.5 Change of basis for transformations

We now investigate how our choice of basis affects matrix representations of linear transformations. We will only consider the special case where \(T\colon V\rightarrow V\) and we are comparing matrix representations \([T]_B\) and \([T]_{B'}\) for two different ordered bases of \(V\text{.}\)

Theorem 6.3.4. Change of basis for transformations.

Let \(V\) be finite-dimensional, let \(T\colon V\rightarrow V\) be linear, and let \(B\) and \(B'\) be two bases for \(V\text{.}\) The matrices \([T]_B\) and \([T]_{B'}\) representing \(T\) with respect to \(B\) and \(B'\text{,}\) respectively, are related as follows:

\begin{align*} [T]_{B'}\amp =\underset{B\rightarrow B'}{P}\, [T]_B\, \underset{B'\rightarrow B}{P}\\ \amp =\underset{B'\rightarrow B}{P}^{-1}\, [T]_B\, \underset{B'\rightarrow B}{P}\text{.} \end{align*}

Remark 6.3.5. Getting change of basis formulas correct.

It is easy to get the various details of the change of basis formula wrong. Here is a potential way to keep things organized in your mind.

We wish to relate \([T]_{B'}\) and \([T]_B\) with an equation of the form \([T]_{B'}=*[T]_B*\text{,}\) where the asterisks are to be replaced with change of basis matrices or their inverses. Think of the three matrices on the right-hand side of this equation as a sequence of three things done to coordinate vectors, reading from right to left.
\([T]_{B'}\) takes as inputs \(B'\)-coordinates of vectors, and outputs \(B'\)-coordinates. Thus the same should be true for \(*[T]_B*\text{.}\)
Since \([T]_B\) takes as inputs \(B\)-coordinates, we must first convert from \(B'\)-coordinates to \(B\)-coordinates. So we should have \([T]_{B'}=*[T]_B\underset{B'\rightarrow B}{P}\text{.}\)
Since \([T]_B\) outputs \(B\)-coordinates, we need to then convert back to \(B'\)-coordinates. Thus \([T]_{B'}=\underset{B\rightarrow B'}{P}[T]_B\underset{B'\rightarrow B}{P}\text{.}\)
If desired you may replace \(\underset{B\rightarrow B'}{P}\) with \(\underset{B'\rightarrow B}{P}^{-1}\text{.}\)

Subsection 6.3.6 Example

Let \(T\colon P_2\rightarrow P_2\) be defined as \(T(p(x))=p(x)+2p'(x)+xp''(x)\text{.}\)

Let \(B=\{1, x, x^2\}\text{.}\) Compute \([T]_B\text{.}\)
Let \(B'=\{1+x+x^2, 1+x, 1+x^2\}\text{.}\) Use the change of basis formula to compute \([T]_{B'}\text{.}\)

We easily compute \([T]_B=\begin{bmatrix}1\amp 2\amp 0\\ 0\amp 1\amp 4\\ 0\amp 0\amp 1 \end{bmatrix}\) using our usual recipe.

We can also easily compute \(\underset{B'\rightarrow B}{P}=\begin{bmatrix}1\amp 1\amp 1\\ 1\amp 1\amp 0\\ 1\amp 0\amp 1 \end{bmatrix}\text{,}\) essentially by inspection.

(In general it is easy to compute the change of basis matrix from a nonstandard basis to the standard basis.)

It follows that

\begin{align*} _B'\amp =\underset{B\rightarrow B'}{P}[T]_B\underset{B'\rightarrow B}{P}=\left( \underset{B'\rightarrow B}{P}\right)^{-1}[T]_B\underset{B'\rightarrow B}{P}\\ \amp =\left( \begin{bmatrix} 1\amp 1\amp 1\\ 1\amp 1\amp 0\\ 1\amp 0\amp 1 \end{bmatrix}\right)^{-1}\begin{bmatrix} 1\amp 2\amp 0\\ 0\amp 1\amp 4\\ 0\amp 0\amp 1 \end{bmatrix} \begin{bmatrix} 1\amp 1\amp 1\\ 1\amp 1\amp 0\\ 1\amp 0\amp 1 \end{bmatrix}= \begin{amatrix}[rrr] 3\amp -2\amp 4\\ 2\amp 3\amp 0\\ -2\amp 2\amp -3 \end{amatrix} \end{align*}

Definition 6.3.6.

Matrices \(A, A'\in M_{nn}\) are similar if there is an invertible matrix \(P\) such that \(A'=P^{-1}AP\text{.}\)

As we will see in coming sections, matrices that are similar in this technical sense do indeed share many of the same properties. We now have the theoretical foundation to understand why this is so: they simply inherit these common properties from the overlying linear transformation \(T\text{,}\) of which they are but earthly shadows.

Figure 6.3.7. Holy commutative tent of linear algebra: \(P=\underset{B\rightarrow B'}{P}\text{,}\) \(B=P^{-1}AP\)

There is but one true \(T\text{!}\)

Subsection 6.3.7 Change of basis for \(V=\R^n\)

Let's consider the special case where \(T\colon \R^n\rightarrow \R^n\text{:}\) that is, when \(V=\R^n\) is a space of \(n\)-tuples. We know from Corollary 4.7.4 that \(T=T_A\) for a unique \(n\times n\) matrix \(A\text{;}\) in fact we know from the proof that \(A=[T]_B\text{,}\) where \(B=(\bolde_1, \bolde_2, \dots, \bolde_n)\) is the standard ordered basis of \(\R^n\text{.}\)

To compute \(A=[T]_B\) directly, we must compute \(T(\bolde_j)\) for each of the standard basis elements \(\bolde_j\text{.}\) However, for many naturally occuring transformations \(T\text{,}\) computing with respect to the standard basis is often not as convenient as computing with respect to some other basis \(B'\text{:}\) i.e., it is often easier to compute \(A'=[T]_{B'}\) for some nonstandard basis \(B'\text{.}\) When this is the case Theorem 6.3.4 allows to us derive the desired matrix \(A\) from the more conveniently computed \(A'\text{:}\) namely, we have

\begin{equation*} A=P^{-1}A'P\text{,} \end{equation*}

where \(P=\underset{B\rightarrow B'}{P}\text{.}\)

This gives us a powerful technique for computing matrix formulas for many interesting geometric linear transformations of \(\R^n\) whose very definitions involve an implicit choice nonstandard basis. Rotations, reflections and orthogonal projections are all examples of such transformations.

Subsection 6.3.8 Example: orthogonal projection revisited

Let \(T\colon \R^3\rightarrow\R^3\) be orthogonal projection onto the plane \(\mathcal{P}: x+y+z=0\text{,}\) as defined earlier. We would like to derive a formula for \(T\text{,}\) which amounts to finding the \(A\) such that \(T=T_A\text{.}\)

As previously observed we have \(A=[T]_B\text{,}\) where \(B\) is the standard basis for \(\R^3\text{.}\) We can compute \([T]_B\) by first computing \([T]_{B'}\) for a cleverly chosen nonstandard basis \(B'\text{,}\) and then using the change of basis formula.

As done previously, we let \(B'=\{(1,-1,0), (1,0,-1), (1,1,1)\text{.}\) Since \(T\) maps the first two vectors to themselves, and the third vector to \((0,0,0)\text{,}\) we have \([T]_{B'}=\begin{bmatrix}1\amp 0\amp 0\\ 0\amp 1\amp 0\\ 0\amp 0\amp 0 \end{bmatrix}\text{.}\) (Go back to original example for details.)

Then

\begin{align*} A\amp =[T]_B=\underset{B'\rightarrow B}{P}[T]_{B'}\underset{B\rightarrow B'}{P}\\ \amp =\begin{amatrix}[rrr] 1\amp 1\amp 1\\ -1\amp 0\amp 1\\ 0\amp -1\amp 1 \end{amatrix} \begin{bmatrix} 1\amp 0\amp 0\\ 0\amp 1\amp 0\\ 0\amp 0\amp 0 \end{bmatrix} \left( \begin{amatrix}[rrr] 1\amp 1\amp 1\\ -1\amp 0\amp 1\\ 0\amp -1\amp 1 \end{amatrix}\right)^{-1} =\frac{1}{3}\begin{amatrix}[rrr] 2\amp -1\amp -1\\ -1\amp 2\amp -1\\ -1\amp -1\amp 2 \end{amatrix} \end{align*}

Lo and behold, we have rediscovered our matrix formula for orthogonal projection onto \(\mathcal{P}\text{!!}\)

(Note: since \(B\) is the standard basis in this case, \(\underset{B'\rightarrow B}{P}\) was easy to compute. )