Section 4.5 Extra Topic: Elementary Matrices
Exploration 4.5.1.
Consider the matrices
\begin{equation*}
B = \begin{bmatrix}
1\amp 0\amp 0\\0\amp 5\amp 0\\0\amp 0\amp 1
\end{bmatrix}
\quad \text{and}\quad
C = \begin{bmatrix}
1\amp 0\amp 0\\0\amp 1\amp 0\\0\amp 0\amp -2
\end{bmatrix}.
\end{equation*}
The two matrices have something in common. Can you figure out what it is? (The answer will be given later in the problem.)
\begin{equation*}
\text{Let}\quad A = \begin{bmatrix}
1\amp 2\amp 3\\4\amp 5\amp 6\\7\amp 8\amp 9
\end{bmatrix}.
\end{equation*}
Problem 4.5.1.
Compute \(BA\) and \(CA\text{.}\)
Answer.
\begin{equation*}
BA=\begin{bmatrix}1 \amp 2 \amp 3 \\ 20 \amp 25 \amp 30\\ 7 \amp 8 \amp 9\end{bmatrix}
\quad
\text{and}
\quad
CA=\begin{bmatrix}1 \amp 2 \amp 3 \\ 4 \amp 5 \amp 6\\ -14 \amp -16 \amp -18\end{bmatrix}.
\end{equation*}
Observe that multiplying \(A\) by \(B\) on the left results in multiplying the second row of \(A\) by \(5\text{,}\) while multiplying \(A\) by \(C\) on the left results in multiplying the third row of \(A\) by \(-2\text{.}\)
Now we need to return to the question of what \(B\) and \(C\) have in common. Both matrices were obtained from the identity matrix by multiplying one row of the identity by a non-zero constant. Matrices \(B\) and \(C\) were obtained from \(I\) by multiplying one row of \(I\) by \(5\) and \(-2\) respectively. Multiplying \(A\) by \(B\) (or \(C\)) on the left affects \(A\) in the same way.
Problem 4.5.2.
Matrix \(A\) does not have to be a square matrix. Try finding \(BA'\) and \(CA'\) for
\begin{equation*}
A'=\begin{bmatrix}1\amp 2\\3\amp 4\\5\amp 6\end{bmatrix}.
\end{equation*}
Answer.
\begin{equation*}
BA'=\begin{bmatrix}1\amp 2\\15\amp 20\\5\amp 6\end{bmatrix}\quad \text{and} \quad CA'=\begin{bmatrix}1\amp 2\\3\amp 4\\-10\amp -12\end{bmatrix}.
\end{equation*}
Observe that \(B\) and \(C\) have the same effect on \(A'\) as they did on \(A\text{.}\)
In general, if a square matrix \(E\) is obtained from the identity matrix \(I\) by multiplying row \(j\) of \(I\) by a non-zero constant \(k\text{,}\) then multiplying an appropriately sized matrix \(A\) on the left by \(E\) results in row \(j\) of \(A\) being multiplied by \(k\text{.}\)
Recall that multiplication of a row of a matrix by a non-zero constant is one of three elementary row operations. Applying such an elementary row operation to \(I\) in order to produce \(E\text{,}\) results in applying the same elementary row operation to \(A\) when \(A\) is multiplied by \(E\) on the left.
Exploration 4.5.2.
Consider the matrices
\begin{equation*}
D = \begin{bmatrix}
1\amp 0\amp 1\\0\amp 1\amp 0\\0\amp 0\amp 1
\end{bmatrix},
\quad \text{and}\quad
F = \begin{bmatrix}
1\amp 0\amp 0\\-2\amp 1\amp 0\\0\amp 0\amp 1
\end{bmatrix}.
\end{equation*}
As in the previous Exploration, the two matrices have something in common. Both \(D\) and \(F\) were obtained from the identity matrix by adding a multiple of one row to another row.
Problem 4.5.3.
Can you guess what will happen if we multiply a matrix \(A\) by \(D\) and \(F\) on the left, were \(A \) is
\begin{equation*}
\text{Let}\quad A = \begin{bmatrix}
1\amp 2\amp 3\\4\amp 5\amp 6\\7\amp 8\amp 9
\end{bmatrix}.
\end{equation*}
Answer.
\begin{equation*}
DA=\begin{bmatrix}8 \amp 10 \amp 12 \\ 4 \amp 5 \amp 6\\ 7 \amp 8 \amp 9\end{bmatrix}
\quad
\text{and}
\quad
FA=\begin{bmatrix}1 \amp 2 \amp 3 \\ 2 \amp 1 \amp 0\\ 7 \amp 8 \amp 9\end{bmatrix}.
\end{equation*}
As you had probably guessed, multiplication by \(D\) resulted in the third row of \(A\) being added to the first, and multiplication by \(F\) produced a matrix by adding \(-2\) times the first row to the second row of \(A\text{.}\) The elementary row operations performed on \(A\) mimic the elementary row operations performed on \(I\) in order to obtain \(D\) and \(F\text{.}\)
In general, if a square matrix \(E\) is obtained from the identity matrix \(I\) by adding \(k\) times row \(j\) of \(I\) to row \(i\text{,}\) then multiplying an appropriately sized matrix \(A\) on the left by \(E\) results in \(k\) times row \(j\) of \(A\) being added to row \(i\) of \(A\text{.}\)
Recall that adding a scalar multiple of one row to another row of a matrix is one of three elementary row operations. Applying such an elementary row operation to \(I\) in order to produce \(E\text{,}\) results in applying the same elementary row operation to \(A\) when \(A\) is multiplied by \(E\) on the left.
Exploration 4.5.3.
The matrices \(B,C,D,F\) above are special because when we multiply them by any appropriately sized matrix \(A\text{,}\) we are performing row operations on \(A\text{.}\)
Problem 4.5.4.
Can you construct a matrix \(G\) such that \(GA\) is the same as \(A\) except that its first and third rows are switched?
Answer.
\begin{equation*}
G=\begin{bmatrix}0 \amp 0 \amp 1 \\ 0 \amp 1 \amp 0\\ 1 \amp 0 \amp 0\end{bmatrix}.
\end{equation*}
Definition 4.5.6.
An elementary matrix is a square matrix formed by applying a single elementary row operation to the identity matrix.
Suppose \(A\) is an \(m \times n\) matrix. If \(E\) is an \(m \times m\) elementary matrix formed by performing a certain row operation on the \(m \times m\) identity matrix, then multiplying any matrix \(A\) on the left by \(E\) is equivalent to performing that same row operation on \(A\text{.}\) As there are three types of elementary row operations, there are three types of elementary matrices.
Elementary matrices give us a new way of looking at Gauss-Jordan elimination. Suppose it takes \(j\) elementary row operations to transform \(A\) into \(R\text{,}\) its reduced row-echelon form. Then we can represent this reduced row-echelon form as
\begin{equation*}
R = E_j \cdots E_2 E_1 A
\end{equation*}
where each \(E_i\) is the elementary matrix corresponding to the \(i\)th row operation performed on \(A\text{.}\)
Subsection 4.5.1 Inverses of Elementary Matrices
It is easy to see that any elementary matrix
\(E\) is invertible, because if
\(E\) is formed by applying a certain row operation to the identity matrix
\(I\text{,}\) then there is a single row operation that may be applied to
\(E\) to get
\(I\) back. For example, in
Exploration 4.5.2,
\(F\) is formed by adding
\(-2\) times the first row of the identity to the second row of the identity. It follows that
\(F^{-1}\) should be the matrix formed by adding
\(2\) times the first row of the identity to the second row of the identity, i.e.
\begin{equation*}
F^{-1} = \begin{bmatrix}
1\amp 0\amp 0\\2\amp 1\amp 0\\0\amp 0\amp 1
\end{bmatrix}.
\end{equation*}
And indeed we can check
\begin{equation*}
F F^{-1} = \begin{bmatrix}
1\amp 0\amp 0\\-2\amp 1\amp 0\\0\amp 0\amp 1
\end{bmatrix} \begin{bmatrix}
1\amp 0\amp 0\\2\amp 1\amp 0\\0\amp 0\amp 1
\end{bmatrix} = \begin{bmatrix}
1\amp 0\amp 0\\0\amp 1\amp 0\\0\amp 0\amp 1
\end{bmatrix}
\end{equation*}
and also \(F^{-1} F = I\text{.}\)
As part of the Practice Problem set you are asked to find the inverse of each of the other elementary matrices in
Exploration 4.5.1,
Exploration 4.5.2 and
Exploration 4.5.3. Once we have accounted for each of the three types of elementary matrices, we will have proven the following theorem.
Theorem 4.5.7.
Elementary matrices are invertible, and the inverse of an elementary matrix is an elementary matrix.
Proof.
Suppose \(E\) is obtained from \(I\) by switching rows \(i\) and \(j\text{.}\) To find the inverse of \(E\text{,}\) we need to find a matrix \(F\) such that \(FE=I\text{.}\) To get from \(E\) back to \(I\text{,}\) rows \(i\) and \(j\) of \(E\) must be switched. This can be accomplished by multiplying \(E\) by itself on the left. So, \(E\) is its own inverse. We can use the same line of reasoning to show that the other two types of elementary matrices are also invertible, and their inverses are also elementary matrices. The details are left to the reader.
Recall that a square matrix \(A\) is called nonsingular provided that \(\mbox{rref}(A)=I\text{.}\)
Theorem 4.5.8.
The following statements are equivalent for an \(n\times n\) matrix \(A\text{.}\)
\(A\) is nonsingular
\(A\) is a product of elementary matrices
\(A\) is invertible
Proof.
We will prove equivalence of the three statements by showing that
Item 1\(\Rightarrow\)Item 2\(\Rightarrow\)Item 3\(\Rightarrow\)Item 1
[Proof of
Item 1\(\Rightarrow\)Item 2]: Suppose
\(\mbox{rref}(A)=I\text{.}\) Then
\(A\) can be carried to the identity by elementary row operations. So, there exist elementary matrices
\(E_1, E_2, \ldots ,E_k\) such that
\begin{equation*}
E_k\ldots E_2E_1A=I
\end{equation*}
By
Theorem 4.5.7, elementary matrices are invertible and their inverses are also elementary matrices. Thus, we can write
\(A\) as a product of elementary matrices as follows:
\begin{equation*}
A=E_1^{-1}E_2^{-1}\ldots E_k^{-1}.
\end{equation*}
[Proof of
Item 2\(\Rightarrow\)Item 3]: Suppose
\(A=E_1E_2\ldots E_k\text{,}\) where
\(E_1, E_2, \ldots , E_k\) are elementary matrices. In
Item 3 we proved that
\((BC)^{-1} = C^{-1} B^{-1}\text{.}\) By repeated applications of this theorem we have
\begin{equation*}
(E_1E_2\ldots E_k)^{-1}=E_k^{-1}\ldots E_2^{-1}E_1^{-1}=A^{-1}
\end{equation*}
We conclude that \(A\) is invertible.
Exercises 4.5.2 Exercises
Exercise Group.
For each elementary matrix \(E\) below, determine the elementary row operation that results from multiplying a \(3\times n\) matrix \(A\) by \(E\) on the left. Write down \(E^{-1}\) without going through the row-reduction procedure.
1.
\begin{equation*}
E=\begin{bmatrix}0\amp 1\amp 0\\1\amp 0\amp 0\\0\amp 0\amp 1\end{bmatrix}
\end{equation*}
Hint.
Think of an elementary row operation that would undo the row operation caused by \(E\text{.}\)
Answer.
\begin{equation*}
E^{-1}=\begin{bmatrix}0\amp 1\amp 0\\1\amp 0\amp 0\\0\amp 0\amp 1\end{bmatrix}
\end{equation*}
2.
\begin{equation*}
E=\begin{bmatrix}1\amp 0\amp 0\\0\amp 1\amp 0\\0\amp 0\amp 5\end{bmatrix}
\end{equation*}
Answer.
\begin{equation*}
E^{-1}=\begin{bmatrix}1\amp 0\amp 0\\0\amp 1\amp 0\\0\amp 0\amp 1/5\end{bmatrix}
\end{equation*}
3.
\begin{equation*}
E=\begin{bmatrix}1\amp 0\amp 0\\0\amp 1\amp 0\\0\amp 4\amp 1\end{bmatrix}
\end{equation*}
Answer.
\begin{equation*}
E^{-1}=\begin{bmatrix}1\amp 0\amp 0\\0\amp 1\amp 0\\0\amp -4\amp 1\end{bmatrix}
\end{equation*}
4.
\begin{equation*}
B = \begin{bmatrix}
1\amp 0\amp 0\\0\amp 5\amp 0\\0\amp 0\amp 1
\end{bmatrix}\quad
C = \begin{bmatrix}
1\amp 0\amp 0\\0\amp 1\amp 0\\0\amp 0\amp -2
\end{bmatrix}
\quad
D = \begin{bmatrix}
1\amp 0\amp 1\\0\amp 1\amp 0\\0\amp 0\amp 1
\end{bmatrix}\quad
G = \begin{bmatrix}
0\amp 0\amp 1\\0\amp 1\amp 0\\1\amp 0\amp 0
\end{bmatrix}
\end{equation*}
5.
6.
Express \(A\) as a product of elementary matrices. Then consider whether the representation unique and prove your claim.
\begin{equation*}
A=\begin{bmatrix}1\amp 2\amp 0\\-1\amp 0\amp 1\\0\amp 4\amp 0\end{bmatrix}
\end{equation*}
Hint.
Row-reduce \(A\) to find \(\mbox{rref}(A)\text{.}\) Record the elementary row operations as you perform row reduction. You will be able to conclude that \(E_j\cdots E_2E_1A=I\text{.}\) Find the inverse of each \(E_i\) and multiply by the inverses on the left.
7.
In
Exploration 4.5.1,
Exploration 4.5.2 and
Exploration 4.5.3 we performed elementary row operations on
\(A\) by multiplying
\(A\) by elementary matrices
\(B, C, D, F, G\) on the left. Compute
\(AB, AC, AD, AF\) and
\(AG\text{.}\) Summarize your findings.
Answer.
\begin{equation*}
AB=\begin{bmatrix}1 \amp 10 \amp 3 \\ 4 \amp 25 \amp 6\\ 7 \amp 40 \amp 9\end{bmatrix}
\quad
AC=\begin{bmatrix}1 \amp 2 \amp -6 \\ 4 \amp 5 \amp -12\\ 7 \amp 8 \amp -18\end{bmatrix}
\end{equation*}
\begin{equation*}
AD=\begin{bmatrix}1 \amp 2 \amp 4 \\ 4 \amp 5 \amp 10\\ 7 \amp 8 \amp 16\end{bmatrix}
\quad
AF=\begin{bmatrix}-3 \amp 2 \amp 3 \\ -6 \amp 5 \amp 6\\ -9 \amp 8 \amp 9\end{bmatrix}
\end{equation*}
\begin{equation*}
AG=\begin{bmatrix}3 \amp 2 \amp 1 \\ 6 \amp 5 \amp 4\\ 9 \amp 8 \amp 7\end{bmatrix}
\end{equation*}
8.
If possible, express
\begin{equation*}
A=\begin{bmatrix}1\amp 4\amp 7\\2\amp 5\amp 8\\3\amp 6\amp 9\end{bmatrix}
\end{equation*}
as a product of elementary matrices.
