Definition 4.1.1. The Zero Matrix.
The \(m\times n\) zero matrix is the \(m\times n\) matrix having every entry equal to zero. The zero matrix is denoted by \(O\text{.}\)
Section 4.1 Matrix Operations
A matrix is a rectangular array of numbers. The plural form of matrix is matrices (not matrixes). You have encountered matrices before in the context of augmented matrices and coefficient matrices associate with linear systems.
Consider the matrix
\begin{equation*}
M=\begin{bmatrix}
1 \amp 2 \amp 3 \amp 4 \\
5 \amp 2 \amp 8 \amp 7 \\
6 \amp -9 \amp 1 \amp 2
\end{bmatrix}.
\end{equation*}
The dimension of a matrix is defined as \(m\times n\) where \(m\) is the number of rows and \(n\) is the number of columns. The above matrix is a \(3\times 4\) matrix because there are three rows and four columns.
A column vector in \(\R^n\) is an \(n\times 1\) matrix. A row vector in \(\R^n\) is a \(1\times n\) matrix.
The individual entries in the matrix are identified according to their position. The \(( i, j)\)-entry of a matrix is the entry in the \(i^{th}\) row and \(j^{th}\) column. For example, in matrix \(M\) above, \(8\) is called the \((2,3)\)-entry because it is in the second row and the third column.
We denote the entry in the \(i^{th}\) row and the \(j^{th}\) column of matrix \(A\) by \(a_{ij}\text{,}\) and write \(A\) in terms of its entries as
\begin{equation*}
A= \begin{bmatrix} a_{ij} \end{bmatrix}=\begin{bmatrix}
a_{11} \amp a_{12}\amp \dots\amp a_{1j}\amp \dots\amp a_{1n}\\
a_{21}\amp a_{22} \amp \dots\amp a_{2j}\amp \dots \amp a_{2n}\\
\vdots \amp \vdots\amp \amp \vdots\amp \amp \vdots\\
a_{i1}\amp a_{i2}\amp \dots \amp a_{ij}\amp \dots \amp a_{in}\\
\vdots \amp \vdots\amp \amp \vdots\amp \amp \vdots\\
a_{m1}\amp a_{m2}\amp \dots \amp a_{mj}\amp \dots \amp a_{mn}
\end{bmatrix}.
\end{equation*}
Occasionally it will be convenient to talk about columns and rows of a matrix \(A\) as vectors. We will use the following notation:
\begin{equation*}
A=\begin{bmatrix}|\amp |\amp \amp |\\\mathbf{c}_1\amp \mathbf{c}_2 \amp \ldots \amp \mathbf{c}_n\\|\amp |\amp \amp |\end{bmatrix}\quad\text{or}\quad A=\begin{bmatrix}\mathbf{c}_1\amp \mathbf{c}_2 \amp \ldots \amp \mathbf{c}_n\end{bmatrix}
\end{equation*}
\begin{equation*}
A=\begin{bmatrix}
- \amp \mathbf{r}_1 \amp - \\ - \amp \mathbf{r}_2 \amp - \\ \amp \vdots \amp \\ - \amp \mathbf{r}_m \amp -
\end{bmatrix}\quad\text{or}\quad A=\begin{bmatrix}\mathbf{r}_1\\\mathbf{r}_2\\\vdots\\\mathbf{r}_m\end{bmatrix}.
\end{equation*}
A matrix is called a square matrix if it has the same number of rows and columns. If \(B=\begin{bmatrix}b_{ij}\end{bmatrix}\) is an \(n \times n\) square matrix, the entries of the form \(b_{ii}\) are said to lie on the main diagonal. For example, if
\begin{equation*}
B=\begin{bmatrix}1\amp 2\amp 3\\4\amp 5\amp 6\\7\amp 4\amp 9\end{bmatrix},
\end{equation*}
then the main diagonal consists of entries \(b_{11}=1\text{,}\) \(b_{22}=5\) and \(b_{33}=9\text{.}\)
There are various operations which are done on matrices of appropriate sizes. Matrices can be added to and subtracted from other matrices, multiplied by a scalar, and multiplied by other matrices. We will never divide a matrix by another matrix, but we will see later how multiplication by a matrix inverse (if an inverse exists) plays a similar role to division.
In doing arithmetic with matrices, we often define the action by what happens in terms of the entries (or components) of the matrices. Before looking at these operations in depth, consider a few general definitions.
Definition 4.1.2. Equality of Matrices.
Let \(A=\begin{bmatrix} a_{ij}\end{bmatrix}\) and \(B=\begin{bmatrix} b_{ij}\end{bmatrix}\) be two \(m \times n\) matrices. Then \(A=B\) means that \(a_{ij}=b_{ij}\) for all \(1\leq i\leq m\) and \(1\leq j\leq n\text{.}\)
Subsection 4.1.1 Addition - and Scalar multiplication of Matrices
Given two matrices of the same dimensions, we can add them together by adding their corresponding entries.
Definition 4.1.3. Addition of Matrices.
Let \(A=\begin{bmatrix} a_{ij}\end{bmatrix} \) and \(B=\begin{bmatrix} b_{ij}\end{bmatrix}\) be two \(m\times n\) matrices. Then the sum of matrices \(A\) and \(B\text{,}\) denoted by \(A+B\text{,}\) is an \(m \times n\) matrix given by
\begin{equation*}
A+B=\begin{bmatrix}a_{ij}+b_{ij}\end{bmatrix}.
\end{equation*}
An example might help unravel the formal definition.
Example 4.1.4.
Find the sum of \(A\) and \(B\text{,}\) if possible.
\begin{equation*}
A = \begin{bmatrix}
1 \amp 2 \amp 3 \\
1 \amp 0 \amp 4
\end{bmatrix},
B = \begin{bmatrix}
5 \amp 2 \amp 3 \\
-6 \amp 2 \amp 1
\end{bmatrix}.
\end{equation*}
Answer.
Notice that both \(A\) and \(B\) are of size \(2 \times 3\text{.}\) Since \(A\) and \(B\) are of the same size, addition is possible.
\begin{align*}
A + B \amp = \begin{bmatrix}
1 \amp 2 \amp 3 \\
1 \amp 0 \amp 4
\end{bmatrix}
+
\begin{bmatrix}
5 \amp 2 \amp 3 \\
-6 \amp 2 \amp 1
\end{bmatrix} \\
\amp =
\begin{bmatrix}
1+5 \amp 2+2 \amp 3+3 \\
1+ -6 \amp 0+2 \amp 4+1
\end{bmatrix} \\
\amp =
\begin{bmatrix}
6 \amp 4 \amp 6 \\
-5 \amp 2 \amp 5
\end{bmatrix}.
\end{align*}
Going forward, whenever we write \(A+B\) it will be assumed that the two matrices are of equal size and addition is possible.
Theorem 4.1.5. Properties of Matrix Addition.
Let \(A,B\) and \(C\) be matrices. Then the following properties hold.
- Commutative Law of Addition:\begin{equation*} A+B=B+A \end{equation*}
- Associative Law of Addition:\begin{equation*} \left( A+B\right) +C=A+\left( B+C\right) \end{equation*}
- Additive Identity: There exists a zero matrix such that\begin{equation*} A+O=A \end{equation*}
- Additive Inverse: There exists a matrix, \(-A\text{,}\) such that\begin{equation*} A+\left( -A\right) =O \end{equation*}
Proof.
\begin{equation*}
a_{ij}+b_{ij}.
\end{equation*}
The \((i,j)\)-entry of \(B+A\) is given by
\begin{equation*}
b_{ij}+a_{ij}.
\end{equation*}
Since \(a_{ij}+b_{ij}=b_{ij}+a_{ij}\text{,}\) for all \(i\text{,}\) \(j\text{,}\) we conclude that \(A+B=B+A\text{.}\)
[Proof of Item 4:] Let \(-A\) be defined by
\begin{equation*}
-A=\begin{bmatrix}-a_{ij}\end{bmatrix}.
\end{equation*}
Then \(A+(-A)=O\text{.}\)
When a matrix is multiplied by a scalar, the new matrix is obtained by multiplying every entry of the original matrix by the given scalar.
Definition 4.1.6. Scalar Multiplication of Matrices.
If \(A=\begin{bmatrix} a_{ij}\end{bmatrix} \) and \(k\) is a scalar, then \(kA=\begin{bmatrix} ka_{ij}\end{bmatrix}\text{.}\)
A hands down example is given below.
Example 4.1.7.
Find \(7A\) if
\begin{equation*}
A=\begin{bmatrix}
2 \amp 0 \\
1 \amp -4
\end{bmatrix}.
\end{equation*}
Answer.
\begin{equation*}
7A =
7\begin{bmatrix}
2 \amp 0 \\
1 \amp -4
\end{bmatrix} =
\begin{bmatrix}
7(2) \amp 7(0) \\
7(1) \amp 7(-4)
\end{bmatrix} =
\begin{bmatrix}
14 \amp 0 \\
7 \amp -28
\end{bmatrix}
\end{equation*}
Theorem 4.1.8. Properties of Scalar Multiplication.
Let \(A, B\) be matrices, and \(k, p\) be scalars. Then, the following properties properties of scalar multiplication hold.
- Distributive Law over Matrix Addition:\begin{equation*} k \left( A+B\right) =k A+ kB \end{equation*}
- Distributive Law over Scalar Addition:\begin{equation*} \left( k +p \right) A= k A+p A \end{equation*}
- Associative Law for Scalar Multiplication:\begin{equation*} k \left( p A\right) = \left( k p \right) A \end{equation*}
- Multiplication by \(1\text{:}\)\begin{equation*} 1A=A \end{equation*}
The proof of this theorem is similar to the proof of Theorem 4.1.5 and is left as an exercise.
Subsection 4.1.2 Matrix Multiplication
We will introduce matrix multiplication by first considering the special case of a matrix-vector product. In other words, the first matrix is \(m \times n\) and the second matrix is \(n \times 1\) for some positive integers \(m,n\text{.}\)
Subsubsection 4.1.1 Matrix-Vector Multiplication
Prior to presenting the formal definition in full generality, it is better to see a concrete example because the formal version is notation heavy.
Example 4.1.9.
Let
\begin{equation*}
A=\begin{bmatrix}2\amp -1\amp 3\amp 2\\0\amp 3\amp -2\amp 1\\-2\amp 4\amp 1\amp 0\end{bmatrix}\quad\text{and}\quad \mathbf{x}=\begin{bmatrix}3\\-1\\4\\1\end{bmatrix}.
\end{equation*}
One way to understand the matrix-vector product \(A\mathbf{x}\) is by thinking of it as a linear combination of the columns of \(A\text{,}\) using the entries in \(x\) as our coefficients.
\begin{align*}
A\mathbf{x} \amp =\begin{bmatrix}\color{red}2\amp \color{blue}-1\amp \color{brown}3\amp 2\\ \color{red}0\amp \color{blue}3\amp \color{brown}-2\amp 1\\ \color{red}-2\amp \color{blue}4\amp \color{brown}1\amp 0\end{bmatrix}\begin{bmatrix}\color{red}3\\ \color{blue}-1\\ \color{brown}4\\1\end{bmatrix} \\
= \color{red}3\begin{bmatrix}\color{red}2\\ \color{red}0\\ \color{red}-2\end{bmatrix}\color{black}+
\color{blue}(-1)\begin{bmatrix}\color{blue}-1\\ \color{blue}3\\ \color{blue}4\end{bmatrix}
\color{black}+\color{brown}4\begin{bmatrix}\color{brown}3\\ \color{brown}-2\\ \color{brown}1\end{bmatrix}
\color{black}+\begin{bmatrix}2\\1\\0\end{bmatrix} \\
=\begin{bmatrix}21\\-10 \\ -6\end{bmatrix}.
\end{align*}
We can also compute the product one entry at a time. First, let’s focus on the first row of \(A\text{.}\)
\begin{align*}
\amp \begin{bmatrix}{\color{red}2}\amp {\color{blue}-1}\amp {\color{brown}3}\amp 2\\0\amp 3\amp -2\amp 1\\-2\amp 4\amp 1\amp 0\end{bmatrix}\begin{bmatrix}{\color{red}3}\\{\color{blue}-1}\\{\color{brown}4}\\1\end{bmatrix} \\
\amp= \begin{bmatrix}{\color{red}(2)( 3)}+{\color{blue}(-1)(-1)}+{\color{brown}(3)(4)}+(2)(1)\\ \\ \\\end{bmatrix}=\begin{bmatrix}21\\ \\ \\\end{bmatrix}.
\end{align*}
Next, let’s look a the second row of \(A\text{.}\)
\begin{align*}
\amp \begin{bmatrix}2\amp -1\amp 3\amp 2\\{\color{red}0}\amp {\color{blue}3}\amp {\color{brown}-2}\amp 1\\-2\amp 4\amp 1\amp 0\end{bmatrix}\begin{bmatrix}{\color{red}3}\\{\color{blue}-1}\\{\color{brown}4}\\1\end{bmatrix}=\begin{bmatrix}21\\{\color{red}(0)( 3)}+{\color{blue}(3)(-1)}+{\color{brown}(-2)(4)}+(1)(1)\\ \\ \end{bmatrix} \\
\amp =\begin{bmatrix}21\\-10 \\ \\\end{bmatrix}.
\end{align*}
Finally, let’s do the third row of \(A\text{.}\)
\begin{align*}
\amp \begin{bmatrix}2\amp -1\amp 3\amp 2\\0\amp 3\amp -2\amp 1\\{\color{red}-2}\amp {\color{blue}4}\amp {\color{brown}1}\amp 0\end{bmatrix}\begin{bmatrix}{\color{red}3}\\{\color{blue}-1}\\{\color{brown}4}\\1\end{bmatrix}=\begin{bmatrix}21\\-10\\{\color{red}(-2)( 3)}+{\color{blue}(4)(-1)}+{\color{brown}(1)(4)}+(0)(1) \end{bmatrix} \\
\amp =\begin{bmatrix}21\\-10 \\ -6\end{bmatrix}.
\end{align*}
Definition 4.1.10.
Let \(A\) be an \(m\times n\) matrix, and let \(\mathbf{x}\) be an \(n\times 1\) vector. The product \(A\mathbf{x}\) is the \(m\times 1\) vector given by:
\begin{align*}
A\mathbf{x} \amp =\begin{bmatrix}
a_{11} \amp a_{12}\amp \dots\amp a_{1n}\\
a_{21}\amp a_{22} \amp \dots \amp a_{2n}\\
\vdots \amp \vdots\amp \ddots \amp \vdots\\
a_{m1}\amp \dots \amp \dots \amp a_{mn}
\end{bmatrix}\begin{bmatrix}x_1\\x_2\\\vdots\\x_n\end{bmatrix} \\
\amp =
x_1\begin{bmatrix}a_{11}\\a_{21}\\ \vdots \\a_{m1}\end{bmatrix}+
x_2\begin{bmatrix}a_{12}\\a_{22}\\ \vdots \\a_{m2}\end{bmatrix}+\dots+
x_n\begin{bmatrix}a_{1n}\\a_{2n}\\ \vdots \\a_{mn}\end{bmatrix}
\end{align*}
or, equivalently,
\begin{equation*}
A\mathbf{x}=\begin{bmatrix}a_{11}x_1+a_{12}x_2+\ldots +a_{1n}x_n\\a_{21}x_1+a_{22}x_2+\ldots +a_{2n}x_n\\\vdots\\a_{m1}x_1+a_{m2}x_2+\ldots +a_{mn}x_n\end{bmatrix}.
\end{equation*}
We can now make a couple of observations about the matrix-vector product. The first observation is part of the definition, but it is still worth pointing out.
Observation 4.1.11.
In order for the product \(A\mathbf{x}\) to exist, \(A\) and \(\mathbf{x}\) must have compatible dimensions. In particular, vector \(\mathbf{x}\) must have as many components as the number of columns of \(A\) (otherwise, we would not be have a well-defined linear combination of the columns). So, if \(A\) is an \(m\times n\) matrix, \(\mathbf{x}\) must be an \(n\times 1\) vector. If we write these dimensions next to each other, we will notice that the inner dimensions (\(n\)) must match, while the outer dimensions, \(m\) and \(1\text{,}\) give us the dimensions of the product.
\begin{equation*}
\begin{array}{ccccc}
A\amp \amp \mathbf{x} \amp =\amp A\mathbf{x}\\
(m\times n) \amp \amp (n\times 1) \amp \amp (m\times 1)
\end{array}
\end{equation*}
Observation 4.1.12.
If you are familiar with the dot product, you may have noticed that each individual entry in the product matrix \(A\mathbf{x}\) is the dot product of a row of \(A\) with \(\mathbf{x}\text{.}\) Thus, if the rows of \(A\) are vectors \(\mathbf{r}_1\text{,}\) \(\mathbf{r}_2,\ldots ,\mathbf{r}_n\) we can restate Definition Definition 4.1.10 as follows:
\begin{equation*}
A\mathbf{x}=\begin{bmatrix}-\amp \mathbf{r}_1\amp -\\-\amp \mathbf{r}_2\amp -\\ \amp \vdots \amp \\-\amp \mathbf{r}_n \amp -\end{bmatrix}\mathbf{x}=\begin{bmatrix}\mathbf{r}_1\cdot\mathbf{x}\\\mathbf{r}_2\cdot\mathbf{x}\\\vdots\\\mathbf{r}_n\cdot\mathbf{x}\end{bmatrix}.
\end{equation*}
Let’s find another matrix-vector product.
Example 4.1.13.
Let
\begin{equation*}
A=\begin{bmatrix}1\amp -1\\2\amp 3\\-2\amp 1\\4\amp 0\end{bmatrix}\quad\text{and}\quad\mathbf{x}=\begin{bmatrix}-3\\5\end{bmatrix}.
\end{equation*}
Find \(A\mathbf{x}\text{.}\)
Answer.
\begin{equation*}
A\mathbf{x}=\begin{bmatrix}1\amp -1\\2\amp 3\\-2\amp 1\\4\amp 0\end{bmatrix}\begin{bmatrix}-3\\5\end{bmatrix}=\begin{bmatrix}-8\\9\\11\\-12\end{bmatrix}.
\end{equation*}
Subsubsection 4.1.2 Matrix-Matrix Multiplication
Matrix-matrix multiplication is simply an extension of the idea of matrix-vector multiplication. In order for the product definition to work, matrix dimensions must be compatible. Let \(A\) be an \(m\times n\) matrix, and let \(B\) be an \(n\times p\) matrix, then the product \(AB\) will be an \(m\times p\) matrix.
\begin{equation*}
\begin{array}{ccccc}
A\amp \amp B \amp =\amp AB\\
(m\times n) \amp \amp (n\times p) \amp \amp (m\times p)
\end{array},
\end{equation*}
Just like with vector products, the inner dimensions must be the same, while the outer dimensions, \(m\) and \(p\text{,}\) give us the dimensions of the product.
Definition 4.1.14.
Let \(A\) be an \(m\times n\) matrix whose rows are vectors \(\mathbf{r}_1\text{,}\) \(\mathbf{r}_2,\ldots ,\mathbf{r}_n\text{.}\) Let \(B\) be an \(n\times p\) matrix with columns \(\mathbf{b}_1, \mathbf{b}_2, \ldots, \mathbf{b}_p\text{.}\) Then the entries of the matrix product \(AB\) are given by the dot products
\begin{align*}
\end{align*}
So the \((i,j)\)-entry of \(AB\) is the dot product of the \(i^{th}\) row of \(A\) and the \(j^{th}\) column of \(B\text{.}\)
In terms of components, if the \(i^{th}\) row of \(A\) is
\begin{equation*}
\begin{bmatrix}a_{i1}\amp a_{i2} \amp \ldots \amp a_{in}\end{bmatrix}
\end{equation*}
and the \(j^{th}\) column of \(B\) is
\begin{equation*}
\begin{bmatrix}b_{1j}\\b_{2j}\\\vdots\\b_{nj}\end{bmatrix}
\end{equation*}
then the \((i,j)\)-entry of \(AB\) is given by
\begin{equation}
a_{i1}b_{1j}+a_{i2}b_{2j}+\dots +a_{in}b_{nj}=\sum_{k=1}^na_{ik}b_{kj}\tag{4.1.1}
\end{equation}
Example 4.1.15.
Let
\begin{equation*}
A=\begin{bmatrix}3 \amp 2 \amp -1 \amp 1\\0 \amp 3 \amp 1 \amp 1\\1 \amp 4 \amp -1 \amp 0\end{bmatrix}\quad\text{and}\quad B=\begin{bmatrix}1 \amp -2 \\-1 \amp 4 \\2 \amp 3 \\2 \amp 0\end{bmatrix}
\end{equation*}
Find \(AB\text{.}\)
Answer.
Observe that \(A\) is a \(3\times 4\) matrix and \(B\) is a \(4\times 2\) matrix. So, we expect the product \(AB\) to be a \(3\times 2\) matrix.
\begin{align*}
AB\amp =\begin{bmatrix}3 \amp 2 \amp -1 \amp 1\\0 \amp 3 \amp 1 \amp 1\\1 \amp 4 \amp -1 \amp 0\end{bmatrix}\begin{bmatrix}1 \amp -2 \\-1 \amp 4 \\2 \amp 3 \\2 \amp 0\end{bmatrix} \\
\amp =\begin{bmatrix}(3)(1)+(2)(-1)+(-1)(2)+(1)(2) \amp (3)(-2)+(2)(4)+(-1)(3)+(1)(0)\\(0)(1)+(3)(-1)+(1)(2)+(1)(2) \amp (0)(-2)+(3)(4)+(1)(3)+(1)(0)\\(1)(1)+(4)(-1)+(-1)(2)+(0)(2) \amp (1)(-2)+(4)(4)+(-1)(3)+(0)(0) \end{bmatrix} \\
\amp =\begin{bmatrix}1 \amp -1 \\ 1 \amp 15\\ -5 \amp 11\end{bmatrix}
\end{align*}
Remark 4.1.16.
It is possible to use linear combinations rather than dot products to compute a matrix-matrix product. If the columns of matrix \(B\) are given by \({\mathbf{b}_1, \mathbf{b}_2, ..., \mathbf{b}_p}\text{,}\) then the matrix product consists of \(p\) columns, each of which is a matrix-vector product:
\begin{equation*}
AB = \begin{bmatrix} | \amp | \amp \dots \amp |\\
A\mathbf{b}_1 \amp A\mathbf{b}_2 \amp \dots \amp A\mathbf{b}_p\\
| \amp | \amp \dots \amp |
\end{bmatrix}.
\end{equation*}
If we return to Example Example 4.1.15 above, we may use this approach to compute
\begin{equation*}
A\mathbf{b}_1=\begin{bmatrix}3\\0\\1\end{bmatrix}
-\begin{bmatrix}2\\3\\4\end{bmatrix}
+2\begin{bmatrix}-1\\1\\-1\end{bmatrix}
+2\begin{bmatrix}1\\1\\0\end{bmatrix}
= \begin{bmatrix}1\\1\\-5\end{bmatrix}
\end{equation*}
and
\begin{equation*}
A\mathbf{b}_2=-2\begin{bmatrix}3\\0\\1\end{bmatrix}
+4\begin{bmatrix}2\\3\\4\end{bmatrix}
+3\begin{bmatrix}-1\\1\\-1\end{bmatrix}+0 = \begin{bmatrix}-1\\15\\11\end{bmatrix}.
\end{equation*}
Let us put this into explicit examples.
Example 4.1.17.
Let
\begin{equation*}
A=\begin{bmatrix}-2 \amp 1\\3 \amp 0\\1 \amp -3\end{bmatrix}\quad\text{and}\quad B=\begin{bmatrix}4 \amp -2 \amp 3 \amp -2 \amp 2\\1 \amp -3 \amp 1 \amp 1 \amp 0\end{bmatrix}.
\end{equation*}
Find \(AB\text{.}\)
Answer.
Fill in the blanks below.
\begin{align*}
AB \amp=\begin{bmatrix}-2 \amp 1\\3 \amp 0\\1 \amp -3\end{bmatrix}\begin{bmatrix}4 \amp -2 \amp 3 \amp -2 \amp 2\\1 \amp -3 \amp 1 \amp 1 \amp 0\end{bmatrix} \\
\amp =\begin{bmatrix}-7 \amp 1 \amp -5 \amp 5 \amp -4\\12 \amp -6 \amp 9 \amp -6 \amp 6\\1 \amp 7 \amp 0 \amp -5 \amp 2\end{bmatrix}.
\end{align*}
Example 4.1.18.
Suppose that we know that \(\mathbf{x}=\begin{bmatrix} -2\\5\end{bmatrix}\) satisfies
\begin{equation*}
\begin{bmatrix}
-2\amp 3\\
4\amp -1
\end{bmatrix}\mathbf{x}=\begin{bmatrix} 19\\-13\end{bmatrix}
\end{equation*}
Use this information to express \(\begin{bmatrix} 19\\-13\end{bmatrix}\) as a linear combination of the columns of \(\begin{bmatrix}
-2\amp 3\\
4\amp -1
\end{bmatrix}\text{.}\)
Answer.
\begin{equation*}
-2\begin{bmatrix} -2\\4\end{bmatrix}+5\begin{bmatrix} 3\\-1\end{bmatrix}=\begin{bmatrix} 19\\-13\end{bmatrix}
\end{equation*}
Subsection 4.1.3 Properties of Matrix Multiplication
Exploration 4.1.1.
Let
\begin{equation*}
A=\begin{bmatrix}1\amp 2\\3\amp 4\end{bmatrix}\quad\text{and}\quad B=\begin{bmatrix}5\amp 6\\7\amp 8\end{bmatrix}.
\end{equation*}
Observe that both \(AB\) and \(BA\) are defined, and both products are \(2\times 2\) matrices. Let’s compute the two products
\begin{equation*}
AB=\begin{bmatrix}19\amp 22\\43\amp 50\end{bmatrix}\quad\text{and}\quad BA=\begin{bmatrix}23\amp 34\\31\amp 46\end{bmatrix}.
\end{equation*}
Clearly \(AB\neq BA\text{.}\) We say that \(A\) and \(B\) do not commute.
Warning 4.1.19.
Matrix multiplication is not commutative. While it is possible to find specific matrices that commute, matrix multiplication is not commutative in general.
One example of an \(n\times n\) square matrix that commutes with all \(n\times n\) matrices is the matrix \(I_n\) defined by
\begin{equation*}
I_n=\begin{bmatrix}1\amp 0\amp \ldots \amp 0\\0\amp 1\amp \ldots \amp 0\\\vdots \amp \vdots \amp \ddots \amp \vdots \\0 \amp 0 \amp \ldots \amp 1\end{bmatrix}
\end{equation*}
\(I_n\) has 1’s along the main diagonal and 0’s everywhere else. It is often useful to think of \(I_n\) as a matrix whose \(j^{th}\) column (and \(j^{th}\) row) is \(\mathbf{e}_j\text{,}\) the \(j^{th}\) standard unit vector of \(\R^n\text{.}\) When the dimensions of \(I_n\) are clear from the context, or irrelevant, we will omit the subscript \(n\) and simply refer to this matrix as \(I\text{.}\)
You can easily convince yourself that \(I\) commutes with all square matrices of appropriate dimensions. Let
\begin{equation*}
A=\begin{bmatrix}a\amp b\amp c\\d\amp e\amp f\\g\amp h\amp i\end{bmatrix}\quad\text{and}\quad I=\begin{bmatrix}1\amp 0\amp 0\\0\amp 1\amp 0\\0\amp 0\amp 1\end{bmatrix}.
\end{equation*}
Verify that \(AI=A\) and \(IA=A\text{.}\)
Because \(I\) acts like the multiplicative identity \(1\) in regular multiplication, \(I\) (or \(I_n\)) is called the identity matrix.
Next we list several important properties of matrix multiplication. These properties hold only when matrix sizes are such that the products are defined.
Theorem 4.1.20. Properties of Matrix Multiplication.
The following hold for matrices \(A,B,\) and \(C\) and for scalar \(c\text{,}\)
- Left Distributive Property\begin{equation*} A\left( B+C\right) =AB +AC \end{equation*}
- Right Distributive Property:\begin{equation*} \left( B+C\right) A=BA+CA \end{equation*}
- Associativity:\begin{equation*} A\left( BC\right) =\left( AB\right) C \end{equation*}
- Matrix multiplication behaves well with scalar multiplication:\begin{equation*} c(AB)=(cA)B=A(cB) \end{equation*}
- Multiplicative Identity:\begin{equation*} AI=IA=A \end{equation*}
Proof.
We prove Item 1 using the expression in ((4.1.1)) for the \((i,j)\)-entry of a matrix product. (The proof of Item 2 is similar.) The \((i,j)\)-entry of \(A(B+C)\) is given by
\begin{equation*}
\sum_{k}a_{ik}( b_{kj}+c_{kj})=\sum_{k}a_{ik}b_{kj}+\sum_{k}a_{ik}c_{kj}.
\end{equation*}
We recognize the right hand side as the \((i,j)\)-entry of \(AB+AC\text{.}\) Thus \(A(B+C) =AB+AC\text{.}\)
For Item 4,
\begin{equation*}
c\left(\sum_{k}a_{ik}b_{kj}\right) = \sum_{k}\left(c a_{ik}\right)b_{kj} = \sum_{k}a_{ik} \left(b_{kj} c\right).
\end{equation*}
For Item 5, the \((i,j)\)-entry of the product \(AI\) is given by the dot product of the \(i^{th}\) row of \(A\) with the standard unit vector \(\mathbf{e}_j\text{.}\) Clearly, this dot product is \(a_{ij}\text{.}\) Because the \((i,j)\)-entry of the product \(AI\) is equal to the \((i,j)\)-entry \(A\text{,}\) we conclude that \(AI=A\text{.}\) The proof that \(IA=A\) is similar.
Note that we skipped the proof of Item 3, which is quite cumbersome using sigma notation. We will easily tackle this proof later in the course when we cover composition of linear Transformations.
Subsection 4.1.4 Transpose of a Matrix
Another important operation on matrices is that of taking the transpose. For a matrix \(A\text{,}\) we denote the transpose of \(A\) by \(A^T\text{.}\) Before formally defining the transpose, we explore this operation on the following matrix.
\begin{equation*}
\begin{bmatrix}
1 \amp 4 \\
3 \amp 1 \\
2 \amp 6
\end{bmatrix}^{T}=
\begin{bmatrix}
1 \amp 3 \amp 2 \\
4 \amp 1 \amp 6
\end{bmatrix}
\end{equation*}
What happened? The first column became the first row and the second column became the second row. Thus the \(3\times 2\) matrix became a \(2\times 3\) matrix. The number \(4\) was in the first row and the second column and it ended up in the second row and first column.
The definition of the transpose is as follows.
Definition 4.1.21. The Transpose of a Matrix.
Let \(A=\begin{bmatrix} a _{ij}\end{bmatrix}\) be an \(m\times n\) matrix. Then the transpose of \(A\), denoted by \(A^{T}\text{,}\) is the \(n\times m\) matrix given by
\begin{equation*}
A^{T} = \begin{bmatrix} a _{ij}\end{bmatrix}^{T}= \begin{bmatrix} a_{ji} \end{bmatrix}
\end{equation*}
The \(( i, j)\)-entry of \(A\) becomes the \(( j,i)\)-entry of \(A^T\text{.}\)
Here is a short exercise to warm you up to the transpose matrix.
Problem 4.1.22.
Calculate \(A^T\) for the following matrix
\begin{equation*}
A =
\begin{bmatrix}
1 \amp 2 \amp -6 \\
3 \amp 5 \amp 4
\end{bmatrix}.
\end{equation*}
Answer.
\begin{equation*}
A^T =
\begin{bmatrix}
1 \amp 3 \\
2 \amp 5 \\
-6 \amp 4
\end{bmatrix}.
\end{equation*}
Note that \(A\) is a \(2 \times 3\) matrix, while \(A^T\) is a \(3 \times 2\) matrix. The columns of \(A\) are the rows of \(A^T\text{,}\) and the rows of \(A\) are the columns of \(A^T\text{.}\)
Theorem 4.1.23. Properties of the Transpose of a Matrix.
Let \(A\) be an \(m\times n\) matrix, \(B\) an \(n\times p\) matrix, and \(k\) a scalar. Then
- \(\displaystyle \left(A^{T}\right)^{T} = A\)
- \(\displaystyle \left( AB\right) ^{T}=B^{T}A^{T} \)
- \(\displaystyle \left( A+ B\right) ^{T}=A^{T}+ B^{T}\)
- \(\displaystyle \left(kA\right)^T=kA^T\)
We will prove Item 2. The remaining properties are left as exercises.
Proof.
[Proof of Item 2:] Note that \(A\) and \(B\) have compatible dimensions, so that \(AB\) is defined and has dimensions \(m\times p\text{.}\) Thus, \((AB)^T\) has dimensions \(p\times m\text{.}\) On the right side of the equality, \(A^T\) has dimensions \(n\times m\text{,}\) and \(B^T\) has dimensions \(p\times n\text{.}\) Therefore \(B^TA^T\) is defined and has dimensions \(p\times m\text{.}\)
Now we know that \((AB)^T\) and \(B^TA^T\) have the same dimensions.
To show that \((AB)^T=B^TA^T\) we need to show that their corresponding entries are equal. Recall that the \((i,j)\)-entry of \(AB\) is given by the dot product of the \(i^{th}\) row of \(A\) and the \(j^{th}\) column of \(B\text{.}\) The same dot product is also the \((j,i)\)-entry of \((AB)^T\text{.}\)
The \((j,i)\)-entry of \(B^TA^T\) is given by the dot product of the \(j^{th}\) row of \(B^T\) and the \(i^{th}\) column of \(A^T\text{.}\) But the \(j^{th}\) row of \(B^T\) is has the same entries as the \(j^{th}\) column of \(B\text{,}\) and the \(i^{th}\) column of \(A^T\) has the same entries as the \(i^{th}\) row of \(A\text{.}\) Therefore the \((j,i)\)-entry of \(B^TA^T\) is also equal to the \((i,j)\)-entry of \(AB\text{.}\)
Thus, the corresponding components of \((AB)^T\) are equal and we conclude that \((AB)^T=B^TA^T\text{.}\)
The transpose of a matrix is related to other important topics. Consider the following definition.
Definition 4.1.24. Symmetric and Skew Symmetric Matrices.
An \(n\times n\) matrix \(A\) is said to be symmetric if \(A=A^{T}.\) It is said to be skew symmetric if \(A=-A^{T}.\)
We will explore these definitions in the following examples.
Example 4.1.25.
Let
\begin{equation*}
A=
\begin{bmatrix}
2 \amp 1 \amp 3 \\
1 \amp 5 \amp -3 \\
3 \amp -3 \amp 7
\end{bmatrix}.
\end{equation*}
Show that \(A\) is symmetric.
Answer.
\begin{equation*}
A^{T} =
\begin{bmatrix}
2 \amp 1 \amp 3 \\
1 \amp 5 \amp -3 \\
3 \amp -3 \amp 7
\end{bmatrix}.
\end{equation*}
Hence, \(A = A^{T}\text{,}\) so \(A\) is symmetric.
Example 4.1.26.
Let
\begin{equation*}
A=
\begin{bmatrix}
0 \amp 1 \amp 3 \\
-1 \amp 0 \amp 2 \\
-3 \amp -2 \amp 0
\end{bmatrix}.
\end{equation*}
Show that \(A\) is skew symmetric.
Answer.
\begin{equation*}
A^{T} =
\begin{bmatrix}
0 \amp -1 \amp -3\\
1 \amp 0 \amp -2\\
3 \amp 2 \amp 0
\end{bmatrix}.
\end{equation*}
Each entry of \(A^T\) is equal to \(-1\) times the same entry of \(A\text{.}\) Hence, \(A^{T} = - A\) and so by Definition 4.1.24, \(A\) is skew symmetric.
A special case of a symmetric matrix is a diagonal matrix. A diagonal matrix is a square matrix whose entries outside of the main diagonal are all zero. The identity matrix \(I\) is a diagonal matrix. Here is another example.
\begin{equation*}
\begin{bmatrix}2\amp 0\amp 0\amp 0\\0\amp -3\amp 0\amp 0\\0\amp 0\amp 1\amp 0\\0\amp 0\amp 0\amp 4\end{bmatrix}.
\end{equation*}
Exercises 4.1.7 Exercises
1.
If
\begin{equation*}
A=\begin{bmatrix}2\amp -1\\3\amp 4\end{bmatrix}\quad\text{and}\quad B=\begin{bmatrix}3\amp 0\\-2\amp 1\end{bmatrix}
\end{equation*}
then what is the matrix \(2(A+B)\text{?}\)
Answer.
\begin{equation*}
2(A+B) = \begin{bmatrix}10\amp -2\\2\amp 10\end{bmatrix}
\end{equation*}
2.
If
\begin{equation*}
A=\begin{bmatrix}2\amp -1\\3\amp 4\end{bmatrix}\quad\text{and}\quad B=\begin{bmatrix}3\amp 0\\-2\amp 1\end{bmatrix}
\end{equation*}
then what is the matrix \(3A-2B\text{?}\)
Answer.
\begin{equation*}
3A-2B=\begin{bmatrix}0\amp -3\\13\amp 10\end{bmatrix}
\end{equation*}
3.
4.
Prove Theorem 4.1.8.
5.
Explain why the following product is not defined.
\begin{equation*}
\begin{bmatrix}a\amp b\amp c\\d\amp e\amp f\\g\amp h\amp i\end{bmatrix}\begin{bmatrix}1\\2\\3\\4\end{bmatrix}
\end{equation*}
6.
Express the given product as a linear combination of the columns of the matrix.
\begin{equation*}
\begin{bmatrix}1\amp 2\amp 3\amp 4\\5\amp 6\amp 7\amp 8\end{bmatrix}\begin{bmatrix}-2\\3\\-5\\7\end{bmatrix}.
\end{equation*}
Answer.
\begin{equation*}
\begin{bmatrix}1\amp 2\amp 3\amp 4\\5\amp 6\amp 7\amp 8\end{bmatrix}\begin{bmatrix}-2\\3\\-5\\7\end{bmatrix}=-2\begin{bmatrix}1\\5\end{bmatrix}+3\begin{bmatrix}2\\6\end{bmatrix}+-5\begin{bmatrix}3\\7\end{bmatrix}+7\begin{bmatrix}4\\8\end{bmatrix}
\end{equation*}
Exercise Group.
Predict the dimensions of each product.
7.
\begin{equation*}
\begin{bmatrix}1\amp 2\amp 3\end{bmatrix}\begin{bmatrix}4\\5\\6\end{bmatrix}
\end{equation*}
Answer.
Dimensions of product: \(1\times 1\text{.}\)
8.
\begin{equation*}
\begin{bmatrix}1\amp 2\\3\amp 4\\5\amp 6\end{bmatrix}\begin{bmatrix}1\amp 2\amp 3\amp 4\\5\amp 6\amp 7\amp 8\end{bmatrix}
\end{equation*}
Answer.
Dimensions of product: \(3\times 4\)
Exercise Group.
Find each product.
9.
\begin{equation*}
\begin{bmatrix}1\amp 3\amp -2\amp 1\\-2\amp 1\amp 0\amp 4\end{bmatrix}\begin{bmatrix}4\\-2\\1\\1\end{bmatrix}
\end{equation*}
Answer.
\begin{equation*}
\begin{bmatrix}-3\\-6\end{bmatrix}
\end{equation*}
10.
\begin{equation*}
\begin{bmatrix}1\amp 2\amp 3\end{bmatrix}\begin{bmatrix}-1\amp 2\amp -3\amp 1\\1\amp 1\amp -1\amp -2\\0\amp 1\amp 2\amp 1\end{bmatrix}
\end{equation*}
Answer.
\begin{equation*}
\begin{bmatrix}1\amp 7\amp 1\amp 0\end{bmatrix}
\end{equation*}
11.
\begin{equation*}
\begin{bmatrix}1\amp 2\\-1\amp 0\\2\amp 3\\-4\amp -1\end{bmatrix}\begin{bmatrix}-1\amp 1\\2\amp -3\end{bmatrix}
\end{equation*}
Answer.
\begin{equation*}
\begin{bmatrix}3\amp -5\\1\amp -1\\4\amp -7\\2\amp -1\end{bmatrix}
\end{equation*}
12.
13.
Let \(A\) be an arbitrary matrix. What can you say about the dimensions of the product \(A^TA\text{?}\)
Exercise Group.
Classify each matrix as symmetric, skew symmetric, or neither.
14.
- Symmetric
- Skew symmetric
- Neither
\begin{equation*}
\begin{bmatrix}
0 \amp -1 \amp -3\\
1 \amp -1 \amp -2\\
3 \amp 2 \amp 0
\end{bmatrix}
\end{equation*}
15.
- Symmetric
- Skew symmetric
- Neither
\begin{equation*}
\begin{bmatrix}
0 \amp 1 \amp -3\\
-1 \amp 0 \amp -2\\
3 \amp 2 \amp 0
\end{bmatrix}
\end{equation*}
16.
- Symmetric
- Skew symmetric
- Neither
\begin{equation*}
\begin{bmatrix}
1 \amp 1 \amp 3\\
1 \amp -1 \amp 2\\
3 \amp 2 \amp 4
\end{bmatrix}
\end{equation*}
17.
Give your own example of a \(4\times 4\) skew symmetric matrix.
18.
Make a conjecture about the main diagonal entries of a skew symmetric matrix. Prove your conjecture.
