Math Insight

Dot product in matrix notation

Given the rules of matrix multiplication, we cannot multiply two vectors when they are both viewed as column matrices. If we try to multiply an $n \times 1$ matrix with another $n \times 1$ matrix, this product is not defined. The number of columns of the first matrix (1) does not match the number of rows of the second matrix ($n$). To rectify this problem, we can take the transpose of the first vector, turning it into a $1 \times n$ row matrix. With this change, the product is well defined; the product of a $1 \times n$ matrix with an $n \times 1$ matrix is a $1 \times 1$ matrix, i.e., a scalar.

If we multiply $\vc{x}^T$ (a $1 \times n$ matrix) with any $n$-dimensional vector $\vc{y}$ (viewed as an $n \times 1$ matrix), we end up with a matrix multiplication equivalent to the familiar dot product of $\vc{x} \cdot \vc{y}$: \begin{align*} \vc{x}^T \vc{y} = \left[ \begin{array}{ccccc} x_1& x_2& x_3& \cdots& x_n \end{array} \right] \left[ \begin{array}{c} y_1\\ y_2\\ y_3\\ \vdots\\ y_n \end{array} \right] =x_1y_1+x_2y_2+x_3y_3 + \ldots + x_ny_n = \vc{x} \cdot \vc{y}. \end{align*} Although we won't typically write a dot product as $\vc{x}^T \vc{y}$, you may see it elsewhere. Moreover, you can view this dot product as forming the building block for the general matrix multiplication.