Mathematics is always fascinating when we understand it by visualizing its effect. Especially, when you already know something and prove it using mathematics, that ‘AHA’ moment hits differently. The same thing happened to me today when I understood what orthogonal projection is. Well, actually I did know the concept of it from my university days but I understood its significance while I was reading the topic in the book Mathematics for Machine Learning (great book btw, worth a read).
Today in this blog, I won’t be going into detail about explaining the very fundamentals of orthogonal projections, you’d understand the topic better by referring the book I mentioned above. But I would like to share how to share its significance and its effects on vectors.
Orthogonal Projections are vector transformations
We understand the basic meaning of projection as mapping something onto another plane. For example, the video elements from our laptop can be projected on another plane (wall) through a projector. The concept is similar in mathematics. We transform a vector v from a higher-dimensional space onto a lower-dimensional space.

The figure above illustrates the blue vector, from 3-dimensional space, projected onto a 2-dimensional vector space. The projected vector is shown as red in the figure. This is the outcome of orthogonal projection of a vector. Orthogonal because the angle between the projected vector and the distance between the projection and the orioginal vector (shown in green) is 90°.
| So what is the math behind this? How do we make this projection happen?
Understanding the math behind orthogonal projections
Projection is a form of vector transformation. In mathematics, we know that when we transform a vector, we multiply it with the corresponding matrix. For example, if we want to rotate a vector, we multiply it with a rotation matrix. Similarly, in this case, we have a projection matrix that help us know the projected vector. The formula for projection matrix is given by:
\(
P = B (B \cdot B^T)^{-1}B^T
\)
B in the above equation represents the basis of the subspace on which the vector is projected. Multiplying this projection matrix P with some vector x results in its projected vector. Let us go ahead and try this on a simple example.
Example of orthogonal projection
Consider a \((2,2,2)^T\) in a 3-dimensional space as shown in the figure below.

We want to project this vector onto XY plane. The basis of XY plane can be assumed to be \((1,0,0)^T\) and \((0,1,0)^T\). Hence, we can say
\(
B = \left[\begin{array}{cc}
1 & 0 \\ 0 & 1\\ 0 & 0
\end{array}\right]
\)
Using the formula above to calculate projection matrix, we get
\(
P = \left[\begin{array}{ccc}
1 & 0 & 0 \\ 0 & 1 & 0\\ 0 & 0 &0
\end{array}\right]
\)
Ultimately, multiplying this projection matrix P with the original vector \((2,2,2)^T\) gives us \((2,2,0)^T\).

Let us look at this from a simpler perspective. The vector \((2,2,2)^T\) when projected on XY plane loses the value of its Z-coordinate, i.e. the value of Z-coordinate Hence, the “shadow” on XY plane will be \((2,2,0)^T\). It’s fascinating to see the effect of the projection matrix on the vector.
We can conclude from this that if we know the basis of the subspace on which a vector is projected, we can calculate the projection matrix, with the help of which, we can find the projection of any vector on that subspace. Go ahead and try this yourself.
Visualize the original and the projected vector using this online tool for plotting vectors: https://www.math3d.org/lBTsWL2S9