In this article on linear algebra, we are going to understand about linear independence between vectors. Furthermore, we will use Gaussian elimination to check if a set of vectors is linearly dependent. But before you read any further, are you familiar with Gaussian elimination? If yes, feel free to go ahead with this article. If you haven’t heard of Gaussian elimination, I strongly recommend you to first go through this article on How to solve a system of linear equations, because it contains the explanation that will be necessary later in this article.
What is linear independence?
Linear independence tells us about the relationship between vectors. It helps us to determine the correlation between the vectors, if any. Consider a given vector space \(V\). If there exists a vector such that it is the result of a linear combination of other vectors, then it can be said that the set of vectors is linearly dependent. Let us understand the meaning of this with an example.:
\(
\left[\begin{array}{ccc}
1 & -1 & 2 \\
2 & 3 & 4 \\
3 & -2 & 6
\end{array}\right]
\)
In the above matrix, we have 3 vectors stacked together. Let us denote them as vectors \(v_1, v_2\) and \(v_3\). There exists no relationship between vectors \(v_1\) and \(v_2\). So as far as only these two vectors are concerned, they are linearly independent. However, there does exist a relationship between vectors \(v_1\) and \(v_3\), that is, \(v_3\) is twice the value of \(v_1\). Mathematically, Hence, this can be described as \(v_3 = 2*v_1\). Thus, ultimately the vectors are said to be linearly dependent.
Let us consider another example through different analogy. Illustrated below is a map of Germany. For instance, you want to travel from Frankfurt to Munich. For that, you take the route shown as vectors \(v1\) (Frankfurt to Stuttgart) and \(v2\) (Stuttgart to Munich) in the image below.
Note that vectors \(v1\) and \(v2\) are completely independent of each other as there exists no relationship between them. After a few days, you get to know a different route that directly connects Frankfurt and Munich (vector \(v3\)).
In this case, the third vector can be described as combination of the first two vectors. In fact, any vector can be described as a combination of the other two (recall vector operations).
Hence, we hereby understand the basic meaning of linear (in)dependence. However, the set of vectors available to determine the dependence is not always trivial. It could get challenging to verify any correlation between vectors. This is where Gaussian elimination comes to the rescue!
Determining linear (in)dependence using Gaussian elimination
[inlinetweet]When we reduce a matrix to its row-echelon form, the vectors are independent if and only if all the columns are pivot columns.[/inlinetweet] Similarly, if there exists non-pivot column, the vectors have dependence between them. Let us consider the matrix we saw above to understand the two statements above.
\(
\left[\begin{array}{ccc}
1 & -1 & 2 \\
2 & 3 & 4 \\
3 & -2 & 6
\end{array}\right]
\)
We know that the vectors are dependent. So, according to the statements above, all the columns must be pivot columns when we reduce the matrix to its row-echelon form.
\(
\begin{equation} R_2 – 2*R_1 = R_2\end{equation} \)
\(
\begin{equation}R_3 – 3*R_1 = R_3 \end{equation} \)
\(\Downarrow\)
\(
\left[\begin{array}{ccc}
1 & -1 & 2 \\
0 & 5 & 0 \\
0 & 1 & 0
\end{array}\right]
\)
Further,
\(
\begin{equation} R_2 \leftrightarrow R_3 \end{equation} \)
\(
\begin{equation}R_3 – 5*R_2 = R_3 \end{equation} \)
\(\Downarrow\)
\(
\left[\begin{array}{ccc}
1 & -1 & 2 \\
0 & 1 & 0 \\
0 & 0 & 0
\end{array}\right]
\)
The first and the second columns are pivot columns, but not the third. Hence, the set of vectors do share relation between them. It is clear that the third column is twice the first column. Déjà–vu?
Conclusion
In this article, we understood the meaning of linear dependence between vectors, and how we can use Gaussian elimination to determine it. In the example above, we proved that the vectors are dependent. Now, its your turn to go ahead and check whether the following vectors are dependent or independent.
\(
\left[\begin{array}{ccc}
2 & -4 & 7 \\
4 & 5 & 3 \\
1 & 9 & 5
\end{array}\right]
\)
Let me know your conclusion in the comments below. Have fun doing math!