Exercises — Lesson 3: Span, Linear Independence, Basis, Dimension
Exercise 3.1. [Hand] Determine whether each list is independent; if dependent, exhibit a dependence relation. (a) \((2,1), (4,3)\) in \(\mathbb{R}^2\). (b) \((1,1,1), (1,2,3), (0,1,2)\) in \(\mathbb{R}^3\). (c) any list containing the zero vector.
Solution.
(a) \(\{(2,1), (4,3)\} \subseteq \mathbb{R}^2\).
\[ \begin{bmatrix} 2 & 4 \\ 1 & 3 \end{bmatrix} \to \begin{bmatrix} 1 & 3 \\ 2 & 4 \end{bmatrix} \to \begin{bmatrix} 1 & 3 \\ 0 & -2 \end{bmatrix} \to \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \]
Linearly independent.
(b) \(\{(1,1,1), (1,2,3), (0,1,2)\}\).
\[ \begin{bmatrix} 1 & 1 & 0 \\ 1 & 2 & 1 \\ 1 & 3 & 2 \end{bmatrix} \to \begin{bmatrix} 1 & 1 & 0 \\ 0 & 1 & 1 \\ 0 & 2 & 2 \end{bmatrix} \to \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 1 \\ 0 & 2 & 2 \end{bmatrix} \to \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 1 \\ 0 & 0 & 0 \end{bmatrix} \]
Linearly dependent. A dependence relation:
\[ -1(1,1,1) + 1(1,2,3) = (0,1,2) \quad \Longrightarrow \quad (0,1,2) \in \operatorname{span}\{(1,1,1), (1,2,3)\}. \]
(c) Any set containing the zero vector is linearly dependent, because \(0 \in \operatorname{span}\{\text{other vectors}\}\).
Exercise 3.2. [Hand] Find a basis for the line \(\{(t, 2t, -t) : t \in \mathbb{R}\}\) and for the plane \(x_1 - x_3 = 0\) in \(\mathbb{R}^3\). State the dimension of each.
Solution.
\[ \begin{aligned} \operatorname{span}\{(1,2,-1)\} &= \{(t, 2t, -t) : t \in \mathbb{R}\}, && \dim = 1 \\[6pt] \operatorname{span}\{(1,0,1), (0,1,0)\} &= \{(x_1, x_2, x_3) : x_1 - x_3 = 0\}, && \dim = 2 \end{aligned} \]
Exercise 3.3. [Proof, \(\star\)] Prove that any list of \(n+1\) vectors in \(\mathbb{R}^n\) is linearly dependent. (One line, given a theorem from this lesson.) Conclude that at most \(n\) features on a dataset of \(n\)-dimensional inputs can be “genuinely new” in the sense of enlarging the span at every step.
Proof. Prove any list of \((n+1)\) vectors in \(\mathbb{R}^n\) is linearly dependent.
The standard basis of \(\mathbb{R}^n\) is \(\{e_1, e_2, \dots, e_n\}\), so \(\dim \mathbb{R}^n = n\) by Theorem 3.6. Hence by Theorem 3.4 any list of vectors of length greater than \(n\) is linearly dependent. \(\square\)
Exercise 3.4. [Proof] Prove that \(\operatorname{span}(v_1, \dots, v_m)\) is a subspace, and that any subspace containing \(v_1, \dots, v_m\) contains \(\operatorname{span}(v_1, \dots, v_m)\).
Proof. Let \(u_1, u_2 \in \operatorname{span}(v_1, \dots, v_m)\) and \(\lambda \in \mathbb{F}\), \(\mathbb{F}\) a field.
\[ \begin{aligned} u_1 + u_2 &\in \operatorname{span}(v_1, \dots, v_m) && \Longrightarrow \ \text{closed under addition} \\[2pt] \lambda u_1 &\in \operatorname{span}(v_1, \dots, v_m) && \Longrightarrow \ \text{closed under scalar multiplication} \\[2pt] 0 &\in \operatorname{span}(v_1, \dots, v_m) && \end{aligned} \]
\(\Longrightarrow \operatorname{span}(v_1, \dots, v_m)\) is a subspace.
By definition of span and subspace, any subspace containing \(v_1, \dots, v_m\) contains \(\operatorname{span}(v_1, \dots, v_m)\). \(\square\)