Learn Linear Algebra

Theorem

The solution set for a matrix equation is the same as the solution set for the associated vector equation. This, in turn, is the same as the solution set for the underlying system of linear equations

Proof:

Let some mxnmxn matrix A=[a1 a2  an],x=[x1x2xn]RnA = [\vec{a}_1 \ \vec{a}_2 \ \cdots \ \vec{a}_n], \vec{x} = \left[\begin{array}{c} x_1 \\x_2 \\\vdots \\x_n\end{array}\right] \in \mathbb{R}^{n} and let the matrix equation Ax=bA\vec{x}=\vec{b} have a valid solution.

Since AxA\vec{x} has a valid solution b\vec{b} we can write b\vec{b} as a linear combination of the columns of AA and the weights in x\vec{x}.

x1a1+x2a2++xnan=bx_1\vec{a}_1 + x_2\vec{a}_2 + \cdots + x_n\vec{a}_n = \vec{b} since the matrix equation has a valid solution it must follow that the corresponding vector equation has the same solution set.

Now, we can construct an augmented matrix from this:

[a11a21an1b1a12a22an2b2a1ma2manmbn]\left[ \begin{array}{cccc|c} a_{11} & a_{21} & \cdots & a_{n1} & b_1 \\ a_{12} & a_{22} & \cdots & a_{n2} & b_2 \\ \vdots & \vdots & \ddots & \vdots & \vdots \\ a_{1m} & a_{2m} & \cdots & a_{nm} & b_n \end{array} \right] From this we can now construct a system of linear equations {x1a11+x2a21++xnan1=b1x2a12+x2a22++xnan2=b2xna1m+x2a2m++xnanm=bn \left\{ \begin{array}{l} x_{1}a_{11} + x_{2}a_{21} + \dots + x_{n}a_{n1} = b_1 \\\\ x_{2}a_{12} + x_{2}a_{22} + \dots + x_{n}a_{n2} = b_2 \\\\ \dots \\\\ x_{n}a_{1m} + x_{2}a_{2m} + \dots + x_{n}a_{nm} = b_n \\\\ \end{array} \right.

Since the solution (x1,x2,,xnx_1, x_2,\cdots,x_n) is valid for the matrix equation it must also follow that it is valid for the assocaited vector equation and for the associated system of equations since we showed that they are all equivelant.

Theorem

The matrix equation Ax=bA\vec{x}=\vec{b} has a solution if and only if b\vec{b} is a linear combination of the columns of AA.

Proof \rightarrow:

Suppose the matrix equation Ax=bA\vec{x}=\vec{b} has a solution. Let AA be an mxnmxn matrix where A=[a1 a2  an]A = [\vec{a}_1 \ \vec{a}_2 \ \cdots \ \vec{a}_{n}] and let x=[x1x2xn]x = \left[ \begin{array}{c} x_1 \\ x_2 \\ \vdots \\ x_n \end{array} \right].

We can rewrite this as the following vector equation: x1a1+x2a2++xnan=bx_1\vec{a}_1 + x_2\vec{a}_2 + \cdots + x_n\vec{a}_n = \vec{b} b\vec{b} is generated as a linear comibination of the columns of AA and the weights (x1,x2,,xn)(x_1, x_2,\cdots,x_n). Since the matrix equation has a solution, b\vec{b} can be generated as a linear combination of the columns of AA.

Proof \leftarrow:

Suppose b\vec{b} can be generated by a linear combination of the columns of some matrix AA. Let A=[a1 a2  an]A = [\vec{a}_1 \ \vec{a}_2 \ \cdots \ \vec{a}_{n}], then there exist some weights (x1,x2,,xn)(x_1, x_2, \cdots, x_n) that satisfy the following vector equation: x1a1+x2a2++xnan=bx_1\vec{a}_1 + x_2\vec{a}_2 + \cdots + x_n\vec{a}_n = \vec{b} We can rewrite this as: [a1 a2  an][x1x2xn]=b[\vec{a}_1 \ \vec{a}_2 \ \cdots \ \vec{a}_n]\left[ \begin{array}{c} x_1 \\ x_2 \\ \vdots \\ x_n \end{array} \right]=\vec{b} By definition this is a matrix equation and can be written in the following form: Ax=bA\vec{x}=\vec{b} Since b\vec{b} can be genereated by a linear combination of the columns of A it must also follow that the matrix equation has a solution because we showed that they are equivelant.

Theorem

Let AA be an m×nm \times n matrix. Then the following statements are logically equivalent:
  1. For each bRm\vec{b} \in \mathbb{R}^{m}, the equation Ax=bA\vec{x}=\vec{b} has a solution.
  2. For each bRm\vec{b} \in \mathbb{R}^{m}, b\vec{b} is a linear combination of the columns of AA.
  3. The columns of AA span Rm\mathbb{R}^{m}.
  4. AA has a pivot position in every row.
Please note that we have proved statement 1 given statement 2 and we have proved statement 2 given statement 1 in a previous theorem. All other proof combinations will be shown.

Proof (statement 1 given statement 3):

Given the columns of some mxnm x n matrix AA span Rm\mathbb{R}^{m} lets assume the matrix equation Ax=bA\vec{x}=\vec{b} does not have a solution. Lets denote the columns of AA as [a1 a2  an\vec{a}_1 \ \vec{a}_2 \ \cdots \ \vec{a}_n]. By definiton Span{a1,a2,,an\vec{a}_1, \vec{a}_2, \cdots, \vec{a}_n} is the set of all linear combinations of a1,a2,,an\vec{a}_1, \vec{a}_2, \cdots, \vec{a}_n and since the columns of AA span Rm\mathbb{R}^{m} it must be true that we can generate a vector b\vec{b} as a linear combination of the columns of AA. More specifically, there exists a solution x1,x2,,xnx_1, x_2,\cdots,x_n for the vector equation x1a1+x2a2++xnan=bx_1\vec{a}_1 + x_2\vec{a}_{2} + \cdots + x_n\vec{a}_n = \vec{b}. Since we previously showed that a vector equation has the same solution set to its associated matrix equation then x1a1+x2a2++xnan=bx_1\vec{a}_1 + x_2\vec{a}_{2} + \cdots + x_n\vec{a}_n = \vec{b} has the same solution set as Ax=bA\vec{x}=\vec{b} but this is a contradiction to our original assumption that the matrix equation does not have a solution.

Proof (statement 3 given statement 1):

Given that for each bRm\vec{b} \in \mathbb{R}^{m}, Ax=bA\vec{x}=\vec{b} has a solution lets denote the columns of some mxnmxn matrix AA as a1 a2 anRm\vec{a}_1 \ \vec{a}_2 \ \cdots \vec{a}_n \in \mathbb{R}^m and let x=[x1x2xn]x = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix}. We can rewrite this equation now in the following form: [a1 a2  an][x1x2xn]=b[\vec{a}_1 \ \vec{a}_2 \ \cdots \ \vec{a}_n] \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix}=\vec{b} Which in turn can now be written in the following form: x1a1+x2a2++xnan=bx_1\vec{a}_1 + x_2\vec{a}_2 + \cdots + x_n\vec{a}_n = \vec{b} Since the solution holds for the intial matrix equation, it must also hold for the vector equation because we have shown that they are equivelant. Furthermore, since we can generate some b\vec{b} as a linear combination of the columns of AA it must also be true that the columns of AA Span Rm\mathbb{R}^{m}.

Proof (statement 1 given statement 4):

Given that some mxnmxn matrix AA has a pivot in each row, lets denote the columns of AA as a1 a2 an\vec{a}_1 \ \vec{a}_2 \ \cdots \vec{a}_n. Since we have a pivot in each row of the coefficient matrix then it must also be true that the associated augmented matrix of AA will never have a pivot in the augmented column. Since no contradictions can arise from this, the system will always have a consistent solution. Then there exists weights x1,x2,,xnRx_1, x_2, \cdots, x_n \in \mathbb{R} such that x1a1+x2a2++xnan=bx_1\vec{a}_1 + x_2\vec{a}_2 + \cdots + x_n\vec{a}_n = \vec{b} will always have a solution for each bRm\vec{b} \in \mathbb{R}^m because we have m pivot columns. Since the vector equation will always have a valid solution and since we have already shown that a vector equation has the same solution set as the associated matrix equation, then it must follow that the matrix equation has a solution for each bRm\vec{b} \in \mathbb{R}^m

Proof (statement 4 given statement 1):

Given that for each bRm\vec{b} \in \mathbb{R}^{m}, Ax=bA\vec{x}=\vec{b} has a solution lets assume that AA does not have a pivot position in each row. Then there exists a row in the form of [0 0  0 b0 \ 0 \ \cdots \ 0 \ b] where b is the augmented column. Now, we are given that for each bRm\vec{b} \in \mathbb{R}^{m}, Ax=bA\vec{x}=\vec{b} has a solution this means that there exists a solution where b is non-zero but this is a contradiction to our original assumption because if b is non-zero then our system is inconsistent and we already stated that the matrix equation has a solution for each bRm\vec{b} \in \mathbb{R}^m

Theorem

If AA is an m×nm \times n matrix, u\vec{u} and v\vec{v} are vectors in Rn\mathbb{R}^n, and cc is a scalar, then the following statements are true:

1. A(u+v)=Au+AvA(\vec{u} + \vec{v}) = A\vec{u} + A\vec{v}

2. A(cu)=c(Au)A(c\vec{u}) = c(A\vec{u})

Proof(Statement one):

Let A=[a1 a2  an]A = [\vec{a}_1 \ \vec{a}_2 \ \cdots \ \vec{a}_n], u=(u1 u2  un)\vec{u} = (u_1 \ u_2 \ \cdots \ u_n) and v=(v1 v2  _n)\vec{v} = (v_1 \ v_2\ \cdots \ \_n)

Theorem

If xRn\vec{x} \in \mathbb{R}^{n}, then Inx=xI_{n}\vec{x} = \vec{x}

Proof:

Suppose xRn\vec{x} \in \mathbb{R}^n lets assume that InxxI_n \vec{x} \ne \vec{x}, where InI_n is the n×nn \times n identity matrix. This implies that there is at least one component of InxI_n \vec{x} that is different from the corresponding component of x\vec{x}. By definition, InI_n has 1s on the main diagonal and 0s elsewhere. Therefore, each component of InxI_n \vec{x} is simply the corresponding component of x\vec{x}. There cannot exist a value in the main diagonal of InI_n that is not 1, nor can there be a value elsewhere that affects x\vec{x} due to the 0s. Then it must be true that Inx=xI_n \vec{x} = \vec{x}, which is a contradiction to our assumption that InxxI_n \vec{x} \ne \vec{x}. It must be true that Inx=xI_n \vec{x} = \vec{x} for all xRn\vec{x} \in \mathbb{R}^n.