Solving Systems of Linear Equations (3)

Pseudoinverse of a Matrix

The pseudoinverse introduced here is the Moore-Penrose inverse matrix, defined as follows: Given a matrix $A\in R^{m*n}$, if a matrix $A^{\dagger}\in R^{n*m}$ satisfies $AA^{\dagger}A=A$, and there exist two matrices $U\in R^{n*n},V\in R^{m*m}$ such that

$$A^{\dagger}=UA^{T},A^{\dagger}=A^{T}V$$

then $A^{\dagger}$ is called the pseudoinverse of matrix $A$. It can be proven that the pseudoinverse of a matrix is unique.

For a matrix $A\in R^{m*n},m\ge n$ and $rank(A)=n$, according to the above definition, the pseudoinverse of $A$ can be verified as

$$A^{\dagger}=(A^{T}A)^{-1}A^{T}$$

For a matrix $A\in R^{m*n},m\le n$ and $rank(A)=m$, similarly, according to the above definition, the pseudoinverse of $A$ can be verified as

$$A^{\dagger}=A^{T}(AA^{T})^{-1}$$

The above two cases are the pseudoinverses when the matrix is full column rank or full row rank. For a general matrix $A\in R^{m*n},rank(A)=r,r\le min(m,n)$, we can use the full rank decomposition method to find its pseudoinverse.

For any matrix $A\in R^{m*n},rank(A)=r,r\le min(m,n)$, it can be decomposed into the product of a full row rank matrix and a full column rank matrix: That is, $A=BC,B\in R^{m*r},c\in R^{r*n},rank(A)=rank(B)=rank(C)=r$

It can be proven that: $A^{\dagger}=C^{\dagger}B^{\dagger}$, where $B^{\dagger}=(B^{T}B)^{-1}B^{T},C^{\dagger}=C^{T}(CC^{T})^{-1}$, which is the method for finding the pseudoinverse of a general matrix.

Solving Linear Equations in General Cases

For a linear equation system $Ax=b,A\in R^{m*n},rank(A)=r$, the vector $x^{*}=A^{\dagger}b$ minimizes $||Ax-b||^{2}$ in the space $R^{n}$; among all vectors in $R^{n}$ that can minimize $||Ax-b||^{2}$, the vector $x^{*}=A^{\dagger}b$ has the smallest norm and is unique.

When $r=m$, $A$ is a full row rank matrix, and in this case, $x^{*}=A^{\dagger}b=A^{T}(AA^{T})^{-1}b$, which is the minimum norm solution of the equation system $Ax=b$.

When $r=n$, $A$ is a full column rank matrix, and in this case, $x^{*}=A^{\dagger}b=(A^{T}A)^{-1}A^{T}b$, which is the least squares solution of the equation system $Ax=b$.