MatrixSolveLs

Solves one or more linear least-squares problems.

`matrix` is a tensor of shape `[..., M, N]` whose inner-most 2 dimensions form real or complex matrices of size `[M, N]`. `Rhs` is a tensor of the same type as `matrix` and shape `[..., M, K]`. The output is a tensor shape `[..., N, K]` where each output matrix solves each of the equations `matrix[..., :, :]` * `output[..., :, :]` = `rhs[..., :, :]` in the least squares sense.

We use the following notation for (complex) matrix and right-hand sides in the batch:

`matrix`=\\(A \in \mathbb{C}^{m \times n}\\), `rhs`=\\(B \in \mathbb{C}^{m \times k}\\), `output`=\\(X \in \mathbb{C}^{n \times k}\\), `l2_regularizer`=\\(\lambda \in \mathbb{R}\\).

If `fast` is `True`, then the solution is computed by solving the normal equations using Cholesky decomposition. Specifically, if \\(m \ge n\\) then \\(X = (A^H A + \lambda I)^{-1} A^H B\\), which solves the least-squares problem \\(X = \mathrm{argmin}_{Z \in \Re^{n \times k} } ||A Z - B||_F^2 + \lambda ||Z||_F^2\\). If \\(m \lt n\\) then `output` is computed as \\(X = A^H (A A^H + \lambda I)^{-1} B\\), which (for \\(\lambda = 0\\)) is the minimum-norm solution to the under-determined linear system, i.e. \\(X = \mathrm{argmin}_{Z \in \mathbb{C}^{n \times k} } ||Z||_F^2 \\), subject to \\(A Z = B\\). Notice that the fast path is only numerically stable when \\(A\\) is numerically full rank and has a condition number \\(\mathrm{cond}(A) \lt \frac{1}{\sqrt{\epsilon_{mach} } }\\) or \\(\lambda\\) is sufficiently large.

If `fast` is `False` an algorithm based on the numerically robust complete orthogonal decomposition is used. This computes the minimum-norm least-squares solution, even when \\(A\\) is rank deficient. This path is typically 6-7 times slower than the fast path. If `fast` is `False` then `l2_regularizer` is ignored.