Was going down a rabbit hole on the singular value decomposition and its differences from the eigendecomposition. I’d known both were used pretty extensively in computational improvements but I had a fuzzy image on how SVD actually worked. I think I’ve got a better handle on it now though, although I wouldn’t cite this.
The primary part of SVD reduces a matrix to a transformation (Σ) from one specific orthogonal grid to another specific orthogonal grid, via scaling and rotating. These specific orthogonal grids are set into place by an orthogonal (U) and switch-back, transpose orthogonal (VT) matrix before and after Σ. These rotational matrices are like padding and are of different sizes, allowing for SVD to target non-square matrices.
Meanwhile, the primary part of Eigendecomp reduces a matrix to a transformation (Λ) of it’s eigenvectors, via scaling by its eigenvalues. These eigenvectors are padded into place by eigenvector matrices, which are inverse to one another (S and S-1). The need here for invertible matrices requires Eigendecomp to target square matrices.
Sources for better reference:
Good SVD breakdown: https://gregorygundersen.com/blog/2018/12/10/svd/
SVD vs Eigendecomp thread: https://math.stackexchange.com/questions/320220/intuitively-what-is-the-difference-between-eigendecomposition-and-singular-valu
wow