Of course while we are at it we may chose to optimize over the tangent space (what will be called V).

The tangent space is a sub-space of the image space. So we are looking for a matrix which is not full rank.

The non-full rank matrix that will do the best job of approximating the full space is built from the first few eigenvectors of the covariance matrix.