Skip to main content

Subsection 11.1.1 Linking the Singular Value Decomposition to the Spectral Decomposition

Week 2 introduced us to the Singular Value Decomposition (SVD) of a matrix. For any matrix \(A \in \mathbb C^{m \times n} \text{,}\) there exist unitary matrix \(U \in \mathbb C^{m \times m} \text{,}\) unitary matrix \(V \in \mathbb C^{n \times n} \text{,}\) and \(\Sigma \in \mathbb R^{m \times n} \) of the form

\begin{equation} \Sigma = \left( \begin{array}{c | c} \Sigma_{TL} \amp 0_{r \times (n-r)} \\ \hline 0_{(m-r) \times r} \amp 0_{(m-r) \times ( n - r )} \end{array} \right), \mbox{ with } \begin{array}[t]{l} \Sigma_{TL} = \diag{ \sigma_0, \ldots , \sigma_{r-1} }, \\ \mbox{ and } \sigma_0 \geq \sigma_1 \geq \cdots \geq \sigma_{r-1} \gt 0 \end{array}\label{chapter11-sigma}\tag{11.1.1} \end{equation}

such that \(A = U \Sigma V^H \text{,}\) the SVD of matrix \(A \text{.}\) We can correspondingly partition \(U = \left( \begin{array}{c | c} U_L \amp U_R \end{array} \right) \) and \(V = \left( \begin{array}{c | c} V_L \amp V_R \end{array} \right) \text{,}\) where \(U_L \) and \(V_L \) have \(r \) columns, in which case

\begin{equation*} A = U_L \Sigma_{TL} V_L^H \end{equation*}

equals the Reduced Singular Value Decomposition. We did not present practical algorithms for computing this very important result in Week 2, because we did not have the theory and practical insights in place to do so. With our discussion of the QR algorithm in the last week, we can now return to the SVD and present the fundamentals that underlie its computation.

In Week 10, we discovered algorithms for computing the Spectral Decomposition of a Hermitian matrix. The following exercises link the SVD of \(A \) to the Spectral Decomposition of \(B = A^H A \text{,}\) providing us with a first hint as to how to practically compute the SVD.

Homework 11.1.1.1.

Let \(A \in \mathbb C^{m \times n} \) and \(A = U \Sigma V^H \) its SVD, where \(\Sigma \) has the structure indicated in (11.1.1). Give the Spectral Decomposition of the matrix \(A^H A \text{.}\)

Solution
\begin{equation*} \begin{array}{l} A^H A \\ ~~~=~~~~ \lt A = U \Sigma V^H \gt \\ ( U \Sigma V^H)^H ( U \Sigma V^H ) \\ ~~~=~~~~\lt ( B C)^H = C^H B^H; U^H U = I \gt \\ V \Sigma^T \Sigma V^H \\ ~~~=~~~~\\ V \left( \begin{array}{c | c} \Sigma_{TL}^2 \amp 0_{r \times (n-r)} \\ \hline 0_{(n-r) \times r} \amp 0_{(n-r) \times ( n - r )} \end{array} \right) V^H. \end{array} \end{equation*}
Homework 11.1.1.2.

Let \(A \in \mathbb C^{m \times n} \) and \(A = U \Sigma V^H \) its SVD, where \(\Sigma \) has the structure indicated in (11.1.1). Give the Spectral Decomposition of the matrix \(A A^H \text{.}\)

Solution
\begin{equation*} \begin{array}{l} A A^H \\ ~~~=~~~~\\ ( U \Sigma V^H ) ( U \Sigma V^H )^H \\ ~~~=~~~~\\ U \Sigma \Sigma^T U^H \\ ~~~=~~~~\\ U \left( \begin{array}{c | c} \Sigma_{TL}^2 \amp 0_{r \times (m-r)} \\ \hline 0_{(m-r) \times r} \amp 0_{(m-r) \times ( m - r )} \end{array} \right) U^H. \end{array} \end{equation*}

Last two homeworks expose how to compute the Spectral Decomposition of \(A^H A \) or \(A A^H \) from the SVD of matrix \(A \text{.}\) We already discovered practical algorithms for computing the Spectral Decomposition in the last week. What we really want to do is to turn this around: How do we compute the SVD of \(A \) from the Spectral Decomposition of \(A^H A \) and/or \(A A^H \text{?}\)

login