Masteries · Jason GlowneyArithmoí Book 1

Masteries

1.1.10 Calculating the SVD

As we will be deal­ing with square ma­tri­ces, we choose the ex­am­ple for a matrix M∈ℂM\in \mathbb{C}, to be

M=(22ii3).\begin{array}{r} M = \begin{pmatrix} 2 & 2i \\ i & 3 \end{pmatrix}. \end{array}
(1.1.36)

We start by com­put­ing M†M^{\dagger}

M†=(2−i−2i3),\begin{array}{r} M^{\dagger}\mathbf{=}\begin{pmatrix} 2 & - i \\ - 2i & 3 \end{pmatrix}, \end{array}
(1.1.37)

then cal­cu­late the matrix prod­uct M†MM^{\dagger}M

M†M=(2−i−2i3)(22ii3)=(5i−i13).\begin{array}{r} M^{\dagger}M\mathbf{=}\begin{pmatrix} 2 & - i \\ - 2i & 3 \end{pmatrix}\begin{pmatrix} 2 & 2i \\ i & 3 \end{pmatrix} = \begin{pmatrix} 5 & i \\ - i & 13 \end{pmatrix}. \end{array}
(1.1.38)

To de­ter­mine our sin­gu­lar value matrix Σ\Sigma, we find the eigen­val­ues of M†MM^{\dagger}M, take their square roots, and arrange them in de­scend­ing order

Σ=eig(M∗M)≈(4.87690013.1231)≈(2.2084003.6226),\begin{array}{r} \Sigma = \sqrt{eig\left( M^{*}M \right)} \approx \sqrt{\begin{pmatrix} 4.8769 & 0 \\ 0 & 13.1231 \end{pmatrix}} \approx \begin{pmatrix} 2.2084 & 0 \\ 0 & 3.6226 \end{pmatrix}, \end{array}
(1.1.39)

and de­ter­mine the in­verse of Σ\Sigma, equals

Σ−1≈(0.4528000.2760).\begin{array}{r} \Sigma^{- 1} \approx \begin{pmatrix} 0.4528 & 0 \\ 0 & 0.2760 \end{pmatrix}. \end{array}
(1.1.40)

To de­ter­mine the VV matrix, we solve for each eigen­val­ue of M†MM^{\dagger}M, in order to find the values of the columns of VV, en­sur­ing these are nor­mal­ized. For the first eigen­val­ue 4.8769,4.8769,we have

(5−4.8769i−i13−4.8769)(v1v2)=(00)→(0.1231v1+iv2−iv1+8.1231v2)=(00)∴V1≈(0.9925i−0.1222).\begin{array}{r} \begin{pmatrix} 5 - 4.8769 & i \\ - i & 13 - 4.8769 \end{pmatrix}\begin{pmatrix} v_{1} \\ v_{2} \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix} \rightarrow \begin{pmatrix} 0.1231v_{1} + iv_{2} \\ - iv_{1} + 8.1231v_{2} \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix}\therefore V_{1} \approx \begin{pmatrix} 0.9925i \\ - 0.1222 \end{pmatrix}. \end{array}
(1.1.41)

A sim­i­lar cal­cu­la­tion for the second eigen­val­ue 13.123113.1231, re­sults in

V2≈(0.1222i0.9925),\begin{array}{r} V_{2} \approx \begin{pmatrix} 0.1222i \\ 0.9925 \end{pmatrix}, \end{array}
(1.1.43)

where the VV matrix is then

V=(V1 V2)≈(0.9925i0.1222i−0.12220.9925).\begin{array}{r} V = \left( V_{1}\ V_{2} \right) \approx \begin{pmatrix} 0.9925i & 0.1222i \\ - 0.1222 & 0.9925 \end{pmatrix}. \end{array}
(1.1.44)

Taking the SVD de­com­po­si­tion

M=UΣV†,M = U\Sigma V^{\dagger},

we solve the SVD for UU, to find

M=UΣV†→MV=UΣV†V→MV=UΣ→U=MVΣ−1,\begin{array}{r} M = U\Sigma V^{\dagger} \rightarrow MV = U\Sigma V^{\dagger}V \rightarrow MV = U\Sigma \rightarrow U = MV\Sigma^{- 1}, \end{array}
(1.1.45)

which for the case ex­am­ple, we find UU equals

U≈(22ii3)(0.9925i0.1222i−0.12220.9925)(0.452800 0.2760)≈(0.7882i0.6154i−0.61540.7882),\begin{array}{r} U \approx \begin{pmatrix} 2 & 2i \\ i & 3 \end{pmatrix}\begin{pmatrix} 0.9925i & 0.1222i \\ - 0.1222 & 0.9925 \end{pmatrix}\begin{pmatrix} 0.4528 & 0 \\ 0 & \ 0.2760 \end{pmatrix} \approx \begin{pmatrix} 0.7882i & 0.6154i \\ - 0.6154 & 0.7882 \end{pmatrix}, \end{array}
(1.1.46)

and the SVD of the matrix MM, is

M=UΣV∗≈(0.7882i0.6154i−0.61540.7882)(2.2084003.6226)(−0.9925i−0.1222−0.1222i0.9925).\begin{array}{r} M = U\Sigma V^{*} \approx \begin{pmatrix} 0.7882i & 0.6154i \\ - 0.6154 & 0.7882 \end{pmatrix}\begin{pmatrix} 2.2084 & 0 \\ 0 & 3.6226 \end{pmatrix}\begin{pmatrix} - 0.9925i & - 0.1222 \\ - 0.1222i & 0.9925 \end{pmatrix}. \end{array}
(1.1.47)

To finish, we look at how the SVD de­com­po­si­tion of MM acts on the vector v=(i,1)Tv = (i,1)^{T} ∈ℂ\in \mathbb{C}. Start­ing with the prod­uct

V†v≈(−0.9925i−0.1222−0.1222i0.9925)(i1)≈(0.87031.1147),\begin{array}{r} V^{\dagger}v \approx \begin{pmatrix} - 0.9925i & - 0.1222 \\ - 0.1222i & 0.9925 \end{pmatrix}\begin{pmatrix} i \\ 1 \end{pmatrix} \approx \begin{pmatrix} 0.8703 \\ 1.1147 \end{pmatrix}, \end{array}
(1.1.48)

we note that the two-norm of the start­ing and re­sul­tant vec­tors is 2\sqrt{2}, mean­ing in polar co­or­di­nates, the start­ing vector (i,1)T(i,1)^{T} has the same radial dis­tance (mod­u­lus) from the origin as the re­sult­ing vector V†v≈(0.8703, 1.1147)TV^{\dagger}v \approx (0.8703,\ 1.1147)^{T}. The trans­for­ma­tion per­formed by the matrix V†V^{\dagger}on vv, is a ro­ta­tion that maps both com­po­nents of the vector into the real field. Scal­ing V†vV^{\dagger}v by Σ\Sigma, we find

ΣV∗v≈(2.2084003.6226)(0.87031.1147)≈(1.92204.0381),\begin{array}{r} \Sigma V^{*}v \approx \begin{pmatrix} 2.2084 & 0 \\ 0 & 3.6226 \end{pmatrix}\begin{pmatrix} 0.8703 \\ 1.1147 \end{pmatrix} \approx \begin{pmatrix} 1.9220 \\ 4.0381 \end{pmatrix}, \end{array}
(1.1.49)

noting that our first vector el­e­ment v1(0.8703) has been scaled by σ1(2.2084)v_{1}(0.8703)\ has\ been\ scaled\ by\ \sigma_{1}(2.2084), equal­ing 1.92201.9220, and our second vector el­e­ment v2(1.1147) has been scaled by σ2(3.6226)v_{2}(1.1147)\ has\ been\ scaled\ by\ \sigma_{2}(3.6226), to now equal to 4.03814.0381. The last step is to mul­ti­ply the above result by the matrix UU, find­ing

UΣV†v≈(0.7882i0.6154i−0.61540.7882)(1.92204.0381)=(4i2),\begin{array}{r} U\Sigma V^{\dagger}v \approx \begin{pmatrix} 0.7882i & 0.6154i \\ - 0.6154 & 0.7882 \end{pmatrix}\begin{pmatrix} 1.9220 \\ 4.0381 \end{pmatrix} = \begin{pmatrix} 4i \\ 2 \end{pmatrix}, \end{array}
(1.1.50)

where the UU ro­ta­tion gives us the real and imag­i­nary vector el­e­ments once again.