Differences

This shows you the differences between two versions of the page.

--- notes:dan:contrast-matrix [2022/03/20 03:56] – created cjj
+++ notes:dan:contrast-matrix [2022/03/20 08:37] (current) – cjj
@@ Line 1: / Line 1: @@
 ======Contrast Matrix Formulation======
-The constraint can be written as
+The constraints on the full model can be written as
 $$\mathbf{L} 𝛃 = \mathbf{c} $$
 Using Lagrange multiplier, we can get the estimate for null hypothesis $\widehat{𝛃}_0$ by minimizing
+$$\mathcal{L}\equiv\left(\mathbf{y}-\mathbf{X}𝛃\right)^{T}\left(\mathbf{y}-\mathbf{X}𝛃\right)+\left(\mathbf{L}𝛃-\mathbf{c}\right)^{T}𝛌,$$
+where $𝛌$ is a column vector of multipliers.
+The minimization leads to
+$$ -2\mathbf{X}^{T}\left(\mathbf{y}-\mathbf{X}𝛃_{0}\right)+\mathbf{L}^{T}𝛌_{0}=0 $$
+and
+$$ \mathbf{L}𝛃_{0}=\mathbf{c}. $$
+These equations give
+\begin{align*}
+& \mathbf{L}^{T}𝛌_{0}=2\mathbf{X}^{T}\left(\mathbf{y}-\mathbf{X}𝛃_{0}\right)=2\mathbf{X}^{T}\mathbf{y}-2\mathbf{X}^{T}\mathbf{X}𝛃_{0}\\
+\Rightarrow\, & \left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T}𝛌_{0}=2\widehat{𝛃}-2𝛃_{0}\\
+\Rightarrow\, & \mathbf{L}\left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T}𝛌_{0}=2\mathbf{L}\widehat{𝛃}-2\mathbf{c}
+\end{align*}
+or
+$$ 𝛌_{0}=2\left(\mathbf{L}\left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T}\right)^{-1}\left(\mathbf{L}\widehat{𝛃}-\mathbf{c}\right), $$
+as well as
+$$ 𝛃_{0}=\widehat{𝛃}-\left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T}𝛌_{0}/2, $$
+or
+\begin{align*}
+\widehat{𝛃}-𝛃_{0} & = \left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T}𝛌_{0}/2\\
+& = \left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T}\left(\mathbf{L}\left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T}\right)^{-1}\left(\mathbf{L}\widehat{𝛃}-\mathbf{c}\right).
+\end{align*}
+As the the error of the restricted model is given by the square of the residual,
+$$ \mathbf{r}_{\mathrm{restr}} = \mathbf{y}-\mathbf{X}𝛃_{0} = \mathbf{r}_{\mathrm{full}}+\mathbf{X}\left(\widehat{𝛃}-𝛃_{0}\right), $$
+we have
+\begin{eqnarray*}
+\mathrm{Err}_{\mathrm{restr}} & = & \mathbf{r}_{\mathrm{restr}}^{T}\mathbf{r}_{\mathrm{restr}}\\
+ & = & \mathbf{r}_{\mathrm{full}}^{T}\mathbf{r}_{\mathrm{full}}+\left(\widehat{𝛃}-𝛃_{0}\right)^{T}\color{#00a}{\mathbf{X}^{T}\mathbf{X}\left(\widehat{𝛃}-𝛃_{0}\right)}\\
+ & = & \mathrm{Err}_{\mathrm{full}}+\color{#800}{\left(\widehat{𝛃}-𝛃_{0}\right)^{T}}\color{#00a}{\mathbf{L}^{T}\left(\mathbf{L}\left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T}\right)^{-1}\left(\mathbf{L}\widehat{𝛃}-\mathbf{c}\right)}\\
+ & = & \mathrm{Err}_{\mathrm{full}}+\color{#800}{\left(\mathbf{L}\widehat{𝛃}-\mathbf{c}\right)^{T}\left(\mathbf{L}\left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T}\right)^{-1}\mathbf{L}\left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}}\\
+ & & \times\mathbf{L}^{T}\left(\mathbf{L}\left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T}\right)^{-1}\left(\mathbf{L}\widehat{𝛃}-\mathbf{c}\right)\\
+ & = & \mathrm{Err}_{\mathrm{full}}+\left(\mathbf{L}\widehat{𝛃}-\mathbf{c}\right)^{T}\left(\mathbf{L}\left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T}\right)^{-1}\left(\mathbf{L}\widehat{𝛃}-\mathbf{c}\right)
+\end{eqnarray*}
+where the cross terms vanish since $ \mathbf{X}^{T}\mathbf{r}_{\mathrm{full}} = 0 $ by the condition of the full model, and we observe that $ \mathbf{X}^{T}\mathbf{X} $ and $ \mathbf{L}\left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T} $ are both symmetric matrices.

CBN Courses

User Tools

Site Tools

Differences

Page Tools