======Contrast Matrix Formulation====== The constraints on the full model can be written as $$\mathbf{L} 𝛃 = \mathbf{c} $$ Using Lagrange multiplier, we can get the estimate for null hypothesis $\widehat{𝛃}_0$ by minimizing $$\mathcal{L}\equiv\left(\mathbf{y}-\mathbf{X}𝛃\right)^{T}\left(\mathbf{y}-\mathbf{X}𝛃\right)+\left(\mathbf{L}𝛃-\mathbf{c}\right)^{T}𝛌,$$ where $𝛌$ is a column vector of multipliers. The minimization leads to $$ -2\mathbf{X}^{T}\left(\mathbf{y}-\mathbf{X}𝛃_{0}\right)+\mathbf{L}^{T}𝛌_{0}=0 $$ and $$ \mathbf{L}𝛃_{0}=\mathbf{c}. $$ These equations give \begin{align*} & \mathbf{L}^{T}𝛌_{0}=2\mathbf{X}^{T}\left(\mathbf{y}-\mathbf{X}𝛃_{0}\right)=2\mathbf{X}^{T}\mathbf{y}-2\mathbf{X}^{T}\mathbf{X}𝛃_{0}\\ \Rightarrow\, & \left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T}𝛌_{0}=2\widehat{𝛃}-2𝛃_{0}\\ \Rightarrow\, & \mathbf{L}\left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T}𝛌_{0}=2\mathbf{L}\widehat{𝛃}-2\mathbf{c} \end{align*} or $$ 𝛌_{0}=2\left(\mathbf{L}\left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T}\right)^{-1}\left(\mathbf{L}\widehat{𝛃}-\mathbf{c}\right), $$ as well as $$ 𝛃_{0}=\widehat{𝛃}-\left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T}𝛌_{0}/2, $$ or \begin{align*} \widehat{𝛃}-𝛃_{0} & = \left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T}𝛌_{0}/2\\ & = \left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T}\left(\mathbf{L}\left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T}\right)^{-1}\left(\mathbf{L}\widehat{𝛃}-\mathbf{c}\right). \end{align*} As the the error of the restricted model is given by the square of the residual, $$ \mathbf{r}_{\mathrm{restr}} = \mathbf{y}-\mathbf{X}𝛃_{0} = \mathbf{r}_{\mathrm{full}}+\mathbf{X}\left(\widehat{𝛃}-𝛃_{0}\right), $$ we have \begin{eqnarray*} \mathrm{Err}_{\mathrm{restr}} & = & \mathbf{r}_{\mathrm{restr}}^{T}\mathbf{r}_{\mathrm{restr}}\\ & = & \mathbf{r}_{\mathrm{full}}^{T}\mathbf{r}_{\mathrm{full}}+\left(\widehat{𝛃}-𝛃_{0}\right)^{T}\color{#00a}{\mathbf{X}^{T}\mathbf{X}\left(\widehat{𝛃}-𝛃_{0}\right)}\\ & = & \mathrm{Err}_{\mathrm{full}}+\color{#800}{\left(\widehat{𝛃}-𝛃_{0}\right)^{T}}\color{#00a}{\mathbf{L}^{T}\left(\mathbf{L}\left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T}\right)^{-1}\left(\mathbf{L}\widehat{𝛃}-\mathbf{c}\right)}\\ & = & \mathrm{Err}_{\mathrm{full}}+\color{#800}{\left(\mathbf{L}\widehat{𝛃}-\mathbf{c}\right)^{T}\left(\mathbf{L}\left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T}\right)^{-1}\mathbf{L}\left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}}\\ & & \times\mathbf{L}^{T}\left(\mathbf{L}\left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T}\right)^{-1}\left(\mathbf{L}\widehat{𝛃}-\mathbf{c}\right)\\ & = & \mathrm{Err}_{\mathrm{full}}+\left(\mathbf{L}\widehat{𝛃}-\mathbf{c}\right)^{T}\left(\mathbf{L}\left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T}\right)^{-1}\left(\mathbf{L}\widehat{𝛃}-\mathbf{c}\right) \end{eqnarray*} where the cross terms vanish since $ \mathbf{X}^{T}\mathbf{r}_{\mathrm{full}} = 0 $ by the condition of the full model, and we observe that $ \mathbf{X}^{T}\mathbf{X} $ and $ \mathbf{L}\left(\mathbf{X}^{T}\mathbf{X}\right)^{-1}\mathbf{L}^{T} $ are both symmetric matrices.