Aggregation Models

这是Coursera上《机器学习技法》的课程笔记。

　　Aggregation models: mix or combine hypotheses for better performance, and it‘s a rich family.

　　Suppose we have $T$ hypotheses ,denoted by $g_1$, $g_2$, ... ,$g_T$. There are four different approachs to get a appregation model:

1.Select the best one $g_{t_*}$ from validation error $$G(x)=g_{t_*}(x) with t_*=argmin_{t \in \{1,2,...,T\}}E_{val}(g^-_t)$$

2.Mix all hypotheses uniformly $$G(x)=sign(\sum_{t=1}^T1*g_t(x))$$

3.mix all hypotheses non-uniformly $$G(x)=sign(\sum_{t=1}^T\alpha_t*g_t(x)) with \alpha_t \gep 0$$

　　NOTE: conclude select and mix uniformly.

4.Combine all hypotheses conditionally $$G(x)=sign(\sum_{t=1}^Tq_t(x)*g_t(x)) with q_t(x)\gep 0$$

　　NOTE: conclude non-uniformly