SVregression

  In kernel ridge regression we have seen the final solution was not sparse in the variables ® . We will now formulate a regression method that is sparse, i.e. it has the concept of support vectors that determine the solution. The thing to notice is that the sparseness arose from complementary slackness conditions which in turn came from the fact that we had inequality constraints. In the SVM the penalty that was paid for being on the wrong side of the support plane was given by C P i » k i for positive integers k, where » i is the orthogonal distance away from the support plane. Note that the term jjwjj2 was there to penalize large w and hence to regularize the solution. Importantly, there was no penalt

SVregression.pdf