There will have many problems when training machine learning algorithm.

  1. under fitting problem Feature Polynomial too low for fitting target training set, unable to meet training set’s performance, let alone future data.
  2. overfitting problem Feature polynomial too high for fitting target training set, means fit training set’s performance too well, but unable to predict future data.

Regularization could ameliorate or to reduce over-fitting problem.

premise

  1. $m$ is the number of training set records.
  2. $n$ is the number of features
  3. $\lambda$ is penality value for reducing high polynomial features’ effect, with larger this value is, smaller the effect is, but learning algorithm turn out to under fitting if $\lambda$ too high.

regularized Linear regression

Cost function

Gradient descend

regularized logistic regression

Cost function

Gradient descend



Published

27 July 2014

Category

development

Tags