Beginner little rookie , I hope it's like taking notes and recording what I've learned , Also hope to help the same entry-level people , I hope the big guys can help correct it ~ Tort made delete .

Catalog

One 、 Problem description

Two 、 Problem analysis

3、 ... and 、 solve the problem  —— look for w and b

1、 Vector form transformation

2、 Target type

3、 Derivative is 0 Come to the conclusion

4、 Final model results

Four 、 Hidden problems —— It may not be a full rank matrix

5、 ... and 、 Solutions to hidden problems —— Regularization

1、L1 Regularization ——Lasso Return to

2、L2 Regularization —— Ridge return

6、 ... and 、 Changes and applications of linear regression

7、 ... and 、python Realization

1、 Multiple linear regression

2、 Ridge return

3、lasso Return to

8、 ... and 、 Linear model —— Regression problem classification problem

One 、 Problem description

We now have a data set on hand D： Each sample is composed of d Attributes to describe , namely  , among Is the sample x In the i Values on attributes . And each sample The final corresponding result value is .

Now comes a new sample , Want to know its result value

Two 、 Problem analysis

We need data sets D To find a linear model for forecast , Find out The right one w and b.

3、 ... and 、 solve the problem  —— look for w and b

We can use the least square method to solve this problem .

1、 Vector form transformation

First turn on the w and b Synthesize a vector form , The size is （d+1）* 1;

Then rewrite the data matrix X：, The size is （d+1）*  m.

Then mark y Also written as vector pattern ： 

2、 Target type

Make  

3、 Derivative is 0 Come to the conclusion

4、 Final model results

Four 、 Hidden problems —— It may not be a full rank matrix

It may not be a full rank matrix , There will be more than one Optimal solution , Which solution should be chosen as

for instance ： The number of samples is small , There are many characteristic attributes , It has even exceeded the number of samples , So at this time Not a full rank matrix , Can solve multiple Solution .

5、 ... and 、 Solutions to hidden problems —— Regularization

The role of regularization is to select models with less empirical risk and model complexity

1、L1 Regularization ——Lasso Return to

Add a term after the objective function  

be , The objective function becomes  

The first is the experience risk mentioned above , The second control is the complexity of the model .

among , Control the punishment ：;

This is also called Lasso Return to .

As shown in the figure below （ Suppose there are only two properties ）：L1 The regularized square error term isoline and the regularized isoline often intersect on the coordinate axis , This means discarding one of these attributes , It embodies the characteristics of feature selection , It is easier to get sparse solution （ Compared with the following L2 Regularization ）—— Namely obtained W There will be fewer non-zero values in the vector .

2、L2 Regularization —— Ridge return

Add a term after the objective function

be , The objective function becomes  

This is also known as ridge regression .

L2 Regularization uniform selection parameters , Let all coefficients of the fitting curve be the same , Although it failed to reduce the number of items , But the coefficients are balanced , This is in principle related to L1 Regularization is different .

6、 ... and 、 Changes and applications of linear regression

If the model of the problem is not linear regression , Then try to make the predicted value of the model approximate y Derivatives of .

such as —— Log linear regression  

A more general ：, This is called generalized linear model .

7、 ... and 、python Realization

1、 Multiple linear regression

from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
X_train,X_test,Y_train,Y_test=train_test_split(x,y,test_size=0.3,random_state=1)//x,y They are divided into attribute data and tag data 
model = LinearRegression()
model.fit(X_train, Y_train)
score = model.score(X_test, Y_test)
print(' Model test score ：'+str(score))
Y_pred = model.predict(X_test)
print(Y_pred)

2、 Ridge return

from sklearn.linear_model import Ridge
from sklearn.model_selection import train_test_split
X_train,X_test,Y_train,Y_test=train_test_split(x,y,test_size=0.3,random_state=1)//x,y They are divided into attribute data and tag data 
model = Ridge(alpha=1)
model.fit(X_train, Y_train)
score = model.score(X_test, Y_test)
print(' Model test score ：'+str(score))
Y_pred = model.predict(X_test)
print(Y_pred)

3、lasso Return to

from sklearn.linear_model import Lasso
from sklearn.model_selection import train_test_split
X_train,X_test,Y_train,Y_test=train_test_split(x,y,test_size=0.3,random_state=1)//x,y They are divided into attribute data and tag data 
model = Lasso(alpha=0.1)
model.fit(X_train, Y_train)
score = model.score(X_test, Y_test)
print(' Model test score ：'+str(score))
Y_pred = model.predict(X_test)
print(Y_pred)

8、 ... and 、 Linear model —— The return question Classification problem

All of the above are linear models used to solve regression problems , In fact, linear models can also be used to solve classification problems —— Logical regression （ Log probability regression ）.

See Logical regression （Logistic Regression）_tt Ya's blog -CSDN Blog _ Logical regression csdn

You are welcome to criticize and correct in the comment area , Thank you. ~