mirror of
https://github.com/microsoft/ai-edu.git
synced 2026-08-28 17:41:25 +08:00
Translating ai-edu/A-基础教程/A2-神经网络基本原理/en-us/Step7 - DNN/14.2-回归任务真实案例.md (#651)
* Create 14.1-Regression And Functional Testing * Update and rename 14.1-Regression And Functional Testing to 14.1-Regression and Functional Testing * Rename 14.1-Regression and Functional Testing to 14.1-Regression and Functional Testing.md * Create 14.2-Regression example in real life.md * Update 14.2-Regression example in real life.md * Update 14.2-Regression example in real life.md
This commit is contained in:
Mark Chen
@@ -0,0 +1,150 @@
|
||||
<!--Copyright © Microsoft Corporation. All rights reserved.
|
||||
适用于[License](https://github.com/Microsoft/ai-edu/blob/master/LICENSE.md)版权许可-->
|
||||
|
||||
## 14.2 Regression Task - Housing Price Prediction
|
||||
|
||||
### 14.2.1 Data
|
||||
|
||||
The data set comes from：https://www.kaggle.com/harlfoxem/housesalesprediction
|
||||
|
||||
This data set shares information about house sales from May 2014 to May 2015 in King County, suitable for training regression models.
|
||||
|
||||
#### Data fields interpretation
|
||||
|
||||
-id: unique id
|
||||
-date: date of sale
|
||||
-price: sale price (label value)
|
||||
-bedrooms: the number of bedrooms
|
||||
-bathrooms: the number of bathrooms
|
||||
-sqft_living: living area
|
||||
-sqft_lot: parking lot area
|
||||
-floors: number of floors
|
||||
-waterfront: swimming pool
|
||||
-view: number of times the house has been viewed
|
||||
-condition: housing condition
|
||||
-grade: grade
|
||||
-sqft_above: ground area
|
||||
-sqft_basement: basement area
|
||||
-yr_built: construction year
|
||||
-yr_renovated: renovation year
|
||||
-zipcode: postal code
|
||||
-lat: latitude
|
||||
-long: longitude
|
||||
-sqft_living15: living area after renovation in 2015
|
||||
-sqft_lot15: parking lot area after renovation in 2015
|
||||
|
||||
Some considerations:
|
||||
|
||||
-The unique id is useful in the database but is not a feature during training, so it should be removed
|
||||
-The date of saleis useless because they are all within one year of data
|
||||
-The value of sqft_living15 should replace sqft_living if it is not 0
|
||||
-The value of sqft_lot15 should be replaced if it is not 0
|
||||
-The geographic location corresponding to the postal code is too broad, which can only cause noise and should be removed
|
||||
-The year of repair could replace the year of construction if it is a non-zero value (author's belief)
|
||||
-The number of house viewings for a record does not mean that the price of the house is high. It is because the geographical location, price, configuration, etc. meet the requirements of specific groups of people. The author believes that it is not a necessary feature value
|
||||
|
||||
In the end, only 13 fields are left.
|
||||
|
||||
#### Data Processing
|
||||
|
||||
The original data is in one data set, so we need to divide it into a training and test set by ourselves with a ratio of about 4:1. This data set is in a `csv` file format. For convenience, we converted it into two compressed forms of `numpy` using `npz`:
|
||||
|
||||
- `house_Train.npz`，Training data set
|
||||
- `house_Test.npz`，Test data set
|
||||
|
||||
#### Download Data
|
||||
|
||||
The code is similar to the code in the first example above. Normalization is required, as the house data attributes are complicated. The house price is also at least 6 digits, so normalization is also required.
|
||||
|
||||
Note that the data of the training and test sets need to be merged together for normalization, and then used separately. Why merge first? Assume that the area of the house in the training set sample ranges from 150 to 220, while the area of the house in the test set may be 160 to 230. The two ranges are not consistent. If they are normalized separately, 150 becomes 0 and 160 becomes 0, so the prediction will produce errors.
|
||||
|
||||
Finally, you need to use `GenerateValidaionSet(k=10)` to separate a 1:9 validation set in the training data set.
|
||||
|
||||
### 14.2.2 Build a model
|
||||
|
||||
Before we know a problem's actual complexity, we might as well further complicate the design the model. As shown in the figure below, this model contains four sets of fully connected Relu layer combinations, and finally a single output for fitting.
|
||||
|
||||
<img src="https://aiedugithub4a2.blob.core.windows.net/a2-images/Images/14/non_linear_regression.png" />
|
||||
|
||||
Figure 14-5 The abstract model for completing the housing price prediction task
|
||||
|
||||
```Python
|
||||
def model():
|
||||
dr = LoadData()
|
||||
|
||||
num_input = dr.num_feature
|
||||
num_hidden1 = 32
|
||||
num_hidden2 = 16
|
||||
num_hidden3 = 8
|
||||
num_hidden4 = 4
|
||||
num_output = 1
|
||||
|
||||
max_epoch = 1000
|
||||
batch_size = 16
|
||||
learning_rate = 0.1
|
||||
|
||||
params = HyperParameters_4_0(
|
||||
learning_rate, max_epoch, batch_size,
|
||||
net_type=NetType.Fitting,
|
||||
init_method=InitialMethod.Xavier,
|
||||
stopper=Stopper(StopCondition.StopDiff, 1e-7))
|
||||
|
||||
net = NeuralNet_4_0(params, "HouseSingle")
|
||||
|
||||
fc1 = FcLayer_1_0(num_input, num_hidden1, params)
|
||||
net.add_layer(fc1, "fc1")
|
||||
r1 = ActivationLayer(Relu())
|
||||
net.add_layer(r1, "r1")
|
||||
......
|
||||
fc5 = FcLayer_1_0(num_hidden4, num_output, params)
|
||||
net.add_layer(fc5, "fc5")
|
||||
|
||||
net.train(dr, checkpoint=10, need_test=True)
|
||||
|
||||
output = net.inference(dr.XTest)
|
||||
real_output = dr.DeNormalizeY(output)
|
||||
mse = np.sum((dr.YTestRaw - real_output)**2)/dr.YTest.shape[0]/10000
|
||||
print("mse=", mse)
|
||||
|
||||
net.ShowLossHistory()
|
||||
|
||||
ShowResult(net, dr)
|
||||
```
|
||||
|
||||
Hyperparameter description:
|
||||
|
||||
1. Learning rate=0.1
|
||||
2. Maximum `epoch=1000`
|
||||
3. Batch size=16
|
||||
4. Fit the network
|
||||
5. Initialization method: Xavier
|
||||
6. The stop condition is relative error `1e-7`
|
||||
|
||||
The `net.train()` function is a blocking function and returns only when the training is complete.
|
||||
|
||||
After training, the test set is used to test the accuracy of the model using the official evaluation method of Data Castle and the mean square error, which is divided by 10000. The smaller the number, the better. The general model gives a result of about 7-digits, and a slightly better 6-digit result.
|
||||
|
||||
### 14.2.3 Training result
|
||||
|
||||
<img src="https://aiedugithub4a2.blob.core.windows.net/a2-images/Images/14/house_loss.png" />
|
||||
|
||||
Figure 14-6 Changes in loss function value and accuracy during training
|
||||
|
||||
Since the label data is also normalized and converted to decimals between 0 and 1, the value of the mean square error is very small, and the 4th place after the decimal point needs to be observed. From Figure 14-6, we see that the loss function values quickly drop below 0.0002, and then it drops very slowly. Since the accuracy value is constantly rising, I believe that more iterations will bring higher accuracy.
|
||||
|
||||
Look at the following printout. The value obtained by the R2_Score method is 0.841. The official evaluation standard of Data Castle, the MSE value obtained is 2384411, which is still relatively large. This is indicating that the model accuracy should have room for improvement.
|
||||
|
||||
```
|
||||
......
|
||||
epoch=999, total_iteration=972999
|
||||
loss_train=0.000079, accuracy_train=0.740406
|
||||
loss_valid=0.000193, accuracy_valid=0.857289
|
||||
time used: 193.5549156665802
|
||||
testing...
|
||||
0.8412989144927305
|
||||
mse= 2384411.5840510926
|
||||
```
|
||||
|
||||
### Code location
|
||||
|
||||
ch14, Level2
|
||||
Reference in New Issue
Block a user