机器学习CV代码练习（九）之深度学习调参

不念不忘少年蓝@ 2022-08-30 05:26 307阅读 0赞

### 机器学习CV代码练习（九）之深度学习调参 ###

*  判断过/欠拟合的一般代码
 *  欠拟合
 *   *  欠拟合(一)消除方法
     *  欠拟合(二)消除方法
 *  好的拟合loss变化曲线
 *  过拟合
 *  补充

# 判断过/欠拟合的一般代码 #

model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])
    history = model.fit(X,Y,epochs=100,validation_split=0.33)
    print(history.history['loss'])
    print(history.history['acc'])
    print(history.history['val_loss'])
    print(history.history['val_acc'])

# 欠拟合 #

## 欠拟合(一)消除方法 ##

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM3NDg2NTAx_size_16_color_FFFFFF_t_70]方法

1.  增加训练epoch
2.  增大batch-size
3.  调整激活函数(使用relu)
4.  调整优化算法  
    使用Adam  
    增大learning rate

from keras.models import Sequential
    from keras.layers import Dense
    from keras.layers import LSTM
    from matplotlib import pyplot
    from numpy import array
    
    # return training data
    def get_train():
    	seq = [[0.0, 0.1], [0.1, 0.2], [0.2, 0.3], [0.3, 0.4], [0.4, 0.5]]
    	seq = array(seq)
    	X, y = seq[:, 0], seq[:, 1]
    	X = X.reshape((len(X), 1, 1))
    	return X, y
    
    # return validation data
    def get_val():
    	seq = [[0.5, 0.6], [0.6, 0.7], [0.7, 0.8], [0.8, 0.9], [0.9, 1.0]]
    	seq = array(seq)
    	X, y = seq[:, 0], seq[:, 1]
    	X = X.reshape((len(X), 1, 1))
    	return X, y
    
    # define model
    model = Sequential()
    model.add(LSTM(10, input_shape=(1,1)))
    model.add(Dense(1, activation='linear'))
    # compile model
    model.compile(loss='mse', optimizer='adam')
    # fit model
    X,y = get_train()
    valX, valY = get_val()
    history = model.fit(X, y, epochs=100, validation_data=(valX, valY), shuffle=False)
    # plot train and validation loss
    pyplot.plot(history.history['loss'])
    pyplot.plot(history.history['val_loss'])
    pyplot.title('model train vs validation loss')
    pyplot.ylabel('loss')
    pyplot.xlabel('epoch')
    pyplot.legend(['train', 'validation'], loc='upper right')
    pyplot.show()

## 欠拟合(二)消除方法 ##

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM3NDg2NTAx_size_16_color_FFFFFF_t_70 1]方法

1.  增加网络复杂度  
    增加层数  
    增加卷积层输出的通道数  
    增加全连接层的节点数
2.  检测训练数据集和测试数据是否有相对应的特征  
    增加训练数据的种类, 使得训练数据覆盖所有测试数据的特性  
    使用数据增强

from keras.models import Sequential
    from keras.layers import Dense
    from keras.layers import LSTM
    from matplotlib import pyplot
    from numpy import array
    
    # return training data
    def get_train():
    	seq = [[0.0, 0.1], [0.1, 0.2], [0.2, 0.3], [0.3, 0.4], [0.4, 0.5]]
    	seq = array(seq)
    	X, y = seq[:, 0], seq[:, 1]
    	X = X.reshape((5, 1, 1))
    	return X, y
    
    # return validation data
    def get_val():
    	seq = [[0.5, 0.6], [0.6, 0.7], [0.7, 0.8], [0.8, 0.9], [0.9, 1.0]]
    	seq = array(seq)
    	X, y = seq[:, 0], seq[:, 1]
    	X = X.reshape((len(X), 1, 1))
    	return X, y
    
    # define model
    model = Sequential()
    model.add(LSTM(1, input_shape=(1,1)))
    model.add(Dense(1, activation='linear'))
    # compile model
    model.compile(loss='mae', optimizer='sgd')
    # fit model
    X,y = get_train()
    valX, valY = get_val()
    history = model.fit(X, y, epochs=300, validation_data=(valX, valY), shuffle=False)
    # plot train and validation loss
    pyplot.plot(history.history['loss'])
    pyplot.plot(history.history['val_loss'])
    pyplot.title('model train vs validation loss')
    pyplot.ylabel('loss')
    pyplot.xlabel('epoch')
    pyplot.legend(['train', 'validation'], loc='upper right')
    pyplot.show()

# 好的拟合loss变化曲线 #

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM3NDg2NTAx_size_16_color_FFFFFF_t_70 2]

from keras.models import Sequential
    from keras.layers import Dense
    from keras.layers import LSTM
    from matplotlib import pyplot
    from numpy import array
    
    # return training data
    def get_train():
    	seq = [[0.0, 0.1], [0.1, 0.2], [0.2, 0.3], [0.3, 0.4], [0.4, 0.5]]
    	seq = array(seq)
    	X, y = seq[:, 0], seq[:, 1]
    	X = X.reshape((5, 1, 1))
    	return X, y
    
    # return validation data
    def get_val():
    	seq = [[0.5, 0.6], [0.6, 0.7], [0.7, 0.8], [0.8, 0.9], [0.9, 1.0]]
    	seq = array(seq)
    	X, y = seq[:, 0], seq[:, 1]
    	X = X.reshape((len(X), 1, 1))
    	return X, y
    
    # define model
    model = Sequential()
    model.add(LSTM(10, input_shape=(1,1)))
    model.add(Dense(1, activation='linear'))
    # compile model
    model.compile(loss='mse', optimizer='adam')
    # fit model
    X,y = get_train()
    valX, valY = get_val()
    history = model.fit(X, y, epochs=800, validation_data=(valX, valY), shuffle=False)
    # plot train and validation loss
    pyplot.plot(history.history['loss'])
    pyplot.plot(history.history['val_loss'])
    pyplot.title('model train vs validation loss')
    pyplot.ylabel('loss')
    pyplot.xlabel('epoch')
    pyplot.legend(['train', 'validation'], loc='upper right')
    pyplot.show()

# 过拟合 #

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM3NDg2NTAx_size_16_color_FFFFFF_t_70 3]方法

1.  检测训练数据集和测试数据是否有相对应的特征  
    增加训练数据的种类, 使得训练数据覆盖所有测试数据的特性  
    使用数据增强
2.  减小网络复杂度
3.  增加正则化  
    L1 Regularization  
    L2 Regularization
4.  增加Drop层
5.  添加Batch Normalizatin

from keras.models import Sequential
    from keras.layers import Dense
    from keras.layers import LSTM
    from matplotlib import pyplot
    from numpy import array
    
    # return training data
    def get_train():
    	seq = [[0.0, 0.1], [0.1, 0.2], [0.2, 0.3], [0.3, 0.4], [0.4, 0.5]]
    	seq = array(seq)
    	X, y = seq[:, 0], seq[:, 1]
    	X = X.reshape((5, 1, 1))
    	return X, y
    
    # return validation data
    def get_val():
    	seq = [[0.5, 0.6], [0.6, 0.7], [0.7, 0.8], [0.8, 0.9], [0.9, 1.0]]
    	seq = array(seq)
    	X, y = seq[:, 0], seq[:, 1]
    	X = X.reshape((len(X), 1, 1))
    	return X, y
    
    # define model
    model = Sequential()
    model.add(LSTM(10, input_shape=(1,1)))
    model.add(Dense(1, activation='linear'))
    # compile model
    model.compile(loss='mse', optimizer='adam')
    # fit model
    X,y = get_train()
    valX, valY = get_val()
    history = model.fit(X, y, epochs=1200, validation_data=(valX, valY), shuffle=False)
    # plot train and validation loss
    pyplot.plot(history.history['loss'][500:])
    pyplot.plot(history.history['val_loss'][500:])
    pyplot.title('model train vs validation loss')
    pyplot.ylabel('loss')
    pyplot.xlabel('epoch')
    pyplot.legend(['train', 'validation'], loc='upper right')
    pyplot.show()

# 补充 #

1.  文本合成: [https://noon99jaki.github.io/publication/2017-text-gen.pdf][https_noon99jaki.github.io_publication_2017-text-gen.pdf]
2.  过拟合/欠拟合: [https://machinelearningmastery.com/diagnose-overfitting-underfitting-lstm-models/][https_machinelearningmastery.com_diagnose-overfitting-underfitting-lstm-models]
3.  Early stopping: [https://machinelearningmastery.com/how-to-stop-training-deep-neural-networks-at-the-right-time-using-early-stopping/][https_machinelearningmastery.com_how-to-stop-training-deep-neural-networks-at-the-right-time-using-early-stopping]

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM3NDg2NTAx_size_16_color_FFFFFF_t_70]: /images/20220829/34a54bccae094cf8833844fdc935dcdf.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM3NDg2NTAx_size_16_color_FFFFFF_t_70 1]: /images/20220829/7e2eaba43e734503a24a13356d067106.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM3NDg2NTAx_size_16_color_FFFFFF_t_70 2]: /images/20220829/250765b1007d48c7a814b1ef5522b419.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM3NDg2NTAx_size_16_color_FFFFFF_t_70 3]: /images/20220829/be1f1595a15a4bc69ff6e06f00c654e2.png
[https_noon99jaki.github.io_publication_2017-text-gen.pdf]: https://noon99jaki.github.io/publication/2017-text-gen.pdf
[https_machinelearningmastery.com_diagnose-overfitting-underfitting-lstm-models]: https://machinelearningmastery.com/diagnose-overfitting-underfitting-lstm-models/
[https_machinelearningmastery.com_how-to-stop-training-deep-neural-networks-at-the-right-time-using-early-stopping]: https://machinelearningmastery.com/how-to-stop-training-deep-neural-networks-at-the-right-time-using-early-stopping/