Pytorch-模型参数：named_parameters()、parameters()、state_dict()区别

骑猪看日落 2023-09-29 13:09 21阅读 0赞

torch中存在3个功能极其类似的方法，它们分别是model.parameters()、model.named\_parameters()、model.state\_dict()，下面就具体来说说这三个函数的差异：

## 一、model.parameters()和model.named\_parameters()差别 ##

*  named\_parameters()返回的list中，每个元组（与list相似，只是数据不可修改）打包了2个内容，分别是`layer-name`和`layer-param`（网络层的名字和参数的迭代器）；
 *  parameters()只有后者layer-param（参数的迭代器）

### 1、model.named\_parameters()里的网络层名字 ###

import torchvision.models as models
    
    model = models.resnet18()
    for param_tuple in model.named_parameters():
        name, param = param_tuple
        print("name = ", name)
        print("-" * 100)

打印结果：

name =  conv1.weight
    ----------------------------------------------------------------------------------------------------
    name =  bn1.weight
    ----------------------------------------------------------------------------------------------------
    name =  bn1.bias
    ----------------------------------------------------------------------------------------------------
    name =  layer1.0.conv1.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer1.0.bn1.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer1.0.bn1.bias
    ----------------------------------------------------------------------------------------------------
    name =  layer1.0.conv2.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer1.0.bn2.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer1.0.bn2.bias
    ----------------------------------------------------------------------------------------------------
    name =  layer1.1.conv1.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer1.1.bn1.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer1.1.bn1.bias
    ----------------------------------------------------------------------------------------------------
    name =  layer1.1.conv2.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer1.1.bn2.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer1.1.bn2.bias
    ----------------------------------------------------------------------------------------------------
    name =  layer2.0.conv1.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer2.0.bn1.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer2.0.bn1.bias
    ----------------------------------------------------------------------------------------------------
    name =  layer2.0.conv2.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer2.0.bn2.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer2.0.bn2.bias
    ----------------------------------------------------------------------------------------------------
    name =  layer2.0.downsample.0.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer2.0.downsample.1.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer2.0.downsample.1.bias
    ----------------------------------------------------------------------------------------------------
    name =  layer2.1.conv1.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer2.1.bn1.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer2.1.bn1.bias
    ----------------------------------------------------------------------------------------------------
    name =  layer2.1.conv2.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer2.1.bn2.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer2.1.bn2.bias
    ----------------------------------------------------------------------------------------------------
    name =  layer3.0.conv1.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer3.0.bn1.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer3.0.bn1.bias
    ----------------------------------------------------------------------------------------------------
    name =  layer3.0.conv2.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer3.0.bn2.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer3.0.bn2.bias
    ----------------------------------------------------------------------------------------------------
    name =  layer3.0.downsample.0.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer3.0.downsample.1.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer3.0.downsample.1.bias
    ----------------------------------------------------------------------------------------------------
    name =  layer3.1.conv1.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer3.1.bn1.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer3.1.bn1.bias
    ----------------------------------------------------------------------------------------------------
    name =  layer3.1.conv2.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer3.1.bn2.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer3.1.bn2.bias
    ----------------------------------------------------------------------------------------------------
    name =  layer4.0.conv1.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer4.0.bn1.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer4.0.bn1.bias
    ----------------------------------------------------------------------------------------------------
    name =  layer4.0.conv2.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer4.0.bn2.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer4.0.bn2.bias
    ----------------------------------------------------------------------------------------------------
    name =  layer4.0.downsample.0.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer4.0.downsample.1.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer4.0.downsample.1.bias
    ----------------------------------------------------------------------------------------------------
    name =  layer4.1.conv1.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer4.1.bn1.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer4.1.bn1.bias
    ----------------------------------------------------------------------------------------------------
    name =  layer4.1.conv2.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer4.1.bn2.weight
    ----------------------------------------------------------------------------------------------------
    name =  layer4.1.bn2.bias
    ----------------------------------------------------------------------------------------------------
    name =  fc.weight
    ----------------------------------------------------------------------------------------------------
    name =  fc.bias
    ----------------------------------------------------------------------------------------------------
    
    Process finished with exit code 0

### 2、model.named\_parameters()里的网络层名字、参数 ###

打印结果：

C:\Program_Files_AI\Anaconda3531\python.exe C:/Users/Admin/OneDrive/WorkSpace_AI/0-基于知识库的智能问答系统-华控智加/01-意图识别/test.py
    name =  conv1.weight
    ----------------------------------------------------------------------------------------------------
    param_tuple =  ('conv1.weight', Parameter containing:
    tensor([[[[-1.4115e-05,  2.9187e-02,  2.9325e-03,  ..., -4.2247e-02,
                1.7490e-02, -4.5253e-02],
              [-2.4594e-02, -3.0836e-02,  3.8604e-02,  ...,  3.5473e-02,
               -4.7046e-03, -2.9440e-02],
              [ 2.4811e-02,  1.2679e-02,  1.0070e-02,  ..., -8.3476e-03,
                1.7960e-02, -1.7406e-02],
              ...,
              [-1.3021e-02,  2.9023e-02, -6.1800e-02,  ..., -5.2802e-02,
               -4.7817e-02, -2.2377e-02],
              [-3.8513e-03, -1.0603e-02, -3.9712e-02,  ...,  5.1941e-03,
                8.2868e-03, -8.3469e-03],
              [ 3.8993e-03,  3.2017e-02, -3.6292e-02,  ..., -2.0210e-02,
               -4.0358e-02,  1.7709e-02]],
    
             [[-1.0894e-03,  1.5720e-02,  7.0129e-03,  ..., -1.2024e-02,
                1.8644e-02,  1.7892e-02],
              [-2.3866e-02,  9.1136e-03,  3.5243e-02,  ..., -1.6756e-02,
                1.4441e-03,  4.7943e-02],
              [-2.0514e-03,  4.3022e-02,  2.6358e-02,  ..., -2.3662e-02,
               -7.8241e-04,  1.0167e-02],
            ...
    
             [[-4.6689e-02, -1.1407e-03,  1.8674e-02,  ...,  1.2649e-03,
               -2.9532e-02,  6.4535e-04],
              [ 1.4171e-03, -1.9274e-02, -8.6811e-03,  ...,  2.4428e-02,
                6.9516e-03,  4.3715e-02],
              [ 1.9982e-02,  1.3124e-02,  9.1508e-03,  ...,  2.5405e-02,
               -1.3132e-02,  4.0835e-02],
              ...,
              [-3.4174e-03,  1.8623e-02, -1.4386e-02,  ...,  1.0627e-03,
               -5.1297e-04,  2.2055e-02],
              [ 2.7333e-02,  2.4858e-02, -5.4305e-02,  ..., -1.2139e-02,
                1.7735e-03, -3.4184e-03],
              [ 1.1412e-03,  1.5794e-02, -2.0699e-02,  ..., -1.7846e-02,
                3.7425e-02, -1.6059e-02]]],
    
    
            ...,
    
    
            [[[-2.7389e-02, -3.8327e-02, -2.9043e-02,  ..., -7.6396e-03,
               -1.6519e-02,  3.9659e-02],
              [ 2.8740e-03, -1.0621e-02, -9.2430e-03,  ...,  2.2581e-02,
                5.1526e-03, -2.0006e-02],
              [ 1.3575e-02,  1.5290e-02, -1.7260e-02,  ...,  6.3830e-03,
               -1.9759e-02,  1.5501e-02],
              ...,
              [ 1.6091e-02,  2.4038e-02,  2.4507e-02,  ..., -4.5613e-02,
               -3.6233e-02,  2.1632e-02],
              [-1.1573e-02, -3.6514e-02,  4.1576e-02,  ...,  1.8090e-02,
               -2.3350e-02, -8.7074e-03],
              [-1.5837e-02, -3.1353e-02,  1.8726e-02,  ...,  9.3698e-03,
                3.0781e-02,  1.0976e-02]],
    
             [[-2.7063e-02,  8.7158e-03,  2.7193e-03,  ..., -1.6670e-03,
               -4.3033e-03,  7.2011e-04],
              [ 2.7870e-03,  1.4264e-02, -5.0581e-02,  ...,  2.5463e-02,
                7.6864e-03, -4.9655e-02],
              [ 2.6030e-03,  2.5918e-02,  2.9615e-02,  ...,  3.0676e-02,
               -2.7723e-02, -7.3628e-03],
              ...,
              [ 2.5969e-02, -1.4247e-02,  1.2516e-02,  ...,  5.9602e-03,
               -3.2843e-02,  3.5822e-02],
              [ 1.2845e-02, -2.0035e-02,  9.9398e-04,  ..., -3.1800e-02,
                5.7984e-03,  2.8756e-02],
              [ 2.3458e-02,  3.8193e-02, -2.3754e-03,  ..., -1.3867e-02,
                8.0831e-03, -3.2438e-02]],
    ...
    
             [[-9.9291e-03, -5.6023e-03, -1.7064e-02,  ...,  8.8544e-03,
               -5.8145e-03,  2.3248e-02],
              [ 1.2148e-02, -1.0730e-02, -1.2682e-02,  ...,  9.4389e-03,
                1.2149e-02,  3.8613e-03],
              [ 3.5913e-02, -5.2048e-04, -8.7133e-02,  ..., -2.0969e-03,
               -5.4117e-03,  5.4637e-05],
              ...,
              [ 4.0351e-03, -1.3189e-02,  3.1229e-02,  ...,  3.2340e-02,
               -2.8351e-02,  1.0634e-02],
              [ 2.6041e-02, -3.0633e-04, -1.2732e-02,  ...,  2.9417e-02,
               -7.3859e-03,  1.7207e-02],
              [ 6.9960e-04,  3.8486e-03,  1.0397e-02,  ...,  1.4535e-03,
               -3.6449e-02,  3.4848e-02]]]], requires_grad=True))
    ********************************************************************************************************************************************************************************************************
    name =  bn1.weight
    ----------------------------------------------------------------------------------------------------
    param_tuple =  ('bn1.weight', Parameter containing:
    tensor([1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
            1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
            1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
            1., 1., 1., 1., 1., 1., 1., 1., 1., 1.], requires_grad=True))
    ********************************************************************************************************************************************************************************************************
    name =  bn1.bias
    ----------------------------------------------------------------------------------------------------
    param_tuple =  ('bn1.bias', Parameter containing:
    tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
           requires_grad=True))
    ********************************************************************************************************************************************************************************************************
    
    ********************************************************************************************************************************************************************************************************
    name =  layer1.0.conv2.weight
    ----------------------------------------------------------------------------------------------------
    param_tuple =  ('layer1.0.conv2.weight', Parameter containing:
    tensor([[[[-8.6159e-02,  1.8507e-04,  5.4006e-03],
              [-6.3063e-03,  3.9225e-03, -6.3141e-02],
              [-7.0145e-02, -3.9266e-02,  1.9724e-03]],
    
             [[ 4.6454e-02,  2.1519e-02,  5.3696e-02],
              [ 1.1086e-02,  1.6269e-01, -7.0579e-02],
              [-1.1220e-01, -4.9811e-02, -7.5515e-02]],
    
             [[ 5.5275e-02, -8.2407e-02, -8.9807e-02],
              [ 5.8418e-02,  4.4029e-02,  3.0584e-03],
              [ 5.2371e-02, -1.5983e-02,  5.1494e-02]],
    
             ...,
    
             [[ 7.3441e-02,  4.5401e-02, -1.9175e-02],
              [-6.2500e-02, -8.5905e-03, -7.4856e-02],
              [-1.6170e-02,  3.7529e-02, -5.1231e-02]],
    
             [[ 7.7501e-04, -5.7506e-02,  1.8422e-01],
              [ 2.4594e-02,  1.7378e-02,  4.0000e-02],
              [-8.6796e-02, -6.0548e-02,  2.6795e-02]],
    
             [[ 5.3264e-02, -8.6190e-02,  4.2443e-02],
              [-6.8029e-03, -1.6581e-02,  7.8568e-02],
              [ 3.2037e-02, -7.3002e-02,  4.9353e-02]]]], requires_grad=True))
    ...，
    ********************************************************************************************************************************************************************************************************
    name =  layer4.1.bn2.bias
    ----------------------------------------------------------------------------------------------------
    param_tuple =  ('layer4.1.bn2.bias', Parameter containing:
    tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
    ...,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
            0., 0., 0., 0., 0., 0., 0., 0.], requires_grad=True))
    ********************************************************************************************************************************************************************************************************
    name =  fc.weight
    ----------------------------------------------------------------------------------------------------
    param_tuple =  ('fc.weight', Parameter containing:
    tensor([[-0.0125,  0.0437, -0.0014,  ..., -0.0230,  0.0280,  0.0249],
            [-0.0105,  0.0242,  0.0291,  ...,  0.0153,  0.0366, -0.0236],
            [-0.0315,  0.0306, -0.0216,  ...,  0.0387,  0.0403,  0.0056],
            ...,
            [-0.0068, -0.0222, -0.0027,  ..., -0.0243,  0.0260,  0.0065],
            [ 0.0213,  0.0167, -0.0379,  ..., -0.0140,  0.0037, -0.0372],
            [ 0.0180,  0.0101, -0.0341,  ..., -0.0295, -0.0146,  0.0416]],
           requires_grad=True))
    ********************************************************************************************************************************************************************************************************
    name =  fc.bias
    ----------------------------------------------------------------------------------------------------
    param_tuple =  ('fc.bias', Parameter containing:
    tensor([ 3.5711e-02,  3.2682e-02,  7.5932e-03, -3.1623e-02, -9.6316e-03,
            -2.4051e-02, -1.0393e-02,  2.3210e-02, -3.6044e-02,  2.3099e-02,
            -3.5723e-02, -3.9482e-02,  4.8526e-03, -3.2688e-02,  3.7720e-03,
            -2.2014e-02, -4.0935e-02,  4.0533e-02, -4.1172e-02,  3.9513e-02,
            -3.0332e-02,  3.2777e-02,  1.3342e-02,  2.3394e-02,  8.2328e-03,
             1.3757e-02, -1.7578e-02, -2.7165e-02,  3.8495e-03, -3.2116e-02,
             7.9903e-03,  9.9640e-04, -8.3106e-03,  2.5033e-02, -3.0446e-02,
            -1.8282e-02, -3.8420e-03, -8.6129e-03, -4.2712e-03,  1.7169e-02,
           ...,
            -5.3570e-05, -3.7353e-02, -9.8633e-03, -9.1069e-03,  3.2688e-02,
             2.2457e-02,  7.6379e-03, -3.6287e-02, -1.0444e-02,  2.1669e-02,
             2.5270e-02, -4.3881e-02,  2.1960e-02,  2.6293e-02, -3.5049e-02,
            -2.0074e-02, -9.7686e-03, -2.3766e-02, -5.0265e-03, -2.1095e-02,
             2.0981e-02, -3.5132e-02,  8.6407e-03,  1.8453e-02,  2.4282e-02,
             3.8392e-02, -1.7470e-02,  3.6958e-02, -3.7590e-02, -4.1951e-02,
            -1.8246e-02,  9.0818e-03,  3.8774e-02,  7.3408e-03,  1.7728e-02,
             3.5547e-02, -7.2857e-03, -2.7015e-02, -8.6983e-03, -2.3785e-02],
           requires_grad=True))
    ********************************************************************************************************************************************************************************************************
    
    Process finished with exit code 0

## 二、model.named\_parameters()和model.state\_dict()差别 ##

它们的差异主要体现在3方面：

*  返回值类型不同
 *  存储的模型参数的种类不同
 *  返回的值的require\_grad属性不同

<table> 
 <thead> 
  <tr> 
   <th>named_parameters()</th> 
   <th>state_dict()</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>将layer_name : layer_param的键值信息打包成一个元祖然后再存到list当中</td> 
   <td>将layer_name : layer_param的键值信息存储为dict形式</td> 
  </tr> 
  <tr> 
   <td>只保存可学习、可被更新的参数，model.buffer()中的参数不包含在model.named_parameters()中</td> 
   <td>存储的是该model中包含的所有layer中的所有参数</td> 
  </tr> 
  <tr> 
   <td>require_grad属性都是True</td> 
   <td>存储的模型参数tensor的require_grad属性都是False</td> 
  </tr> 
 </tbody> 
</table>

为何model.parameters()迭代出来的所有参数的require\_grad属性都是True，因为它们在被创建时，默认的require\_grad就是True。这也符合逻辑，即，使用nn.Parameter()创建的变量是模型参数，本就是要参与学习和更新的；而不参与模型学习和更新的呢？我们再看一个例子：

--------------------

参考资料：  
[pytorch中state\_dict()和named\_parameters()的差别，以及model.buffer/model.parameter][pytorch_state_dict_named_parameters_model.buffer_model.parameter]  
[【pytorch】named\_parameters()、parameters()、state\_dict()==＞给出网络的名字和参数的迭代器][pytorch_named_parameters_parameters_state_dict]  
[【pytorch】named\_parameters()和parameters()][pytorch_named_parameters_parameters]

[pytorch_state_dict_named_parameters_model.buffer_model.parameter]: https://blog.csdn.net/weixin_41712499/article/details/110198423
[pytorch_named_parameters_parameters_state_dict]: https://blog.csdn.net/weixin_43135178/article/details/117688370
[pytorch_named_parameters_parameters]: https://blog.csdn.net/qq_36530992/article/details/102729585