数据分析——算法——K-means聚类（天池：汽车产品聚类分析）

灰太狼 2023-01-22 09:50 69阅读 0赞

# K-means聚类 #

**目录**

K-means聚类

1 简介

2 Python实战

--------------------

## 1 简介 ##

**原理**：通过计算不同样本间的距离来判断他们的相近关系的，相近的就会放到同一个类别中去。

**适用数据**：数值数据

**优点**：思想简单，容易实现，可解释度比较强

**缺点**：对噪音和异常点比较的敏感。k-means是在做凸优化，因此处理不了非凸的分布,对于条形或不规则形状的数据，效果较差。如果两个类别距离比较近，k-means的效果也不会太好。初始中心点的选择以及k值的选择对结果影响较大，可能每次聚类结果都不一样。结果可能只是局部最优而不是全局最优

## 2 Python实战 ##

**数据集:**天池学习赛数据集——汽车产品聚类分析（有兴趣的同学可以直接参赛学习一下）

**任务**：对该汽车数据进行聚类分析，并找到vokswagen汽车的相应竞品。

**数据介绍**：car\_price.csv，数据包括了205款车的26个字段

<table> 
 <thead> 
 <tr> 
 <th>1</th> 
 <th>Car_ID</th> 
 <th>Unique id of each observation (Interger)</th> 
 </tr> 
 </thead> 
 <tbody> 
 <tr> 
 <td>2</td> 
 <td>Symboling</td> 
 <td>Its assigned insurance risk rating, A value of +3 indicates that the auto is risky, -3 that it is probably pretty safe.(Categorical)</td> 
 </tr> 
 <tr> 
 <td>3</td> 
 <td>carCompany</td> 
 <td>Name of car company (Categorical)</td> 
 </tr> 
 <tr> 
 <td>4</td> 
 <td>fueltype</td> 
 <td>Car fuel type i.e gas or diesel (Categorical)</td> 
 </tr> 
 <tr> 
 <td>5</td> 
 <td>aspiration</td> 
 <td>Aspiration used in a car (Categorical)</td> 
 </tr> 
 <tr> 
 <td>6</td> 
 <td>doornumber</td> 
 <td>Number of doors in a car (Categorical)</td> 
 </tr> 
 <tr> 
 <td>7</td> 
 <td>carbody</td> 
 <td>body of car (Categorical)</td> 
 </tr> 
 <tr> 
 <td>8</td> 
 <td>drivewheel</td> 
 <td>type of drive wheel (Categorical)</td> 
 </tr> 
 <tr> 
 <td>9</td> 
 <td>enginelocation</td> 
 <td>Location of car engine (Categorical)</td> 
 </tr> 
 <tr> 
 <td>10</td> 
 <td>wheelbase</td> 
 <td>Weelbase of car (Numeric)</td> 
 </tr> 
 <tr> 
 <td>11</td> 
 <td>carlength</td> 
 <td>Length of car (Numeric)</td> 
 </tr> 
 <tr> 
 <td>12</td> 
 <td>carwidth</td> 
 <td>Width of car (Numeric)</td> 
 </tr> 
 <tr> 
 <td>13</td> 
 <td>carheight</td> 
 <td>height of car (Numeric)</td> 
 </tr> 
 <tr> 
 <td>14</td> 
 <td>curbweight</td> 
 <td>The weight of a car without occupants or baggage. (Numeric)</td> 
 </tr> 
 <tr> 
 <td>15</td> 
 <td>enginetype</td> 
 <td>Type of engine. (Categorical)</td> 
 </tr> 
 <tr> 
 <td>16</td> 
 <td>cylindernumber</td> 
 <td>cylinder placed in the car (Categorical)</td> 
 </tr> 
 <tr> 
 <td>17</td> 
 <td>enginesize</td> 
 <td>Size of car (Numeric)</td> 
 </tr> 
 <tr> 
 <td>18</td> 
 <td>fuelsystem</td> 
 <td>Fuel system of car (Categorical)</td> 
 </tr> 
 <tr> 
 <td>19</td> 
 <td>boreratio</td> 
 <td>Boreratio of car (Numeric)</td> 
 </tr> 
 <tr> 
 <td>20</td> 
 <td>stroke</td> 
 <td>Stroke or volume inside the engine (Numeric)</td> 
 </tr> 
 <tr> 
 <td>21</td> 
 <td>compressionratio</td> 
 <td>compression ratio of car (Numeric)</td> 
 </tr> 
 <tr> 
 <td>22</td> 
 <td>horsepower</td> 
 <td>Horsepower (Numeric)</td> 
 </tr> 
 <tr> 
 <td>23</td> 
 <td>peakrpm</td> 
 <td>car peak rpm (Numeric)</td> 
 </tr> 
 <tr> 
 <td>24</td> 
 <td>citympg</td> 
 <td>Mileage in city (Numeric)</td> 
 </tr> 
 <tr> 
 <td>25</td> 
 <td>highwaympg</td> 
 <td>Mileage on highway (Numeric)</td> 
 </tr> 
 <tr> 
 <td>26</td> 
 <td>price(Dependent variable)</td> 
 <td>Price of car (Numeric)</td> 
 </tr> 
 </tbody> 
</table>

**数据集以及源码下载**：链接：https://pan.baidu.com/s/1yGpMJIu\_vS31GjM8mc\_8UA 提取码：5n6x

**代码以及结果展示：**

import pandas as pd
    
    # 数据加载
    data = pd.read_csv('./car_price.csv')
    data

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQyMzIwMDQ4_size_16_color_FFFFFF_t_70][]

# 使用KMeans进行聚类,导入库
    from sklearn.cluster import KMeans
    #预处理
    from sklearn import preprocessing
    from sklearn.preprocessing import LabelEncoder
    import pandas as pd
    #矩阵运算
    import numpy as np

# 建立训练集
    train_x = data[["car_ID","symboling","CarName", "fueltype", "aspiration", "doornumber",
                        "carbody", "drivewheel", "enginelocation", "wheelbase", "carlength","carwidth",
                        "carheight", "curbweight", "enginetype", "cylindernumber", "enginesize","fuelsystem",
                        "boreratio", "stroke", "compressionratio", "horsepower", "peakrpm","citympg",
                        "highwaympg", "price"]]
    train_x

#将非数值字段转化为数值
    le = LabelEncoder()
    columns = ['CarName','fueltype','aspiration','doornumber','carbody','drivewheel','enginelocation','enginetype','cylindernumber','fuelsystem']
    for column in columns:
        #LabelEncoder().transform将标签转换为归一化的编码。fit_transform安装标签编码器并返回编码的标签。
        train_x[column] = le.fit_transform(train_x[column])
    train_x

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQyMzIwMDQ4_size_16_color_FFFFFF_t_70 1][]

# 规范化到 [0,1] 空间
    min_max_scaler=preprocessing.MinMaxScaler()
    #MinMaxScaler()将每个要素缩放到给定范围，拟合数据，然后对其进行转换。
    train_x=min_max_scaler.fit_transform(train_x)
    pd.DataFrame(train_x).to_csv('temp.csv', index=False)

#选择聚类组数
    import matplotlib.pyplot as plt
    sse = []
    for k in range(1, 11):
    	kmeans = KMeans(n_clusters=k)
    	kmeans.fit(train_x)
    	# 计算inertia簇内误差平方和
    	sse.append(kmeans.inertia_)
    x = range(1, 11)
    #将图像嵌入在结果中
    %matplotlib inline
    plt.xlabel('K')
    plt.ylabel('SSE')
    plt.plot(x, sse, 'o-')

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQyMzIwMDQ4_size_16_color_FFFFFF_t_70 2][]

# 使用KMeans聚类,分成4类
    kmeans = KMeans(n_clusters=5)
    kmeans.fit(train_x)  # 也可以直接fit+predict
    #predict计算聚类中心并预测每个样本的聚类索引。
    predict_y = kmeans.predict(train_x)
    predict_y

![20210519111102550.png][]

# 合并聚类结果，插入到原数据中,axis： 需要合并链接的轴，0是行，1是列 
    result = pd.concat((data,pd.DataFrame(predict_y)),axis=1)
    # 将结果列重命名为'聚类结果'
    result.rename({0:u'聚类结果'},axis=1,inplace=True)
    result

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQyMzIwMDQ4_size_16_color_FFFFFF_t_70 3][]

# 找出vokswagen汽车的聚类结果
    label = result[result.CarName.str.contains('vokswagen')]['聚类结果']
    label
    #以下此段仅为练习，可忽略。多个关键词搜索，join() 方法用于将序列中的元素（必须是str）以指定的字符连接生成一个新的字符串。
    List = ['vokswagen','volkswagen']
    mask = result.CarName.str.contains('|'.join(List))
    selected_data = result[mask]
    selected_data

![20210519111315333.png][]

# Vokswagen 轿车 竞品按价格排序
    #lambda 是为了减少单行函数的定义而存在的,lambda作为一个表达式，定义了一个匿名函数，上例的代码x为入口参数，x['聚类结果']==3and 'sedan' in x['carbody']为函数体，
    #将函数应用到由各列或行形成的一维数组上。DataFrame的apply方法可以实现此功能。默认情况下会以列为单位,axis = 1以行为单位
    #[[]]选择多列时用双括号
    result[result.apply(lambda x : x['聚类结果'] ==3 and 'sedan' in x['carbody'],axis = 1)][['CarName',"wheelbase", "price",'horsepower','carbody','fueltype','聚类结果']].sort_values('price',ascending = False)

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQyMzIwMDQ4_size_16_color_FFFFFF_t_70 4][]

# Vokswagen wagon 竞品按价格排序
    result[result.apply(lambda x : x['聚类结果'] ==3 and 'wagon' in x['carbody'],axis =1)][['CarName',"wheelbase", "price",'horsepower','carbody','fueltype','聚类结果']].sort_values('price',ascending = False)

![20210519111445837.png][]

#显示竞品车总体聚类结果
    benchmark = result[result['聚类结果']==3].CarName
    print("竞品车如下所示")
    print(benchmark)

![20210519111529534.png][]

**注明：以上代码仅为学习，代码参考比赛论坛的某帖子，实名感谢，另外还有很多不完善的地方，大家可以提出，一起探讨。**

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQyMzIwMDQ4_size_16_color_FFFFFF_t_70]: /images/20221020/457b3f469a2c454d900a90c43e01f46f.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQyMzIwMDQ4_size_16_color_FFFFFF_t_70 1]: /images/20221020/be70ba805ec244fca2f0b3f02cbd60a0.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQyMzIwMDQ4_size_16_color_FFFFFF_t_70 2]: /images/20221020/c162e9f5b1ed43b2a76cbacb4c2dfb7d.png
[20210519111102550.png]: /images/20221020/89629ec39ef148d9a02c7599f76f8f6f.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQyMzIwMDQ4_size_16_color_FFFFFF_t_70 3]: /images/20221020/802be88184194956acd0aac44bd3428a.png
[20210519111315333.png]: /images/20221020/8daff0cd7d844f5e9378ffe27f626adb.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQyMzIwMDQ4_size_16_color_FFFFFF_t_70 4]: /images/20221020/dba0bae90ea14d12bde216a58912ef03.png
[20210519111445837.png]: /images/20221020/554f525ae57045eca5efd96d5dfe91b4.png
[20210519111529534.png]: /images/20221020/2587604902174bd4b6d6e33313ac43ad.png