Python-爬虫-处理数据-lxml库

向右看齐 2024-04-17 05:39 134阅读 0赞

## lxml介绍： ##

对xpath和html/xml的解析

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9odWFuZ2hhaXRhby5ibG9nLmNzZG4ubmV0_size_16_color_FFFFFF_t_70][]

## 基本使用： ##

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9odWFuZ2hhaXRhby5ibG9nLmNzZG4ubmV0_size_16_color_FFFFFF_t_70 1][]

代码：

lxml的使用

#--coding:utf-8--
    
    from lxml import etree
    
    text = '''
    <div>
        <ul>
             <li class="item-0"><a href="link1.html">first item</a></li>
             <li class="item-1"><a href="link2.html">second item</a></li>
             <li class="item-inactive"><a href="link3.html">third item</a></li>
             <li class="item-1"><a href="link4.html">fourth item</a></li>
             <li class="item-0"><a href="link5.html">fifth item</a>
         </ul>
     </div>
    '''
    # 将字符串解析为html文档
    html = etree.HTML(text)
    print(html)
    # 按字符串序列化html
    result = etree.tostring(html).decode('utf-8')
    print(result)

lxml读取本地文件使用

#--coding:utf-8--
    
    from lxml import etree
    
    
    #读取
    html = etree.parse('hello.html')
    
    result = etree.tostring(html).decode('utf-8')
    print(result)

hello.html文件

![20190823095948173.png][]

<html><body>
    <div>
        <ul>
             <li class="item-0"><a href="link1.html">first item</a></li>
             <li class="item-1"><a href="link2.html">second item</a></li>
             <li class="item-inactive"><a href="link3.html"><span class="bold">third item</span></a></li>
             <li class="item-1"><a href="link4.html">fourth item</a></li>
             <li class="item-0"><a href="link5.html">fifth item</a></li>
    </ul>
     </div>
    </body></html>

xpath在python中的练习代码：

#--coding:utf-8--
    
    from lxml import etree
    html = etree.parse('hello.html')
    # # 获取所有li标签：
    result = html.xpath('//li')
    print(result)
    for i in result:
        print(etree.tostring(i).decode('utf-8'))
    
    # 获取所有li元素下的所有class属性的值：
    result = html.xpath('//li/@class')
    print(result)
    
    # 获取li标签下href为www.baidu.com的a标签：
    result = html.xpath('//li/a[@href="www.baidu.com"]')
    print(result)
    
    # 获取li标签下所有span标签：
    result = html.xpath('//li//span')
    print(result)
    
    # 获取li标签下的a标签里的所有class：
    result = html.xpath('//li/a//@class')
    print(result)
    
    # 获取最后一个li的a的href属性对应的值：
    result = html.xpath('//li[last()]/a/@href')
    print(result)
    
    # 获取倒数第二个li元素的内容：
    result = html.xpath('//li[last()-1]/a')
    print(result)
    print(result[0].text)
    
    # 获取倒数第二个li元素的内容的第二种方式：
    result = html.xpath('//li[last()-1]/a/text()')
    print(result)

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9odWFuZ2hhaXRhby5ibG9nLmNzZG4ubmV0_size_16_color_FFFFFF_t_70]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/11/c118d76ed21242418f1a33a87ca7f635.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9odWFuZ2hhaXRhby5ibG9nLmNzZG4ubmV0_size_16_color_FFFFFF_t_70 1]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/11/b1cad1eb16334783819a7e75d4c435eb.png
[20190823095948173.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/11/7e337d6f41be42ddbd8956937bdc3092.png