Python爬取全民小视频网站视频，可爬取全站视频

桃扇骨 2022-12-17 12:53 384阅读 0赞

## 前言 ##

本文的文字及图片来源于网络,仅供学习、交流使用,不具有任何商业用途,如有问题请及时联系我们以作处理。

PS：如有需要Python学习资料的小伙伴可以加点击下方链接自行获取

[python免费学习资料以及群交流解答点击即可加入][python]

## 开发工具 ##

*  python 3.6.5
 *  pycharm

import requests

## 确定目标网页 ##

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA_size_16_color_FFFFFF_t_70_pic_center]

*  选择影视类目

![在这里插入图片描述][20201023165248986.png_pic_center]

*  右键选择检查，开发开发者工具

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA_size_16_color_FFFFFF_t_70_pic_center 1]

*  选择 Network，网页往下滑，可以看到数据加载  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA_size_16_color_FFFFFF_t_70_pic_center 2]
 *  视频地址
 *  视频名字
 *  视频ID

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA_size_16_color_FFFFFF_t_70_pic_center 3]  
![在这里插入图片描述][20201023165716686.png_pic_center]  
接口数据链接是通过pn的变化，实现翻页效果

如果想要爬取其他分类视频，把参数中tab\_name 更改即可

## 完整代码 ##

import pprint
    import requests
    
    url = 'https://quanmin.baidu.com/wise/growth/api/home/tabmorelist'
    for page in range(1, 100):
        params = { 
            'rn': '12',
            'pn': '{}'.format(page),
            'timestamp': '1603442730716',
            'session_id': '1603442719704',
            'tab_name': 'movie',
            '_format': 'json',
        }
        headers = { 
            'user-agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36'
        }
        response = requests.get(url=url, params=params, headers=headers)
        html_data = response.json()
        lis = html_data['data']['list']['video_list']
        for li in lis:
            play_url = li['play_url']
            title = li['title']
            filename = 'D:\\python\\demo\\全民小视频\\视频\\' + title + '.mp4'
            response_2 = requests.get(url=play_url, headers=headers)
            with open(filename, mode='wb') as f:
                f.write(response_2.content)
                print(title, play_url)

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA_size_16_color_FFFFFF_t_70_pic_center 4]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA_size_16_color_FFFFFF_t_70_pic_center 5]

[python]: https://jq.qq.com/?_wv=1027&k=kQrKxXXR
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA_size_16_color_FFFFFF_t_70_pic_center]: /images/20221122/bef0ec31a7044c15add3edbe6e889903.png
[20201023165248986.png_pic_center]: /images/20221122/ad9e1eb2e9f0404793fd8ad11ffca302.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA_size_16_color_FFFFFF_t_70_pic_center 1]: /images/20221122/7ef6ba03d0ae4ebda2ce9dad2befa217.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA_size_16_color_FFFFFF_t_70_pic_center 2]: /images/20221122/b9061f6fcaf74ac7b58f1bce1fddcfe5.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA_size_16_color_FFFFFF_t_70_pic_center 3]: /images/20221122/3b58eb957f3041bc8dc32b692eab0120.png
[20201023165716686.png_pic_center]: /images/20221122/dabcbcbc29a24a408858ef52b0a2ff1a.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA_size_16_color_FFFFFF_t_70_pic_center 4]: /images/20221122/a62908543ae24ce084cffa2c664e5bb1.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaTM0Nzc5NTc5MA_size_16_color_FFFFFF_t_70_pic_center 5]: /images/20221122/896b44ad2e3748b9a7ac8b80c2988e65.png