50 行 Python 代码抓取 divnil 动漫妹子图！

雨点打透心脏的1/2处 2021-09-19 03:08 384阅读 0赞

目标网站 **https://divnil.com**

*  首先看看这网站是怎样加载数据的;  
    打开网站后发现底部有下一页的按钮，ok，爬这个网站就很简单了;
 *   
    
    哦对了，在分享这篇文字前，我先说一下，我这里有一份python学习资料，直接加我的python直播学习群：975188894就能免费领取，长期真实有效。
    
    ![20190427154013295.png][]
 *  我们目标是获取每张图片的高清的源地址，并且下载图片到桌面;  
    先随便打开一张图片看看详细;  
    emmm，只有一张图
    
    ![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDgxMTQxNw_size_16_color_FFFFFF_t_70][]

看起来还挺清晰的，单击新窗口打开图片

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDgxMTQxNw_size_16_color_FFFFFF_t_70 1][]

然后下载图片，说实话，这图片很小，我很担心不是高清原图(管他的);

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDgxMTQxNw_size_16_color_FFFFFF_t_70 2][]

*PS:一定要禁用广告拦截插件，不然加载不出图，我就在这被坑T\_T;*

![20190427154052149.png][]

**接着分析我们从何入手**

1、**先去主页面获取每个图片的详细页面的链接**

这链接还是比较好获取的，直接 F12 审核元素，或者右键查看代码，手机上chrome和firefox在url前面加上 "view-source"

比如： **`view-source:https://www.baidu.com/`**

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDgxMTQxNw_size_16_color_FFFFFF_t_70 3][]

2、**从详细页面获取图片大图地址**  
随便打开一个图片详细页面如图：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDgxMTQxNw_size_16_color_FFFFFF_t_70 4][]

接着按 F12 审核元素，我们需要定位该图片的链接，首先单击左上角的这玩意儿，像一个鼠标的图标：

![20190427154108732.png][]

接着只需要单击网页上的图片就能定位到代码了：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDgxMTQxNw_size_16_color_FFFFFF_t_70 5][]

3、**用大图地址下载该图片**

这个很简单，看代码

*  先安装 Requests 和 BeautifulSoup 库

pip install requests bs4

*  导入库

import requestsfrom bs4 import BeautifulSoupimport sys

*  请求获取网页源代码

url = "https://divnil.com/wallpaper/iphone8/%E3%82%A2%E3%83%8B%E3%83%A1%E3%81%AE%E5%A3%81%E7%B4%99\_2.html"  
headers = \{  
"User-Agent": "Mozilla/5.0 (X11; Linux x86\_64; rv:58.0) Gecko/20100101 Firefox/58.0",  
\}  
resp = requests.get(url, headers=headers)  
if resp.status\_code != requests.codes.OK:  
print("Request Error, Code: %d"% resp.status\_code)  
sys.exit()

*  然后解析出所有图片的详细地址

soup = BeautifulSoup(resp.text, "html.parser")  
contents = soup.findAll("div", id="contents")\[0\]  
wallpapers = contents.findAll("a", rel="wallpaper")  
links = \[\]  
for wallpaper in wallpapers:  
 links.append(wallpaper\['href'\])

*  接着在详细网页里获取那个看似高清的图片的不确定是否为真实图片链接并下载(/滑稽)

import os  
  
head = "https://divnil.com/wallpaper/iphone8/"  
if os.path.exists("./Divnil") != True:  
 os.mkdir("./Divnil")  
  
for url in links:  
 url = head + url  
 resp = requests.get(url, headers=headers)  
 if  resp.status\_code != requests.codes.OK:  
   print("URL: %s REQUESTS ERROR. CODE: %d" % (url, resp.status\_code))  
   continue  
 soup = BeautifulSoup(resp.text, "html.parser")  
 img =  soup.find("div", id="contents").contents.find("img", id="main\_content")  
 img\_url = head + img\['"original'\].replace("../", "")  
 img\_name = img\['alt'\]  
 print("start download %s ..." % img\_url)  
  
 resp = requests.get(img\_url, headers=headers)  
 if resp.status\_code != requests.codes.OK:  
   print("IMAGE %s DOWNLOAD FAILED." % img\_name)  
  
 with open("./Divnil/" + img\_name + ".jpg", "wb") as f:  
   f.write(resp.content)

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDgxMTQxNw_size_16_color_FFFFFF_t_70 6][]

*  主要代码

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDgxMTQxNw_size_16_color_FFFFFF_t_70 7][]

*  完成，贴上所有代码

import requests  
from bs4 import BeautifulSoup  
import sys  
import os  
  
  
class Divnil:  
  
   def \_\_init\_\_(self):  
       self.url = "https://divnil.com/wallpaper/iphone8/%E3%82%A2%E3%83%8B%E3%83%A1%E3%81%AE%E5%A3%81%E7%B4%99.html"  
       self.head = "https://divnil.com/wallpaper/iphone8/"  
       self.headers = \{  
           "User-Agent": "Mozilla/5.0 (X11; Linux x86\_64; rv:58.0) Gecko/20100101 Firefox/58.0",  
       \}  
     
  
   def getImageInfoUrl(self):  
  
       resp = requests.get(self.url, headers=self.headers)  
       if resp.status\_code != requests.codes.OK:  
           print("Request Error, Code: %d"% resp.status\_code)  
           sys.exit()  
  
       soup = BeautifulSoup(resp.text, "html.parser")  
         
       contents = soup.find("div", id="contents")  
       wallpapers = contents.findAll("a", rel="wallpaper")  
         
       self.links = \[\]  
       for wallpaper in wallpapers:  
           self.links.append(wallpaper\['href'\])  
  
     
   def downloadImage(self):  
  
       if os.path.exists("./Divnil") != True:  
           os.mkdir("./Divnil")  
  
       for url in self.links:  
             
           url = self.head + url  
             
           resp = requests.get(url, headers=self.headers)  
           if  resp.status\_code != requests.codes.OK:  
               print("URL: %s REQUESTS ERROR. CODE: %d" % (url, resp.status\_code))  
               continue  
             
           soup = BeautifulSoup(resp.text, "html.parser")  
             
           img = soup.find("div", id="contents").find("img", id="main\_content")  
           img\_url = self.head + img\['original'\].replace("../", "")  
           img\_name = img\['alt'\]  
             
           print("start download %s ..." % img\_url)  
  
           resp = requests.get(img\_url, headers=self.headers)  
           if resp.status\_code != requests.codes.OK:  
               print("IMAGE %s DOWNLOAD FAILED." % img\_name)  
               continue  
  
           if '/' in img\_name:  
               img\_name = img\_name.split('/')\[1\]  
  
           with open("./Divnil/" + img\_name + ".jpg", "wb") as f:  
               f.write(resp.content)  
  
  
   def main(self):  
       self.getImageInfoUrl()  
       self.downloadImage()  
  
  
if \_\_name\_\_ == "\_\_main\_\_":  
   divnil = Divnil()  
   divnil.main()

如果在学习中遇到问题有不明白的地方，推荐加小编微信：**Pythonsusu**

有视频教程 ，直播课程 ，等学习资料，期待你的加入

**-END-**

[20190427154013295.png]: /images/20210812/51c2cf141d7c4e0d9481bfd19cb92216.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDgxMTQxNw_size_16_color_FFFFFF_t_70]: /images/20210812/c368b0b4382e47de8ebeef9f925ddd14.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDgxMTQxNw_size_16_color_FFFFFF_t_70 1]: /images/20210812/a59e34f017854c0497f72e386b781e2d.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDgxMTQxNw_size_16_color_FFFFFF_t_70 2]: /images/20210812/aef9c1ce854844689976a4f0433df254.png
[20190427154052149.png]: /images/20210812/fa5928b5e93e46fe848d169eb16ba766.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDgxMTQxNw_size_16_color_FFFFFF_t_70 3]: /images/20210812/da9691728eda4c13980d61d94496c737.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDgxMTQxNw_size_16_color_FFFFFF_t_70 4]: /images/20210812/ca7c0d169f7e46f4925ce9834d7ec725.png
[20190427154108732.png]: /images/20210812/d24e62ffd8de4943b6b35311ef80209b.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDgxMTQxNw_size_16_color_FFFFFF_t_70 5]: /images/20210812/0e5ef615b65249ea926d98463e10df33.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDgxMTQxNw_size_16_color_FFFFFF_t_70 6]: /images/20210812/2bd6563593a14f10bf198cdc4c06d5cc.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDgxMTQxNw_size_16_color_FFFFFF_t_70 7]: /images/20210812/d19db438fdfe4ebb91864f4f36676e1c.png