其他
爬取斗鱼4300W热度的小姐姐是一种什么样的体验？
https://www.douyu.com/gapi/rknc/directory/yzRec/1https://www.douyu.com/gapi/rknc/directory/yzRec/2https://www.douyu.com/gapi/rknc/directory/yzRec/3https://www.douyu.com/gapi/rknc/directory/yzRec/4找到规律一会我们就可以照此爬取当前在线的6页美女主播。
5. 我们使用JSON插件打开页面观察数据们需要的数据如下，它们都是位于data下的rl列表中。
6. 编码实现
url = f'https://www.douyu.com/gapi/rknc/directory/yzRec/1'
headers = {
'cookie':'acf_did=9e27372460247e4aba9f762900071601; dy_did=9e27372460247e4aba9f762900071601; acf_auth=cf37zSEUJmMW8ZnLUicTZng%2FHg7%2F%2FXwXaGUx5XRFfkiTZxiQqXqEdt6ciewk%2B0ZudB0VDPtl339VKg2am5RoxV9aaw2wwYv0aFEHGY7Y5EJrwlQyRsHIFM0; dy_auth=9643Kl5KYaOKVu7Z%2BkbLbWkLDRCWUOayvVZCx21pJexn5QfumzonjR8vpwPRsKiQouZQ9EsYftUsXRewnWxOWZX3hnYZNLlXCHpwMHXdT%2F7nzryDyvyHPIw; wan_auth37wan=177c41f40328We%2BbP8pHuZa6rMZxrXPLGk5u4ywuBp4L8oMVpWZAquYVI371KXp6oaHp%2BnY9eLOalWDflPTeuUyJJnmqNqC2D31bX732GsP15o1uHU0; acf_uid=139162360; acf_username=139162360; acf_nickname=%E5%B8%95%E5%8D%9A%E9%9B%B7%E5%85%8B666; acf_own_room=0; acf_groupid=1; acf_phonestatus=1; acf_ct=0; acf_ltkid=10823007; acf_biz=1; acf_stk=4bceac5e967a4731; loginrefer=pt_en5984hidln9; Hm_lvt_e99aee90ec1b2106afe7ec3b199020a7=1619091301,1619518542,1620738434,1620781653; acf_avatar=//apic.douyucdn.cn/upload/avanew/face/201705/07/20/3d9f7186451336f7b97992c2fba06bf8_; Hm_lpvt_e99aee90ec1b2106afe7ec3b199020a7=1620781665',
'referer':'https://www.douyu.com/g_yz',
'user-agent':'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4484.7 Safari/537.36'
}
#获取浏览器返回的JSON结果
resp = requests.get(url,headers = headers).json()
#获取每一页的120个颜值小姐姐信息
girls = resp['data']['rl']
'''
[{'rid': 3477773, 'rn': '唱歌主播 无人听懂曲中意', 'uid': 160088008, 'nn': '鹿鹿酱in', 'cid1': 2, 'cid2': 311, 'cid3': 1749, 'iv': 0, 'av': 'avatar_v3/202104/31e0b77f8fa84f779f5e965c9ae3acf2', 'ol': 178516, 'url': '/3477773', 'c2url': '/directory/game/XX', 'c2name': '颜值', 'icdata': {'548': {'url': '', 'w': 0, 'h': 0}}, 'dot': 2103, 'subrt': 0, 'topid': 0, 'oaid': 0, 'bid': 0, 'gldid': 0, 'rs1': 'https://rpic.douyucdn.cn/live-cover/roomCover/2020/12/22/fbc91f6c4baf895bc7c91e6fa299803d_small.png/dy2', 'rs16': 'https://rpic.douyucdn.cn/live-cover/roomCover/2020/12/22/fbc91f6c4baf895bc7c91e6fa299803d_big.png/dy1', 'utag': [], 'rpos': 0, 'rgrpt': 1, 'rkic': '', 'rt': 2103, 'ot': 0, 'clis': 2, 'chanid': 0, 'icv1': [[{'id': 548, 'url': 'https://sta-op.douyucdn.cn/dy-listicon/4d1d7ef013162713c5f1f601697bcb2f.png', 'score': 1001, 'w': 0, 'h': 0}], [], [], []], 'ioa': 0, 'od': '', 'isShowUp': 0, 'authInfo': None}]
'''
7.接下来一步就是循环获取美女主播的照片、房间号、房间名称和主播分类。
for girl in girls:
pic_links = girl['rs1'] # 所有照片链接
room_title = girl['rn'] # 房间标题
room_name = girl['nn'] # 房间名称
category = girl['c2name'] # 主播分类
print(pic_links,room_title,room_name,category)
'''
https://rpic.douyucdn.cn/live-cover/appCovers/2021/03/07/7750351_20210307090509_big.jpg/dy2 这是个靓女的直播间~ Thea柒清 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2020/10/11/638575_20201011212022_big.jpg/dy2 小姨子： 幸运之神 丽亚celiahunne 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2021/03/24/7909303_20210324172547_big.jpg/dy2 有颜有才，尽在斗歌台 斗歌台官方直播间
https://rpic.douyucdn.cn/live-cover/appCovers/2020/07/27/8922441_20200727190717_big.jpg/dy2 5月15日月庆聚会 睡懵的渣皇 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2021/03/04/9384662_20210304004743_big.jpg/dy2 想要成为那个被你保护的人。 新来的恩宠 颜值
https://rpic.douyucdn.cn/live-cover/roomCover/2021/04/21/7fb646bba059b6a9f78b094d13fcde68_small.png/dy2 身怀绝技 百里挑一 星势力官方直播间
https://rpic.douyucdn.cn/live-cover/appCovers/2021/04/15/9737095_20210415195744_big.jpg/dy2 08年妹妹第一天直播求关注9737095 一只酸奶牛new 颜值
https://rpic.douyucdn.cn/live-cover/roomCover/2020/07/22/f865fc38018b758f92ca65413d01f611_small.png/dy2 我学会孤独 娇儿的36D吖 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2020/07/23/4125556_20200723164324_big.jpg/dy2 我回来啦！！681888 Skr丶大伟伟 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2021/02/22/1010383_20210222125342_big.jpg/dy2 双倍亲密，三倍上限，来了不想走呦~ 王羽杉abc 颜值
https://rpic.douyucdn.cn/live-cover/roomCover/2021/04/13/42e209e1cb9f15b27b3f40e5b3224fb4_small.png/dy2 我和我的猫都很想你 小初困了吗 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2021/05/03/8086045_20210503111144_big.jpg/dy2 时间不语，但会回答一切 豆乐乐咩 颜值
https://rpic.douyucdn.cn/live-cover/roomCover/2021/05/07/5b840cb58099af5f0912c716d30ded13_small.png/dy2 一条小虞er~想被哥哥带上岸 虞Oxygen 颜值
https://rpic.douyucdn.cn/live-cover/roomCover/2021/05/07/946df9dfa50623ac9fcb10d943d5678b_small.png/dy2 520周年庆欢迎大家来玩喔❤️Day8 林芊又 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2021/05/11/9720412_20210511195844_big.jpg/dy2 新人主播请多关照！！9720412 开心市民小邵 颜值
https://rpic.douyucdn.cn/live-cover/roomCover/2021/04/16/4c932e400f6e74c4fe62219013f43e7a_small.png/dy2 医生讲，你不要熬夜了 奉奉儿 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2019/04/08/5271808_20190408082749_big.jpg/dy2 消费查询，恭喜空空荣升超帝 酥小A 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2020/05/12/2243218_20200512082834_big.jpg/dy2 哦 那我走呗 八分饱ww 颜值
https://rpic.douyucdn.cn/live-cover/roomCover/2021/04/29/cfd9f29d923b5e4159ef3e77402fde76_small.png/dy2 每一天都期待能和你见面 9775190 尤田T 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2021/02/06/9064900_20210206191857_big.jpg/dy2 今天是你喜欢的美女子吗 玉玉子sway 颜值
https://rpic.douyucdn.cn/live-cover/roomCover/2021/04/09/c90f603c9305f85dc5efbd3d24134ea6_small.png/dy2 四川新主播略略略略略略 翼遥遥丶 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2020/11/26/4566947_20201126195330_big.jpg/dy2 可可爱爱 没有脑袋 4566947 关晓羽 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2020/06/20/4652236_20200620175813_big.jpg/dy2 身高170长发及腰的武汉小姐姐~~ 小火郭丶 颜值
https://rpic.douyucdn.cn/live-cover/roomCover/2019/04/01/6df644b55178d672257996028418bf3b_small.jpg/dy2 2021 一切顺利~ 林cute 颜值
https://rpic.douyucdn.cn/live-cover/roomCover/2021/04/06/2796d0dfbbaff1b4a7f537829658b6a4_small.png/dy2 小猪开播了 金善智 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2021/04/27/6532591_20210427105108_big.jpg/dy2 随意来去 开心就好 林以安阿 颜值
.........
'''
8.我们将获取到的这些信息先存入到Excel中。
ws = op.Workbook()
wb = ws.create_sheet(index=0)
wb.cell(row=1, column=1, value='照片链接')
wb.cell(row=1, column=2, value='房间标题')
wb.cell(row=1, column=3, value='房间名称')
wb.cell(row=1, column=4, value='主播分类')
ws.save('斗鱼颜值.xlsx')
9.然后我们再定义一个专门保存图片的函数down_dy()。
def down_dy(save_pic):
count = 0
for num,(pic_img,pic_name) in enumerate(save_pic):
r = requests.get(pic_img)
pic = r.content
try:
with open('./pictures/{}.jpg'.format(pic_name), 'wb') as fin:
print(f'正在爬取第{count}张图片')
fin.write(pic)
print('{}.jpg----下载成功'.format(pic_name))
except:
print('下载失败！')
10.最后一步，因为我们要获取的小姐姐照片总共有6页，所以我们定义page变量来获取全部页数的小姐姐照片。
for page in range(1,6+1):
print(f'--------正在爬取第{page}页--------')
url = f'https://www.douyu.com/gapi/rknc/directory/yzRec/{page}'
最终效果如下：
。。。。。。。。
想要获取源码学习的同学关注微信公众号回复'斗鱼'即可文章部分内容参考：https://mp.weixin.qq.com/s/ZZ332SPW4TgRh_8zw7WzAQ