The spiders2 from coder-fly

整理一下自己做过的具有一定的反爬手段的爬虫，一时半会也记不清都有哪些了，总之先放这，以后慢慢更新

plugins

parse_ttf 借助百度ORC 自动化解析ttf文件中的字体

spiders2's People

Contributors

Stargazers

Watchers

spiders2's Issues

?

头条网页版貌似失效了？

试用了下头条网页版，生成signature，爬取指定用户的feed列表

输入：https://www.toutiao.com/c/user/token/MS4wLjABAAAAP09LrX61xFpIWrgGdBDqkp-5om9Lans_kuIZ_ipAGRE/?source=feed&log_from=eaf124025aa9e_1651888131770

生成的signature: _02B4Z6wo00101Bz3DRAAAIDCpOVyFIga6AQc3wmAAGVOb8

import requests

headers = {
    "user-agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36",
    'accept': 'application/json, text/plain, */*',
    'referer': 'https://www.toutiao.com/?wid=1641028484634',
}

uid = 'MS4wLjABAAAAP09LrX61xFpIWrgGdBDqkp-5om9Lans_kuIZ_ipAGRE'
sign = '_02B4Z6wo00101Bz3DRAAAIDCpOVyFIga6AQc3wmAAGVOb8'
url = f'https://www.toutiao.com/api/pc/list/user/feed?category=pc_profile_article&token={uid}&max_behot_time=0&aid=24&app_name=toutiao_web&_signature={sign}'

response = requests.get(url,headers=headers)
print(response.url)
print(response.json())

结果打印

https://www.toutiao.com/api/pc/list/user/feed?category=pc_profile_article&token=MS4wLjABAAAAP09LrX61xFpIWrgGdBDqkp-5om9Lans_kuIZ_ipAGRE&max_behot_time=0&aid=24&app_name=toutiao_web&_signature=_02B4Z6wo00101Bz3DRAAAIDCpOVyFIga6AQc3wmAAGVOb8

{'data': [], 'decision': '{"code":"10000","from":"","type":"verify","version":"","region":"cn","subtype":"3d","detail":"k87OAt17RgTAQ7Gl7Z4DLmcP8UURvubqqf0M*SsHcDHF29s0jJwVleRhMU2ukEC7*FBoO7nCJ0rrc17JWBp55kpbrqUmS3C*T04Y*MB5dIjaJlBYWFj5bCJUChb6rc2e7tvqsIRD69Sw8r0-ujhZYK*SBnf7hc9nDBmTSiYXxaSgXcwOvu*hGfFGt0y-9HAJ9rn6l9T2sGgxkFPgFNP1Dkf98JdH6qhfpAw7R*fmFN*msRrQPBXxu3loRxBxxhXrxU40QljhRfbC*hMr36vaqEmkU*6yo-rYpoDAKgXKmW5s9v*9krQBYilpzUt8L27LXuHaEkYgBzKtdKyIxveczg1MZSsb73AZgvX5PGs49st9hPmADukgFW2v1GzwZCVd2WIFJWOtDl9uEC69AGmDV2vU55km1jtdqALvXsLXpN3T3Q..","verify_event":"","fp":"verify_7675712d2130cc97b00d5c9ebc5664ca","server_sdk_env":"{\\"idc\\":\\"lf\\",\\"region\\":\\"CN\\",\\"server_type\\":\\"business\\"}"}', 'has_more': False, 'message': 'success'}

是否哪里需要改造呢？

Recommend Projects