mikubill / pixivpy-async Goto Github PK
View Code? Open in Web Editor NEWPure Python 3 Async Pixiv API
License: The Unlicense
pixivpy-async's People
Contributors
Stargazers
Watchers
Forkers
kyle2142 shengurun synodriver rexexe ldoubil lunez99 leon92101 mzdluo123 ethpch lxy1226 asd666q9 nimsdue lei-mu solomon-edwards blabra 5l1v3r1 quulpixivpy-async's Issues
Question: how to sort illustrations by popularity and filter r18 illusts
As the title says, I have a premium pixiv accounts so I can search illustrations sorted by popularity. But I didn't see there are any corresponding parameters in search_illusts method. Also I want to filter r18 illustrations, is there any way to achieve it?
请求添加请求时检查access_token有效性的功能
如题，有时请求时会产生以下错误
{'user_message': '', 'message': 'Error occurred at the OAuth process. Please check your Access Token to fix this. Error Message: invalid_grant', 'reason': '', 'user_message_details': {}}
能否在调用时就检查返回结果是否属于access_token无效并更新access_token后再次请求？
请问这里session关闭前执行asyncio.sleep(0)有什么特别的作用吗？
pixivpy-async/pixivpy_async/client.py
Line 72 in ef877db
一些教程提到asyncio.sleep(0)是交出控制权，那这里为什么要交出而不是直接close呢？
bapi.py 中download方法如果传入的path没有拓展名会报错
Rate Limit error
Hello, do you know how to avoid this error?
TOTAL_ILLUSTS json_result = {'error': {'user_message': '', 'message': 'Rate Limit', 'reason': '', 'user_message_details': {}}}
I'm looping through a list of artists (1 artist every 100 seconds) and getting their new artworks. After the loop has been running for a while, I get this error. I can share more code if needed.
download方法使用后缀自动匹配位于判断文件是否存在之后，此时可能存在重复下载的问题
如下代码，初始化 name 变量时仅使用 url 最后的字符串，
pixivpy-async/pixivpy_async/bapi.py
Lines 137 to 138 in 4e110ed
而在如下代码中，初始化
img_path 变量时使用了 name 变量，在之后的判断里，直接使用 img_path 变量来判断文件是否存在，pixivpy-async/pixivpy_async/bapi.py
Lines 142 to 144 in 4e110ed
其后又使用后缀自动匹配方法修改了
img_path 的后缀并且作为文件名写入图片文件，pixivpy-async/pixivpy_async/bapi.py
Lines 145 to 154 in 4e110ed
此时即存在标题所描述的问题：例如下载 webp 格式的文件，下载后得到了
image.webp 文件，若使用 url 初始化得到的 name 以及 img_path 后缀名并不是 webp （例如 jpg 格式），此时判断了 image.jpg 文件不存在，因而重复下载文件并重新改名为 image.webp 。将后缀自动匹配的代码提前到 142行与143行之间应该能修复此问题
i get errros when i use pixivpy-async running on wl2-kali and win11
win11:
login_web() get a wrong
wls2-kali
Traceback (most recent call last): File "/mnt/c/Users/Administrator/Desktop/test.py", line 5, in <module> from pixivpy_async import * File "/usr/local/lib/python3.9/dist-packages/pixivpy_async/__init__.py", line 6, in <module> from .aapi import AppPixivAPI File "/usr/local/lib/python3.9/dist-packages/pixivpy_async/aapi.py", line 2, in <module> from .bapi import BasePixivAPI File "/usr/local/lib/python3.9/dist-packages/pixivpy_async/bapi.py", line 9, in <module> from .net import Net File "/usr/local/lib/python3.9/dist-packages/pixivpy_async/net.py", line 1, in <module> import aiohttp File "/usr/lib/python3/dist-packages/aiohttp/__init__.py", line 6, in <module> from .client import ( File "/usr/lib/python3/dist-packages/aiohttp/client.py", line 35, in <module> from . import hdrs, http, payload File "/usr/lib/python3/dist-packages/aiohttp/http.py", line 7, in <module> from .http_parser import ( File "/usr/lib/python3/dist-packages/aiohttp/http_parser.py", line 15, in <module> from .helpers import NO_EXTENSIONS, BaseTimerContext File "/usr/lib/python3/dist-packages/aiohttp/helpers.py", line 667, in <module> class CeilTimeout(async_timeout.timeout): TypeError: function() argument 'code' must be code, not str
`AppPixivAPI(bypass=True)` 时是否可以仅使用 IPv4 DoH？
使用 AppPixivAPI(bypass=True) 在 DNS over HTTP(s) 解析 Pixiv IP 的时候尝试请求 IPv6 dns-query 接口，在 IPv6 支持不完全的服务器上会引起 OSError: [Errno 101] Network is unreachable，虽然不影响使用，但是报错信息较长，有点反感，是否可以将 IPv6 地址的 dns-query 接口与 IPv4 的区分开？
pixivpy-async/pixivpy_async/bypass_sni.py
Lines 14 to 24 in ef877db
下面是我参考仓库里的文件 /demo/demo_api_proxy.py 修改的代码以及一些报错信息。环境为 Python 3.8.10 & PixivPy-Async 1.2.14，运行在 Docker 容器内（在主机上运行也存在类似报错）。
代码
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import asyncio
# import uuid
# import sentry_sdk
from pixivpy_async import AppPixivAPI
# _USERNAME = "userbay"
# _PASSWORD = "UserPay"
_TOKEN = "uXooTT7xz9v4mflnZqJUO7po9W5ciouhKrIDnI2Dv3c"
# sentry_sdk.init(
# dsn="https://[email protected]/111",
# traces_sample_rate=1.0
# )
async def _main(aapi):
"""
Tips: How to create a proxy for Pixiv api? Just throw the following code into a CloudFlare worker.
```js
addEventListener("fetch", e => {
let t = e.request, d = new URL(t.url);
d.hostname = "app-api.pixiv.net"; // Or public-api.secure.pixiv.net, oauth.secure.pixiv.net
e.respondWith(fetch(d, {body: t.body, headers: t.headers, method: t.method}))
});
```
"""
# aapi.set_api_proxy(app_hosts="", auth_hosts="", pub_hosts="")
# await aapi.login(_USERNAME, _PASSWORD)
await aapi.login(refresh_token=_TOKEN)
# result = await aapi.illust_ranking('day')
result = await aapi.user_bookmarks_illust(
12345678, restrict="private", req_auth=True
)
for item in result.illusts:
print(item.image_urls['large'])
# await aapi.download(item.image_urls['large'])
def main():
loop = asyncio.get_event_loop()
loop.run_until_complete(_main(AppPixivAPI(bypass=True)))
if __name__ == '__main__':
main()命令行原始报错
Task exception was never retrieved
future: <Task finished name='Task-5' coro=<ByPassResolver._resolve() done, defined at /home/pwuser/.local/lib/python3.8/site-packages/pixivpy_async/bypass_sni.py:88> exception=ClientConnectorError(ConnectionKey(host='2606:4700:4700::1001', port=443, is_ssl=True, ssl=None, proxy=None, proxy_auth=None, proxy_headers_hash=None), OSError(99, 'Cannot assign requested address'))>
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/aiohttp/connector.py", line 980, in _wrap_create_connection
return await self._loop.create_connection(*args, **kwargs) # type: ignore[return-value] # noqa
File "/usr/lib/python3.8/asyncio/base_events.py", line 1025, in create_connection
raise exceptions[0]
File "/usr/lib/python3.8/asyncio/base_events.py", line 1010, in create_connection
sock = await self._connect_sock(
File "/usr/lib/python3.8/asyncio/base_events.py", line 924, in _connect_sock
await self.sock_connect(sock, address)
File "/usr/lib/python3.8/asyncio/selector_events.py", line 496, in sock_connect
return await fut
File "/usr/lib/python3.8/asyncio/selector_events.py", line 501, in _sock_connect
sock.connect(address)
OSError: [Errno 99] Cannot assign requested address
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/pwuser/.local/lib/python3.8/site-packages/pixivpy_async/bypass_sni.py", line 98, in _resolve
async with session.get(endpoint, params=params, headers={"accept": "application/dns-json"}, timeout=ClientTimeout(total=timeout)) as resp:
File "/usr/local/lib/python3.8/dist-packages/aiohttp/client.py", line 1141, in __aenter__
self._resp = await self._coro
File "/usr/local/lib/python3.8/dist-packages/aiohttp/client.py", line 536, in _request
conn = await self._connector.connect(
File "/usr/local/lib/python3.8/dist-packages/aiohttp/connector.py", line 540, in connect
proto = await self._create_connection(req, traces, timeout)
File "/usr/local/lib/python3.8/dist-packages/aiohttp/connector.py", line 901, in _create_connection
_, proto = await self._create_direct_connection(req, traces, timeout)
File "/usr/local/lib/python3.8/dist-packages/aiohttp/connector.py", line 1206, in _create_direct_connection
raise last_exc
File "/usr/local/lib/python3.8/dist-packages/aiohttp/connector.py", line 1175, in _create_direct_connection
transp, proto = await self._wrap_create_connection(
File "/usr/local/lib/python3.8/dist-packages/aiohttp/connector.py", line 988, in _wrap_create_connection
raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host 2606:4700:4700::1001:443 ssl:default [Cannot assign requested address]
Task exception was never retrieved
future: <Task finished name='Task-6' coro=<ByPassResolver._resolve() done, defined at /home/pwuser/.local/lib/python3.8/site-packages/pixivpy_async/bypass_sni.py:88> exception=ClientConnectorError(ConnectionKey(host='2606:4700:4700::1111', port=443, is_ssl=True, ssl=None, proxy=None, proxy_auth=None, proxy_headers_hash=None), OSError(99, 'Cannot assign requested address'))>
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/aiohttp/connector.py", line 980, in _wrap_create_connection
return await self._loop.create_connection(*args, **kwargs) # type: ignore[return-value] # noqa
File "/usr/lib/python3.8/asyncio/base_events.py", line 1025, in create_connection
raise exceptions[0]
File "/usr/lib/python3.8/asyncio/base_events.py", line 1010, in create_connection
sock = await self._connect_sock(
File "/usr/lib/python3.8/asyncio/base_events.py", line 924, in _connect_sock
await self.sock_connect(sock, address)
File "/usr/lib/python3.8/asyncio/selector_events.py", line 496, in sock_connect
return await fut
File "/usr/lib/python3.8/asyncio/selector_events.py", line 501, in _sock_connect
sock.connect(address)
OSError: [Errno 99] Cannot assign requested address
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/pwuser/.local/lib/python3.8/site-packages/pixivpy_async/bypass_sni.py", line 98, in _resolve
async with session.get(endpoint, params=params, headers={"accept": "application/dns-json"}, timeout=ClientTimeout(total=timeout)) as resp:
File "/usr/local/lib/python3.8/dist-packages/aiohttp/client.py", line 1141, in __aenter__
self._resp = await self._coro
File "/usr/local/lib/python3.8/dist-packages/aiohttp/client.py", line 536, in _request
conn = await self._connector.connect(
File "/usr/local/lib/python3.8/dist-packages/aiohttp/connector.py", line 540, in connect
proto = await self._create_connection(req, traces, timeout)
File "/usr/local/lib/python3.8/dist-packages/aiohttp/connector.py", line 901, in _create_connection
_, proto = await self._create_direct_connection(req, traces, timeout)
File "/usr/local/lib/python3.8/dist-packages/aiohttp/connector.py", line 1206, in _create_direct_connection
raise last_exc
File "/usr/local/lib/python3.8/dist-packages/aiohttp/connector.py", line 1175, in _create_direct_connection
transp, proto = await self._wrap_create_connection(
File "/usr/local/lib/python3.8/dist-packages/aiohttp/connector.py", line 988, in _wrap_create_connection
raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host 2606:4700:4700::1111:443 ssl:default [Cannot assign requested address]
caught: TimeoutError()
caught: ClientConnectorError(ConnectionKey(host='2606:4700:4700::1111', port=443, is_ssl=True, ssl=None, proxy=None, proxy_auth=None, proxy_headers_hash=None), OSError(99, 'Cannot assign requested address'))
caught: ClientConnectorError(ConnectionKey(host='2606:4700:4700::1001', port=443, is_ssl=True, ssl=None, proxy=None, proxy_auth=None, proxy_headers_hash=None), OSError(99, 'Cannot assign requested address'))
https://i.pximg.net/c/600x1200_90_webp/img-master/img/2022/06/19/23/21/09/99169234_p0_master1200.jpg
https://i.pximg.net/c/600x1200_90_webp/img-master/img/2022/06/12/22/46/48/99012724_p0_master1200.jpg
https://i.pximg.net/c/600x1200_90_webp/img-master/img/2022/07/10/20/37/14/99635521_p0_master1200.jpg
...
我用 ip addr 检查了一下主机网络，参考 IPv6 Network is unreachable (os error 101) - StackOverflow 的回答认为是由于服务器 IPv6 is enabled but dynamic assignment of IPv6 addresses is not enabled.
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 52:54:00:c2:56:5f brd ff:ff:ff:ff:ff:ff
inet 10.0.4.4/22 brd 10.0.7.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fec2:565f/64 scope link
valid_lft forever preferred_lft forever
4: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:0b:e1:24:5b brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
inet6 fe80::42:bff:fee1:245b/64 scope link
valid_lft forever preferred_lft forever
illust_recommended请求下一页时返回“不正なリクエストです。”
复现代码：
import asyncio
from pixivpy_async import *
async def _main():
async with PixivClient(proxy="socks5://127.0.0.1:7890") as client:
api = AppPixivAPI(client=client)
api.set_auth(access_token="access_token",
refresh_token="refresh_token")
result = await api.illust_recommended()
next_qs = api.parse_qs(result["next_url"])
result = await api.illust_recommended(**next_qs)
print(result)
def main():
loop = asyncio.get_event_loop()
loop.run_until_complete(_main())
if __name__ == '__main__':
main()对比pixivpy3和pixivpy-async的代码发现pixivpy3的illust_recommended并未在请求中添加viewed参数，将next_qs中viewed参数删去后工作正常。
有计划向pixivpy一样提供一个bypass类似的类么？
大陆地区访问比较难，有点麻烦（
Client regularly stops responding to requests
I'm building a discord bot which periodically accesses Pixiv using this API and it works great... for a few hours. Every time we restart the bot, it logs in and locates images just fine, but then it mysteriously stops working.
I suspect this is because we require a new token, but it's not clear how to obtain one when the old token expires. We're currently logging in using a username and password.
Would you be able to shed any light on this matter? I've been looking for documentation on the Pixiv API but it's rather sparse.
json request error 400 in papi.search_works
I'm trying to get illustrations by tag name but I got JSON requests error 400 like there upbit/pixivpy#72.
But update pixivpy-async and aiohttp packages don't solve my problem.
Im using code like in examples:
async with PixivClient() as client:
pixiv_api_ = PixivAPI(client=client)
json_response = await pixiv_api_.search_works("五航戦 姉妹", page=1, mode='text')
and I got this JSON response:
{'status': 'failure', 'has_error': True, 'errors': {'system': {'message': '400 Bad Request', 'code': None}}}
Thank you in advance)
装了依赖仍然报错
login操作异常
我写了一个只有登录功能的程序，运行到login()的时候会打印
Exception during <function Net.auth at 0x000001369B8ECEE8> execution. 1 of 10 retries attempted
为了保证不是我的网络或者账号问题，我运行pixivpy3时候是可以正常访问和正常结束的，下面是简单的测试程序
# -*- coding: utf-8 -*-
import asyncio
from pixivpy_async import AppPixivAPI
_USERNAME = "**"
_PASSWORD = "**“
async def _main(papi):
await papi.login(_USERNAME, _PASSWORD)
def main():
loop = asyncio.get_event_loop()
loop.run_until_complete(_main(AppPixivAPI()))
if __name__ == '__main__':
main()
Cannot find declaration to go to
Am I missing a library or something else? Here is my full code.
import discord
import asyncio
import logging
import sys
from pixivpy_async import *
class MyClient(discord.Client):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.bg_task = self.loop.create_task(self.my_background_task())
async def on_ready(self):
print('Logged in as')
print(self.user.name)
print(self.user.id)
print('------')
async def my_background_task(self):
await self.wait_until_ready()
channel = self.get_channel(830874147513040968)
async with PixivClient() as pClient:
aapi = AppPixivAPI(client=pClient)
await aapi.login(refresh_token='WzxapVvg2G-m9AdZ0B00UVEeXwYvgg9f93vCp-_gsck')
while not self.is_closed():
print('hi')
await self.appapi_illust(aapi)
await asyncio.sleep(1)
async def appapi_illust(self, aapi):
print('hi1')
json_result = aapi.illust_detail(89135896)
print('hi2')
illust = await json_result.illust
print('hi3')
print(">>> %s, origin url: %s" % (illust.title, illust.image_urls['large']))
client = MyClient()
logging.basicConfig(level=logging.INFO)
client.run('xxx')
And here's the libraries I have installed.
(venv) M:\coding\discord-bot\venv\Scripts>pip list
Package Version
----------------- -----------
aiodns 2.0.0
aiofiles 0.6.0
aiohttp 3.7.4.post0
aiohttp-socks 0.6.0
async-timeout 3.0.1
asyncbs4 1.0.4
asyncio 3.4.3
attrs 20.3.0
brotlipy 0.7.0
cchardet 2.1.7
certifi 2020.12.5
cffi 1.14.5
chardet 4.0.0
cloudscraper 1.2.48
discord.py 1.7.1
idna 2.10
multidict 5.1.0
pip 21.0.1
PixivPy 3.5.11
PixivPy-Async 1.2.12
pycares 3.1.1
pycparser 2.20
pyparsing 2.4.7
python-socks 1.2.4
requests 2.25.1
requests-toolbelt 0.9.1
selenium 3.141.0
setuptools 56.0.0
soupsieve 2.2.1
typing-extensions 3.7.4.3
urllib3 1.26.4
yarl 1.6.3
My code stops doing anything when it gets to illust = await json_result.illust. Here's the code output
Logged in as
bot
830647009744125972
------
hi
hi1
hi2
And I was following the demo here https://github.com/Mikubill/pixivpy-async/blob/master/demo/demo_appapi_illust.py
Json server response. Error: Offset must be no more than 5000
I'm trying to get 11.000 followings from my pixiv account but at a middle way got this error in json request:
{'error': {'user_message': '', 'message': '{"offset":["Offset must be no more than 5000"]}', 'reason': '', 'user_message_details': {}}}
Is there any way to get over 5000 followings from my account?
How I do that:
next_qs = 31545655 while next_qs: if type(next_qs) is not int: user_following_json = await app_api.user_following(**next_qs) else: user_following_json = await app_api.user_following(next_qs) next_qs = app_api.parse_qs(user_following_json.next_url)
当使用proxy时，设置cookies发送请求会请求失败
不知道应不应该反馈到这里，如果打扰了还请见谅。
因为本项目中的搜索结果的数量很明显少于实际值，我自己基于本项目中的网络请求做了一套浏览器的P站爬虫，因为需要设置cookie来绕过登录，在使用中就发现了如题所述的问题。
设置的cookie是P站返回的 PHPSSEID 。以下是我使用的一些记录。
测试：
1.使用 requests 的 proxies 后设置 cookies ，成功返回P站在已登录情况下的结果。
2.使用 pixivpy-async 的 proxies 后不设置 cookies，成功返回P站在未登录情况下的结果
3.使用 pixivpy-async 的 proxies 后设置 cookies，报错（返回403）。
4.使用 pixivpy-async 的 bypass 后设置 cookies，成功返回P站在已登录情况下的结果。
测试代码如下：
async def search_illusts(self,keyword:str,page:int,order="date_d", s_mode="s_tag_full"):
url="https://www.pixiv.net/ajax/search/illustrations/{}".format(keyword)
params={
"word":keyword,
"order":order,
"mode":"all",
"p":page,
"s_mode":s_mode,
"type":"illust_and_ugoira",
"lang":"zh",
}
res = await self.requests_api('GET',url,params=params)
return res.body.illust.data
本人推测是因为使用了 ProxyConnector 导致的报错，如果 TCPConnector 会不会报错尚不可知，因为不使用代理的情况下无法访问P站。如果能回答我的话将不胜感激。
How to filter illusts by number of bookmarks?
Can I filter the result by the number of bookmarks while searching by tag? Like in this image(image from the pixiv site).
Getting my user ID
If I want to get my own bookmarks I need to provide a user ID but it seems that there is no endpoint to get info about the authenticated user directly.
The Pixiv website doesn't use an endpoint to get your own user info but rather embeds it in the main html document so that doesn't really help.
Translated tag is always None
Exemple: https://imgur.com/a/47BmyQf
请问能否增加对popular-preview的api支持
其实是看别的项目的时候发现的一个api，可以支持非vip用户查看一部分 sort by popular_desc 的结果（
definition here
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.