marshalx / atproto Goto Github PK
View Code? Open in Web Editor NEWThe AT Protocol (π¦ Bluesky) SDK for Python π
Home Page: https://atproto.blue
License: MIT License
The AT Protocol (π¦ Bluesky) SDK for Python π
Home Page: https://atproto.blue
License: MIT License
After transition to pydantic I am no longer able to fetch threads. It might very well be a user error, but I can't see where.
Running:
bsky.app.bsky.feed.get_post_thread(params={"uri": uri})
and getting response:
File "/home/linus/.local/lib/python3.8/site-packages/atproto/xrpc_client/namespaces/sync_ns.py", line 403, in get_post_thread
return get_response_model(response, models.AppBskyFeedGetPostThread.Response)
File "/home/linus/.local/lib/python3.8/site-packages/atproto/xrpc_client/models/utils.py", line 98, in get_response_model
return get_or_create(response.content, model)
File "/home/linus/.local/lib/python3.8/site-packages/atproto/xrpc_client/models/utils.py", line 64, in get_or_create
raise e
File "/home/linus/.local/lib/python3.8/site-packages/atproto/xrpc_client/models/utils.py", line 57, in get_or_create
model_instance = _get_or_create(model_data, model, strict=strict)
File "/home/linus/.local/lib/python3.8/site-packages/atproto/xrpc_client/models/utils.py", line 89, in _get_or_create
raise ModelError(str(e)) from e
atproto.exceptions.ModelError: 1 validation error for Response
thread.`app.bsky.feed.defs#threadViewPost`.viewer
Extra inputs are not permitted [type=extra_forbidden, input_value={'canReply': True}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.3/v/extra_forbidden
I'm not sure what "extra inputs" it's receiving.
Hello,
Recently, Bluesky added access tokens (#151), and right now, the library allows exporting a session string (client.export_session_string()
). This string includes the handle, DID, their access token, and a refresh token.
Previously, the refresh token had been implemented with a 2-month lifetime (and then developer should generate another refresh token with client.login(login=..., password=...)
), but now, the CreateSession function returns another refresh token with another 2-month lifetime.
In my codebase, it's something like this:
def app_init():
# On app init
client = Client()
client.login(BLUESKY_USER_HANDLE, BLUESKY_USER_APP_PASSWORD)
with open(BLUESKY_SECRET_FILE, "w") as f:
f.write(client.export_session_string())
def post_message(text):
session_string = "..." # content from file
client = Client()
client.login(session_string=session_string)
if client._access_jwt and client._should_refresh_session():
client._refresh_and_set_session()
# Update file with new session
with open(BLUESKY_SECRET_FILE, "w") as f:
f.write(client.export_session_string())
# client.send_post(...)
My point here is that protected methods are being used to resolve this problem with autorefresh. Is it better to make them public (if the token needs to be invalidated in this way)?
My initial expectation is that methods like client.send_post()
/client.com.atproto.repo.create_record()
/etc will take care of token invalidation (if needed), but I receive an error, and it looks like invalidation doesn't work for those methods.
Official client behavior:
.refresh_session()
with the auth header equals to refreshJwt
field.Now, the high-level client doesn't save refresh_token
at all. The client should save it and process error handling on UnauthorizedError
. Also, check the expired date before performing a new request.
Example of how to get and refresh the session:
session = client.com.atproto.server.create_session({'identifier': 'my-handle', 'password': 'my-pass'})
token = session.accessJwt
refresh_token = session.refreshJwt
refreshed_session = client.com.atproto.server.refresh_session(headers={'Authorization': f'Bearer {refresh_token}'})
new_token = refreshed_session.accessJwt
We can get the token "expired at" from decoding of Access JWT.
Header:
{
"alg": "HS256",
"typ": "JWT"
}
Payload:
{
"scope": "com.atproto.appPass",
"sub": "did:plc:PUP",
"iat": 1683662608,
"exp": 1683669808
}
The "exp" field. We can use it to refresh the token a bit later.
iat β 2023-05-09T20:03:28Z
exp β 2023-05-09T22:03:28Z
The access token lives for 2 hours.
The refresh token lives for 3 months.
Please see the interaction with a bluesky dev here:
"Hang on to that access token! It's good for 2 hrs!"
https://bsky.app/profile/dholms.xyz/post/3k7ieei53a326
Your CLIENT.login provides no token, and without it (or a way to use it to login instead of using CLIENT.login), my script is essentially broken.
Thank you!
Nick
Provide autogenerated Record Namespaces with more high-level work with basic operations upon records (CRUD + list records)
this is how we work with records now:
self.com.atproto.repo.delete_record(
models.ComAtprotoRepoDeleteRecord.Data(
collection=ids.AppBskyFeedPost,
repo=uri.hostname,
rkey=uri.rkey,
)
)
this is how it could be (.post
is autogenerated PostRecordNamespace
):
self.bsky.feed.post.delete(repo, rkey)
Is this intended? I am not able to use Python 3.12 which is the latest stable?
The current project's supported Python range (>=3.12,<4.0) is not compatible with some of the required packages Python requirement:
- atproto requires Python >=3.7.1,<3.13, so it will not be satisfied for Python >=3.13,<4.0
Since around midnight last night I have started getting the following error when using atproto:
Traceback (most recent call last):
File "/[PATH REDACTED]/.local/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 87, in _get_or_create
return model(**model_data)
File "/[PATH REDACTED]/.local/lib/python3.10/site-packages/pydantic/main.py", line 165, in __init__
__pydantic_self__.__pydantic_validator__.validate_python(data, self_instance=__pydantic_self__)
pydantic_core._pydantic_core.ValidationError: 1 validation error for Response
emailConfirmed
Extra inputs are not permitted [type=extra_forbidden, input_value=False, input_type=bool]
For further information visit https://errors.pydantic.dev/2.3/v/extra_forbidden
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "[PATH REDACTED]", line 15, in <module>
bsky.login(bsky_handle, bsky_password)
File "/[PATH REDACTED]/.local/lib/python3.10/site-packages/atproto/xrpc_client/client/client.py", line 74, in login
session = self._get_and_set_session(login, password)
File "/[PATH REDACTED]/.local/lib/python3.10/site-packages/atproto/xrpc_client/client/client.py", line 37, in _get_and_set_session
session = self.com.atproto.server.create_session(
File "/[PATH REDACTED]/.local/lib/python3.10/site-packages/atproto/xrpc_client/namespaces/sync_ns.py", line 1757, in create_session
return get_response_model(response, models.ComAtprotoServerCreateSession.Response)
File "/[PATH REDACTED]/.local/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 98, in get_response_model
return get_or_create(response.content, model)
File "/[PATH REDACTED]/.local/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 64, in get_or_create
raise e
File "/[PATH REDACTED]/.local/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 57, in get_or_create
model_instance = _get_or_create(model_data, model, strict=strict)
File "/[PATH REDACTED]/.local/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 89, in _get_or_create
raise ModelError(str(e)) from e
atproto.exceptions.ModelError: 1 validation error for Response
emailConfirmed
Extra inputs are not permitted [type=extra_forbidden, input_value=False, input_type=bool]
For further information visit https://errors.pydantic.dev/2.3/v/extra_forbidden
Since no changes have been made on my end, and I haven't downloaded a new version of atproto, I assume something must have changed with the API? Or I'm missing something obvious on my end, also a possibility.
Thanks for sharing. The mutation is happening in your
DotDict
validator. If I add some prints, and adjust your__repr__
ofDotDict
to show it's aDotDict
:def validate_from_dict(value: dict) -> DotDict: print("start", value) result = DotDict(value) print("result", result) print("end", value) return resultThen I see that calling
DotDict(value)
is replacing nested dictionaries insidevalue
withDotDict
instances.The reason why this has only broken with the bump to 2.5 is that on 2.5 the new union behaviour is doing a little bit more work than before to check that
DotDict
is not a better match than your model instances. On 2.4 theDotDict
validation was never run, on 2.5 it is now being run which is why you see the mutation bug arising.
Now recently there has started to appear the following issue
Traceback (most recent call last):
File "/home/pi/bots/posters/crossposter/blueskypost.py", line 80, in <module>
client._set_session(ses)
File "/home/pi/.local/lib/python3.9/site-packages/atproto/xrpc_client/client/methods_mixin/session.py", line 68, in _set_session
self._refresh_jwt_payload = get_jwt_payload(session.refresh_jwt)
File "/home/pi/.local/lib/python3.9/site-packages/atproto/xrpc_client/client/auth.py", line 18, in get_jwt_payload
return JwtPayload(**plain_payload)
TypeError: __init__() got an unexpected keyword argument 'aud'
Traceback (most recent call last):
File "/home/pi/bots/posters/crossposter/blueskypost.py", line 80, in <module>
client._set_session(ses)
File "/home/pi/.local/lib/python3.9/site-packages/atproto/xrpc_client/client/methods_mixin/session.py", line 68, in _set_session
self._refresh_jwt_payload = get_jwt_payload(session.refresh_jwt)
File "/home/pi/.local/lib/python3.9/site-packages/atproto/xrpc_client/client/auth.py", line 18, in get_jwt_payload
return JwtPayload(**plain_payload)
TypeError: __init__() got an unexpected keyword argument 'aud'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/pi/bots/posters/crossposter/blueskypost.py", line 88, in <module>
client.login(username, password)
File "/home/pi/.local/lib/python3.9/site-packages/atproto/xrpc_client/client/client.py", line 74, in login
File "/home/pi/.local/lib/python3.9/site-packages/atproto/xrpc_client/client/client.py", line 40, in _get_and_set_session
session = self._get_and_set_session(login, password)
self._set_session(session)
File "/home/pi/.local/lib/python3.9/site-packages/atproto/xrpc_client/client/methods_mixin/session.py", line 68, in _set_session
self._refresh_jwt_payload = get_jwt_payload(session.refresh_jwt)
File "/home/pi/.local/lib/python3.9/site-packages/atproto/xrpc_client/client/auth.py", line 18, in get_jwt_payload
return JwtPayload(**plain_payload)
TypeError: __init__() got an unexpected keyword argument 'aud'
I think it has to do something with the JwtPayload
Originally posted by Scrxtchy July 27, 2023
I've had this problem for a while and had it swept under the rug for a while. I still cannot think of a solution to this.
When retrieving a post though feed.getposts()
if the post has an image, but the post has an embed of another record, the images in the maid record as returned with the type of app.bsky.embed.images#image
, which is in contrast to when the Post does not have a record embed, it includes the app.bsky.embed.images#view
in the embeds, giving back the impageproxy urls for the media.
I am confused as to why this has happened, as when looking at the network requests on the website, the app.bsky.embed.recordWithMedia#view
record type contains both a app.bsky.embed.record#viewRecord
to contain the record of the quoted post, as well as the app.bsky.embed.images#view
images containing the urls that I have been seeking out
I am in Central Daylight Time, GMT-5. When calling send_image
, it appears in my timeline 5 hours in the past.
send_image
calls send_post
. It looks like the cause for this bug is in send_post
:
atproto/atproto/xrpc_client/client/client.py
Line 120 in 2bdb5b0
in which it calls datetime.now().isoformat()
. now()
returns the local time zone; atproto uses UTC.
A fix would be to change line 2 to from datetime import datetime, timezone
and line 120 to
createdAt=datetime.now(timezone.utc).isoformat(), text=text, reply=reply_to, embed=embed, langs=langs
It looks like like
and repost
have the same issue, but could be fixed similarly.
It happens rarely. About 1 frame per few hours
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/dag_cbor/decoding/__init__.py", line 269, in _decode_dict
v, _ = _decode_item(stream, options)
File "/usr/local/lib/python3.8/dist-packages/dag_cbor/decoding/__init__.py", line 149, in _decode_item
value, num_bytes_further_read = _decoders[major_type](stream, arg, options)
File "/usr/local/lib/python3.8/dist-packages/dag_cbor/decoding/__init__.py", line 208, in _decode_bytes
raise CBORDecodingError(_err._unexpected_eof(stream, f"{length} bytes of bytestring", length))
dag_cbor.decoding.err.CBORDecodingError: Unexpected EOF while attempting to read 1047 bytes of bytestring.
At byte #248: 04...75 (last byte #993)
^^^^^^^ 744 bytes read, out of 1047 expected.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/atproto/firehose/client.py", line 175, in start
self._process_raw_frame(raw_frame)
File "/usr/local/lib/python3.8/dist-packages/atproto/firehose/client.py", line 101, in _process_raw_frame
frame = Frame.from_bytes(data)
File "/usr/local/lib/python3.8/dist-packages/atproto/firehose/models.py", line 114, in from_bytes
decoded_parts = decode_dag_multi(data)
File "/usr/local/lib/python3.8/dist-packages/atproto/cbor/__init__.py", line 54, in decode_dag_multi
decoded_part = decode_dag(data, allow_concat=True, callback=bytes_read_counter)
File "/usr/local/lib/python3.8/dist-packages/atproto/cbor/__init__.py", line 30, in decode_dag
return _dag_cbor.decode(data, allow_concat=allow_concat, callback=callback)
File "/usr/local/lib/python3.8/dist-packages/dag_cbor/decoding/__init__.py", line 127, in decode
data, _ = _decode_item(stream, options)
File "/usr/local/lib/python3.8/dist-packages/dag_cbor/decoding/__init__.py", line 149, in _decode_item
value, num_bytes_further_read = _decoders[major_type](stream, arg, options)
File "/usr/local/lib/python3.8/dist-packages/dag_cbor/decoding/__init__.py", line 271, in _decode_dict
raise CBORDecodingError(_err._dict_item(dict_head_snapshot, "value", i, length, e)) # pylint: disable = raise-missing-from
dag_cbor.decoding.err.CBORDecodingError: Error while decoding dict.
At byte #0: aa...
^^ dict of length 10
Error occurred while decoding value at position 6: further details below.
\ Unexpected EOF while attempting to read 1047 bytes of bytestring.
At byte #248: 04...75 (last byte #993)
^^^^^^^ 744 bytes read, out of 1047 expected.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/home/marshal/bluesky-feed-generator/src/data_stream.py", line 87, in run
client.start(on_message_handler)
File "/usr/local/lib/python3.8/dist-packages/atproto/firehose/client.py", line 179, in start
should_stop = _handle_firehose_error_or_stop(e)
File "/usr/local/lib/python3.8/dist-packages/atproto/firehose/client.py", line 63, in _handle_firehose_error_or_stop
raise FirehoseError from exception
atproto.exceptions.FirehoseError
I'm running into an issue with auth token handling that hopefully I can explain.
atproto handles scenario 1 and 2, but does not handle scenario 3. _should_refresh_session
will check if the access token is nearly expired, and if so, will refresh the access token using the refresh token. However, atproto does not check if the refresh token is expired and handle refreshing it.
Right now Client.login()
accepts either the handle/password or the session string, but not both. If the session string is passed then the handle/password are ignored.
Client.login
)._refresh_and_set_session
. This will allow me to know when the session string has changed so I can store it in the database and use it in the future when creating new Clients. Otherwise I have to call export_session_string
after every atproto method because I don't know which one may have created a new session.now i support backward compatibility and show warnings. but i want to delete it rly soon
uri
and cid
args requiredwas added here: #255
When I try to get user's followers with the given code:
from atproto import Client, models
client = Client()
client.login('some_handle', 'app_password')
params = models.AppBskyGraphGetFollowers.Params(actor='some_actor')
client.bsky.graph.get_followers(params=params)
I get the following error:
atproto.exceptions.WrongTypeError: wrong value type for field "followers.viewer.mutedByList.purpose" - should be "Literal" instead of value "app.bsky.graph.defs#modlist" of type "str"
The underlying cause of this error is config.check_types
procedure:
dacite.exceptions.WrongTypeError: wrong value type for field "followers.viewer.mutedByList.purpose" - should be "Literal" instead of value "app.bsky.graph.defs#modlist" of type "str"
If type checking (lines 68-69 here) is commented out, list of followers is successfully received.
atproto
version is 0.0.24
was added here: #264
Hi, sorry for the novice question but Iβm struggling to work out how to access alt-text for embedded images in posts. How is this done, assuming there may be multiple imagesβ text to parse per post.
Thanks
Now the records in the responses are plain dicts. We need to parse it as Record Models. The models are generated already.
It doesn't work now because in the lexicons all records are marked as "unknown" instead of the reference to the record model.
For now this fields are typehinted as "Any":
https://github.com/MarshalX/atproto/blob/0826a31a3668eec9d85f79c3e9861ed394ab1165/atproto/codegen/models/generator.py#L156C1-L158
Many other packages depend on typing-extensions>=4.6.1.
Please consider supporting >=4.6.0 to avoid dependency conflicts.
Example: AtUri.from_str("at://100ideas.bsky.social").hostname
returns: at:
should return: 100ideas.bsky.social
Fix: This implementation is based on a REGEX string from bluesky-social and they've updated and changed this string. Making similar change here fixes the problem.
In packages/atproto_core/uri/uri.py change _ATP_URI_REGEX
from: _ATP_URI_REGEX = r'^(at:\/\/)?((?:did:[a-z0-9:%-]+)|(?:[a-z][a-z0-9.:-]*))(\/[^?#\s]*)?(\?[^#\s]+)?(#[^\s]+)?$'
to : _ATP_URI_REGEX = r'^(at:\/\/)?((?:did:[a-z0-9:%-]+)|(?:[a-z0-9][a-z0-9.:-]*))(\/[^?#\s]*)?(\?[^#\s]+)?(#[^\s]+)?$'
based on ref: https://github.com/bluesky-social/atproto/blob/8c19ce991a766fd9cff5023160853ab1cb106f21/packages/uri/src/index.ts#LL5C38-L5C38
Thanks
Hi. After recent update no invite history for most accounts.
Returns codes=[]
res = client.com.atproto.server.get_account_invite_codes(
models.ComAtprotoServerGetAccountInviteCodes.Params(
createAvailable=True,
includeUsed=True)
)
As of today, I switched the astronomy feeds FirehoseSubscribeReposClient
to use the new BGS at wss://bsky.network/xrpc
. I am getting intermittent errors on a very small fraction of posts when atproto.CAR.from_bytes
is called on certain commits. This happens once every ~10 minutes (so must be caused by only a small fraction of posts); otherwise, the feed is running fine.
This is on Ubuntu 20.04, with Python 3.11.4 and atproto 0.0.30. This is the commit in the GitHub repo where issues started happening.
Stack trace of the error:
[firehose] [2023-11-10 16:43:23] thread '<unnamed>' panicked at src/lib.rs:164:67:
[firehose] [2023-11-10 16:43:23] called `Result::unwrap()` on an `Err` value: Parsing("failed to parse uvarint for header")
[firehose] [2023-11-10 16:43:23] note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
[firehose] [2023-11-10 16:43:23] Process Process-1:
[firehose] [2023-11-10 16:43:23] Traceback (most recent call last):
[firehose] [2023-11-10 16:43:23] File "/workspace/.heroku/python/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
[firehose] [2023-11-10 16:43:23] self.run()
[firehose] [2023-11-10 16:43:23] File "/workspace/.heroku/python/lib/python3.11/multiprocessing/process.py", line 108, in run
[firehose] [2023-11-10 16:43:23] self._target(*self._args, **self._kwargs)
[firehose] [2023-11-10 16:43:23] File "/workspace/server/firehose.py", line 99, in worker_main
[firehose] [2023-11-10 16:43:23] _worker_loop(receiver)
[firehose] [2023-11-10 16:43:23] File "/workspace/server/firehose.py", line 85, in _worker_loop
[firehose] [2023-11-10 16:43:23] operations_callback(_get_ops_by_type(commit))
[firehose] [2023-11-10 16:43:23] ^^^^^^^^^^^^^^^^^^^^^^^^
[firehose] [2023-11-10 16:43:23] File "/workspace/server/firehose.py", line 30, in _get_ops_by_type
[firehose] [2023-11-10 16:43:23] car = CAR.from_bytes(commit.blocks)
[firehose] [2023-11-10 16:43:23] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[firehose] [2023-11-10 16:43:23] File "/workspace/.heroku/python/lib/python3.11/site-packages/atproto/car/__init__.py", line 51, in from_bytes
[firehose] [2023-11-10 16:43:23] header, blocks = libipld.decode_car(data)
[firehose] [2023-11-10 16:43:23] ^^^^^^^^^^^^^^^^^^^^^^^^
[firehose] [2023-11-10 16:43:23] pyo3_runtime.PanicException: called `Result::unwrap()` on an `Err` value: Parsing("failed to parse uvarint for header")
I've added more logging on the feed and set RUST_BACKTRACE=1
. Will update this issue if I can work out which commits are causing the problem.
my thoughts about impl: #93 (comment)
The regexp in extract_url_byte_positions
at https://github.com/MarshalX/atproto/blob/main/examples/advanced_usage/auto_hyperlinks.py does not appear to detect all valid URLs. Take for example:
https://www.cell.com/matter/fulltext/S2590-2385(23)00409-5?rss=yes
This is misdetected, and the URL is stopped before (
Hi,
I want to automatically associate an image with a preview card inside a publication and so I tried:
`
blob = bsky.com.atproto.repo.upload_blob(image_bytes)
pprint(blob)
blob.blob.mime_type = 'image/*'
embed_external = models.AppBskyEmbedExternal.Main(
external=models.AppBskyEmbedExternal.External(
title=preview_data.title,
description=preview_data.description,
uri=url,
thumb=blob.blob,
)
)
`
with the following log:
2023-12-18 16:42:47,328 HTTP Request: POST https://bsky.social/xrpc/com.atproto.repo.uploadBlob "HTTP/1.1 200 OK"
Response(blob=BlobRef(mime_type='*/*', size=2483, ref=BlobRefLink(link='bafkreihg7iiatouv6hiqb7t5uharq3ikgcfd2ejbspw3bhxqp3led7xjea'), py_type='blob'))
2023-12-18 16:42:47,456 HTTP Request: POST https://bsky.social/xrpc/com.atproto.repo.createRecord "HTTP/1.1 400 Bad Request"
2023-12-18 16:42:47,457 Response(success=False, status_code=400, content=XrpcError(error='InvalidMimeType', message='Wrong type of file. It is */* but it must match image/*.'), headers=Headers({'date': 'Mon, 18 Dec 2023 16:42:46 GMT', 'content-type': 'application/json; charset=utf-8', 'content-length': '96', 'connection': 'keep-alive', 'x-powered-by': 'Express', 'access-control-allow-origin': '*', 'ratelimit-limit': '5000', 'ratelimit-remaining': '4961', 'ratelimit-reset': '1702921208', 'ratelimit-policy': '5000;w=3600', 'etag': 'W/"60-naTgCWHhjZ5FGkdXs552SxW2Nug"', 'vary': 'Accept-Encoding'}))
Apparently, the image is uploaded with the content-type of '/' but when associating the blob object returned from the upload_blob
method with the thumb
field it does not work, since it expects the blob of being of content-type 'image/*' . I tried to force the mime_type of blob object to be of this value but it does not work either !
Thanks for your attention!
Hi. Please help.
What is wrong with this code?
I get an 'AuthenticationRequired' error when I try it:
new_email = '[email protected]'
client = Client()
profile = client.login('handle', 'password')
res = client.com.atproto.admin.update_account_email(
models.ComAtprotoAdminUpdateAccountEmail.Data(
account=profile.did,
email=new_email)
)
Hello @MarshalX , Firstly I have to say you did a great job with this project. I am working on a custom feed using it.
Do you have any idea why I get an error on the app even though the feed was published and these endpoints work on my domain?
- /.well-known/did.json
- /xrpc/app.bsky.feed.describeFeedGenerator
- /xrpc/app.bsky.feed.getFeedSkeleton
Although a similar issue was raised here on the official bluesky repo - bluesky-social/feed-generator#35 but I could really use a second eye.
Hey @MarshalX (and others contributing to this repo)!
Wanted to share a resource and see if you had any feedback. We have started collecting some cross-language test vectors at: https://github.com/bluesky-social/atproto/tree/main/interop-test-files
For example, long lists of valid and invalid identifiers, trying to hit corner cases. These are intended to be easy to copy directly in to other implementations. So far they are used in both the typescript code and golang implementation (indigo).
Would be curious what other test files would be helpful for ensuring inter-operation between implementations, particularly if there have been any sharp edges that you have run in to.
We will probably add:
app.bsky
record Lexicons, both valid and invalidHello!
I am not able to fetch some user profiles β looks like this issue happens on profiles where author defined language of "skeets". For example you can check it with "uavibes.bsky.social" profile.
def main():
client = Client()
client.login(...)
profile_feed = client.bsky.feed.get_author_feed({'actor': 'uavibes.bsky.social'})
for feed_view in profile_feed.feed:
print('-', feed_view.post.record.text)
Traceback (most recent call last):
File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/models/utils.py", line 101, in _get_or_create
model(**model_data)
TypeError: Main.__init__() got an unexpected keyword argument 'langs'
Traceback (most recent call last):
File "/Users/vadym/Projects/vadym/uavibes/main.py", line 14, in <module>
main()
File "/Users/vadym/Projects/vadym/uavibes/main.py", line 8, in main
profile_feed = client.bsky.feed.get_author_feed({'actor': 'uavibes.bsky.social'})
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/namespaces/sync_ns.py", line 249, in get_author_feed
return get_response_model(response, models.AppBskyFeedGetAuthorFeed.Response)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/models/utils.py", line 132, in get_response_model
return get_or_create_model(response.content, model)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/models/utils.py", line 119, in get_or_create_model
model_instance = get_or_create(model_data, model)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/models/utils.py", line 78, in get_or_create
raise e
File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/models/utils.py", line 75, in get_or_create
return _get_or_create(model_data, model, strict=strict)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/models/utils.py", line 103, in _get_or_create
return from_dict(model, model_data, config=_DACITE_CONFIG)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/dacite/core.py", line 64, in from_dict
value = _build_value(type_=field_type, data=field_data, config=config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/dacite/core.py", line 97, in _build_value
data = _build_value_for_collection(collection=type_, data=data, config=config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/dacite/core.py", line 154, in _build_value_for_collection
return data_type(_build_value(type_=item_type, data=item, config=config) for item in data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/dacite/core.py", line 154, in <genexpr>
return data_type(_build_value(type_=item_type, data=item, config=config) for item in data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/dacite/core.py", line 99, in _build_value
data = from_dict(data_class=type_, data=data, config=config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/dacite/core.py", line 64, in from_dict
value = _build_value(type_=field_type, data=field_data, config=config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/dacite/core.py", line 99, in _build_value
data = from_dict(data_class=type_, data=data, config=config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/dacite/core.py", line 64, in from_dict
value = _build_value(type_=field_type, data=field_data, config=config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/dacite/core.py", line 91, in _build_value
data = config.type_hooks[type_](data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/models/utils.py", line 31, in _record_model_type_hook
return get_or_create_model(data, RECORD_TYPE_TO_MODEL_CLASS[record_type])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/models/utils.py", line 119, in get_or_create_model
model_instance = get_or_create(model_data, model)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/models/utils.py", line 78, in get_or_create
raise e
File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/models/utils.py", line 75, in get_or_create
return _get_or_create(model_data, model, strict=strict)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/models/utils.py", line 107, in _get_or_create
raise UnexpectedFieldError(msg) from e
atproto.exceptions.UnexpectedFieldError: Main.Main got an unexpected keyword argument 'langs'
When Main
class is extended with langs: t.Optional[t.List[str]] = None
field β it works fine. However, I can't find this langs
field in atproto docs, so that's why I didn't create PR for this change yet.
Replit: Updating package configuration
--> poetry add 'atproto 0.0.17'
Updating dependencies
Resolving dependencies...
SolverProblemError
Because no versions of replit match >3.2.4,<3.2.5 || >3.2.5,<3.2.6 || >3.2.6,<3.2.7 || >3.2.7,<3.2.8 || >3.2.8,<3.3.0 || >3.3.0,<3.3.1 || >3.3.1,<4.0.0
and replit (3.2.4) depends on typing_extensions (>=3.7.4,<4.0.0), replit (>=3.2.4,<3.2.5 || >3.2.5,<3.2.6 || >3.2.6,<3.2.7 || >3.2.7,<3.2.8 || >3.2.8,<3.3.0 || >3.3.0,<3.3.1 || >3.3.1,<4.0.0) requires typing_extensions (>=3.7.4,<4.0.0).
And because replit (3.2.6) depends on typing_extensions (>=3.7.4,<4.0.0), replit (>=3.2.4,<3.2.5 || >3.2.5,<3.2.7 || >3.2.7,<3.2.8 || >3.2.8,<3.3.0 || >3.3.0,<3.3.1 || >3.3.1,<4.0.0) requires typing_extensions (>=3.7.4,<4.0.0).
And because replit (3.2.7) depends on typing_extensions (>=3.7.4,<4.0.0)
and replit (3.2.8) depends on typing_extensions (>=3.7.4,<4.0.0), replit (>=3.2.4,<3.2.5 || >3.2.5,<3.3.0 || >3.3.0,<3.3.1 || >3.3.1,<4.0.0) requires typing_extensions (>=3.7.4,<4.0.0).
And because replit (3.3.0) depends on typing_extensions (>=3.7.4,<4.0.0)
and replit (3.3.1) depends on typing_extensions (>=3.7.4,<4.0.0), replit (>=3.2.4,<3.2.5 || >3.2.5,<4.0.0) requires typing_extensions (>=3.7.4,<4.0.0).
And because replit (3.2.5) depends on typing_extensions (>=3.7.4,<4.0.0)
and atproto (0.0.17) depends on typing-extensions (>=4.5.0,<4.6.0), atproto (0.0.17) is incompatible with replit (>=3.2.4,<4.0.0).
So, because python-template depends on both replit (^3.2.4) and atproto (0.0.17), version solving failed.
at venv/lib/python3.10/site-packages/poetry/puzzle/solver.py:241 in _solve
237β packages = result.packages
238β except OverrideNeeded as e:
239β return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
240β except SolveFailure as e:
β 241β raise SolverProblemError(e)
242β
243β results = dict(
244β depth_first_search(
245β PackageNode(self._package, packages), aggregate_package_nodes
exit status 1
Replit: Package operation failed.
Hey there! Thanks so much for creating this package. I've just started using it and it is working great for me.
This issue is to request the ability to include labels when submitting posts. From what I can tell, labels are used to apply content warnings on posts when submitting on https://bsky.app/.
Please let me know if this is already possible and I just missed it!
Here's what the request looks like when submitting via bsky.app and including a content warning:
{
"collection": "app.bsky.feed.post",
"record": {
"$type": "app.bsky.feed.post",
"createdAt": "2024-01-13T16:28:19.938Z",
"embed": {
"$type": "app.bsky.embed.images",
"images": [
{
"alt": "",
"aspectRatio": {
"height": 626,
"width": 640
},
"image": {
"$type": "blob",
"mimeType": "image/jpeg",
"ref": {
"$link": "bafkreichm4wf4ru2omi2v7iefx5f63kejfycs5ggnltf4g6zxyn3or3kxe"
},
"size": 299493
}
}
]
},
"labels": {
"$type": "com.atproto.label.defs#selfLabels",
"values": [
{
"val": "sexual"
}
]
},
"langs": [
"en"
],
"text": "Test content warning"
},
"repo": "did:plc:b76akgyhvhtykcp7j7jsehrc"
}
Every time I try to manually run gunicorn against my test feed, it dies about 30 seconds.
I'm not sure if a standardized gunicorn config file would work for every feed, or the configuration would really be dependent on whatever a specific feed is doing.
If I go ahead and deploy it to Fly with the Dockerfile using "flask run" as the CMD though it works, I'd just prefer to make it work with gunicorn.
This is my stub of a gunicorn config file:
workers = 2
# threads = 4
# timeout = 120
bind = '0.0.0.0:8080'
# forwarded_allow_ips = '*'
# secure_scheme_headers = { 'X-Forwarded-Proto': 'https' }
Thanks for all the work on this library!! π
I've been using it to run a bsky bot that posts twice daily and everything had been going smoothly until yesterday, when importing atproto
started failing with the following Pydantic error:
File "/home/runner/work/bsky-nycasp/bsky-nycasp/bsky_nycasp.py", line 6, in <module>
from atproto import Client
File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/atproto/__init__.py", line 3, in <module>
from .firehose import models as firehose_models
File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/atproto/firehose/__init__.py", line 3, in <module>
from atproto.firehose.client import AsyncFirehoseClient, FirehoseClient
File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/atproto/firehose/client.py", line 21, in <module>
from atproto.firehose.models import ErrorFrame, Frame, MessageFrame
File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/atproto/firehose/models.py", line 7, in <module>
from atproto.xrpc_client.models.utils import get_or_create
File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/atproto/xrpc_client/models/__init__.py", line 265, in <module>
load_models()
File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/atproto/xrpc_client/models/models_loader.py", line 58, in load_models
__on_load()
File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/atproto/xrpc_client/models/models_loader.py", line 54, in __on_load
__rebuild_all_models()
File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/atproto/xrpc_client/models/models_loader.py", line 50, in __rebuild_all_models
__model.model_rebuild()
File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/pydantic/main.py", line 470, in model_rebuild
return _model_construction.complete_model_class(
File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/pydantic/_internal/_model_construction.py", line 498, in complete_model_class
cls.__pydantic_core_schema__ = schema = validate_core_schema(schema)
File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/pydantic/_internal/_core_utils.py", line 631, in validate_core_schema
return _validate_core_schema(schema)
pydantic_core._pydantic_core.SchemaError: Invalid Schema:
definitions.definitions.48.model.schema.model-fields.fields.parent.schema.default.schema.nullable.schema.default.schema.tagged-union.choices.`app.bsky.feed.defs#threadViewPost`
Recursion error - cyclic reference detected [type=recursion_loop, input_value={'type': 'model', 'cls': ...iewPost:94259874186816'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.4/v/recursion_loop
definitions.definitions.48.model.schema.model-fields.fields.replies.schema.default.schema.nullable.schema.list.items_schema.tagged-union.choices.`app.bsky.feed.defs#threadViewPost`
Recursion error - cyclic reference detected [type=recursion_loop, input_value={'type': 'model', 'cls': ...iewPost:94259874186816'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.4/v/recursion_loop
I've tested v0.0.26 and v0.0.28 and both seem to consistently fail. FWIW, my code runs on GitHub Actions with a fresh environment and installation of atproto
every day.
I'm not totally sure how to diagnose or debug this issue, so any pointers would be much appreciated! Thanks!
When using listRecords, if a post is using custom records (like "via" for example), the SDK returns the original dict which is not fully compatible with the proper instance because of the snake/camelCase notation difference on some fields (createdAt, indexedAt, likeCount...).
See original discussion here:
https://bsky.app/profile/mwyann.fr/post/3k757t5scw62l
Maybe other methods are affected too, but it only happened to me with this method anyway.
Rewrite this example using our SDK models: https://gist.github.com/DavidBuchanan314/f92b3925c5480e35076773613cb87bdd
All our examples are under CC0. The gist above was originally MIT, but the author allowed to relicense it to CC0:
Would it be possible to utilise TypeGuard on this function in order to leverage the return value at the callsite for static type checking?
atproto/atproto/xrpc_client/models/utils.py
Line 202 in d5e2293
I'd do it myself but I don't feel I understand enough about the details of the library at this point to get it right
backward compatibility was added here #269
Hi MarshalX.
Strange new error since this morning:
from atproto import Client, models, exceptions
client = Client()
profile = client.login('handle', 'pwd')
try:
res = client.com.atproto.server.get_account_invite_codes(
models.ComAtprotoServerGetAccountInviteCodes.Params(
createAvailable=True,
includeUsed=True)
)
except exceptions.BadRequestError as err:
print(err)
It throws an error with status code 400
'InvalidToken'
'Token could not be verified'
I looked at https://atproto.blue/en/latest/namespace.html#atproto.xrpc_client.namespaces.sync_ns.ServerNamespace.get_account_invite_codes again, there is nothing about a token?
What am I missing?
Please help.
I have started to get the following error whenever I try to post anything at all. I have not made any change in the code but both reposts and normal posts will throw the following error:
BadRequestError(Response(success=False, status_code=400, content=XrpcError(error='InvalidRequest', message='Invalid app.bsky.feed.repost record: createdAt must be an valid atproto datetime (both RFC-3339 and ISO-8601)'), headers=Headers({'date': 'Tue, 05 Dec 2023 01:34:08 GMT', 'content-type': 'application/json; charset=utf-8', 'content-length': '148', 'connection': 'keep-alive', 'x-powered-by': 'Express', 'access-control-allow-origin': '*', 'ratelimit-limit': '5000', 'ratelimit-remaining': '4997', 'ratelimit-reset': '1701743648', 'ratelimit-policy': '5000;w=3600', 'etag': 'W/"94-6peGxeMN8ENNthf4a1NzrR9XOkg"', 'vary': 'Accept-Encoding'})))
I think Bsky has changed something that causes this issue
The error suggests to me that the underlying semantic changed out from under this codebase.
Traceback (most recent call last):
File "/home/temujin9/.local/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 87, in _get_or_create
return model(**model_data)
File "/home/temujin9/.local/lib/python3.10/site-packages/pydantic/main.py", line 164, in __init__
__pydantic_self__.__pydantic_validator__.validate_python(data, self_instance=__pydantic_self__)
pydantic_core._pydantic_core.ValidationError: 8 validation errors for Response
suggestions.0.banner
Extra inputs are not permitted [type=extra_forbidden, input_value='https://cdn.bsky.app/img...u6kla7qb4fhafdihti@jpeg', input_type=str]
For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.0.followsCount
Extra inputs are not permitted [type=extra_forbidden, input_value=84, input_type=int]
For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.0.followersCount
Extra inputs are not permitted [type=extra_forbidden, input_value=50, input_type=int]
For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.0.postsCount
Extra inputs are not permitted [type=extra_forbidden, input_value=593, input_type=int]
For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.1.banner
Extra inputs are not permitted [type=extra_forbidden, input_value='https://cdn.bsky.app/img...7dftfyvh6j46diolje@jpeg', input_type=str]
For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.1.followsCount
Extra inputs are not permitted [type=extra_forbidden, input_value=279, input_type=int]
For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.1.followersCount
Extra inputs are not permitted [type=extra_forbidden, input_value=381, input_type=int]
For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.1.postsCount
Extra inputs are not permitted [type=extra_forbidden, input_value=2529, input_type=int]
For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/temujin9/Projects/T9_Productions/code/bsky_scripts/suggested_follows", line 24, in <module>
result = graph_ns.get_suggested_follows_by_actor( params )
File "/home/temujin9/.local/lib/python3.10/site-packages/atproto/xrpc_client/namespaces/sync_ns.py", line 817, in get_suggested_follows_by_actor
return get_response_model(response, models.AppBskyGraphGetSuggestedFollowsByActor.Response)
File "/home/temujin9/.local/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 98, in get_response_model
return get_or_create(response.content, model)
File "/home/temujin9/.local/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 64, in get_or_create
raise e
File "/home/temujin9/.local/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 57, in get_or_create
model_instance = _get_or_create(model_data, model, strict=strict)
File "/home/temujin9/.local/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 89, in _get_or_create
raise ModelError(str(e)) from e
atproto.exceptions.ModelError: 8 validation errors for Response
suggestions.0.banner
Extra inputs are not permitted [type=extra_forbidden, input_value='https://cdn.bsky.app/img...u6kla7qb4fhafdihti@jpeg', input_type=str]
For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.0.followsCount
Extra inputs are not permitted [type=extra_forbidden, input_value=84, input_type=int]
For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.0.followersCount
Extra inputs are not permitted [type=extra_forbidden, input_value=50, input_type=int]
For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.0.postsCount
Extra inputs are not permitted [type=extra_forbidden, input_value=593, input_type=int]
For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.1.banner
Extra inputs are not permitted [type=extra_forbidden, input_value='https://cdn.bsky.app/img...7dftfyvh6j46diolje@jpeg', input_type=str]
For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.1.followsCount
Extra inputs are not permitted [type=extra_forbidden, input_value=279, input_type=int]
For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.1.followersCount
Extra inputs are not permitted [type=extra_forbidden, input_value=381, input_type=int]
For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.1.postsCount
Extra inputs are not permitted [type=extra_forbidden, input_value=2529, input_type=int]
For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
Usage that lead to this was unremarkable:
params = graph_m.get_suggested_follows_by_actor.Params(actor=profile.did)
result = graph_ns.get_suggested_follows_by_actor( params )
Would like to atproto use it with fastapi, but getting:
The conflict is caused by:
The user requested typing_extensions
atproto 0.0.31 depends on typing-extensions<4.8.0 and >=4.6.1
fastapi 0.104.1 depends on typing-extensions>=4.8.0
Thanks for your work!
Related to #102
Hello there,
the inline example in atproto/car/init.py stopped working last week.
The line repo = client.com.atproto.sync.get_repo({'did': client.me.did})
atproto/atproto/car/__init__.py
Line 40 in 1e1829e
throw an atproto.exceptions.BadRequestError:
Response(success=False, status_code=400, content=XrpcError(error='InvalidRequest', message='Could not find repo for DID: '), headers=Headers({'date': 'Mon, 20 Nov 2023 14:09:40 GMT', 'content-type': 'application/json; charset=utf-8', 'content-length': '100', 'connection': 'keep-alive', 'x-powered-by': 'Express', 'access-control-allow-origin': '*', 'ratelimit-limit': '3000', 'ratelimit-remaining': '2998', 'ratelimit-reset': '1700489680', 'ratelimit-policy': '3000;w=300', 'etag': 'W/"64-u+z2SVfRs2zfn34PuZ9u5A4P55k"', 'vary': 'Accept-Encoding'}))
I could verify the correctness of the DID in other ways.
Additional information: the more or less 'official' indigo-implementation seems to have a similar issue. Did they change the handling in the protocol?
Thanks for looking into it.
Harald
Each change of lexicons (API schemes) may break SDK. This requires model regeneration (this is automated) and a new SDK release on PyPi (this is what i do manually). So, the old versions are dying after changes on the backend.
From one side is good to support only the latest version and keep it updated. Because of the unstable state of the whole protocol. From another side, i understand my users. They want to write 1 simple script/bsky bot, run it, and forget about it, and consume the results. Without fixing it every few weeks.
One of the popular issues is adding new fields to the models. For example #201. Ofc could be problems with data types, etc, but it's too much to ignore. So let's think about how we can fix unknown fields.
@tonycsoka did good research about it in #176
highlight:
Unexpected fields in data which otherwise conforms to the Lexicon should be ignored. When doing schema validation, they should be treated at worst as warnings. This is necessary to allow evolution of the schema by the controlling authority, and to be robust in the case of out-of-date Lexicons.
i want to continue it here
dict
. To be more precise as DotDict. DotDict
exists to unify the codebase of end users to work with SDK models (to be able to use dot notation always).DotDict
for each model instead of Record only. but it leads to problems with types. isinstance
and so on. we have the solution for Records, but for any model idk.allow
extra
fields: https://docs.pydantic.dev/latest/api/config/#pydantic.config.ConfigDict.extra$type
field to understand what is it. but idk will it cover all edge cases?DotDict
is gone. it could serve only for unknown records in the data repositories. Fine?I'm reading from the firehouse using the synchronous client and I am encountering an error when processing a frame. A CBORDecodingError
is raised and the logic when handling this error appears to be to reconnect to the feed. The client continues to read from the feed from the initial cursor and shortly after encounters the same unprocessable frame.
The cursor that the error occurs at is just after 93278360
I'm not sure what the solution should be. Should the client reconnect just after the problematic cursor and continue reading the feed, or is there a deeper issue in processing the data that needs to be resolved here?
Describe the bug
Everything was working fine, up until two days ago, when I got this error, without any change in my code:
Traceback (most recent call last):
File ~\Anaconda3\anaconda3\lib\site-packages\spyder_kernels\py3compat.py:356 in compat_exec
exec(code, globals, locals)
File my_script.py:41
bsky.login(login=bsky_handle, password=bsky_password)
File ~\Anaconda3\anaconda3\lib\site-packages\atproto\xrpc_client\client\client.py:74 in login
session = self._get_and_set_session(login, password)
File ~\Anaconda3\anaconda3\lib\site-packages\atproto\xrpc_client\client\client.py:40 in _get_and_set_session
self._set_session(session)
File ~\Anaconda3\anaconda3\lib\site-packages\atproto\xrpc_client\client\methods_mixin\session.py:68 in _set_session
self._refresh_jwt_payload = get_jwt_payload(session.refresh_jwt)
File ~\Anaconda3\anaconda3\lib\site-packages\atproto\xrpc_client\client\auth.py:18 in get_jwt_payload
return JwtPayload(**plain_payload)
TypeError: __init__() got an unexpected keyword argument 'aud'
To Reproduce
Steps to reproduce the behavior:
from atproto import Client, models
bsky_handle = "mybskyaccount.bsky.social"
bsky_password = "xxxx-xxxx-xxxx-xxxx"
bsky = Client()
bsky.login(login=bsky_handle, password=bsky_password)
Expected behavior
Well, it should be able to connect π
Details
Additional context
Probably related to this change to bluesky-social / atproto which occurred around the same time:
bluesky-social/atproto#1959
When this was working a bit over an hour ago the "seen_at" parameter was documented as optional but including it always generated the error message: 'The seenAt parameter is unsupported'. Removing this parameter allowed app.bsky.notification.list_notifications() to work.
Now the same call generates the following error:
Traceback (most recent call last):
File "/home/max/.local/share/virtualenvs/p20_bs-CytYnhbM/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 87, in _get_or_create
return model(**model_data)
File "/home/max/.local/share/virtualenvs/p20_bs-CytYnhbM/lib/python3.10/site-packages/pydantic/main.py", line 164, in __init__
__pydantic_self__.__pydantic_validator__.validate_python(data, self_instance=__pydantic_self__)
pydantic_core._pydantic_core.ValidationError: 1 validation error for Response
seenAt
Extra inputs are not permitted [type=extra_forbidden, input_value='2023-12-12T21:40:05.894Z', input_type=str]
For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/max/zz_dev/projects/p20_bs/main.py", line 54, in <module>
example_42_appbskynotification_list_notifications()
File "/home/max/zz_dev/projects/p20_bs/example.py", line 509, in example_42_appbskynotification_list_notifications
FEED = client.app.bsky.notification.list_notifications({'limit': limit, 'cursor': cursor})
File "/home/max/.local/share/virtualenvs/p20_bs-CytYnhbM/lib/python3.10/site-packages/atproto/xrpc_client/namespaces/sync_ns.py", line 970, in list_notifications
return get_response_model(response, models.AppBskyNotificationListNotifications.Response)
File "/home/max/.local/share/virtualenvs/p20_bs-CytYnhbM/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 98, in get_response_model
return get_or_create(response.content, model)
File "/home/max/.local/share/virtualenvs/p20_bs-CytYnhbM/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 64, in get_or_create
raise e
File "/home/max/.local/share/virtualenvs/p20_bs-CytYnhbM/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 57, in get_or_create
model_instance = _get_or_create(model_data, model, strict=strict)
File "/home/max/.local/share/virtualenvs/p20_bs-CytYnhbM/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 89, in _get_or_create
raise ModelError(str(e)) from e
atproto.exceptions.ModelError: 1 validation error for Response
seenAt
Extra inputs are not permitted [type=extra_forbidden, input_value='2023-12-12T21:40:05.894Z', input_type=str]
For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
traceback.print_exc()
only works as expected within an except
block, otherwise it prints NoneType: None
which is really difficult to debug. I've been hitting this issue in the wild and don't have a minimal example to actually trigger this line.
atproto/atproto/firehose/client.py
Line 204 in 3a68ade
Using traceback.print_exception(type(exception), exception, exception.__traceback__)
instead gave me some reasonable debugging information.
Running on Python 3.8.10
Haven't actually tested, but the logic in is_record_type
in atproto.xrpc_client.models.utils
will likely fail for dicts:
def is_record_type(model: t.Union[ModelBase, dict], types_module) -> bool:
if isinstance(model, RecordModelBase) and hasattr(types_module, 'Main'):
# for now records in Main. could be broken late
if isinstance(model, dict): # custom record
return types_module.Main._type == model.get('$type')
return types_module.Main._type == model._type
return False
I think it should be:
def is_record_type(model: t.Union[ModelBase, dict], types_module) -> bool:
if isinstance(model, RecordModelBase) and hasattr(types_module, 'Main'):
# for now records in Main. could be broken late
return types_module.Main._type == model._type
if isinstance(model, dict): # custom record
return types_module.Main._type == model.get('$type')
return False
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.