Hi, sorry for the novice question but I’m struggling to work out how to access alt-text for embedded images in posts. How is this done, assuming there may be multiple images’ text to parse per post.

Thanks

https://github.com/MarshalX/atproto/blob/f55fea61550074028cbe2e3b978e020b47d9442e/pyproject.toml#L51C25-L51C25

Is this intended? I am not able to use Python 3.12 which is the latest stable?

The current project's supported Python range (>=3.12,<4.0) is not compatible with some of the required packages Python requirement:
  - atproto requires Python >=3.7.1,<3.13, so it will not be satisfied for Python >=3.13,<4.0

Example: AtUri.from_str("at://100ideas.bsky.social").hostname
returns: at:
should return: 100ideas.bsky.social

Fix: This implementation is based on a REGEX string from bluesky-social and they've updated and changed this string. Making similar change here fixes the problem.

In packages/atproto_core/uri/uri.py change _ATP_URI_REGEX

from: _ATP_URI_REGEX = r'^(at:\/\/)?((?:did:[a-z0-9:%-]+)|(?:[a-z][a-z0-9.:-]*))(\/[^?#\s]*)?(\?[^#\s]+)?(#[^\s]+)?$'
to  : _ATP_URI_REGEX = r'^(at:\/\/)?((?:did:[a-z0-9:%-]+)|(?:[a-z0-9][a-z0-9.:-]*))(\/[^?#\s]*)?(\?[^#\s]+)?(#[^\s]+)?$'

based on ref: https://github.com/bluesky-social/atproto/blob/8c19ce991a766fd9cff5023160853ab1cb106f21/packages/uri/src/index.ts#LL5C38-L5C38

Thanks

Since around midnight last night I have started getting the following error when using atproto:

Traceback (most recent call last):
  File "/[PATH REDACTED]/.local/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 87, in _get_or_create
    return model(**model_data)
  File "/[PATH REDACTED]/.local/lib/python3.10/site-packages/pydantic/main.py", line 165, in __init__
    __pydantic_self__.__pydantic_validator__.validate_python(data, self_instance=__pydantic_self__)
pydantic_core._pydantic_core.ValidationError: 1 validation error for Response
emailConfirmed
  Extra inputs are not permitted [type=extra_forbidden, input_value=False, input_type=bool]
    For further information visit https://errors.pydantic.dev/2.3/v/extra_forbidden

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "[PATH REDACTED]", line 15, in <module>
    bsky.login(bsky_handle, bsky_password)
  File "/[PATH REDACTED]/.local/lib/python3.10/site-packages/atproto/xrpc_client/client/client.py", line 74, in login
    session = self._get_and_set_session(login, password)
  File "/[PATH REDACTED]/.local/lib/python3.10/site-packages/atproto/xrpc_client/client/client.py", line 37, in _get_and_set_session
    session = self.com.atproto.server.create_session(
  File "/[PATH REDACTED]/.local/lib/python3.10/site-packages/atproto/xrpc_client/namespaces/sync_ns.py", line 1757, in create_session
    return get_response_model(response, models.ComAtprotoServerCreateSession.Response)
  File "/[PATH REDACTED]/.local/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 98, in get_response_model
    return get_or_create(response.content, model)
  File "/[PATH REDACTED]/.local/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 64, in get_or_create
    raise e
  File "/[PATH REDACTED]/.local/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 57, in get_or_create
    model_instance = _get_or_create(model_data, model, strict=strict)
  File "/[PATH REDACTED]/.local/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 89, in _get_or_create
    raise ModelError(str(e)) from e
atproto.exceptions.ModelError: 1 validation error for Response
emailConfirmed
  Extra inputs are not permitted [type=extra_forbidden, input_value=False, input_type=bool]
    For further information visit https://errors.pydantic.dev/2.3/v/extra_forbidden

Since no changes have been made on my end, and I haven't downloaded a new version of atproto, I assume something must have changed with the API? Or I'm missing something obvious on my end, also a possibility.

I have started to get the following error whenever I try to post anything at all. I have not made any change in the code but both reposts and normal posts will throw the following error:

BadRequestError(Response(success=False, status_code=400, content=XrpcError(error='InvalidRequest', message='Invalid app.bsky.feed.repost record: createdAt must be an valid atproto datetime (both RFC-3339 and ISO-8601)'), headers=Headers({'date': 'Tue, 05 Dec 2023 01:34:08 GMT', 'content-type': 'application/json; charset=utf-8', 'content-length': '148', 'connection': 'keep-alive', 'x-powered-by': 'Express', 'access-control-allow-origin': '*', 'ratelimit-limit': '5000', 'ratelimit-remaining': '4997', 'ratelimit-reset': '1701743648', 'ratelimit-policy': '5000;w=3600', 'etag': 'W/"94-6peGxeMN8ENNthf4a1NzrR9XOkg"', 'vary': 'Accept-Encoding'})))

I think Bsky has changed something that causes this issue

Provide autogenerated Record Namespaces with more high-level work with basic operations upon records (CRUD + list records)

this is how we work with records now:

self.com.atproto.repo.delete_record(
            models.ComAtprotoRepoDeleteRecord.Data(
                collection=ids.AppBskyFeedPost,
                repo=uri.hostname,
                rkey=uri.rkey,
            )
        )

this is how it could be (.post is autogenerated PostRecordNamespace):

self.bsky.feed.post.delete(repo, rkey)

Would like to atproto use it with fastapi, but getting:

The conflict is caused by:
    The user requested typing_extensions
    atproto 0.0.31 depends on typing-extensions<4.8.0 and >=4.6.1
    fastapi 0.104.1 depends on typing-extensions>=4.8.0

Thanks for your work!

Related to #102

The error suggests to me that the underlying semantic changed out from under this codebase.

Traceback (most recent call last):
  File "/home/temujin9/.local/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 87, in _get_or_create
    return model(**model_data)
  File "/home/temujin9/.local/lib/python3.10/site-packages/pydantic/main.py", line 164, in __init__
    __pydantic_self__.__pydantic_validator__.validate_python(data, self_instance=__pydantic_self__)
pydantic_core._pydantic_core.ValidationError: 8 validation errors for Response
suggestions.0.banner
  Extra inputs are not permitted [type=extra_forbidden, input_value='https://cdn.bsky.app/img...u6kla7qb4fhafdihti@jpeg', input_type=str]
    For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.0.followsCount
  Extra inputs are not permitted [type=extra_forbidden, input_value=84, input_type=int]
    For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.0.followersCount
  Extra inputs are not permitted [type=extra_forbidden, input_value=50, input_type=int]
    For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.0.postsCount
  Extra inputs are not permitted [type=extra_forbidden, input_value=593, input_type=int]
    For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.1.banner
  Extra inputs are not permitted [type=extra_forbidden, input_value='https://cdn.bsky.app/img...7dftfyvh6j46diolje@jpeg', input_type=str]
    For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.1.followsCount
  Extra inputs are not permitted [type=extra_forbidden, input_value=279, input_type=int]
    For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.1.followersCount
  Extra inputs are not permitted [type=extra_forbidden, input_value=381, input_type=int]
    For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.1.postsCount
  Extra inputs are not permitted [type=extra_forbidden, input_value=2529, input_type=int]
    For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/temujin9/Projects/T9_Productions/code/bsky_scripts/suggested_follows", line 24, in <module>
    result = graph_ns.get_suggested_follows_by_actor( params )
  File "/home/temujin9/.local/lib/python3.10/site-packages/atproto/xrpc_client/namespaces/sync_ns.py", line 817, in get_suggested_follows_by_actor
    return get_response_model(response, models.AppBskyGraphGetSuggestedFollowsByActor.Response)
  File "/home/temujin9/.local/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 98, in get_response_model
    return get_or_create(response.content, model)
  File "/home/temujin9/.local/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 64, in get_or_create
    raise e
  File "/home/temujin9/.local/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 57, in get_or_create
    model_instance = _get_or_create(model_data, model, strict=strict)
  File "/home/temujin9/.local/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 89, in _get_or_create
    raise ModelError(str(e)) from e
atproto.exceptions.ModelError: 8 validation errors for Response
suggestions.0.banner
  Extra inputs are not permitted [type=extra_forbidden, input_value='https://cdn.bsky.app/img...u6kla7qb4fhafdihti@jpeg', input_type=str]
    For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.0.followsCount
  Extra inputs are not permitted [type=extra_forbidden, input_value=84, input_type=int]
    For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.0.followersCount
  Extra inputs are not permitted [type=extra_forbidden, input_value=50, input_type=int]
    For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.0.postsCount
  Extra inputs are not permitted [type=extra_forbidden, input_value=593, input_type=int]
    For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.1.banner
  Extra inputs are not permitted [type=extra_forbidden, input_value='https://cdn.bsky.app/img...7dftfyvh6j46diolje@jpeg', input_type=str]
    For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.1.followsCount
  Extra inputs are not permitted [type=extra_forbidden, input_value=279, input_type=int]
    For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.1.followersCount
  Extra inputs are not permitted [type=extra_forbidden, input_value=381, input_type=int]
    For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden
suggestions.1.postsCount
  Extra inputs are not permitted [type=extra_forbidden, input_value=2529, input_type=int]
    For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden

Usage that lead to this was unremarkable:

  params = graph_m.get_suggested_follows_by_actor.Params(actor=profile.did)
  result = graph_ns.get_suggested_follows_by_actor( params )

Thanks for sharing. The mutation is happening in your DotDict validator. If I add some prints, and adjust your __repr__ of DotDict to show it's a DotDict:

        def validate_from_dict(value: dict) -> DotDict:
            print("start", value)
            result = DotDict(value)
            print("result", result)
            print("end", value)
            return result

Then I see that calling DotDict(value) is replacing nested dictionaries inside value with DotDict instances.

The reason why this has only broken with the bump to 2.5 is that on 2.5 the new union behaviour is doing a little bit more work than before to check that DotDict is not a better match than your model instances. On 2.4 the DotDict validation was never run, on 2.5 it is now being run which is why you see the mutation bug arising.

ref: pydantic/pydantic#8349 (comment)

Hello,

Recently, Bluesky added access tokens (#151), and right now, the library allows exporting a session string (client.export_session_string()). This string includes the handle, DID, their access token, and a refresh token.

Previously, the refresh token had been implemented with a 2-month lifetime (and then developer should generate another refresh token with client.login(login=..., password=...)), but now, the CreateSession function returns another refresh token with another 2-month lifetime.

In my codebase, it's something like this:

def app_init():
    # On app init
    client = Client()
    client.login(BLUESKY_USER_HANDLE, BLUESKY_USER_APP_PASSWORD)
    with open(BLUESKY_SECRET_FILE, "w") as f:
        f.write(client.export_session_string())

def post_message(text):
    session_string = "..."  # content from file
    client = Client()
    client.login(session_string=session_string)

    if client._access_jwt and client._should_refresh_session():
        client._refresh_and_set_session()
        # Update file with new session
        with open(BLUESKY_SECRET_FILE, "w") as f:
            f.write(client.export_session_string())

    # client.send_post(...)

My point here is that protected methods are being used to resolve this problem with autorefresh. Is it better to make them public (if the token needs to be invalidated in this way)?

My initial expectation is that methods like client.send_post()/client.com.atproto.repo.create_record()/etc will take care of token invalidation (if needed), but I receive an error, and it looks like invalidation doesn't work for those methods.

now i support backward compatibility and show warnings. but i want to delete it rly soon

• delete subject arg
• make uri and cid args required
• delete compatibility tests

was added here: #255

I am in Central Daylight Time, GMT-5. When calling send_image, it appears in my timeline 5 hours in the past.

send_image calls send_post. It looks like the cause for this bug is in send_post:

in which it calls datetime.now().isoformat(). now() returns the local time zone; atproto uses UTC.

A fix would be to change line 2 to from datetime import datetime, timezone and line 120 to

createdAt=datetime.now(timezone.utc).isoformat(), text=text, reply=reply_to, embed=embed, langs=langs

It looks like like and repost have the same issue, but could be fixed similarly.

Hello @MarshalX , Firstly I have to say you did a great job with this project. I am working on a custom feed using it.

Do you have any idea why I get an error on the app even though the feed was published and these endpoints work on my domain?

- /.well-known/did.json
- /xrpc/app.bsky.feed.describeFeedGenerator
- /xrpc/app.bsky.feed.getFeedSkeleton

Although a similar issue was raised here on the official bluesky repo - bluesky-social/feed-generator#35 but I could really use a second eye.

my thoughts about impl: #93 (comment)

Hello there,

the inline example in atproto/car/init.py stopped working last week.

The line repo = client.com.atproto.sync.get_repo({'did': client.me.did})

throw an atproto.exceptions.BadRequestError:

Response(success=False, status_code=400, content=XrpcError(error='InvalidRequest', message='Could not find repo for DID: '), headers=Headers({'date': 'Mon, 20 Nov 2023 14:09:40 GMT', 'content-type': 'application/json; charset=utf-8', 'content-length': '100', 'connection': 'keep-alive', 'x-powered-by': 'Express', 'access-control-allow-origin': '*', 'ratelimit-limit': '3000', 'ratelimit-remaining': '2998', 'ratelimit-reset': '1700489680', 'ratelimit-policy': '3000;w=300', 'etag': 'W/"64-u+z2SVfRs2zfn34PuZ9u5A4P55k"', 'vary': 'Accept-Encoding'}))

I could verify the correctness of the DID in other ways.

Additional information: the more or less 'official' indigo-implementation seems to have a similar issue. Did they change the handling in the protocol?

Thanks for looking into it.
Harald

Looks like httpcore changed the location of httpcore.backends in v0.17.3

traceback.print_exc() only works as expected within an except block, otherwise it prints NoneType: None which is really difficult to debug. I've been hitting this issue in the wild and don't have a minimal example to actually trigger this line.

Using traceback.print_exception(type(exception), exception, exception.__traceback__) instead gave me some reasonable debugging information.

Running on Python 3.8.10

Rewrite this example using our SDK models: https://gist.github.com/DavidBuchanan314/f92b3925c5480e35076773613cb87bdd

All our examples are under CC0. The gist above was originally MIT, but the author allowed to relicense it to CC0:

After transition to pydantic I am no longer able to fetch threads. It might very well be a user error, but I can't see where.

Running:
bsky.app.bsky.feed.get_post_thread(params={"uri": uri})

and getting response:

 File "/home/linus/.local/lib/python3.8/site-packages/atproto/xrpc_client/namespaces/sync_ns.py", line 403, in get_post_thread      
    return get_response_model(response, models.AppBskyFeedGetPostThread.Response)
  File "/home/linus/.local/lib/python3.8/site-packages/atproto/xrpc_client/models/utils.py", line 98, in get_response_model
    return get_or_create(response.content, model)
  File "/home/linus/.local/lib/python3.8/site-packages/atproto/xrpc_client/models/utils.py", line 64, in get_or_create
    raise e
  File "/home/linus/.local/lib/python3.8/site-packages/atproto/xrpc_client/models/utils.py", line 57, in get_or_create
    model_instance = _get_or_create(model_data, model, strict=strict)
  File "/home/linus/.local/lib/python3.8/site-packages/atproto/xrpc_client/models/utils.py", line 89, in _get_or_create
    raise ModelError(str(e)) from e
atproto.exceptions.ModelError: 1 validation error for Response
thread.`app.bsky.feed.defs#threadViewPost`.viewer
  Extra inputs are not permitted [type=extra_forbidden, input_value={'canReply': True}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.3/v/extra_forbidden

I'm not sure what "extra inputs" it's receiving.

Hi. Please help.
What is wrong with this code?
I get an 'AuthenticationRequired' error when I try it:

new_email = '[email protected]'
client = Client()
profile = client.login('handle', 'password')
res = client.com.atproto.admin.update_account_email(
    models.ComAtprotoAdminUpdateAccountEmail.Data(
        account=profile.did,
        email=new_email)
)

Every time I try to manually run gunicorn against my test feed, it dies about 30 seconds.

I'm not sure if a standardized gunicorn config file would work for every feed, or the configuration would really be dependent on whatever a specific feed is doing.

If I go ahead and deploy it to Fly with the Dockerfile using "flask run" as the CMD though it works, I'd just prefer to make it work with gunicorn.

This is my stub of a gunicorn config file:

workers = 2
# threads = 4
# timeout = 120
bind = '0.0.0.0:8080'

# forwarded_allow_ips = '*'
# secure_scheme_headers = { 'X-Forwarded-Proto': 'https' }

Please see the interaction with a bluesky dev here:
"Hang on to that access token! It's good for 2 hrs!"
https://bsky.app/profile/dholms.xyz/post/3k7ieei53a326

Your CLIENT.login provides no token, and without it (or a way to use it to login instead of using CLIENT.login), my script is essentially broken.

Thank you!
Nick

Discussed in #119

Hey there! Thanks so much for creating this package. I've just started using it and it is working great for me.

This issue is to request the ability to include labels when submitting posts. From what I can tell, labels are used to apply content warnings on posts when submitting on https://bsky.app/.

Please let me know if this is already possible and I just missed it!

Here's what the request looks like when submitting via bsky.app and including a content warning:

{
    "collection": "app.bsky.feed.post",
    "record": {
        "$type": "app.bsky.feed.post",
        "createdAt": "2024-01-13T16:28:19.938Z",
        "embed": {
            "$type": "app.bsky.embed.images",
            "images": [
                {
                    "alt": "",
                    "aspectRatio": {
                        "height": 626,
                        "width": 640
                    },
                    "image": {
                        "$type": "blob",
                        "mimeType": "image/jpeg",
                        "ref": {
                            "$link": "bafkreichm4wf4ru2omi2v7iefx5f63kejfycs5ggnltf4g6zxyn3or3kxe"
                        },
                        "size": 299493
                    }
                }
            ]
        },
        "labels": {
            "$type": "com.atproto.label.defs#selfLabels",
            "values": [
                {
                    "val": "sexual"
                }
            ]
        },
        "langs": [
            "en"
        ],
        "text": "Test content warning"
    },
    "repo": "did:plc:b76akgyhvhtykcp7j7jsehrc"
}

The regexp in extract_url_byte_positions at https://github.com/MarshalX/atproto/blob/main/examples/advanced_usage/auto_hyperlinks.py does not appear to detect all valid URLs. Take for example:

https://www.cell.com/matter/fulltext/S2590-2385(23)00409-5?rss=yes

This is misdetected, and the URL is stopped before (

was added here: #264

• remove backward compatibility from codegen
• remove backward compatibility tests
• remove backward compatibility from is_record_type

Now the records in the responses are plain dicts. We need to parse it as Record Models. The models are generated already.

It doesn't work now because in the lexicons all records are marked as "unknown" instead of the reference to the record model.

For now this fields are typehinted as "Any":
https://github.com/MarshalX/atproto/blob/0826a31a3668eec9d85f79c3e9861ed394ab1165/atproto/codegen/models/generator.py#L156C1-L158

Many other packages depend on typing-extensions>=4.6.1.
Please consider supporting >=4.6.0 to avoid dependency conflicts.

Replit: Updating package configuration

--> poetry add 'atproto 0.0.17'

Updating dependencies
Resolving dependencies...

SolverProblemError

Because no versions of replit match >3.2.4,<3.2.5 || >3.2.5,<3.2.6 || >3.2.6,<3.2.7 || >3.2.7,<3.2.8 || >3.2.8,<3.3.0 || >3.3.0,<3.3.1 || >3.3.1,<4.0.0
and replit (3.2.4) depends on typing_extensions (>=3.7.4,<4.0.0), replit (>=3.2.4,<3.2.5 || >3.2.5,<3.2.6 || >3.2.6,<3.2.7 || >3.2.7,<3.2.8 || >3.2.8,<3.3.0 || >3.3.0,<3.3.1 || >3.3.1,<4.0.0) requires typing_extensions (>=3.7.4,<4.0.0).
And because replit (3.2.6) depends on typing_extensions (>=3.7.4,<4.0.0), replit (>=3.2.4,<3.2.5 || >3.2.5,<3.2.7 || >3.2.7,<3.2.8 || >3.2.8,<3.3.0 || >3.3.0,<3.3.1 || >3.3.1,<4.0.0) requires typing_extensions (>=3.7.4,<4.0.0).
And because replit (3.2.7) depends on typing_extensions (>=3.7.4,<4.0.0)
and replit (3.2.8) depends on typing_extensions (>=3.7.4,<4.0.0), replit (>=3.2.4,<3.2.5 || >3.2.5,<3.3.0 || >3.3.0,<3.3.1 || >3.3.1,<4.0.0) requires typing_extensions (>=3.7.4,<4.0.0).
And because replit (3.3.0) depends on typing_extensions (>=3.7.4,<4.0.0)
and replit (3.3.1) depends on typing_extensions (>=3.7.4,<4.0.0), replit (>=3.2.4,<3.2.5 || >3.2.5,<4.0.0) requires typing_extensions (>=3.7.4,<4.0.0).
And because replit (3.2.5) depends on typing_extensions (>=3.7.4,<4.0.0)
and atproto (0.0.17) depends on typing-extensions (>=4.5.0,<4.6.0), atproto (0.0.17) is incompatible with replit (>=3.2.4,<4.0.0).
So, because python-template depends on both replit (^3.2.4) and atproto (0.0.17), version solving failed.

at venv/lib/python3.10/site-packages/poetry/puzzle/solver.py:241 in _solve
237│ packages = result.packages
238│ except OverrideNeeded as e:
239│ return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
240│ except SolveFailure as e:
→ 241│ raise SolverProblemError(e)
242│
243│ results = dict(
244│ depth_first_search(
245│ PackageNode(self._package, packages), aggregate_package_nodes
exit status 1

Replit: Package operation failed.

As of today, I switched the astronomy feeds FirehoseSubscribeReposClient to use the new BGS at wss://bsky.network/xrpc. I am getting intermittent errors on a very small fraction of posts when atproto.CAR.from_bytes is called on certain commits. This happens once every ~10 minutes (so must be caused by only a small fraction of posts); otherwise, the feed is running fine.

This is on Ubuntu 20.04, with Python 3.11.4 and atproto 0.0.30. This is the commit in the GitHub repo where issues started happening.

Stack trace of the error:

[firehose] [2023-11-10 16:43:23] thread '<unnamed>' panicked at src/lib.rs:164:67:
[firehose] [2023-11-10 16:43:23] called `Result::unwrap()` on an `Err` value: Parsing("failed to parse uvarint for header")
[firehose] [2023-11-10 16:43:23] note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
[firehose] [2023-11-10 16:43:23] Process Process-1:
[firehose] [2023-11-10 16:43:23] Traceback (most recent call last):
[firehose] [2023-11-10 16:43:23]   File "/workspace/.heroku/python/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
[firehose] [2023-11-10 16:43:23]     self.run()
[firehose] [2023-11-10 16:43:23]   File "/workspace/.heroku/python/lib/python3.11/multiprocessing/process.py", line 108, in run
[firehose] [2023-11-10 16:43:23]     self._target(*self._args, **self._kwargs)
[firehose] [2023-11-10 16:43:23]   File "/workspace/server/firehose.py", line 99, in worker_main
[firehose] [2023-11-10 16:43:23]     _worker_loop(receiver)
[firehose] [2023-11-10 16:43:23]   File "/workspace/server/firehose.py", line 85, in _worker_loop
[firehose] [2023-11-10 16:43:23]     operations_callback(_get_ops_by_type(commit))
[firehose] [2023-11-10 16:43:23]                         ^^^^^^^^^^^^^^^^^^^^^^^^
[firehose] [2023-11-10 16:43:23]   File "/workspace/server/firehose.py", line 30, in _get_ops_by_type
[firehose] [2023-11-10 16:43:23]     car = CAR.from_bytes(commit.blocks)
[firehose] [2023-11-10 16:43:23]           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[firehose] [2023-11-10 16:43:23]   File "/workspace/.heroku/python/lib/python3.11/site-packages/atproto/car/__init__.py", line 51, in from_bytes
[firehose] [2023-11-10 16:43:23]     header, blocks = libipld.decode_car(data)
[firehose] [2023-11-10 16:43:23]                      ^^^^^^^^^^^^^^^^^^^^^^^^
[firehose] [2023-11-10 16:43:23] pyo3_runtime.PanicException: called `Result::unwrap()` on an `Err` value: Parsing("failed to parse uvarint for header")

I've added more logging on the feed and set RUST_BACKTRACE=1. Will update this issue if I can work out which commits are causing the problem.

It happens rarely. About 1 frame per few hours

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/dag_cbor/decoding/__init__.py", line 269, in _decode_dict
    v, _ = _decode_item(stream, options)
  File "/usr/local/lib/python3.8/dist-packages/dag_cbor/decoding/__init__.py", line 149, in _decode_item
    value, num_bytes_further_read = _decoders[major_type](stream, arg, options)
  File "/usr/local/lib/python3.8/dist-packages/dag_cbor/decoding/__init__.py", line 208, in _decode_bytes
    raise CBORDecodingError(_err._unexpected_eof(stream, f"{length} bytes of bytestring", length))
dag_cbor.decoding.err.CBORDecodingError: Unexpected EOF while attempting to read 1047 bytes of bytestring.
At byte #248: 04...75 (last byte #993)
              ^^^^^^^ 744 bytes read, out of 1047 expected.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/atproto/firehose/client.py", line 175, in start
    self._process_raw_frame(raw_frame)
  File "/usr/local/lib/python3.8/dist-packages/atproto/firehose/client.py", line 101, in _process_raw_frame
    frame = Frame.from_bytes(data)
  File "/usr/local/lib/python3.8/dist-packages/atproto/firehose/models.py", line 114, in from_bytes
    decoded_parts = decode_dag_multi(data)
  File "/usr/local/lib/python3.8/dist-packages/atproto/cbor/__init__.py", line 54, in decode_dag_multi
    decoded_part = decode_dag(data, allow_concat=True, callback=bytes_read_counter)
  File "/usr/local/lib/python3.8/dist-packages/atproto/cbor/__init__.py", line 30, in decode_dag
    return _dag_cbor.decode(data, allow_concat=allow_concat, callback=callback)
  File "/usr/local/lib/python3.8/dist-packages/dag_cbor/decoding/__init__.py", line 127, in decode
    data, _ = _decode_item(stream, options)
  File "/usr/local/lib/python3.8/dist-packages/dag_cbor/decoding/__init__.py", line 149, in _decode_item
    value, num_bytes_further_read = _decoders[major_type](stream, arg, options)
  File "/usr/local/lib/python3.8/dist-packages/dag_cbor/decoding/__init__.py", line 271, in _decode_dict
    raise CBORDecodingError(_err._dict_item(dict_head_snapshot, "value", i, length, e)) # pylint: disable = raise-missing-from
dag_cbor.decoding.err.CBORDecodingError: Error while decoding dict.
At byte #0: aa...
            ^^ dict of length 10
Error occurred while decoding value at position 6: further details below.
\ Unexpected EOF while attempting to read 1047 bytes of bytestring.
  At byte #248: 04...75 (last byte #993)
                ^^^^^^^ 744 bytes read, out of 1047 expected.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/marshal/bluesky-feed-generator/src/data_stream.py", line 87, in run
    client.start(on_message_handler)
  File "/usr/local/lib/python3.8/dist-packages/atproto/firehose/client.py", line 179, in start
    should_stop = _handle_firehose_error_or_stop(e)
  File "/usr/local/lib/python3.8/dist-packages/atproto/firehose/client.py", line 63, in _handle_firehose_error_or_stop
    raise FirehoseError from exception
atproto.exceptions.FirehoseError

Hi MarshalX.
Strange new error since this morning:

from atproto import Client, models, exceptions

client = Client()
profile = client.login('handle', 'pwd')
try:
    res = client.com.atproto.server.get_account_invite_codes(
        models.ComAtprotoServerGetAccountInviteCodes.Params(
            createAvailable=True,
            includeUsed=True)
    )
except exceptions.BadRequestError as err:
    print(err)

It throws an error with status code 400
'InvalidToken'
'Token could not be verified'

I looked at https://atproto.blue/en/latest/namespace.html#atproto.xrpc_client.namespaces.sync_ns.ServerNamespace.get_account_invite_codes again, there is nothing about a token?

What am I missing?
Please help.

Hey @MarshalX (and others contributing to this repo)!

Wanted to share a resource and see if you had any feedback. We have started collecting some cross-language test vectors at: https://github.com/bluesky-social/atproto/tree/main/interop-test-files

For example, long lists of valid and invalid identifiers, trying to hit corner cases. These are intended to be easy to copy directly in to other implementations. So far they are used in both the typescript code and golang implementation (indigo).

Would be curious what other test files would be helpful for ensuring inter-operation between implementations, particularly if there have been any sharp edges that you have run in to.

We will probably add:

• JSON records covering all the app.bsky record Lexicons, both valid and invalid
• CAR files with repository content, both valid and invalid
• DID documents (valid and invalid)

Hi. After recent update no invite history for most accounts.
Returns codes=[]

res = client.com.atproto.server.get_account_invite_codes(
models.ComAtprotoServerGetAccountInviteCodes.Params(
createAvailable=True,
includeUsed=True)
)

I'm running into an issue with auth token handling that hopefully I can explain.

My understanding of Bluesky auth

1. A user handle + app password creates a new session, which provides an access token and refresh token.
2. The access token expires in 2 hours. To refresh it you use the refresh token.
3. The refresh token expires in 2 months. To refresh it you use the handle + app password to create a new session again.

How atproto handles this

atproto handles scenario 1 and 2, but does not handle scenario 3. _should_refresh_session will check if the access token is nearly expired, and if so, will refresh the access token using the refresh token. However, atproto does not check if the refresh token is expired and handle refreshing it.

Right now Client.login() accepts either the handle/password or the session string, but not both. If the session string is passed then the handle/password are ignored.

My request

1. Change auth to handle both access token refreshing and refresh token refreshing.
   2. That will mean checking that both tokens are not expired. If the access token is expired then it is refreshed with the refresh token. If the refresh token is expired then it is refreshed using the username/password via CreateSession (assuming that the handle/password are passed into Client.login).
2. Add some sort of callback functionality when a token is refreshed by _refresh_and_set_session. This will allow me to know when the session string has changed so I can store it in the database and use it in the future when creating new Clients. Otherwise I have to call export_session_string after every atproto method because I don't know which one may have created a new session.

Now recently there has started to appear the following issue

Traceback (most recent call last):
  File "/home/pi/bots/posters/crossposter/blueskypost.py", line 80, in <module>
    client._set_session(ses)
  File "/home/pi/.local/lib/python3.9/site-packages/atproto/xrpc_client/client/methods_mixin/session.py", line 68, in _set_session
    self._refresh_jwt_payload = get_jwt_payload(session.refresh_jwt)
  File "/home/pi/.local/lib/python3.9/site-packages/atproto/xrpc_client/client/auth.py", line 18, in get_jwt_payload
    return JwtPayload(**plain_payload)
TypeError: __init__() got an unexpected keyword argument 'aud'
Traceback (most recent call last):
  File "/home/pi/bots/posters/crossposter/blueskypost.py", line 80, in <module>
    client._set_session(ses)
  File "/home/pi/.local/lib/python3.9/site-packages/atproto/xrpc_client/client/methods_mixin/session.py", line 68, in _set_session
    self._refresh_jwt_payload = get_jwt_payload(session.refresh_jwt)
  File "/home/pi/.local/lib/python3.9/site-packages/atproto/xrpc_client/client/auth.py", line 18, in get_jwt_payload
    return JwtPayload(**plain_payload)
TypeError: __init__() got an unexpected keyword argument 'aud'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/pi/bots/posters/crossposter/blueskypost.py", line 88, in <module>
    client.login(username, password)
  File "/home/pi/.local/lib/python3.9/site-packages/atproto/xrpc_client/client/client.py", line 74, in login
  File "/home/pi/.local/lib/python3.9/site-packages/atproto/xrpc_client/client/client.py", line 40, in _get_and_set_session
    session = self._get_and_set_session(login, password)
    self._set_session(session)
  File "/home/pi/.local/lib/python3.9/site-packages/atproto/xrpc_client/client/methods_mixin/session.py", line 68, in _set_session
    self._refresh_jwt_payload = get_jwt_payload(session.refresh_jwt)
  File "/home/pi/.local/lib/python3.9/site-packages/atproto/xrpc_client/client/auth.py", line 18, in get_jwt_payload
    return JwtPayload(**plain_payload)
TypeError: __init__() got an unexpected keyword argument 'aud'

I think it has to do something with the JwtPayload

When using listRecords, if a post is using custom records (like "via" for example), the SDK returns the original dict which is not fully compatible with the proper instance because of the snake/camelCase notation difference on some fields (createdAt, indexedAt, likeCount...).

See original discussion here:
https://bsky.app/profile/mwyann.fr/post/3k757t5scw62l

Maybe other methods are affected too, but it only happened to me with this method anyway.

Describe the bug

Everything was working fine, up until two days ago, when I got this error, without any change in my code:

Traceback (most recent call last):

  File ~\Anaconda3\anaconda3\lib\site-packages\spyder_kernels\py3compat.py:356 in compat_exec
    exec(code, globals, locals)

  File my_script.py:41
    bsky.login(login=bsky_handle, password=bsky_password)

  File ~\Anaconda3\anaconda3\lib\site-packages\atproto\xrpc_client\client\client.py:74 in login
    session = self._get_and_set_session(login, password)

  File ~\Anaconda3\anaconda3\lib\site-packages\atproto\xrpc_client\client\client.py:40 in _get_and_set_session
    self._set_session(session)

  File ~\Anaconda3\anaconda3\lib\site-packages\atproto\xrpc_client\client\methods_mixin\session.py:68 in _set_session
    self._refresh_jwt_payload = get_jwt_payload(session.refresh_jwt)

  File ~\Anaconda3\anaconda3\lib\site-packages\atproto\xrpc_client\client\auth.py:18 in get_jwt_payload
    return JwtPayload(**plain_payload)

TypeError: __init__() got an unexpected keyword argument 'aud'

To Reproduce

Steps to reproduce the behavior:

from atproto import Client, models
bsky_handle = "mybskyaccount.bsky.social"
bsky_password = "xxxx-xxxx-xxxx-xxxx"
bsky = Client()
bsky.login(login=bsky_handle, password=bsky_password)

Expected behavior

Well, it should be able to connect 😄

Details

• Operating system: Windows 10
• Node version: no idea?

Additional context

• I really don't know, it was working fine, until it wasn't anymore, without making any changes to my python setup, to my script or to my bluesky account.
• I tried updating, but I have atproto v 0.0.29 already.
• I tried using another app password, same error.
• Neither my account nor my password contain "aud".

Hello!

I am not able to fetch some user profiles — looks like this issue happens on profiles where author defined language of "skeets". For example you can check it with "uavibes.bsky.social" profile.

def main():
    client = Client()
    client.login(...)

    profile_feed = client.bsky.feed.get_author_feed({'actor': 'uavibes.bsky.social'})
    for feed_view in profile_feed.feed:
        print('-', feed_view.post.record.text)

Traceback (most recent call last):
  File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/models/utils.py", line 101, in _get_or_create
    model(**model_data)
TypeError: Main.__init__() got an unexpected keyword argument 'langs'


Traceback (most recent call last):
  File "/Users/vadym/Projects/vadym/uavibes/main.py", line 14, in <module>
    main()
  File "/Users/vadym/Projects/vadym/uavibes/main.py", line 8, in main
    profile_feed = client.bsky.feed.get_author_feed({'actor': 'uavibes.bsky.social'})
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/namespaces/sync_ns.py", line 249, in get_author_feed
    return get_response_model(response, models.AppBskyFeedGetAuthorFeed.Response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/models/utils.py", line 132, in get_response_model
    return get_or_create_model(response.content, model)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/models/utils.py", line 119, in get_or_create_model
    model_instance = get_or_create(model_data, model)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/models/utils.py", line 78, in get_or_create
    raise e
  File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/models/utils.py", line 75, in get_or_create
    return _get_or_create(model_data, model, strict=strict)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/models/utils.py", line 103, in _get_or_create
    return from_dict(model, model_data, config=_DACITE_CONFIG)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/dacite/core.py", line 64, in from_dict
    value = _build_value(type_=field_type, data=field_data, config=config)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/dacite/core.py", line 97, in _build_value
    data = _build_value_for_collection(collection=type_, data=data, config=config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/dacite/core.py", line 154, in _build_value_for_collection
    return data_type(_build_value(type_=item_type, data=item, config=config) for item in data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/dacite/core.py", line 154, in <genexpr>
    return data_type(_build_value(type_=item_type, data=item, config=config) for item in data)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/dacite/core.py", line 99, in _build_value
    data = from_dict(data_class=type_, data=data, config=config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/dacite/core.py", line 64, in from_dict
    value = _build_value(type_=field_type, data=field_data, config=config)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/dacite/core.py", line 99, in _build_value
    data = from_dict(data_class=type_, data=data, config=config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/dacite/core.py", line 64, in from_dict
    value = _build_value(type_=field_type, data=field_data, config=config)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/dacite/core.py", line 91, in _build_value
    data = config.type_hooks[type_](data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/models/utils.py", line 31, in _record_model_type_hook
    return get_or_create_model(data, RECORD_TYPE_TO_MODEL_CLASS[record_type])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/models/utils.py", line 119, in get_or_create_model
    model_instance = get_or_create(model_data, model)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/models/utils.py", line 78, in get_or_create
    raise e
  File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/models/utils.py", line 75, in get_or_create
    return _get_or_create(model_data, model, strict=strict)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vadym/Projects/vadym/uavibes/venv/lib/python3.11/site-packages/atproto/xrpc_client/models/utils.py", line 107, in _get_or_create
    raise UnexpectedFieldError(msg) from e
atproto.exceptions.UnexpectedFieldError: Main.Main got an unexpected keyword argument 'langs'

When Main class is extended with langs: t.Optional[t.List[str]] = None field – it works fine. However, I can't find this langs field in atproto docs, so that's why I didn't create PR for this change yet.

Thanks for all the work on this library!! 🎉

I've been using it to run a bsky bot that posts twice daily and everything had been going smoothly until yesterday, when importing atproto started failing with the following Pydantic error:

  File "/home/runner/work/bsky-nycasp/bsky-nycasp/bsky_nycasp.py", line 6, in <module>
    from atproto import Client
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/atproto/__init__.py", line 3, in <module>
    from .firehose import models as firehose_models
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/atproto/firehose/__init__.py", line 3, in <module>
    from atproto.firehose.client import AsyncFirehoseClient, FirehoseClient
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/atproto/firehose/client.py", line 21, in <module>
    from atproto.firehose.models import ErrorFrame, Frame, MessageFrame
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/atproto/firehose/models.py", line 7, in <module>
    from atproto.xrpc_client.models.utils import get_or_create
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/atproto/xrpc_client/models/__init__.py", line 265, in <module>
    load_models()
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/atproto/xrpc_client/models/models_loader.py", line 58, in load_models
    __on_load()
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/atproto/xrpc_client/models/models_loader.py", line 54, in __on_load
    __rebuild_all_models()
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/atproto/xrpc_client/models/models_loader.py", line 50, in __rebuild_all_models
    __model.model_rebuild()
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/pydantic/main.py", line 470, in model_rebuild
    return _model_construction.complete_model_class(
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/pydantic/_internal/_model_construction.py", line 498, in complete_model_class
    cls.__pydantic_core_schema__ = schema = validate_core_schema(schema)
  File "/opt/hostedtoolcache/Python/3.10.13/x64/lib/python3.10/site-packages/pydantic/_internal/_core_utils.py", line 631, in validate_core_schema
    return _validate_core_schema(schema)
pydantic_core._pydantic_core.SchemaError: Invalid Schema:
definitions.definitions.48.model.schema.model-fields.fields.parent.schema.default.schema.nullable.schema.default.schema.tagged-union.choices.`app.bsky.feed.defs#threadViewPost`
  Recursion error - cyclic reference detected [type=recursion_loop, input_value={'type': 'model', 'cls': ...iewPost:94259874186816'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.4/v/recursion_loop
definitions.definitions.48.model.schema.model-fields.fields.replies.schema.default.schema.nullable.schema.list.items_schema.tagged-union.choices.`app.bsky.feed.defs#threadViewPost`
  Recursion error - cyclic reference detected [type=recursion_loop, input_value={'type': 'model', 'cls': ...iewPost:94259874186816'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.4/v/recursion_loop

I've tested v0.0.26 and v0.0.28 and both seem to consistently fail. FWIW, my code runs on GitHub Actions with a fresh environment and installation of atproto every day.

I'm not totally sure how to diagnose or debug this issue, so any pointers would be much appreciated! Thanks!

Problem

Each change of lexicons (API schemes) may break SDK. This requires model regeneration (this is automated) and a new SDK release on PyPi (this is what i do manually). So, the old versions are dying after changes on the backend.

From one side is good to support only the latest version and keep it updated. Because of the unstable state of the whole protocol. From another side, i understand my users. They want to write 1 simple script/bsky bot, run it, and forget about it, and consume the results. Without fixing it every few weeks.

One of the popular issues is adding new fields to the models. For example #201. Ofc could be problems with data types, etc, but it's too much to ignore. So let's think about how we can fix unknown fields.

@tonycsoka did good research about it in #176
highlight:

Unexpected fields in data which otherwise conforms to the Lexicon should be ignored. When doing schema validation, they should be treated at worst as warnings. This is necessary to allow evolution of the schema by the controlling authority, and to be robust in the case of out-of-date Lexicons.

i want to continue it here

Notes

• for record types, for example, Post, we allow unknown fields because of the architecture of the protocol itself. But, we are parsing such "extended" records as dict. To be more precise as DotDict. DotDict exists to unify the codebase of end users to work with SDK models (to be able to use dot notation always).
• we can think about applying DotDict for each model instead of Record only. but it leads to problems with types. isinstance and so on. we have the solution for Records, but for any model idk.
• we don't want to just ignore unknown fields when parsing models. we should provide access to it. because we a not an official bsky application, we want to have access to raw protocol and extra data that it could have, not only that we possibly need for "rendering". SDK should be a tool for scripting and doing random stuff as well.
• we want a solution that could be roll-backed to the strict mode easily
• bsky team doesn't have a 100% answer for the future. read https://atproto.com/specs/lexicon#possible-future-changes

Solution

1. we are using pydantic. so the possible solution is just to allow extra fields: https://docs.pydantic.dev/latest/api/config/#pydantic.config.ConfigDict.extra
2. keep strict mode and just provide user-friendly exception like "pls update your SDK"; make updating of the lexicon in SDK more automatic (one approve click from SDK maintainers; we can setup CI to create pull requests and do new releases)

Open questions for solution No. 1

• as i understand this is a temporal problem until a stable version of the protocol. because later they will care about versioning lexicons, etc.. so should we allow it now to disallow it in the future?
• I'm afraid of type checks, especially in Unions. i try to use tagged unions whenever possible. Lexicon provides $type field to understand what is it. but idk will it cover all edge cases?
• after allowing extra fields the purpose of DotDict is gone. it could serve only for unknown records in the data repositories. Fine?
• will it occur undefined behaviors from old versions? when users will not understand what the magic is happening. now SDK raises exception SDK about forbidden extra fields and users come to GitHub and check if is there a new SDK version that fixes it.
• should we add warnings to the log about extra fields? "maybe you are using outdated SDK because '...' field comes from the backend and doesn't exist in the local described model". These warnings should be enabled/disabled for the user

When I try to get user's followers with the given code:

from atproto import Client, models

client = Client()
client.login('some_handle', 'app_password')

params = models.AppBskyGraphGetFollowers.Params(actor='some_actor')
client.bsky.graph.get_followers(params=params)

I get the following error:

atproto.exceptions.WrongTypeError: wrong value type for field "followers.viewer.mutedByList.purpose" - should be "Literal" instead of value "app.bsky.graph.defs#modlist" of type "str"

The underlying cause of this error is config.check_types procedure:

dacite.exceptions.WrongTypeError: wrong value type for field "followers.viewer.mutedByList.purpose" - should be "Literal" instead of value "app.bsky.graph.defs#modlist" of type "str"

If type checking (lines 68-69 here) is commented out, list of followers is successfully received.

atproto version is 0.0.24

Probably related to this change to bluesky-social / atproto which occurred around the same time:
bluesky-social/atproto#1959

When this was working a bit over an hour ago the "seen_at" parameter was documented as optional but including it always generated the error message: 'The seenAt parameter is unsupported'. Removing this parameter allowed app.bsky.notification.list_notifications() to work.

Now the same call generates the following error:

Traceback (most recent call last):
  File "/home/max/.local/share/virtualenvs/p20_bs-CytYnhbM/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 87, in _get_or_create
    return model(**model_data)
  File "/home/max/.local/share/virtualenvs/p20_bs-CytYnhbM/lib/python3.10/site-packages/pydantic/main.py", line 164, in __init__
    __pydantic_self__.__pydantic_validator__.validate_python(data, self_instance=__pydantic_self__)
pydantic_core._pydantic_core.ValidationError: 1 validation error for Response
seenAt
  Extra inputs are not permitted [type=extra_forbidden, input_value='2023-12-12T21:40:05.894Z', input_type=str]
    For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/max/zz_dev/projects/p20_bs/main.py", line 54, in <module>
    example_42_appbskynotification_list_notifications()
  File "/home/max/zz_dev/projects/p20_bs/example.py", line 509, in example_42_appbskynotification_list_notifications
    FEED = client.app.bsky.notification.list_notifications({'limit': limit, 'cursor': cursor})
  File "/home/max/.local/share/virtualenvs/p20_bs-CytYnhbM/lib/python3.10/site-packages/atproto/xrpc_client/namespaces/sync_ns.py", line 970, in list_notifications
    return get_response_model(response, models.AppBskyNotificationListNotifications.Response)
  File "/home/max/.local/share/virtualenvs/p20_bs-CytYnhbM/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 98, in get_response_model
    return get_or_create(response.content, model)
  File "/home/max/.local/share/virtualenvs/p20_bs-CytYnhbM/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 64, in get_or_create
    raise e
  File "/home/max/.local/share/virtualenvs/p20_bs-CytYnhbM/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 57, in get_or_create
    model_instance = _get_or_create(model_data, model, strict=strict)
  File "/home/max/.local/share/virtualenvs/p20_bs-CytYnhbM/lib/python3.10/site-packages/atproto/xrpc_client/models/utils.py", line 89, in _get_or_create
    raise ModelError(str(e)) from e
atproto.exceptions.ModelError: 1 validation error for Response
seenAt
  Extra inputs are not permitted [type=extra_forbidden, input_value='2023-12-12T21:40:05.894Z', input_type=str]
    For further information visit https://errors.pydantic.dev/2.5/v/extra_forbidden

Would it be possible to utilise TypeGuard on this function in order to leverage the return value at the callsite for static type checking?

I'd do it myself but I don't feel I understand enough about the details of the library at this point to get it right

Hi,

I want to automatically associate an image with a preview card inside a publication and so I tried:

`

blob = bsky.com.atproto.repo.upload_blob(image_bytes)
pprint(blob)
blob.blob.mime_type = 'image/*'  

embed_external = models.AppBskyEmbedExternal.Main(
    external=models.AppBskyEmbedExternal.External(
        title=preview_data.title,
        description=preview_data.description,
        uri=url,
        thumb=blob.blob,
    )
)

`
with the following log:

2023-12-18 16:42:47,328 HTTP Request: POST https://bsky.social/xrpc/com.atproto.repo.uploadBlob "HTTP/1.1 200 OK"
Response(blob=BlobRef(mime_type='*/*', size=2483, ref=BlobRefLink(link='bafkreihg7iiatouv6hiqb7t5uharq3ikgcfd2ejbspw3bhxqp3led7xjea'), py_type='blob'))
2023-12-18 16:42:47,456 HTTP Request: POST https://bsky.social/xrpc/com.atproto.repo.createRecord "HTTP/1.1 400 Bad Request"
2023-12-18 16:42:47,457 Response(success=False, status_code=400, content=XrpcError(error='InvalidMimeType', message='Wrong type of file. It is */* but it must match image/*.'), headers=Headers({'date': 'Mon, 18 Dec 2023 16:42:46 GMT', 'content-type': 'application/json; charset=utf-8', 'content-length': '96', 'connection': 'keep-alive', 'x-powered-by': 'Express', 'access-control-allow-origin': '*', 'ratelimit-limit': '5000', 'ratelimit-remaining': '4961', 'ratelimit-reset': '1702921208', 'ratelimit-policy': '5000;w=3600', 'etag': 'W/"60-naTgCWHhjZ5FGkdXs552SxW2Nug"', 'vary': 'Accept-Encoding'}))

Apparently, the image is uploaded with the content-type of '/' but when associating the blob object returned from the upload_blob method with the thumb field it does not work, since it expects the blob of being of content-type 'image/*' . I tried to force the mime_type of blob object to be of this value but it does not work either !

Thanks for your attention!

I'm reading from the firehouse using the synchronous client and I am encountering an error when processing a frame. A CBORDecodingError is raised and the logic when handling this error appears to be to reconnect to the feed. The client continues to read from the feed from the initial cursor and shortly after encounters the same unprocessable frame.

The cursor that the error occurs at is just after 93278360

I'm not sure what the solution should be. Should the client reconnect just after the problematic cursor and continue reading the feed, or is there a deeper issue in processing the data that needs to be resolved here?

Official client behavior:

1. Try to get info about the session using the access token (according to the "expired at" field).
2. If failed request .refresh_session() with the auth header equals to refreshJwt field.
3. try to get info about the current session again.

Now, the high-level client doesn't save refresh_token at all. The client should save it and process error handling on UnauthorizedError. Also, check the expired date before performing a new request.

Example of how to get and refresh the session:

session = client.com.atproto.server.create_session({'identifier': 'my-handle', 'password': 'my-pass'})
token = session.accessJwt
refresh_token = session.refreshJwt

refreshed_session = client.com.atproto.server.refresh_session(headers={'Authorization': f'Bearer {refresh_token}'})
new_token = refreshed_session.accessJwt

We can get the token "expired at" from decoding of Access JWT.

Header:

{
  "alg": "HS256",
  "typ": "JWT"
}

Payload:

{
  "scope": "com.atproto.appPass",
  "sub": "did:plc:PUP",
  "iat": 1683662608,
  "exp": 1683669808
}

The "exp" field. We can use it to refresh the token a bit later.

iat – 2023-05-09T20:03:28Z
exp – 2023-05-09T22:03:28Z

The access token lives for 2 hours.
The refresh token lives for 3 months.

Haven't actually tested, but the logic in is_record_type in atproto.xrpc_client.models.utils will likely fail for dicts:

def is_record_type(model: t.Union[ModelBase, dict], types_module) -> bool:
    if isinstance(model, RecordModelBase) and hasattr(types_module, 'Main'):
        # for now records in Main. could be broken late
        if isinstance(model, dict):  # custom record
            return types_module.Main._type == model.get('$type')

        return types_module.Main._type == model._type

    return False

I think it should be:

def is_record_type(model: t.Union[ModelBase, dict], types_module) -> bool:
    if isinstance(model, RecordModelBase) and hasattr(types_module, 'Main'):
        # for now records in Main. could be broken late
        return types_module.Main._type == model._type

    if isinstance(model, dict):  # custom record
        return types_module.Main._type == model.get('$type')

    return False

backward compatibility was added here #269