Object Storage for embedded pict-rs on from scratch install? about lemmy-docs HOT 7 CLOSED

Actually, @dessalines you can disregard my last request I'll be ok with it not using a custom subdomain to deliver the uploads. It's working correctly. Very much appreciate the help and confirming I did in fact need to edit the docker-compose.yml file despite building from source. Thanks!

from lemmy-docs.

The pictrs configuration must be in the docker-compose.yml , not the lemmy.hjson

from lemmy-docs.

The pictrs configuration must be in the docker-compose.yml , not the lemmy.hjson

Ah yes @dessalines this was it, thank you! Now, where am I able to add my custom hostname (files.example.com) for the bucket so the uploaded files aren't using localhost address?

I tried changing the http://localhost:8080/ in the hjson file and the 127.0.0.1:8080 in the systemd unit file, but those don't seem to be it.

Or, am I grossly misunderstanding how the object store for pict-rs works? And it's just push/fetch in the background with some form of dynamic loading using the root hostname? Sorry, first time using this.

from lemmy-docs.

Here's some of our vars in the pictrs docker-compose environment block. Outside of that I don't know your question, you'll have to ask in a pictrs chat.

lemmy.hjson :

pictrs: {
  url: "http://pictrs:8080/" # This will be different if you're not using docker.
  ...

docker-compose.yml :

# Object storage config
- PICTRS__STORE__TYPE=object_storage
- PICTRS__STORE__ENDPOINT=https://s3....
- PICTRS__STORE__BUCKET_NAME=...
- PICTRS__STORE__REGION=...
- PICTRS__STORE__ACCESS_KEY=...
- PICTRS__STORE__SECRET_KEY=...
- PICTRS__STORE__USE_PATH_STYLE=false

from lemmy-docs.

@dessalines No worries, I've got the object storage working. I have my s3 compliant bucket using files.example.com subdomain. Would the url param in the lemmy config need to change?

Is there some special config setting that will allow fetching the file hash over this subdomain rather than the localhost and/or apex domain? Thanks!

from lemmy-docs.

@dessalines Sorry to re-open this, but have had a significant update on this.

It appears my initial thought that adding the env vars to the docker-compose.yml file made the object storage work with my Lemmy instance and using Cloudflare R2 bucket (s3 compliant storage).

However, it became apparent that simply adding those to the config file did not allow objects to be uploaded. Instead, with the "from scratch" install and with the embedded pict-rs, I need to add the object storage deets to the the /etc/systemd/system/lemmy.service systemd file.

Only when those env vars were added to the systemd file did the media get uploaded to the object storage bucket. However, when the media is fetched, it presents an error. Below is an example of my systemd file:

[Unit]
Description=Lemmy Server
After=network.target

[Service]
User=lemmy
ExecStart=/opt/lemmy/lemmy-server/lemmy_server
Environment=LEMMY_CONFIG_LOCATION=/opt/lemmy/lemmy-server/lemmy.hjson
Environment=PICTRS_ADDR=127.0.0.1:8080
Environment=PICTRS_OPENTELEMETRY_URL=http://otel:4137
Environment=PICTRS__API_KEY=API_KEY
Environment=PICTRS__MEDIA__VIDEO_CODEC=vp9
Environment=PICTRS__MEDIA__GIF__MAX_WIDTH=256
Environment=PICTRS__MEDIA__GIF__MAX_HEIGHT=256
Environment=PICTRS__MEDIA__GIF__MAX_AREA=65536
Environment=PICTRS__MEDIA__GIF__MAX_FRAME_COUNT=400
Environment=PICTRS__STORE__TYPE=object_storage
Environment=PICTRS__STORE__ENDPOINT=https://redacted.r2.cloudflarestorage.com
Environment=PICTRS__STORE__BUCKET_NAME=lemmy
Environment=PICTRS__STORE__REGION=auto
Environment=PICTRS__STORE__USE_PATH_STYLE=false
Environment=PICTRS__STORE__ACCESS_KEY=redacted
Environment=PICTRS__STORE__SECRET_KEY=redaceted
Environment=RUST_LOG="info"
Restart=on-failure
WorkingDirectory=/opt/lemmy

# Hardening
ProtectSystem=yes
PrivateTmp=true
MemoryDenyWriteExecute=true
NoNewPrivileges=true

[Install]
WantedBy=multi-user.target

Uploaded media shows a {"msg":"Error in store"} response, ex: https://h4x0r.host/pictrs/image/92426ee1-8b2e-4edd-bbd5-b4a31455f78b.jpeg

I suspect that pict-rs is unable to use the sled repo db file to correctly match the hash for the image to the object storage. Any idea how I might be able to resolve this? Thanks!

from lemmy-docs.

Its possible you're using the wrong env vars for that version of pictrs, they've changed recently. I'd suggest asking these in the pictrs chat #pictrs:matrix.asonix.dog

from lemmy-docs.